Communications of the ACM
Home/Magazine Archive/February 2010 (Vol. 53, No. 2)/Practical Intelligence in IT: Assessing Soft Skills.../Full Text
Virtual Extension
Practical Intelligence in IT: Assessing Soft Skills of IT Professionals
What qualities make a successful it professional? Certainty strong technical skills are sine qua non. As a result, the technology geek remains the stereotype of an information technology (IT) professional.5 Indeed, when companies hire IT professionals, their focus is often on the "hard" skills needed to perform the work, such as years of Java programming experience.
However, there is a growing (and gnawing) awareness that technical skills alone are insufficient for success in IT, particularly in today's dynamic, distributed and complex workplace. Companies are exploring outsourcing and offshoring to become more flexible and contain costs while strategically leveraging IT. Consequently, IT professionals (whether onsite or offshore, in-house or outsourced) must acquire a broader set of skills beyond their traditional technical skills. These broader managerial or interpersonal skills are generically labeled "soft skills."2,5
Despite the increasing importance of soft skills, very little systematic research has conceptualized such skills and even less has measured these skills.2 Given this gap in the literature, this article introduces "practical intelligence" as the overarching concept that could provide a better understanding of this broader set of skills required of IT professionals. We describe the development of the SoftSkills for IT (SSIT) instrument for assessing an IT professional's level of practical intelligence, and report the results of a study that validates this instrument by comparing the practical intelligence of experienced IT professionals versus novices. We conclude by drawing out implications of our study for IT recruitment, training and development, and suggest future research directions.
Defining Practical Intelligence of IT Professionals
Wagner and Sternberg, renowned cognitive psychologists, first identified a broader set of managerial, interpersonal or "soft" skills as "practical intelligence."8,11 They observed that graduates who performed well academically in school (school smarts) did not necessarily perform well in the workplace (street smarts). School smarts may not translate technical or academic intelligence into street smarts because the problems faced in everyday work life often have little relation to the knowledge or skills acquired through formal education or the abilities used in classroom activities. Practical intelligence is, therefore, different from the kind of intelligence associated with academic success.8
Based on Wagner and Sternberg's theory, we contend that having technical knowledge and skills are important and necessary; but technical knowledge and skills are insufficient for success in IT. Successful IT professionals require practical intelligence in addition to technical knowledge and skills. We formally define the practical intelligence of IT professionals as the managerial, intrapersonal, and interpersonal skills that are used to resolve IT-related work problems.
Practical intelligence in the workplace is typically obvious when individuals apply their managerial, self-management, interpersonal abilities, knowledge and skills to work-related contexts.8,11 Unlike technical knowledge of a factual and detailed nature, practical intelligence is based on an abstract or summary representation of specific procedural rules. An example of practical intelligence at work is knowing which tasks should be done first. The tacit nature of practical intelligence implies that practical intelligence can be inferred only from actions taken or from statements provided. Moreover, practical intelligence is often tied to a particular context and may not be transferable across domains. Together, the tacit and context-specific nature of practical intelligence helps to distinguish experts from novices.1
Drawing on Wagner and Sternberg11 we propose a taxonomy of IT practical intelligence comprising four dimensions: managing tasks; managing career; managing self; and managing others (Figure 1).
Managing Tasks. Practical intelligence in managing tasks refers to an IT professional's know-how and draws upon procedural knowledge and skills to perform specific tasks related to IT. This practical know-how is typically acquired by "learning the ropes" and "getting your hands dirty."11 In the IT context, managing tasks often involves knowing where and how to obtain non-IT domain knowledge. For example, IT professionals must frequently develop information systems for unfamiliar domains, and to do this successfully they need to know who knows what and which strategies are most effective for searching and obtaining domain knowledge.10
Managing Career. In managing careers, IT professionals know their career goals and understand how an existing assignment fits into the larger context of their entire career.6 IT professionals manage their careers effectively if they can anticipate which skills will be most valuable in the future and can select or shape projects to develop those skills. IT professionals are usually offered an assortment of IT projects. Further, the trend towards outsourcing and IT service management provides IT professionals with an even wider range of IT professional roles and career alternatives. For IT professionals who do not have the opportunity to select a project, it may still be possible to structure a project in order to develop a particular skill; for example, if assigned to maintain systems, an IT professional could nevertheless be proactive in looking for learning opportunities in improving the system. As a result, practical intelligence in managing one's career includes knowing which IT project assignments to accept or decline or how to structure an IT project assignment in order to progress along a successful career path.6
Managing Self. Managing self focuses primarily on the intrapersonal competencies of self-motivation and self-management.11 Self-motivation and self-management are important for achieving performance excellence, as evidenced by studies consistently finding high performers also having high levels of self-awareness.11 Hence, the objective in managing self is to maximize one's productivity. Overcoming procrastination or knowing how to prioritize projects based on importance and urgency are examples of managing self. High performing individuals possessing strong self-management abilities also tend to positively impact fellow employees in task interdependent work by not procrastinating and by completing tasks on time.11
Managing Others. Managing others is perhaps the most challenging dimension of practical intelligence needed by IT professionals. IT professionals face a diverse range of "others" in the course of their work.3,7,10,13 Reflecting this idiosyncratic nature of IT, we identify seven sub-types of "others": superiors, permanent subordinates, contract subordinates, peers, users, clients, and vendors.
Perhaps the most dominant subtype is managing subordinates. Organizations may require IT professionals to manage contract workers (contract subordinates) as well as employees (permanent subordinates).3 Hence, an IT professional managing a systems development project must be aware of the varying interests, motivations, knowledge levels and skills that subordinates bring with them to the IT team.6 While contract subordinates offer technical knowledge that an organization lacks, contractors and permanent employees have work attitudes and practices that may be dissimilar. Thus, an IT professional must be able to motivate and direct the efforts of diverse individuals.
In addition to managing subordinates, IT professionals interact and work with superiors and peers. For example, these may include IT Operations Managers or IT Training and Development Managers. The practical intelligence required to work with superiors and peers covers a broad range of business and political skills.
Users and clients are the most frequently emphasized stakeholders in the IT profession. As such, the ability to maintain productive user and client relationships is a vital interpersonal skill.10 In our framework of IT practical intelligence, we differentiate the terms client and user. Clients are the ultimate sponsors of an IT project, and users are those who actually operate and use the system. An increasingly sophisticated user community adds greater urgency for developing practical intelligence to manage users and clients. The common practice of assigning end-user managers as project leaders requires IT professionals to interact with user managers not only as users, but also as peers or superiors.
Finally, as the IT profession moves towards outsourcing IT functions, strategic alliances and partnerships, IT professionals must work with vendors and consultants who may be onsite or offshore.6,13 As IT organizations explore partnerships and alliances with external service providers, IT professionals must acquire the vital abilities to manage these service providers.4
Developing SoftSkills for IT (SSIT)
The tacit nature of practical intelligence makes it difficult to observe and thus challenging to measure. Wagner and Sternberg11 have proposed using a critical incidents methodology to measure practical intelligence. Accordingly, we adopt this critical incidents approach in developing SoftSkills for IT (SSIT) an instrument that uses work incidents to assess an IT professional's practical intelligence.
Eliciting Incidents. We elicited a variety of incidents faced by IT professionals in the workplace by interviewing 37 senior IT managers, systems analysts and consultants from a variety of user organizations and IT consulting firms. These IT professionals were asked to relate "war stories", i.e. work incidents that were particularly challenging. From these interviews, we collected 132 work incidents, which we subsequently classified according to the ten dimensions of IT practical intelligence in Figure 1. The most frequently reported incidents included managing clients (20%), users (16%), and permanent subordinates (14%), followed by managing tasks (9%), contract subordinates (9%), superiors (8%), self (7%), vendors (7%), careers (6%), and peers (5%).
Developing SSIT. To develop SSIT, we randomly picked one incident for each of the ten dimensions of IT practical intelligence and developed a computerized version to facilitate the validation study. SSIT presents one incident for each of the ten dimensions of IT practical intelligence and requires the participant to generate responses to resolve the incident. In addition to the respondent's answer, SSIT also automatically captures the time elapsed to respond to each incident using the system clock, and collects demographic information including years of work experience, gender and age. Figure 2 shows an example of the managing career incident in SSIT.
Comparing Experienced IT Professionals and Novices with SSIT
As practical intelligence is typically acquired through experience in real-world settings, we would expect that experienced IT professionals have different levels of practical intelligence than novices.9,11 Thus, in our validation of SSIT, we assessed the practical intelligence of both expert and novice IT professionals. Using experts and novices allows the essential differentiator of IT experience to surface.1,11
We administered SSIT to 68 full-time, practicing IT professionals enrolled in a part-time MBA program (average age = 30 years; work experience = 3.9 years). We refer to this group as "experienced IT professionals." We also administered SSIT to 54 first year IT undergraduates (average age =21; work experience = 2.5 months). We refer to this group as "novices."
IT practical intelligence is assessed along three complementary dimensions: the number of responses provided per incident; the average time taken to provide each response; and the quality of responses. The number of responses reflects the breadth of knowledge held by an individual with respect to a situation while the average time taken measures the cognitive accessibility of each response. The quality measure assesses the appropriateness of a response in resolving an incident. An expert panel of five senior IT managers rated the quality of each response in resolving each incident using a 7-point scale ranging from 1 (extremely poor) to 7 (extremely good) (see Figure 2 for example of responses and expert panel ratings). The inter-rater agreement (rwg) across the experts' ratings of quality responses over the ten dimensions of IT practical intelligence ranged from 0.73 to 1.00 with a mean of 0.93, indicating a high level of agreement. The average expert rating for each response was applied to a respondent's specific responses. The respondent's overall quality measure on each dimension was then computed using a geometric mean of the quality ratings for all of the respondent's responses to give higher weight to the quality of more cognitively accessible responses.
Findings
We conducted a multivariate analysis of covariance (MANCOVA), analyzing differences between experienced IT professionals and novices on the three measures, and controlling for the gender, age and work experience of each respondent.
Experienced IT professionals generated significantly more responses than novices (Table 1); specifically, more responses on managing career; permanent subordinates; peers; clients; and vendors. There were no significant differences in the number of responses on incidents relating to managing tasks; self; superiors; contract subordinates; and users.
Time Taken Per Response. We found experienced IT professionals took significantly less time per response to complete each incident compared to novices (Table 1); specifically, less time on managing career; superiors; permanent subordinates; contract subordinates; peers; users; clients; and managing vendors. There were no differences between the experienced IT professionals and novices on managing tasks and self.
Weighted Quality of Responses. The results indicated experienced IT professionals generated responses that were rated significantly higher in quality compared to responses generated by novices (Table 1); specifically, higher in quality on managing tasks; career; permanent subordinates; peers; clients; and managing vendors. For example, experienced IT professionals presented with the managing career scenario (see Figure 2) would accede to top management's request to remain with the legacy systems but would also devise ways to increase their value to the organization. Novice IT professionals would insist on moving to new applications or resign. There were no differences on managing self; superiors; contract subordinates; and users.
What We Learned about the Practical Intelligence of IT Professionals
In totality, we find that experienced IT professionals performed better than novices by providing a significantly larger repertoire of responses, taking significantly less time in generating this larger repertoire, and providing responses that were of significantly higher quality.
On managing tasks, experienced and novice IT professionals generated the same number of responses and took the same amount of time to respond. However, experienced IT professionals provided higher quality responses suggesting that they have successfully refined their responses to suit the IT work context.9 Experienced IT professionals (on the job) and novices (in undergraduate assignments) tend to have similar training or experience in developing information systems for user domains they know little about. The difference in their respective levels of practical intelligence lies in the quality of their responses offered to resolve work incidents. Further analyzing their responses, we find that IT novices are likely to turn to convenient sources of information such as colleagues or friends for domain information while experienced IT professionals are more likely to turn to domain experts such as users.12
On managing careers, experienced IT professionals performed significantly better on all three measures. Experienced IT professionals are more likely to encounter career issues at work. A higher frequency of exposure to similar work incidents equips experienced IT professionals with a larger set of responses that remains salient and allows the individual to respond very quickly.9
Interestingly, we find that novices performed as well as experienced IT professionals on managing self. This finding suggests that, at least with this sample, IT novices have learned to develop and refine appropriate strategies for managing self in their school environment as much as experienced IT professionals have in their workplace.1
Finally, on managing others, experienced IT professionals performed significantly better on all measures for managing permanent subordinates, peers, clients, and vendors. Experienced IT professionals consistently interact with these stakeholders in their daily work. Accordingly, repeated encounters of such incidents have helped them develop and refine a salient set of scripts.9 Moreover, experienced IT professionals are more likely to have been trained by their employers in the softer skills, such as negotiation and client relationship management. Novices, in contrast, have very limited exposure to such incidents or to relevant training and thus, lack sufficient opportunities to develop salient and refined responses.9 The lack of exposure and longer recall times for IT novices may suggest that they are drawing on vicarious experiences and analogical reasoning to develop suitable responses.
In summary, our results suggest that IT practical intelligence can be measured and that SSIT can detect differing levels of practical intelligence between practicing IT professionals and novice IT undergraduates. It appears that work experience, relevant training, and mechanisms such as mentoring, may offer opportunities for experienced IT professionals to develop and refine their practical intelligence.
Practical Implications
This study has implications for recruitment, training and development of future IT professionals. In recruiting IT professionals, SSIT can potentially be included as part of a structured interview process. In structured interviews, hypothetical work-related incidents from SSIT could be used to evaluate applicants' responses against a set of established criteria. If an expert panel is drawn from the company, the experts' assessments of responses on work incidents could help socialize and orient newly-hired IT professionals to the unique culture and norms of the organization.4 In essence, the expert panel's response becomes a corporate library of best practices endorsed by senior IT management that can be used as an effective socialization and training tool for IT professionals joining the organization.
SSIT can similarly be used to help a seasoned IT professional develop practical intelligence along a particular dimension where improvement is needed. For example, organizations may use a 360 degree type assessment to evaluate whether an IT professional exhibits practically intelligent behaviors at work. Then, if an IT professional falls below an established organization norm for a particular IT practical intelligence dimension, human resource specialists could use SSIT to help focus the training and development effort in increasing the professional's level of practical intelligence for that dimension. Subsequent to the training, the IT professional's actual behavior could be assessed using the 360 degree or other type of peer evaluation to discern whether the individual's behaviors reflect an increased level of practical intelligence along that particular dimension.
Finally, SSIT could also be introduced into the IT curriculum to nurture the next generation of IT professionals. By integrating formal academic education with practice-based developmental approaches such as SSIT, IT students may move more quickly from being novices to experienced IT professionals.
Looking Ahead
Several immediate research directions flow from this study. First, although we primarily designed SSIT to examine the practical intelligence of systems development professionals, the instrument can be easily extended to examine the practical intelligence of IT professionals in other roles such as in IT management or in IT infrastructure and support. Future research could then examine the relative importance of each dimension of practical intelligence for the various roles that IT professionals perform in organizations. For example, do IT managers have different types of practical intelligence than IT programmers? Perhaps certain dimensions of practical intelligence are more or less important for performance in different IT jobs. The process to ascertain their relative importance could start with a job analysis to establish the nature and scope of different roles in IT. Further research could examine whether the salience of a particular dimension of practical intelligence for an IT role changes over time and in different geographical contexts.3,4 Finally, future research may test the effectiveness of various training methods for developing practical intelligence. A possible research question is: which is the most effective way to develop IT practical intelligence? This line of research may compare the efficacy of improving practical intelligence through methods such as formal training, mentoring or actual work experiences.
References
1. Adelson, B. When novice surpass experts: The difficulty of a task may increase with expertise. Journal of Experimental Psychology 10, 3 (1984), 483495.
2. Ang, S. and Slaughter, S. A. The missing context of information technology personnel: A review and future directions for research. In Framing The Domains of IT Management: Projecting The Future From The Past, (Pinnaflex Educational Resources, Cincinnati) 2000, 305328.
3. Ang, S., and Slaughter, S.A. Work outcomes and job design for contract versus permanent information systems professionals on software development teams. MIS Quarterly 25, 3 (2001), 321350.
4. Ang, S., Van Dyne, L., Koh, C., Ng, K.Y., Templer, K. J., Tay, C., and Chandrasekar, N. A. Cultural intelligence: Its measurement and effects on cultural judgment and decision making, cultural adaptation, and task performance. Management and Organization Review 3, 3 (2007), 335371.
5. Enns, H.G., Ferratt, T.W. and Prasad, J. Beyond stereotypes of IT professionals: Implications for IT HR practices. Comm. ACM 49, 4 (2006), 105109.
6. O'Mahony, S. and Bechky, B.A. Stretchwork: Managing the career progression paradox in external labor markets. Academy of Management Journal 49, 5 (May 2006), 918941.
7. Slaughter, S. A., & Ang, S. Employment outsourcing in information systems. Comm. ACM 39, 7 (July 1996), 4754.
8. Sternberg, R.J. and Hedlund J. Practical intelligence, g, and work psychology. Human Performance 15, 2 (2002), 143160
9. Tesluk, P.E. and Jacobs, R.R. Toward an integrated model of work experience. Personnel Psychology 51, 2 (1998), 321355.
10. Trauth, E.M., Farwell, D.W. and Lee, D. The IS exception gap: Industry expectations versus academic preparation. MIS Quarterly 17, 3 (1993), 293307.
11. Wagner, R.K. and Sternberg, R.J. Practical intelligence in real-world pursuits - The role of tacit knowledge. Journal of Personality and Social Psychology 42, 2 (1985), 436458.
12. Wegner, D.M., Erber, R. and Raymond, P. Transactive knowledge in close relationships. Journal of Personality and Social Psychology 61, 6 (1991), 923929.
13. Willcocks, L., and Lacity, M. Global Sourcing of Business and IT Services. Palgrave, U.K. 2006.
Figures
Figure 1. Taxonomy of Practical Intelligence of IT Professionals
Figure 2. An Example of Managing-Career Incident with Responses and Expert Ratings
©2010 ACM 0001-0782/10/0200 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.
No entries found