Communications of the ACM

Communications of the ACM

Inconsistencies and Disconnects

By Jeffrey A. Stone, Elinor Madigan
Communications of the ACM, April 2007, Vol. 50 No. 4, Pages 76-79
10.1145/1232743.1232751
Comments

The overwhelming popularity and usefulness of information and communication technology (ICT) poses a critical problem for higher education. Colleges and universities increasingly emphasize ICT for student learning; however, the ICT skill level of incoming freshmen is often less than is needed for academic success. Compounding this problem is the changing workplace, which places increasing value on the ability of workers to be effective consumers of technology, as well as information.

In order to participate in today's high-performance workplace, students must be knowledge workers, solve problems, analyze knowledge, and use technology effectively. A review of the literature highlights the fact that success in the workplace is determined by one's ability to collect, organize, assimilate, and interpret information. A lack of basic ICT skills inhibits one's ability to perform valid research and information analysis, as well as communicate ideas effectively [5]. This lack of basic ICT skills also inhibits other fundamental aspects of knowledge work, including coordination, document sharing, knowledge exchange, and collaboration [4]. Given the nature of knowledge work, any deficiency in skills would tend to hinder both students' and workers' ability to maintain and build their knowledge bases [4].

Universities must ask: How do students become effective consumers of technology? Is it the responsibility of the individual states through their K–12 curricula? Is it the responsibility of higher education, the last stop before students enter the work force? Many states have sought to address the need for technologically literate students through statewide curriculum standards, an approach commonly used in traditional subjects like reading, writing, arithmetic, and science.

In order to measure and understand the value states place on ICT literacy for their graduates, we performed a content analysis of state curriculum standards in 10 states in 2004. Focusing on ICT literacy outcomes for high school students, we found a growing acknowledgment of ICT literacy as a fundamental skill, along with a diverse set of technological skill mandates. We also found that some states (such as New Jersey, Ohio, and Pennsylvania) emphasize technology education. Others (such as Florida, Illinois, and Maryland) seem to have little or no firm ICT literacy standards (see Table 1). In many cases, we found the language of the standards to be so vague, it precluded identification of specific skills as learning outcomes.

Given the ambitious nature of some state curriculum standards, researchers must ask whether or not they translate into learning outcomes. Prior research indicates that a majority of high school graduates throughout the U.S. lack the skills necessary to succeed in both school and the workplace; they also lack the higher-order thinking skills required to use technology effectively [3, 6]. A 2003 report by the Association of American Universities and the Pew Charitable Trusts detailed the skills needed in the core areas—mathematics and English—but paid little attention to ICT literacy. While the report identified information gathering, as well as the ability to identify valid sources of information and assemble information to support an argument, as a fundamental skill, it lacked guidelines for other ICT skills [2].

What about the students themselves? While faculty and administrators often expect incoming freshmen to be technologically literate, how many of them are actually prepared for college? State standards and curriculum guidelines may seem adequate on paper, but are new college students able to use technology and become the knowledge workers of tomorrow? Our study sought to examine these questions. Specifically, we wanted to see whether students' perceptions of their technological prowess aligned with the reality of their skills and with the ICT literacy mandated by state curriculum standards.

The data we examined came from three sources: our survey of technology skills in the summer of 2004, our pilot study of follow-up skills in the fall of 2004, and the content analysis we discussed earlier of the state curriculum standards in a sample of 10 states. We asked all incoming first-year students in a testing and advising program at two branch campuses of the Pennsylvania State University (Schuylkill and Harrisburg) to participate in a survey of technology skills. We used current literature and faculty input to identify 20 such skills in four technology domains—basic computing, applications, Internet, and research—on which the students rated themselves on a scale of 1–5 (1 = "no knowledge" and 5 = "expert knowledge") [5]. The survey, administered during the summer of 2004, included questions on the 20 skills, as well as on demographic information and technology access. Participation included 502 incoming first-year students with a response rate of 57.6 percent (n = 289).

During the fall of 2004, we asked students in a first-year seminar at the Schuylkill campus to participate in a research assignment. The related skills were from three domains: applications, Internet, and research. Security concerns precluded evaluation of basic computing domain skills in this follow-up assignment. We conceived the assignment after analyzing the preliminary results of the survey; consequently, this made it difficult to incorporate the research assignment into the first-year seminar classes. We therefore viewed the resulting study as a pilot study, with 38 of the 350 first-year seminar students who participated.

We calculated a score for the 10 skills for each participant in the survey and for the follow-up assignment, as well as for each state. We used the Kruskal-Wallis H test—the alternative to the one-way ANOVA for nonparametric data—to determine whether there were differences in technology scores among the three groups. We then used the Mann-Whitney U test—a nonparametric test for assessing whether two medians are the same—to determine the locations of any differences.

Our analysis found significant differences in the technology scores among the three groups (Kruskal-Wallis H = 36.271, p=0.000); the Mann-Whitney results (see Table 2) found no significant differences between the state standards and the performance skills of the students in the follow-up assignment group (USTS = 26.37 vs. UFG = 17.40). However, we did find significant differences between the state standards and the students' perceptions of their abilities; student perceptions were much higher than the state standards (USG = 146.35 vs. USTS = 64.85). We also found a significant difference in student perceptions of their ability to perform these skills; the perceptions were much higher than their actual ability (USG = 167.29 vs. UFG = 86.42).

Back to Top

Conclusion

The students who participated in these studies perceived themselves as having excellent technology skills. Based on this perception, it would appear they would be able to navigate the complexities of academic learning. Most if not all universities in the U.S. provide online tools, courses, and curricula. Faculty also may have high expectations of student skills. Clear from the studies is that university students do not generally perform at the level they themselves believe they perform. However, their ability does align with the weak state technology curriculum standards we included in the study.

The gap between perception and reality creates an atmosphere of frustration for students and faculty alike. The results of the follow-up assignment group indicated that students were not able to perform basic research using technology, evaluate their sources, or distinguish between the research value of a library source and an Internet source. Students also have difficulty communicating electronically—sending email, attaching documents to email messages, and using application programs. Incoming first-year students are confronted by a technologically complex environment for which they are ill-prepared. Their lack of a solid foundation of ICT knowledge and skills may well hinder their success in higher education, as well as in their later performance in the workplace.

---

Students do not leave high school with an acceptable level of ICT knowledge necessary to function at the university level or to be able to perform in the workplace.

---

The majority of the 10 state standards we surveyed did not cover the basic skills needed to perform research, communicate, and navigate Web sites. Nor did they cover the basic computing skills needed to install and remove programs or work with applications. Students do not leave high school with an acceptable level of ICT knowledge necessary to perform at the university level or in the workplace.

Moreover, the state standards were not consistent and did not align with the Association of College and Research Libraries' recommendations for information literacy [1]. As technology becomes even more ubiquitous in higher education, as well as in the workplace, students are arriving unprepared for this technological environment.

The inconsistency and disconnect among state ICT standards, student ICT abilities, and student perceptions of their ICT abilities is disturbing and must be addressed. Even though the global economy requires ICT-literate knowledge workers, educators are not preparing them to succeed. There must be a consensus among educators and business and government leaders on the definition of ICT literacy and a generally accepted road map of how to produce an ICT-literate individual. National standards for the ICT knowledge and skills necessary for university success must also be developed to ensure that the U.S. remains economically competitive in the global marketplace.

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2007 ACM, Inc.

---

No entries found