Communications of the ACM
Technical opinion
The Educational Productivity Paradox
Expenditures on technology in K12 public education have skyrocketed in recent years, growing from $2 billion in 19911992 to nearly $6 billion in 20032004 [5]. IT expenditures have been suggested to "contribute significantly to improved educational outcomes" [3]. It was fully anticipated that technology expenditures would show a positive impact on test scores in this study of 2,500 schools in Pennsylvania. I have analyzed data from these 2,500 districts and found no such relationships and little or no positive benefits accrued to schools that spent more on technology. There appears to be a productivity paradox in education.
IT Expenditures
Over the past two decades, a number of researchers have probed the issue of the productivity paradox in industry. Simply stated, this paradox is the perceived lack of productivity gains that have resulted from increased IT expenditures. This topic has become a major research issue and no conclusion has been reached on whether IT positively impacts productivity in the private sector.
Within this framework, recent increased computer and IT expenditures in education have gone relatively unnoticed and unstudied. In recent years large expenditures have been made in IT in educational institutions with little corresponding study of whether there have been productivity gains in the form of better educated students associated with these expenditures. The magnitude of the expenditures in the U.S. spent on IT in schools makes this an important research topic. In just a few sample school districts upgrading technology in schools cost millions of dollars. In Plano, TX, a $44 million bond issue was floated for new computers and software. Austin, TX, approved a bond issue of $26 million for technology and Cobb County, GA, approved a $32 million bond for technology [6]. With billions of dollars of taxpayer money in the balance, the issue of whether there is a productivity paradox in education is perhaps more important than whether there is a productivity paradox in private industry.
Productivity Paradox Research
As noted previously, most research has been undertaken to determine whether there is a productivity paradox in industry. In "The Productivity Paradox of Information Technology," Brynjolfsson [2] studied labor and commerce statistics and found that IT investments did not result in productivity gains. There was a negative relationship between output per hour and IT spending. A recent alternative conclusion was reached by Lehr and Lichtenberg [4], who analyzed firm-level data and found that computers contributed to improved productivity and have excess returns in comparison to other forms of capital. There has been little study done on whether a productivity paradox exists in education despite its significance in education and the economy. One of the significant articles on the association between computers and school performance was prepared by Attewell and Battle [1]. Their study reviewed the effects of home computers on standardized test scores in reading and mathematics, as well as a self-report on grades. Scores increased 3% to 5% for students with computers in their homes. Socioeconomic status was nearly four times as important but cultural and social exposure produced similar returns as computers in the home.
Background of this Study
This study measures the relationship between a variety of technology and non-technology related factors and student performance measured by standardized test scores. The data was analyzed using SPSS 10.0.5 using multiple linear regression and stepwise entry of independent variables. The results indicate the relative importance of technology in schools versus these other influencing factors. The data sets used for this study come from the Commonwealth of Pennsylvania Department of Education.
Included in the data were the results of an annual survey of computers, IT, and other technology, as well as library resources in each school in the state (including separate records for primary, secondary, middle, junior, and senior high schools where appropriate). Specific data from year 20002001 included:
- Internet in the school (yes, no)
- Internet in classrooms (yes, no)
- Broadcast TV in school (yes, no)
- Broadcast TV in classroom (yes, no)
- Cable TV in school (yes, no)
- Cable TV in classroom (yes, no)
- Total number of computers in the school
- Total number of CD-ROM titles in school
- Total number of titles in library
- Total number of titles in library that were checked out
Also included was the public school enrollment file for year 20002001. Total size of a school has been shown by researchers to be a significant factor in student performance.
A third file was integrated and contained the percentage of enrollment from low-income households in the local educational agency. This was used since many prior studies had suggested the income level of the district had a significant direct influence on student performance. The final file was a measure of student performance in the form of required PSSA (Pennsylvania System of School Assessment) scores including mandatory student testing and assessment in reading and mathematics in the 5th grade, 8th grade, and 11th grade. Data in the file included the percentage of students scoring proficient and above in mathematics, and the percentage of students scoring proficient and above in reading.
Results
In both studies, IT played a minor or insignificant role in educational test scores. Also in both studies nearly 50% of the variance in test scores was attributable to the significant factors. The first study used as its dependent variable the number of students who scored proficient and above in math. There are four categories of proficiency according to the PSSA scores: poor, basic, proficient, and superior. This study looked at influences on student achievement to the level of proficiency and above.
IT factors do not show consistent, positive relationships with higher educational math and reading test scores.
The most significant factor affecting math proficiency and above (see Table 1) was the percentage of households from low-income families. The relationship was strongly negative with each increased percentage resulting in a 0.5% decrease in percentage of students scoring proficient and above. Total number of students in the school also negatively impacted performance. Key positive factors included titles checked out from library per student, titles in library, and cable TV in school. If a school had cable TV, its proficiency level and above percentage was 6.2% higher. Computers per student and total number of computers in the school were both significant factors, but surprisingly showed negative impact on scores. As schools increased computers both in absolute numbers and per student, math proficiency actually decreased. One possible reason for this is that more time was spent with computers than actual math activities. Some factors included in the analysis but not deemed significant were Internet access, CD-ROMs, and broadcast TV in the school.
Reading proficiency and above (see Table 2) was also most affected by the percentage of low-income households in the district, with every percentage increase resulting in a half-percentage decline in the percentage of students scoring proficient and above. Cable TV in the school increased the reading percentage by 2.7%. Also positively affecting reading were number of titles in the library, titles checked out from the library per student, and Internet capability in the classrooms. Internet capability in the classroom positively affected the percentage by 1.3%. Negative effects were found with total number of students in the school (size of the school), computers per student, and titles in the library per student.
Conclusion
After reviewing the data from over 2,500 different school/class combinations in the state of Pennsylvania it has been found that IT factors do not show consistent, positive relationships with higher educational math and reading test scores. No IT factor positively affected math proficient level and above. Reading proficiency scores were positively and significantly affected by Internet availability in the classroom. But reading studies were negatively impacted by computers per student and total number of computers in the school. The study also showed that student performance based on test scores was negatively affected by the number of students in the school (school size) and by the percentage of students from low-income households.
It cannot conclusively be determined that the increased IT expenditures taking place in school districts today are positively influencing student performance. Careful review of educational IT spending should be undertaken.
References
1. Attewell, P. and Battle, J. Home computers and school performance. The Information Society 15 (1999), 110.
2. Brynjolfsson, E. The productivity paradox of information technology. Commun. ACM 36, 12 (Dec. 1993), 6777.
3. Glennan, T.K. and Melmed, A. Fostering the use of educational technology: Elements of a national strategy (1996); www.rand.org/publications/MR/MR682/contents.html.
4. Lehr, B. and Lichtenberg, F. Information technology and its impact on productivity: Firm-level evidence from government and private data sources, 19771993. The Canadian Journal of Economics 32, 2 (Apr. 1999), 335362.
5. Quality Education Data Inc. U.S. School Technology Spending Heading For $6 Billion (2003); www.scholastic.com/aboutscholastic/news/press_091603.htm.
6. Rowe, S. Utopia or a scary monsterA discussion of the effectiveness of computer technology. Contemporary Education 69, 3 (1998), 144149.
Tables
Table 1. Factors significantly influencing math students scoring proficient and above.
Table 2. Factors significantly influencing reading students scoring proficient and above.
©2005 ACM 0001-0782/05/1000 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2005 ACM, Inc.
No entries found