Communications of the ACM

BLOG@CACM

MOOCs Need More Work; So Do CS Graduates

By Mark Guzdial, Joel C. Adams
Communications of the ACM, January 2014, Vol. 57 No. 1, Pages 18-19
10.1145/2555813
Comments (4)

http://bit.ly/1gX4AYl
October 18, 2013

MOOCs in the Coursera, Udacity, and edX form are tightly tied to CS. The leaders of the xMOOC movement came out of computer science, and most of the first generation of xMOOCs focused on teaching computer science. Many of the MOOC evaluations so far have been expert reviews. Our Learning Sciences and Technologies seminar at Georgia Tech's College of Computing just read Moti Ben-Ari's travelogue on his experiences in Coursera's and Udacity's introductory CS MOOC. The empirical results of the first rounds of MOOCs on intro courses are now in, so it is worth considering how they are doing.

Karen Head has finished her series of posts in The Chronicle of Higher Education on the freshman-composition MOOC she taught with Gates Foundation funding. The stats were disappointing—only 238 of the approximately 15K students who did the first homework finished the course. That is even less than the ~10% we saw completing other MOOCs.

Karen Head writes:

No, the course was not a success. Of course, the data are problematic: Many people have observed that MOOCs often have terrible retention rates, but is retention an accurate measure of success? We had 21,934 students enrolled, 14,771 of whom were active in the course. Our 26 lecture videos were viewed 95,631 times. Students submitted work for evaluation 2,942 times and completed 19,571 peer assessments (the means by which their writing was evaluated). However, only 238 students received a completion certificate—meaning that they completed all assignments and received satisfactory scores.

Our team is now investigating why so few students completed the course, but we have some hypotheses. For one thing, students who did not complete all three major assignments could not pass the course. Many struggled with technology, especially in the final assignment, in which they were asked to create a video presentation based on a personal philosophy or belief. Some students, for privacy and cultural reasons, chose not to complete that assignment, even when we changed the guidelines to require only an audio presentation with visual elements. There were other students who joined the course after the second week; we cautioned them that they would not be able to pass it because there was no mechanism for doing peer review after an assignment's due date had passed.

Georgia Tech also received funding from the Gates Foundation to develop a MOOC approach for a first-year college physics course. I met with Mike Schatz, the lead teacher on that effort. It is a remarkable course, including a "laboratory" where students take videos of moving objects, then construct computational simulations in Python to match the real-world observations. The completion results were pretty similar to Karen's: 20K students signed up, 3K students completed the first assignment, and only 170 finished.

In terms of empirical studies, Mike had an advantage that Karen did not—there are standardized tests for measuring the physics knowledge he was testing, and he used those tests before and after the course. Mike said the completers fell into three categories: those who came in with a lot of physics knowledge and who ended with relatively little gain, those who came in with very little knowledge and made almost no progress, and a group of students who really did learn a lot. They do not yet know the relative percentages of the three categories. However, it is clear that being a completer does not mean that anything was learned.

I also met with Jason Freeman who finished his Survey of Music Technology MOOC for Coursera. His results were a bit better: 24K signed up, 13K visited, and 900 completed. It seems the more advanced the course, the better the completion rate.

The report from San Jose State University's MOOC experiment with a remedial mathematics course found: The researchers say, perhaps unsurprisingly, that what mattered most was how hard students worked. "Measures of student effort trump all other variables tested for their relationships to student success, including demographic descriptions of the students, course subject matter, and student use of support services."

It is not surprising, but it is relevant. Students need to make an effort to learn. New college students, especially first-generation college students, may not know how much effort is needed. Who will be most effective at communicating the message about effort and motivating that effort—a video of a professor, or an in-person professor who might even learn your name?

MOOC companies have set a goal of democratizing education. They aim to make education available to people who would not otherwise get access to education. MOOCs are not yet succeeding at that goal. The empirical findings (for example, Armando Fox's results at Berkeley and Tucker Balch's demographics) suggest MOOCs draw mostly men (especially in the CS MOOCs) who already hold degrees and are overwhelmingly from the U.S. and the developed world. Right now, xMOOCs seem most successful for professional and continuing education. Gary May, dean of engineering at Georgia Tech, recently wrote in Inside Higher Ed, "The prospect of MOOCs replacing the physical college campus for undergraduates is dubious at best. Other target audiences are likely better-suited for MOOCs." That summarizes the current state pretty well.

Back to Top

Joel C. Adams "Hot Job Market for Computer Science Graduates"

http://bit.ly/JaSMim
April 19, 2012

Back in 2010, companies were hiring computing graduates as fast as we could produce them, but there was a widespread misperception that U.S. computing jobs were in danger of being off-shored. Many people still believe this.

To counteract this, I put together a Web page called the Market for Computing Careers. My basic idea was to create visualizations of the U.S. Bureau of Labor Statistics (US-BLS) employment projections and data on bachelor's degrees awarded, to help people—especially students, parents, teachers, and guidance counselors—understand what the U.S. government was predicting the job market for CS graduates would be like. I had seen fragments of this data reported in piecemeal fashion, but I wanted to collect these pieces in one place to try and tell a more complete story.

I updated this Market For Computing Careers page using the new US-BLS 2010–2020 employment projections, as well as the most recent U.S. STEM graduation data (2008) available from the National Science Foundation.

The new US-BLS projections predict the already hot job market for computing professionals will become even hotter this decade. Excluding health care, these projections predict the five careers in science, technology, engineering, and mathematics (STEM) with the most growth will all be in computing:

Taken collectively, these projections predict computing careers will make up 73% of the jobs in STEM careers this decade compared to 16% in (non-software) engineering, 9% in the natural sciences, and 2% in mathematical sciences.

To try to predict how competitive the job environment will be, we can combine the US-BLS total job projections (new jobs plus retirement-replacements) with graduation data from the NSF. Dividing the total projected computing jobs per year by the number of computing bachelor's degrees awarded in the most recent year yields a jobs/grads ratio of 3.5 computing jobs per person graduating with a bachelor's degree in computing. (In 2010, this computing jobs/grads ratio was 2.9.) By contrast, the total jobs/grads ratio is below 1.0 in every other STEM area.

This data suggests on average, there will be 97,000 more U.S. computing jobs than graduates each year, a shortfall that even the current H1B Visa Quota is insufficient to address. To meet this decade's demand with homegrown talent, U.S. colleges and universities would need to produce 3.5 times as many computing graduates per year as they did in 2008. The Taulbee Survey data has shown modest increases in computing graduation rates the past two years at Ph.D.-granting institutions, but the observed increases do not come close to addressing the projected demand.

Companies seeking U.S. computing professionals will thus be competing with other companies for a limited supply of personnel. We are already seeing this competition, as many of our students are receiving multiple internship offers, and many of our graduates are receiving multiple job offers. The US-BLS projections suggest this competition will likely increase this decade.

For visualizations of this data, see the 2012 Market For Computing Careers page at http://cs.calvin.edu/p/ComputingCareersMarket. For comparison purposes, see the 2010 Market For Computing Careers page at http://cs.calvin.edu/images/department/jobs/2018/.

Now is a great time to be a computing major as the abundance and variety of computing jobs over the next decade should make it relatively easy to find a career that is stimulating, fulfilling, and that compensates well. Help spread the word!

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2014 ACM, Inc.

---

Comments

---

Displaying all 4 comments