Communications of the ACM
Learning Computer Science Is Different Than Learning Other STEM Disciplines
A colleague who is a STEM Education evaluation expert asked me a great question: "Is learning CS really different from other STEM disciplines? If so, why?" That’s a great topic for a blog post! I have three reasons for thinking that learning CS is different than learning other STEM disciplines.
- Our infrastructure for teaching CS is younger, smaller, and weaker;
- We don’t realize how hard learning to program is;
- CS is so valuable that it changes the affective components of learning.
Infrastructure: In November 2016, Briana Morrison and I published an article in CACM about how CS education is different than other STEM education (see it in ACM DL here). Several of our arguments are relevant for this question, too. We don’t have the infrastructure than other STEM disciplines have. We know less about how people learn programming than what we know about how people come to understand algebra, evaporation, bacteria, and Newton’s Laws.
Here’s a simple numerical comparison to drive this point home. The major U.S. teaching organization for mathematics, the National Council for Teachers of Mathematics (NCTM), was established in 1920. The American Association of Physics Teachers (AAPT) was established in 1950. The Computer Science Teachers Association (CSTA) was established in 2005. The other STEM disciplines have decades more experience in defining learning progressions, establishing programs for preparing teachers, defining frameworks and standards, and creating and testing curriculum. We are simply not going to be as good at teaching CS because we have so little experience at it. That makes the learning experience different.
We don't know how hard learning programming is: Briana’s dissertation was a set of educational psychology experiments that she replicated in CS from comparable experiments in other STEM disciplines. She did not always come up with the expected results. She has published other papers explaining why an experiment didn’t work, or even replicating the experiment to show that the unexpected behavior wasn’t a fluke. I don’t think Briana showed that CS is harder than other STEM disciplines. Rather, I think she showed that we do not really understand why CS is so hard. The intro CS examples and problems that she selected for her experiments seemed reasonable, but were actually much more cognitively complex than the physics and mathematics examples in the original experiments. We simply don’t know yet how to measure the complexity of learning CS.
That was the conclusion of both the 2001 McCracken Working group report and the 2013 replication study. Even small changes to a programming assignment can massively increase the cognitive complexity of an assignment. We as experts have a blind spot: we fill in the blanks on programming problems, and we literally don’t see what’s so hard about a problem.
Probably the best example of this is the Rainfall Problem:
Design a program called rainfall that consumes a list of numbers representing daily rainfall amounts as entered by a user. The list may contain the number -999 indicating the end of the data of interest. Produce the average of the non-negative values in the list up to the first -999 (if it shows up). There may be negative numbers other than -999 in the list.
Most CS teachers think this is a pretty easy, pretty typical intro CS problem. But this problem is infamous. The Rainfall Problem has been used in classrooms and experiments for literally decades, and students always do terribly on the problem—nobody gets even half of their first-year students to get the solution right. In fact, Simon argued in 2013 that students were actually doing worse now than they did when the problem was first published in the 1980’s. Kathi Fisler beat the Rainfall Problem in 2014. She regularly gets the majority of her students to get it right. But we’re still not really sure why. Is it because of her curriculum, because she’s using functional programming, because of the data structures she’s using, or because she’s teaching higher-level functions? We don’t really know what makes programming so hard, and we don’t yet have enough theory to explain why it works when we get it right.
CS is so valuable and growing: My colleague Charles Isbell argues that CS is unique in Universities because we have both (a) a dramatically increasing number of students who want to major in CS and (b) a dramatically increasing number of non-majors who want more CS. The data support both parts of his claim.
The CRA "Generation CS" report and the National Academies’ recent report on CS undergraduate enrollment show that our enrollments are exploding with both a growing number of majors and a growing non-majors population. The non-majors are wise in their choices. The recent "Rebooting Jobs" report from Burning Glass and Oracle Academy shows that CS skills are the most rapidly growing skills requested in job ads, but only 18% of those job ads ask for a CS degree. Chris Scaffidi published a fascinating paper last year showing that workers who program (including building spreadsheets) earn more than similar workers who do not.
This makes the effect of teaching and learning in CS classrooms different than other STEM classes. Students want CS skills, but they don’t want be CS professionals—but there are lots of people in the room who do want to be CS professionals. That creates tensions and challenges. Does the teacher teach for the majors (who have values like the teacher and want jobs that the teacher understands) or for the non-majors? Do the non-majors get a sense that they "don’t belong"? CS has had measurably difficult classroom environments, even before we got flooded with students (see the great work on defensive climates in CS classes). Researchers are noticing the problem. The International Computing Education Research conference in 2017 had over a half dozen papers about emotion.
We are increasingly finding that the emotional component of learning computing (e.g., motivation, feeling of belonging, self-efficacy) is among the most critical variables. When you put more and more students in a high-pressure, competitive setting, and some of whom feel "like" the teacher and some don’t, you get emotional complexity that is unlike any other STEM discipline. Not mathematics, any of the sciences, or any of the engineering disciplines are facing growing numbers of majors and non-majors at the same time. That makes learning CS different and harder.
Comments
Briana Morrison
There's another reason that might make learning computing more difficult than other STEM subjects: students have little to no experience with it before arriving at the university level. Students learn math and science beginning with their earliest school experiences and continue throughout primary and secondary education. Most students may be exposed to "literacy" (or keyboarding/productivity) computing before the university, but how many have computer science learning experience. In the US, only around 1% of high schools offer AP computer science courses. If you've _never_ learned something before, it's likely to be more difficult (certainly newer) than subjects you've had years of experience with.
Mark Lewis
I want to start off by saying I appreciate the reference to the functional language study. We use Scala and teach higher-order methods. I'm very tempted to see how our students do with the Rainfall problem.
The headline for this article made me think it might go in a slightly different direction. This last fall I had a discussion with students about the difficulty of full majors. This piece really focuses on the introductory courses in CS, though the two aren't completely independent. I was a double major in Physics and CS, and I told them that when I went through, I definitely felt that Physics was the harder major. However, I'm not certain that is true anymore. The physics major now, roughly 20 years later, is pretty much the same as the one I went through. There are minor changes in emphasis, but the broad topics are the same. That isn't true for CS. The major is now larger and has quite a few courses that cover topics that didn't even exist 20 years ago. A few topics have been thrown out, but not nearly as much as what has been added. The growth of the role of computing in society has lead to many developments that have necessitated these additions to a proper CS education.
I feel that this need to fit more in the major filters down to CS1 as well, and pairs with the changing nature of how students have interacted with computers before they begin programming to make CS1 more complex. To get all the material in the major, we often feel a need to add more to the foundational introduction courses. Even if we didn't though, our students are no longer as easily motivated by simple text programs. To keep students interested, we have to have them do things that they find relevant, and those things often involve complexities that didn't even exist 10-20 years ago.
Mark Guzdial
Hi Briana! I'm not sure that the lack of experience with CS is that different than lack of experience with the deep parts of other STEM fields. Most students never had experience with cells, forces, or electron shells before entering their biology, physics, or chemistry classes.
Hi Mark! That's interesting -- I've noted several other people on Twitter interpreting the question differently than I did. Evidence that CS is harder is that our withdraw-or-fail rate (students who drop out or fail) is higher in CS than Physics or Calculus. More people give up, fail, or don't even try CS than other STEM subjects. That's the "harder" that I was trying to explain. Why don't students succeed in the first class? Other people interpreted that you did -- if you can succeed at both CS and another STEM subject, which is more cognitively challenging? I'm more worried about the people who can't succeed at CS, not the cognitive complexity. I don't know how to measure that latter, between fields.
Ray Toal
I agree with each of the three reasons, and find the second an always-fertile ground for research. The difficulty of first-year programmers finding correct solutions the Rainfall problem is yet another data point in a group of results that include (1) Jon Bentley's reporting of studies showing only 10% of professional programmers were able to write a correct binary search function (documented in his book Programming Pearls), as well as (2) the discovery of a bug in Bentley's own binary search algorithm from the same book (https://research.googleblog.com/2006/06/extra-extra-read-all-about-it-nearly.html)! This problem runs into large numbers of edge cases and the crazy behavior of modular integer arithmetic that our human brains might not handle naturally. Test-driven development and functional programming might help, though it's interesting that we may not yet know to what extent they do, as the cognitive complexity of learning computer science might be hard to measure.
Perhaps one of the themes from Eric S. Roberts' "Loop Exits and Structured Programming:
Reopening the Debate" paper (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125.1211&rep=rep1&type=pdf) may be relevant here. Roberts cites separate studies from Soloway and Shapiro contrasting different strategies for implementing search algorithms with and without mid-loop returns, and finding stunning differences in success rates among programmers taking the different approaches. Perhaps the issues highlighted in the article may not lie so much cognitive complexity but rather in our inability to find ways to line up programming with human intuition (to which is can be argued that mid-loop returns do).
But even human intuition in problem solving can differ greatly among people, as shown by Turkle and Papert (http://www.papert.org/articles/EpistemologicalPluralism.html). In addition to teaching the majors versus teaching the non-majors (with different desired outcomes), we need to learn how to teach to the top-down analysts as well as the bottom-up bricoleurs (with completely different learning styles). That makes for a diverse set of learners that we'll have to figure out how to accommodate well as our discipline's infrastructure matures.
Mark Lewis
Thanks for the response Mark. I think that the attrition rates, which you mentioned in your response to me, might actually relate a lot to what Briana mentioned. It is true that High School math doesn't resemble most of the math major, but most math majors start in Calculus, which does look a lot like their High School Pre-calc or Calc class. The situation is similar in Physics and probably Biology and Chemistry, which are the STEM topics taught broadly in High School in the US. High School Physics doesn't do much with Modern Physics, Quantum Mechanics, or other upper-division courses, but it looks a lot like Physics 101 at the college level, especially if it is a Calculus-based AP course. It would be interesting to compare the attrition rates in CS to those in Engineering, Geoscience, or other STEM fields that aren't well represented in High School curricula.
Of course, all of this is a side note to your post. The fact that small tweaks to a programming problem can turn it from being easy to quite hard, and that we generally aren't good at predicting in advance how much a change will alter the complexity of a problem is something that I really don't think applies to the sciences or Engineering. It might apply to Mathematics courses that involve proofs, but proofs don't commonly appear in the first semesters of the math major, and I have a feeling that the proofs that appear in undergraduate Mathematics are fairly standard and well understood. Thanks to the dynamic nature of CS and the changing role of computers in society, we often change up our introductory problems to make them more relevant and motivating. While well intentioned, and probably needed, these changes can have unexpected consequences when we accidentally produce something that is far more challenging than we intended.
Tom Kyle
I love this article, and pretty much agree with all the points made. I would like to add another perspective. Some students are taking computer science in high school. If students are taught computer science in high school, it will likely be much different that the same course in college, even if they pass an AP test.
I earned a BS in Computer Science in 1986. A LOT has changed since then to be sure. I worked as a computer programmer / analyst / project manager et. al. for over 25 year before getting into teaching at the secondary level. This year I'm teaching computer science at the high school level for the first time. (I taught middle school math the last 4 years). I don't have any concrete results yet but I'll certainly share them when I do. The point I want to make is that high school computer science teachers ARE NOT at all like college computer science teachers. Universities require a PhD to teach, and even community colleges require a master's degree. (I have an MBA). However, bachelor level teachers from other teaching subjects are being recruited to teach computer science in high schools. Even with that effort there are not enough qualified teachers, and more students want (and need) to take computer science. Dual credit programs with colleges will help. But I think we will have people without computer science degrees teaching HS computer science for the foreseeable future. This has both advantages and disadvantages. For starters, teaching children is entirely different that teaching adults. I've done both. I taught computer science at the community college level for 7 years, and I've been an academic coach assisting with the delivery of masters level content online. (I acted much like an online teacher's assistant. I wasn't the teacher of record, but I graded assignments and answered many student questions). In the private sector I often taught users how to use the software written by me or someone on my team. I also did several years of substitute teaching at the primary and secondary level before becoming certified to teach. NONE of that prepared me to be a classroom teacher of 6-12th graders. The classroom management, paperwork, compliance with various programs, etc. is daunting. Experienced teachers in other subjects will in theory have mastered that process. On the other hand, most of the teachers learning to teach computer science are struggling with some of the language concepts themselves. How can you teach what you don't completely understand. The good news is it's possible their experience teaching diverse learners may in fact help us all figure out how to better teach a clearly difficult subject.
Thomas Mcguire
Part of the problem for young people today is the overuse of computers that is pervasive in the classroom. In their mathematics classes most teachers throw slideware up on a screen. The students don't get a chance to copy down the slide in class because it's available online. By not writing the problem out on the board the teacher does not impart how much space or time is required to solve a problem. The students are left with a superficial treatment of the subject matter.
The very bright kids can deal with this situation but you start to lose some of the second tier students. The second tier students who can't teach themselves or don't have time to supplement their learning start to think they aren't cut out for math. Computer science courses have become the same thing. Increasingly they are screen based. The students watch slides and windows of IDEs, they become lost in the trappings of computer presentation and have no concept of how to solve the problem.
How is it that in the days where computer time was at a premium we could send almost all Engineering students through an introductory computer course? I believe it was because you had to solve the problem by hand with pen and paper. You had to draw out how you wanted to represent data and you wrote out the program by hand and rechecked syntax because in the days of the card punch getting access to another card punch machine was a pain in the neck. You learned to be thorough by necessity. Today we throw code into the IDE, it checks our syntax and we become lazy to the details because the debugger is integrated and we can just step through the program.
My own introduction to computers back in 1972 in 7th grade was a teletype terminal with a paper tape I could save Basic programs to. I would spend hours at home designing programs in Basic and then hours after school in the terminal room inputting my programs. They were simple programs of my own design to solve math problems or play simple games. But the point is it takes hours of time with computer programming to become proficient. What student really has time for that?
It is a shame because I believe that Computer Science/Programming teaches students how to plan. There is little difference in programming a computer and setting up a factory. In the end procedures must be worked out, reviewed and refactored to obtain an efficient process to produce a report from a database or a widget from a factory floor. Business processes are no exception to this either.
Finally I went and reviewed Kathi Fisler's paper and have to wonder is it Functional programming embodied by Racket variant of the Scheme language. Or the fact that the language has its roots dating back to the late 1950s and the invention of LISP by Dr. McCarthy (which was an attempt to provide a model of lambda calculus invented even further back by Alonso Church)? In other words the language has its roots in a basic problem solving era that predates the wide spread use of computers.
Jeffrey Yearout
Well I stumbled onto this post after a link to a scholarly article by Briana Morrison was shared at the https://teachingisstem.wordpress.com blog. And glad I did!
I am one of those referred to in a comment, a veteran educator jumping into the world of CS. And while I have enjoyed what I have learned, and helping my students to try and learn in this field, it has certainly come with frustrations as well. Most of these frustrations are centered on best pedagogical practices to help my high school students learn, and it is honestly a relief to know it's not just me when things aren't going well, the research seems to show that we in CS education are still figuring a lot of this out.
I think it will be important to take this information and research that appears to be largely centered on college level CS, and then work to determine what aspects drill down to best practice for teaching CS concepts at the K-12 level. Which in itself will have a lot of variety throughout the different age levels. Add to that the fact that coding is "hot" right now, meaning there are a lot of products and vendors entering the market, and we will have to be very diligent in finding the quality needles in the proverbial haystack.
So for this 50 year old veteran educator, who feels somewhat like a beginning teacher all over again, and while at times I sort of wish I had stayed doing what I used to do because it was easier to deal with, I am glad I am here! Having to learn so much content myself has pushed me to also rethink what works in teaching and learning itself. I am reading the book Make It Stick (https://makeitstick.net/index.php) which is prompting a lot of thought as well.
Displaying all 8 comments