Communications of the ACM

Education

Improving Computing Education Research through Valuing Design

By R. Benjamin Shapiro, Kayla DesPortes, Betsy DiSalvo
Communications of the ACM, August 2023, Vol. 66 No. 8, Pages 24-26
10.1145/3604633
Comments

When most people think about computing education, they imagine students sitting in front of computers, learning to program them. Those imaginary students are probably writing code in a text editor, compiling it, running it, and seeing what happens. The code they write, and the editor, compilers and runtimes they use are all designed. But, remarkably, computing education research (CER) has never had publication venues that invite designers and developers to discuss how to design these tools to better support learning or tell retrospective stories of how they have designed these systems. CER ought to value design more. Thinking and talking about design practices will advance our field. With the support of its editorial board, ACM Transactions on Computing Education (TOCE) is now welcoming design-centered submissions.

Design research has historically not been valued or accepted within CER. In this column, we highlight the need for publications centering design and design methodologies in CER, and welcome computer scientists of all specializations to contribute innovative designs to the field.

Back to Top

Motivation

CER, like CS, is a design science or a science of the artificial.⁵ Unlike, for example, Physics, where the aim is to develop knowledge about the intrinsic nature of the universe, CER aims to develop knowledge about how to create new universes that better support problem-solving, creativity, and communication by people. Computational technologies can be "cultural building material[s]"⁴ for designers, researchers, educators, and everyone else to use to reshape how we live and learn.

Early in the history of CS, creating programmable systems of any sort required enormous ingenuity. The advent of compilers accelerated progress dramatically, enabling program authors to shift their attention from the minutiae of machine design to business logic. Throughout that time, CS concerns shifted from creating more programmers to concerns about making more computers, and making them useful. But today, we have plenty of computers and struggle to develop greater human capacity to author programs—including capacity that includes people of different genders, ethnicities, cultures, and desires. This struggle is fundamentally a design problem, but surprisingly, CER's premier publication venues have done little to welcome design research.

TOCE editor-in-chief Amy Ko² blogged that "Programming languages are the least usable, but most powerful human-computer interfaces ever invented." Yet much work in CER examines students' difficulties of learning to program as if the root issues are cognitive (focused on deficits of the learner) or pedagogical (focused on deficits of the instruction) rather than within the design of the tools we use to program (focused on the deficits of what we are teaching in the first place). We must research the redesign of these technologies to make it easier, while also conducting research that deepens our understanding of human needs and potentials.

The designs of programming tools impact how easy it is to create various types of computational artifacts, and therefore, influence the disciplines within which learners can learn about computing. Technologies such as the Circuit Playground Express, Scratch, and danceON, have opened opportunities for disciplinary integration with fine arts, liberal arts, and social sciences. The design of the technology can thus change the social and cultural context of learning—that is, the knowledge learners can draw on, the types of participation that are valued, the communities their work is situated, the ease in sharing their knowledge outside the learning environment, as well as what is cognitively and pedagogically required.

By design we mean the process and results of imagining new types of computational tools, practices, or learning environments. Design contributions can take different forms: They may be design process innovations, seeking to provide transparency into how to design. Other accounts can focus on the creative outcomes and can illustrate new futures sparking new avenues of exploration. They may critique existing systems and practices. They may also be case studies of practical design implementations, reference points for other designers to learn from.

To achieve that goal, we must learn from human-computer interaction (HCI), which has venues supporting a variety of forms of scholarship, including explorations of new design possibilities. It is time for CER to catch up to HCI by making room for design. TOCE aims to push CER forward by fostering and rewarding greater imagination and discussion of what could be via design papers.

---

Today, we have plenty of computers and struggle to develop greater human capacity to author programs.

---

Within CER, we have always been engaged in questions of design of computational systems that help learners understand how the system's semantics translate to computational behavior. However, these design concerns and processes have been hidden. For example, there is a theme of computing education literature dedicated to "notional machines," or ways of creating a pedagogical model to support learners in being able to understand and reason around the hidden inner workings of a computing system.¹ We have had community discussions surrounding designs, for example: What language to teach first? How to support transfer across languages? Whether we should be language agnostic in our educational designs? These all acknowledge the centricity of design to computing, computing education and what learning means; yet despite these informal conversations, there is no CER publication venue that welcomes design-focused contributions.

Our top conferences (for example, International Computing Education Research) and journals (including Computer Science Education and TOCE) have a long record of publishing empirical research of which design is a part. Specifically, these venues routinely accept papers that evaluate the effects of some designed educational intervention on student outcomes. But these venues have not historically solicited or accepted contributions that dwell in the details of design for computing education. This should change. Given how crucial design is to progress in computing education research, we must create space to discuss design as design within the field.

Back to Top

Illustrations

Consider the case of block-based programming. Numerous empirical studies have shown the benefits of blocks to beginner programmers. Block-based structure editors were first designed and described by Sten Minör in the International Journal of Man-Machine Studies.³ Notably, this paper does not include an empirical evaluation showing its benefits to programmers. Instead, the paper was about the novel design itself, including its theoretical motivation, relevant prior work, details of its implementation, and possible directions for future empirical research. It was published in a journal that welcomed design contributions. Would this same work be publishable in computing education research venues today? As associate editors and senior program committee members for the field's top publication venues, we think not. That is a problem. How can computing education researchers design the next generation of breakthrough programming tools when the primary publication venues within their own academic community do not value design processes? If Minör had been a computing education researcher, his work might have never seen the light of day through scholarly avenues. Later work in CER shows the importance of Minör's innovations— tools including Alice and Scratch, which millions of young people use to learn programming, are usable because they embody Minör's ideas. We do not know whether the Alice or Scratch teams were aware of this prior work, and we suspect few in CER know of Minör's invention of blocks technology. We wish to live in a world with more ingenious inventors like Minör, and making that happen requires creating venues for their work to be shared and valued.

How to Design Programs (HtDP) is another illustration of the need for design papers in CER. HtDP is an approach to designing computing curricula that differs strikingly from the norm. It is common for introductory programming courses to be organized around the progressive introduction of language features and control structures (for example, variables and loops) and how to use them, typically within imperative programming languages popular in industry. HtDP takes a different approach:

• It is language agnostic.
• It emphasizes a functional programming perspective on program and data structure design.
• And most importantly, it teaches a structured approach to problem solving.

The crux of the HtDP design is to focus on helping students learn to read and write problem statements, descriptions of the problems that their programs are meant to solve, and then deconstruct those statements into actionable sub-problems that roughly correspond to functions and data structures that they must define, implement, and write test suites for.

Several things about HtDP are remarkable: Numerous studies evaluating elements of HtDP's approach have been published in CER venues—such as Kathi Fisler's 2014 paper "The Recurring Rainfall Problem" showing that students who learn to program the way HtDP teaches consistently outperform peers who learn programming in more traditional ways, and that these effects arise regardless of programming language used (they hold in Java and Scheme). HtDP's pedagogical philosophy has spawned a range of technological innovations, including programming languages, compiler features, and visualizations, and its spinoff Bootstrap has successfully integrated computing into other subjects such as algebra, physics, and social studies. Yet despite HtDP's remarkable two-decade history of innovation and impact, including large scale impact in urban schools, no design story of HtDP's conception, improvement, and overall impacts have been published in a CER venue. We believe such a story would have been unpublishable until now, because no CER venue's review criteria would accept such a submission. So it is not surprising that we regularly encounter new papers in CER that reinvent elements of HtDP without even acknowledging it. We wish for CER to be a field where innovators from other areas of computing (like the programming languages researchers who created HtDP) can describe their ideas about how to reimagine computing education, even if they have not yet evaluated their ideas empirically. If we are successful, the next idea with as much power and eventual impact as HtDP will be described in the pages of TOCE.

Back to Top

Conclusion

The CER community should do more to value the generation of novel designs for learning. Greater exploration of the space of design alternatives may enable us to reframe longstanding research questions and methods through altering learning processes and ameliorating challenges that students face today. Outside of empirical evaluations of designed things, design papers can support sharing our design visions, conceptions of speculative futures, methods and processes that use design as a tool for inquiry, and our histories and struggles to bring our visions to life.

TOCE now welcomes researchers to share and learn from each others' design stories and methods and explore experimental futures for CS education. We hope that inviting conversations about design will broaden participation in the field of CER by bringing the viewpoints of researchers from across CS fields into shared imagination and reflection on what the tools of computing education could be. We encourage Communications readers to submit design papers, and to volunteer to review them.

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.

---

No entries found