Communications of the ACM
The Pros and Cons of Online Lab Classes for Computer Science - 2020 Pandemic Edition
Teaching online is at the forefront of every college professor's mind nowadays. For those of us who teach computer science, data science, and design classes, we're very much used to the energy and excitement of physical lab environments like the one shown in the photo below.
Computer and design labs are magical places where students huddle around communal tables, mingle with each other, work on programming projects, and get help from TAs (teaching assistants) who wander the space. They're high-energy hands-on experiential learning environments that exemplify some of the best parts of the college experience.
Obviously during current pandemic times, it's impossible for us to get together in these tightly-packed physical spaces to hold lab classes in computer science and related majors. Instead, we're all forced to retreat to our isolated pods at home and awkwardly interface with each other on Zoom and assorted online tools. Millions of words have been written over the past few months about all the downsides of online teaching, especially for hands-on classes of the sort that benefit the most from in-person interactions. But is it all doom and gloom? We think not!
We got a chance to study this firsthand earlier this year because my undergraduate research student Julia Markel was a TA (teaching assistant) for a CS programming lab class of the sort shown in the above picture. During winter quarter (Jan-Mar 2020), it was held in-person as usual. But then when pandemic quarantines started, she TA'ed for the next offering of the exact same class, which was converted to an online format for spring quarter (Apr-Jun 2020). Since the instructor and curriculum remained identical, comparing the in-person and online offerings gave us a chance to reflect on the pros and cons of both formats in a quasi-controlled way.
We wrote up Julia's TA experiences in this pair of classes as a case study for the New Future of Work symposium hosted by Microsoft Research this summer. Here's the link to our full paper:
- Julia M. Markel and Philip J. Guo. Designing the Future of Experiential Learning Environments for a Post-COVID World: A Preliminary Case Study. Symposium on the New Future of Work, 2020.
This blog posts summarizes key excerpts from our paper, most notably the five major ways in which in-person and online experiences differed, and how we can better design future online environments for hands-on lab classes:
1. Comfort and Privacy
First and foremost, students must feel comfortable in their learning environment or else they will not actively participate.
In-person pros and cons: Since everyone is packed into a dense space, some students may feel uncomfortable for physical reasons or self-conscious that many eyes are on them. If they walk into class late or need to step out momentarily, that also feels awkward. But one advantage of a physical space is that one can lurk in the periphery (e.g., the back of the room), whereas everyone is equally visible online in a Zoom videoconference.
Online: In general, online is better than in-person for comfort. Students can participate in Zoom from their own home (assuming they have a private space). They can turn webcams off for greater privacy and also chime in via text chat, which can be more comfortable than speaking via voice. Students also aremore willing to ask for help online (both synchronously and asynchronously) since they can do so semi-privately.
Recommendations to improve online experience: To improve comfort and privacy, videoconferencing tools could support animated avatars such as Apple's Memojis in iOS ...
... which animates a cartoon face in real time based on the student's facial expressions that their webcam continually detects. Students may feel more comfortable projecting an avatar than their real faces and home backgrounds. It would also drastically cut down on bandwidth needs versus sharing dozens of video feeds in Zoom. Videoconferencing could also support anonymous text chat as a side channel for students who want to participate anonymously but still share their thoughts with the entire class. To prevent misuse, instructors should still be able to see everyone's real identities and delete inappropriate chats.
2. Feedback
Giving and receiving feedback on works-in-progress (e.g., design sketches, UI mock-ups, software prototypes) is a central activity in experiential learning environments like those offered in CS and design lab classes.
In-person: In theory, physical presence gives richer affordances for live feedback [Distance Matters by Gary and Judy Olson], but in practice we observed that usually a small number of outspoken individuals dominate the conversations, which made the rest of the class disengage. But one advantage of in-person is that students can quietly whisper to one another and have side conversations before deciding to bring up their ideas publicly in front of the entire class.
Online: Students gave much more feedback online than in-person since they could easily send text chats while their classmates were presenting their work via screenshare. The main benefit of text chatting is that it does not feel like it is interrupting the flow of the presenter; also, other students can chime in with +1's, emojis, or follow-up comments. The downside is that it is harder to have side conversations; there is no analogue of quietly whispering to your neighbor.
Recommendations: Videoconferencing could add a "presenter" role to assign to the student (or group of students) currently presenting their work, as indicated by who is sharing their computer screens. That way, classmates can directly text chat with the presenter(s) to give them private feedback without everyone else in class seeing it. For students who prefer to give richer audiovisual feedback, the tool can allow them to leave short voice messages for presenters alongside an annotated video snippet of part of their screenshare recording (see tools like Loom and Screencastify). To support "whisper mode" it could let students chat privately to classmates without instructors seeing those chats; but we need a way for instructors to prevent this feature from being abused.
3. Shared Context
When an instructor sits down to help a student one-on-one or when students are working together in the lab, it is important for them to have shared context to mediate those interactions.
In-person: In-person provides multiple rich channels of shared context since everyone can see the same computer screen, point to it, hand-draw on paper or the whiteboard, inspect other physical artifacts, and get implicit cues by reading everyone else's faces and gestures.
Online: Although online is generally inferior to in-person here, one benefit is that it forces everyone to look at the same shared screen (e.g., with Zoom screen-sharing) and annotate it with a virtual pen tool. This enforced focus may prevent certain students from getting distracted by having too many different stimuli to focus on in-person (e.g., other people's faces, laptops, hand-drawn sketches, loud voices in the background).
Recommendations: Enhance screen-sharing features in videoconferencing tools to quickly let users switch between multiple display types, such as their laptop screen, a webcam pointed at a physical artifact, and a tablet for hand-drawn sketches. Since many people have smartphones, that could be used as a secondary shared display to provide more context; they can use touch gestures to make deictic references to shared elements.
4. Ambient Awareness
Experiential learning environments like computer labs are often energetic and loud during freeform work times, with students working on their projects in an open space and instructors walking around to provide on-demand feedback to individuals or small groups.
In-person: The main advantage of in-person is that students and instructors have better ambient awareness of their surroundings even when engaged in a conversation; they can still hear what is going on in the background or pause if something notable happens. For instance, if a TA is helping a student on a particular design problem, a neighboring student might overhear the conversation and walk over to participate as well. TAs can also take the pulse of a room by walking around the physical space and glancing to get a sense of how students are doing, which is harder to do online.
Online: In our experience, it is very hard to replicate this kind of spontaneity and ambient awareness online because videoconferencing tools are designed to only hold one focused conversation at once. If multiple people are talking at once, it is impossible to discern what is going on. Zoom has breakout rooms where students and instructors can split off into small private groups; but breakout rooms are too isolating and do not let participants see or hear what is going on in other rooms.
Recommendations: Rather than using standard videoconferencing tools, one idea is to create a location-oriented interface where each person is a virtual avatar who can move around in a 2-D map that looks like a studio classroom. Just like in the physical world, they can see and hear only people who are near them in the virtual map. Gather (formerly Online Town) is a videoconferencing tool that supports this type of map-based interaction:
In the above screenshot, there are five people in a virtual classroom. The two people on the left can see only each other's live webcam feeds (like they are in a temporary Zoom breakout room), as can the three on the right. At any time, they can move around to form other impromptu groups and see/hear only those who are closest to them. This fluid interface lets students work either independently or in small groups, then lets instructors "roam" around the virtual classroom to see what everyone is working on. However, it still can't solve the problem of not being able to hear ambient sounds off in the distance like people in a physical classroom are able to do.
5. Flexibility
In-person: Everyone needs to congregate in a physical space for class sessions, which reduces flexibility for those with arduous commutes or more home-care responsibilities. Also, since lab classrooms are shared between multiple classes, it is harder to hang around after class ends to hold follow-up chats since people from the next class are waiting to use that room.
Online: Not needing to gather in-person gives more flexibility to both instructors and students. Instructors can schedule classes and office hours even at times when they would not be able to commute to campus. It is also easier to stay around after class times to hold follow-up conversations since there is no physical classroom that needs to be vacated.
Recommendations: One way to make online tools even more flexible is to merge asynchronous forums (e.g., Piazza) with synchronous video chat so that if an instructor and student are both online, they can quickly jump from the forum to video chat to discuss harder questions. Another is to allow an instructor to help more students at once by monitoring several live screen-shares and toggling between multiple video chat sessions. This mechanism takes advantage of downtime that arises when an instructor tells a student to try something out and report back on progress; they can switch to helping someone else and return later when the first student is ready.
Parting Thoughts
Why are we so focused on trying to improve the online experience? For one, because we might all be stuck learning remotely for the foreseeable future ... obviously we all want to get back to the classroom as soon as it's deemed universally safe to do so, but for the time being this is our new normal. And two, even when we (hopefully!) return to normal, developing better online learning technologies can make education more universally accessible to the billions of people around the world who can't afford to attend college in-person to participate in these hands-on experiential learning environments. There are still many open challenges, though, including:
- Universities want students to eventually return to socially-distanced classrooms with lower occupancy and with some students participating remotely. For lab courses, how do we set expectations for mixed interactions where some classmates will be in-person and others remote?
- In mixed-mode settings, how can we design technologies and set expectations so that remote students feel just as valued and engaged as in-person ones? There are already best practices for engaging remote workers in office environments, but how do we adapt those for educational settings?
- Students' home environments vary widely, with some having quiet places to work and fast internet connections, and others having little privacy and slow Internet. How can we start to level the playing field here? One idea is to send students equipment such as high-quality webcams and microphones via mail. Another is to pay for subsidized high-speed cellular data plans for their phones. Finally, staffing up virtual lab hours at more times throughout the day can let students find times that fit into their diverse schedules.
- Asynchronous tools (e.g., prerecorded videos, discussion forums) are a known way to make courses more inclusive, but the very essence of experiential learning involves high-touch synchronous interactions in a lab or studio setting. How can we move toward semi-synchronous technologies that keep the benefits of real-time interactions while letting more students participate at times that fit their schedules?
- A major weakness of remote technologies is that they cannot easily foster the sorts of spontaneous interactions that occur in the hallways, by the office watercooler, or when people are hanging out before and after meetings. The analogue for experiential learning is people hanging out around the lab space. How can we create remote technologies that both provide these benefits and surpass them by fitting in better with students' schedules and varied home environments?
- Finally, as argued in Beyond Being There [Hollan and Stornetta 1992], remote technologies have the potential to surpass the fidelity of in-person interactions rather than simply being imperfect imitations. How can we design or adapt technologies to get to Beyond Being There for experiential learning environments? How can we get students to prefer these technologies versus just viewing them as second-rate substitutes for being in-person?
We call on designers to use this year's unexpected challenges as inspiration to create experiential learning environments that are more flexible and inclusive. Doing so will not only help students in existing courses, but also expand educational opportunities around the world to people who cannot ordinarily access them. We believe people will now be more receptive to adopting remote technologies, since everyone has gone through the collective experience of abruptly shifting to remote work in early 2020. We similarly call on researchers and policymakers to consider online-first formats even when thinking about traditionally in-person learning environments. These efforts will be critical for training the next generation of students in creative fields such as engineering, design, and data science to face the challenges of a post-COVID world.
Julia Markel is a fourth-year computer engineering undergraduate student at the University of California San Diego, working on research in HCI and design.
Philip Guo is an associate professor of cognitive science at the University of California San Diego, working on HCI and online learning research.