Communications of the ACM
App Turns Tablet Into Math Aid For Visually Impaired Students
Vanderbilt graduate student Jenna Gorliewicz (right) with Kira, one of two visually impaired high school students who are testing Gorlewicz's new Android app.
Credit: Pat Slattery / Vanderbilt University
Without looking down, Kira runs her index finger across the screen of an Android tablet that she holds in her lap. For the occasion, she has painted her fingernails bright pink. When her finger touches a line drawn on the screen, the tablet vibrates quietly. Scanning her finger back and forth and feeling the vibration come and go allows her to trace the line's path.
When her finger reaches a pink dot, the tablet gives off an electronic tone and she grins delightedly.
Kira is one of two visually impaired high school students who are testing a new Android application, one designed to assist students like her in mastering algebra, geometry, graphing and other subjects that are particularly hard to comprehend without the aid of normal vision.
The app is the brainstorm of Jenna Gorlewicz, a graduate student in the Medical and Electromechanical Design Laboratory (MED Lab) at Vanderbilt University, and her adviser Robert Webster, an assistant professor of mechanical engineering, who directs the lab. Given the enthusiastic reaction of Kira, her classmate Quinn and their teacher, the innovation could have a major impact on how science, technology, engineering and math — the critical STEM subjects — are taught to the visually impaired.
Major Change in Direction
The project is a considerable change from the research projects that Gorlewicz pursued during her first two years in the MED Lab. Supported by a fellowship from the U.S. National Science Foundation, she worked on improving cochlear implant surgery and miniaturizing endoscopic robotic capsules. But she wasn't completely satisfied: "I'm a real people person," she says. "So I wanted a project that would let me interact directly with the people it was helping."
"Jenna kept saying that she wanted to work more with people, so we began exploring other possible projects," says Webster. "Fortunately, her fellowship and my NSF CAREER Award gave us enough flexibility to think outside the box and explore a wide variety of thesis topics for her."
One of the ideas they came up with was a study of the effectiveness of haptic technology: technology that takes advantage of a user's sense of touch by applying forces, vibrations or motions to enhance remote control of machines, devices or virtual objects. The lab was experimenting with haptic interfaces to control some of their biomedical robots.
Matching Haptics With the Visually Impaired
"When I began reading articles about haptic technology being incorporated into these new touchscreen devices, I realized that the people who really need haptics are people with impaired vision because they heavily rely on their sense of touch to 'see' the world around them," Gorlewicz says. "I love math and I love teaching, so I immediately thought of using them for math education, because it has such a strong visual component." After she did some research into the methods that are currently used to teach mathematics to the visually impaired, she became even more enthusiastic about her idea.
When she started the project 18 months ago, the only off-the-shelf tablet that Gorlewicz could find that included tactile feedback was a $2,500 model produced for industrial and commercial applications. Since then haptic feedback has been added in a number of consumer tablets that sell for as little as $300.
Gorlewicz has programmed these tablets so they vibrate or generate a specific tone when the student's fingertip touches a line, curve or shape displayed on the screen. The devices can generate vibrations with a number of different frequencies and hundreds of different sounds. This allows Gorlewicz to assign different tactile or audio signals to different features. For example, in an exercise that includes an X-Y grid, she can set the horizontal and vertical lines to vibrate at different frequencies and set points to make a certain tone. In this way, it's easier for the students to distinguish between the gridlines and the points on the grid.
Teaching STEM Courses
"If one of these tablets were networked wirelessly to the teacher's computer, then, when he or she projects a graph or equation on the screen at the front of the class, the same graph would appear on the student's tablet. They could then use their senses of touch and hearing to follow the content the teacher is discussing," Gorlewicz says.
"When Jenna first approached me with the idea, I thought it would be interesting and might be some small help," says Ann Smith, a teacher of the visually impaired who works with Kira and Quinn at Hillsboro High School in Nashville, TN. "The more experience I have with it, the more valuable I think it could be. It makes the work more accessible. The students are really interested and they talk about it even when Jenna isn't here."
View a video of Kira and Quinn using and discussing the Android touch-screen tablet as a learning aid.
Hillsboro High includes visually impaired students in classes with their sighted peers as much as possible. That means Kira and Quinn attend general education classes. Smith sits with them during their math classes and uses various methods to ensure that they can follow the teacher's presentation. With Quinn, who can read Braille and see large print, this involves duplicating the shapes and figures the teacher is using on a small white board that he can hold close enough to see. In Kira's case Smith must use tactile aids like pins and pipe-cleaners on a bulletin board with raised graphing lines to reproduce complex mathematical shapes that Kira can feel with her fingers.
"Because this takes a certain amount of time, we are always one or two steps behind the teacher," Smith says. "One of these haptic tablets would allow them to keep up much better. If I didn't have to attend class with them, it would also make them feel more independent."
Smith also points out that such a tablet can be used by students with widely differing degrees of impairment because it combines visual, tactile and audio capabilities.
Learning Required
According to Kira and Quinn, there is a little bit of a learning curve with the new device. "At first, I didn't think this would help me," Quinn says. "But after I started using it I found that it can be very helpful." Adds Kira, "The biggest obstacle was getting the correct mental images. However, once you get the knack, it gets fairly easy."
"It was amazing how quickly they caught on and how good they have become," Gorlewicz says.
According to Gorlewicz, every time she meets with the students, they come up with new features that they would like her to add. The configuration that they think would be the most useful is a touch-sensitive version of the graphing calculators that are widely used by their sighted classmates.
"It would really help to have something like this because it makes us equal to everyone else," Kira says.
No entries found