Communications of the ACM
News
Improving Brain-Computer Interfaces
Brain-computer interface researcher Dawn Taylor at the Cleveland VA Functional Electrical Stimulation Center.
Speculative fiction has long entertained the idea of humans interfacing with machines at the level of thought, resulting in enhancement technologies that not only sidestep the limitations associated with the fragile human body, but also supplement the brain's own short-comings in processing information or accessing data. While fictional renderings of human-machine interfaces typically take the form of supplementary enhancements for healthy individuals, scientists doing research in brain-computer interface (BCI) technologies have been developing innovative restorative strategies for those who have lost basic functions, such as sight, hearing, and movement.
BCI research, which draws on several fields, such as neuroscience, computer science, physics, and electrical engineering, has led to new developments in nontraditional approaches to the physiological problems that have been resistant to traditional medical solutions. These developments include deep brain stimulators for those who have Parkinson's disease, cochlear implants for those who have lost their hearing, and communication devices for those who are paralyzed. Today, researchers are demonstrating real-world restorative BCI systems, both invasive and noninvasive, that are giving paralyzed individuals more effective ways to interact with their environment and even move.
The complex issues that scientists deal with in this area are numerous, and include challenges ranging from practical lab logistics, sensor hardware, and data-processing systems to team members who have come to the work from very dissimilar disciplines, making it difficult for the field to establish and maintain consistent terminology. For Dawn Taylor, a research scientist working in this area, such problems are not insurmountable. "With a close-knit lab, everyone learns a common vocabulary fairly quickly," says Taylor, who conducts her research at the Cleveland Clinic Department of Neurosciences and the Cleveland VA Functional Electrical Stimulation Center.
Despite the ongoing challenges, research in BCI technologies is resulting in real help for people with severe disabilities, says Taylor. In her current work, Taylor is focusing on using brain signals to trigger movement in paralyzed individuals, bypassing damage in the spinal cord. "We are getting pretty good at decoding someone's intended arm and hand movements from recorded brain activity," she says, noting that her colleagues are now able to restore arm and hand function in paralyzed individuals by using implanted stimulators to activate paralyzed muscles. The main goal for her at this point is to link these two technologies—on the one hand effectively decoding the brain's movement intentions and on the other hand successfully stimulating paralyzed muscles to produce movement—to enable paralyzed people to move their limbs just by thinking about doing so.
Despite the ongoing challenges, research in BCI technologies is resulting in real help for people with severe disabilities, says Dawn Taylor.
To devise a brain-controlled typing system for a patient who is unable to communicate, one method would be to display a keyboard on a screen and have the patient think about moving his or her fingers to each letter. While it would be possible to decode the movement path the patient is imagining and use that data to control a mouse on the screen, Taylor says it might be more efficient to decode the final goal of each pointing movement and select each letter directly. Or, Taylor explains, decoding each muscle's activation level might be the most efficient strategy when using the brain to control an implanted stimulation system that activates paralyzed muscles to restore arm motion.
Depending on where the electrodes are implanted in the brain's information-processing stream, BCI researchers can decode the goal of the movement itself, the trajectory in 3D space that a hand follows during a movement, the angular motion of the joints, or even the force in each muscle. The type of device that the BCI system controls will impact which aspect of movement to decode and where it would be best to acquire data from the brain's processing stream.
What is clear from Taylor's research and other BCI studies is that users can improve their device-movement skills with practice. "The brain is an adaptable learning machine," says Taylor. "We learn to do a new dance move or play tennis through practice. Learning to control the movements of a device directly with the brain is no different."
Taylor says such learning can be accelerated by using smart algorithms that learn in parallel with the brain. Taylor's latest algorithm, which she calls "co-adaptive," is designed to decode muscle activation levels from the brain to restore arm and hand function via implanted stimulators. The algorithm, which must be set up in an initial supervised training phase where it is clear what the patient is trying to accomplish, is designed to modify itself on the basis of how accurate recent past movements were. Taylor likens this update process to how supervised learning occurs in neural-network algorithms.
However, unlike most neural-network applications, the algorithm's input (that is, the brain activity) is also learning in parallel to the decoding algorithm. "This co-adaptation process can be powerful in that adjusting the decoding algorithm allows the person to explore new and potentially easier ways to think about moving that will generate stronger brain activity patterns and ultimately result in more robust decoding of the appropriate muscle activity," explains Taylor. "In any parallel adapting system, finding an appropriate adaptation rate is key to ensuring stable rapid improvement."
While decoded brain activity has been used to activate muscle stimulators in a few studies and the groundwork is now being laid for U.S. Food and Drug Administration endorsement, suggesting that the techniques and technologies associated with BCI systems are finally maturing, Taylor notes there is still a long way to go. She cites two main technical challenges, in particular, as bottlenecks for further progress. One is that researchers working in this area need better brain-recording technologies to extract high-resolution brain activity reliably. The other is the actual physical components that go into a complete BCI system must be made smaller and more portable, and must also be simple enough to use so they can be adjusted without the aid of a research engineer.
Platform Standardization
Another major issue facing the BCI field as a whole is the lack of platform standardization and interoperability. One scientist who has been working to address this issue is Gerwin Schalk, a researcher at the Wadsworth Center in Albany, NY, and director of the BCI2000 project. Schalk says he became interested in BCI technologies in the late 1990s, when most BCI teams were writing custom software because there was no common platform to facilitate BCI implementations. In 1999, Schalk and his colleagues began working to develop such a platform, which they named BCI2000.
At present, the software is being used by some 600 laboratories around the world. "BCI2000 has supported experiments in about 150 peer-reviewed publications or theses," says Schalk. "Some of these papers are among the most influential studies in the field of BCI research."
"The brain is an adaptable learning machine," says Dawn Taylor. "We learn to do a new dance move or play tennis through practice. Learning to control the movements of a device directly with the brain is no different."
At its core, BCI2000 is a general-purpose software system designed to support many data-acquisition and feedback formats through an interface that allows for interaction with external programs. For example, a robotic arm application that is external to BCI2000 can be controlled in real time with brain signals processed by BCI2000. As another example, BCI2000 can be used to store behavioral-based inputs such as eye-tracker coordinates. The software is free, and is available with full source code. While the application is currently Windows-based, Schalk says he and his team are developing BCI2000 code for other operating systems, including Linux and Mac.
With labs regularly developing both new sensor technologies and new methods for interacting with the brain's data stream, work on BCI2000 remains ongoing. "We are working hard to prepare BCI2000 for the growing complexity of the experiments in BCI research, for the increasing number of clinical applications of BCI technology, and for new applications of BCI technology in areas other than communication and control, such as clinical diagnosis," says Schalk, noting that he and his team are committed to keeping the software evolving to the latest developments in BCI research.
The biggest challenge at this point in the BCI community, says Schalk, is the lack of a noninvasive sensor that can robustly measure brain signals at high fidelity. For noninvasive options, Schalk says he expects this sensor issue to remain a substantial problem for the foreseeable future. But he says he remains optimistic. "Without the limitations associated with current sensors, and with a more complete understanding of brain function, it may be possible to create brain-computer interfaces that seamlessly connect our nervous system with machines," says Schalk. "We are far from this goal, but many in the research community, myself included, dream about this possibility every day."
Such seamless interactions might not be achieved for many years to come, but Schalk says it is realistic to see impressive demonstrations of functional restoration, such as full control of robotic arms with actuation of individual fingers, in the near term. Despite the challenges, the potential benefits to ongoing work in this area remain clear. "Using conventional methods, our brain's normal input and output pathways limit the amount of information that can be communicated to about 50 bits per second, such as for spoken speech," says Schalk. "Using BCI technology, it may be possible to substantially increase this rate of communication, and thus allow for an ideal symbiosis of the human brain and artificial machines."
Taylor also offers a similar perspective on the future of BCI technology, suggesting that more adaptive systems will be emerging in the near term. "When you have a less-than-perfect means of controlling your computer, it is imperative that your computer interface make the most of the limited information coming in from the user," she says. In this sense, necessity may lead to significant interface improvements designed to be more intelligent and dynamic in responding to human input.
The prospect of more intuitive input mechanisms has been a key motivation for interface designers, who have long expressed excitement about the potential for BCI research to change the way humans interact with computers. Still, most BCI scientists are careful to offer only guarded optimism, cautioning that the field is still relatively young in its methods and results, making it unlikely the average person will be using BCIs to interact with computers in the near term. At this point, noninvasive ways of recording brain activity provide only low-fidelity data. But for somebody who is paralyzed, even gaining a slow method of interacting with a computer can make a significant improvement in quality of life.
The biggest challenge in the BCI community, says Gerwin Schalk, is the lack of a noninvasive sensor that can robustly measure brain signals at high fidelity.
"In a truly ideal future, BCI technology will make it so that you wouldn't be able to tell if someone has a physical disability," says Taylor. "Paralyzed individuals would move and interact with their environment just like everyone else."
Further Reading
Berger, T.W., Chapi, J.K., Gerhardt, G.A., McFarland, D.J., Principe, J.C., Soussou, W.V., Taylor, D.M., and Tresco, P.A. Brain-Computer Interfaces: An International Assessment of Research and Development Trends. Springer, New York, NY, 2008.
Schalk, G. Brain-computer symbiosis, Journal of Neural Engineering 5, 1, March 2008.
Schalk, G. and Mellinger, J. A Practical Guide to Brain Computer Interfacing with BCI2000. Springer-Verlag, London, U.K., 2010.
Taylor, D.M., Helms Tillery, S.I., and Schwartz, A.B. Direct cortical control of 3D neuroprosthetic devices, Science 296, 5574, June 2002.
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., and Vaughan T.M. Brain-computer interfaces for communication and control, Clinical Neurophysiology 113, 6, June 2002.
Figures
Figure. Brain-computer interface researcher Dawn Taylor at the Cleveland VA Functional Electrical Stimulation Center.
Figure. Neural activity from a 16-channel electrode array for a BCI system. Each spike indicates a firing neuron. The software determines which neuron generates each detected spike by attributing each spike to the neuron whose wave shape is most similar to its own.
Figure. Diagram showing a brain-computer interface setup with implanted and external components.
©2011 ACM 0001-0782/11/1000 $10.00
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from [email protected] or fax (212) 869-0481.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.
No entries found