Communications of the ACM
Algorithm Replicates Brain Circuitry to Direct a Realistic Prosthetic Arm
Neural signals from a prosthetic virtual arm (top) are mirrored in the robot arm (bottom) which delivers feedback that allows users to feel what they are touching.
Credit: Dura-Bernal et al. / IBM Journal
By applying a novel computer algorithm to mimic how the brain learns, a team of researchers — with the aid of the Comet supercomputer based at the San Diego Supercomputer Center (SDSC) at UC San Diego and the Center's Neuroscience Gateway — has identified and replicated neural circuitry that resembles the way an unimpaired brain controls limb movement.
The research lays the groundwork to develop realistic "biomimetic neuroprosthetics" — brain implants that replicate brain circuits and their function — that one day could replace lost or damaged brain cells or tissue from tumors, stroke, or other diseases. The work is described in "Evolutionary Algorithm Optimization of Biological Learning Parameters in a Biomimetic Neuroprosthesis," published in the IBM Journal of Research and Development.
"In patients with motor paralysis, the biomimetic neuroprosthetic could be used to replace the deteriorated motor cortex where it could interact directly with healthy brain pre-motor regions, and send commands and receive feedback via the spinal cord to a prosthetic arm," says W.W. Lytton, a professor of physiology and pharmacology at State University of New York Downstate Medical Center in Brooklyn, N.Y., and the study's principal investigator.
This scenario required high-performance computing and expertise to simulate and evaluate potential computer models in an automated way, along with the Neuroscience Gateway (NSG) based at SDSC, which provided an entrance to these resources.
"The increasing complexity of the virtual arm, which included many realistic biomechanical processes, and the more challenging dynamics of the neural system, called for more sophisticated methods and highly parallel computing in a system such as Comet to tackle thousands of model possibilities," says Amit Majumdar, director of the Data Enabled Scientific Computing division at SDSC, principal investigator of the NSG, and co-author of the IBM Journal paper.
"Combining these computational advantages can be an effective approach to build even more realistic biomimetic neuroprostheses for future clinical applications," he says.
Fusing Principles
Over the past decade or so, researchers have been trying to fuse computational and biological principles to create realistic computer models that would form the basis for silicon-based neural circuits or implants that would replace damaged brain tissue. In this emerging field, a primary goal has been the decoding of electrical signals recorded from the brain to move, for example, a prosthetic arm. In scenarios once considered science fiction, techniques that encode neural signals from a prosthetic virtual arm to the brain are now allowing users to feel what they are touching.
That said, researchers now recognize that controlling even more realistic and complex systems, including prosthetic limbs involving larger numbers of bones, joints and muscles, require computer models that more closely resemble real brain circuitry.
To get closer to this goal, the researchers in the IBM Journal study relied on several concepts inspired by biology to create a more realistic artificial neural network that allows the motor cortex to learn to direct a virtual arm — consisting of eight bones, seven joints, and 14 muscle branches — to a specified target.
The biomimetic model in question involved more than 8,000 spiking neurons and about 500,000 synaptic connections. The main component consisted of primary motor cortex microcircuits based on brain activity mapping, connected to a circuitry model of the spinal cord and the virtual arm.
"We argue that for the model to respond in a biophysiologically realistic manner to ongoing dynamic inputs from the real brain, it needs to reproduce as closely as possible the structure and function or actual cortical cells and microcircuits," says Salvador Dura-Bernal, a research assistant professor in physiology and pharmacology with Downstate, and the paper's first author.
As outlined, the researchers trained their model using spike-timing dependent plasticity and reinforcement learning, believed to be the basis for memory and learning in mammalian brains. Briefly, the process refers to the ability of synaptic connections to become stronger based on when they are activated in relation to each other, meshed with a system of biochemical rewards or punishments that are tied to correct or incorrect decisions.
In this case, reward signal is based on the ability of the computer model to control how close a virtual hand comes to a target. If the hand got close to the target, synapses generating that movement were rewarded; if the hand was further away, those synapses were punished.
Identifying the best reinforcement learning model required the identification of an optimal set of characteristics or parameters; among others, these include learning and exploratory movement rates, duration of training, and motor command threshold measured in spikes.
'Evolutionary' Algorithms
To isolate reinforcement learning parameters that yielded the best control over a virtual arm, the researchers turned to "evolutionary algorithms." The methodology follows the principles of biological evolution, where a population of individuals, each representing a set of genes or parameters, evolves over generations until one of them reaches a desired fitness level. With every generation, individuals are evaluated and selected for reproduction, produce new offspring by crossing their genes and applying random mutations, and subsequently are replaced by fitter offspring.
"Only the fittest individuals remain," says Dura-Bernal, "those models that are able to learn better, survive, and propagate their genes."
Applied to this system, each individual gene represents a model with a particular set of learning parameters. The fitness of the individual is the ability of the model to learn to control the virtual arm based on real brain signals.
For their study, the researchers evolved a population of 60 individuals (models with different learning parameters) over 1,000 generations, where at every generation a new set of characteristics was evaluated to measure its fitness. Measuring the fitness required training the whole system (pre-motor input, motor cortex, spinal cord, virtual arm) for a period of time and then testing each of the two directions (left and right) to check how it performs. The process needed to be repeated for every individual and every generation.
Six-Year Strategy?
To further increase the chances of finding an optimal solution, the researchers turned to an "island model" approach, which divided the total population of 60 individuals into sub-groups, each of which evolved independently. Again, following principles of biological evolution, an individual periodically would be migrated to a different island, introducing new genes to increase the chances of finding a new combination of genes with better fitness or performance.
"Integrating the parallel 'island model' on Comet required some work, but in the end we made it work to further speed the process," says Subhashini Sivagnanam, a senior scientific computing specialist at SDSC, co-principal investigator of the National Science Foundation-funded NSG project, and a co-author of the IBM Journal paper.
"Since thousands of parameter combinations need to be evaluated, this is only possible by running the simulations using HPC resources such as those provided by SDSC," Dura-Bernal says. "We estimated that using a single processor instead of the Comet system would have taken almost six years to obtain the same results."
Future studies will focus on developing even more realistic models of the primary motor cortex microcircuits to help understand and decipher the neural code — how information is encoded and transmitted in the brain.
"We emphasize the concept of covering multiple scales, from the molecular, through the cellular, up to the network level," says Dura-Bernal. "This will be instrumental in understanding and treating brain disorders such as epilepsy, schizophrenia, Parkinson's, motor paralysis, depression, or amnesia."
Other participants in the research and co-authors of the IBM Journal paper are Samuel A. Neymotin from Brown University; Cliff Kerr from the University of Sydney, Australia; and Joseph T. Francis from the University of Houston.
The study received financial support from DARPA grant N66001-10-C-2008, NIH U01EB017695; NSF DBI 1146949; NSF DB1 1458840; and NYS SCIRB DOH01-C30838GG-350000.
No entries found