| Health / Health News |
Composing thoughts: mental handwriting produces brain activity that can be turned into text
Scientists have developed a brain-computer interface (BCI) designed to restore the ability to communicate in people with spinal cord injuries and neurological disorders such as amyotrophic lateral sclerosis (ALS). This system has the potential to work more quickly than previous BCIs, and it does so by tapping into one of the oldest means of communications we have — handwriting.
Thoughts turned into text: Two implanted electrode arrays record the brain activity produced by thinking about writing letters. This information is then collected and processed in real-time by a computer, which converts that data into words on a screen. Photo: Shenoy lab & Erika Woodrum (artist)
Researchers focused on the part of the brain that is responsible for fine movement and recorded the signals generated when the participant attempted to write individual letters by hand. In doing so, the participant, who is paralyzed from the neck down following a spinal cord injury, trained a machine learning computer algorithm to identify neural patterns representing individual letters. While demonstrated as a proof of concept in one patient so far, this system appears to be more accurate and more efficient than existing communication BCIs and could help people with paralysis rapidly type without needing to use their hands.
When a person becomes paralyzed due to spinal cord injury, the part of the brain that controls movement still works. This means that, while the participant could not move his hand or arm to write, his brain still produced similar signals related to the intended movement. Similar BCI systems have been developed to restore motor function through devices like robotic arms.
“Just think about how much of your day is spent on a computer or communicating with another person,” said study co-author Krishna Shenoy, Ph.D., a Howard Hughes Medical Institute (HHMI) Investigator and the Hong Seh and Vivian W. M. Lim Professor at Stanford University. “Restoring the ability of people who have lost their independence to interact with computers and others is extremely important, and that is what is bringing projects like this one front and center.”
First, the participant was asked to copy letters that were displayed on the screen, which included the 26 lower-case letters along with some punctuation. At the same time, implanted electrodes recorded the brain activity from approximately 200 individual neurons that responded differently while he mentally “wrote” each individual character. After a series of training sessions, the BCI’s computer algorithms learned how to recognize neural patterns corresponding to individual letters, allowing the participant to “write” new sentences that hadn’t been printed out before, with the computer displaying the letters in real time.
“This method is a marked improvement over existing communication BCIs that rely on using the brain to move a cursor to “type” words on a screen,” said Frank Willett, Ph.D., an HHMI Research Scientist at Stanford University and the study’s lead author. “Attempting to write each letter produces a unique pattern of activity in the brain, making it easier for the computer to identify what is being written with much greater accuracy and speed.”
Using this system, the participant was able to compose sentences and communicate with others at a speed of about 90 characters per minute, comparable to someone of a similar age typing on a smartphone. In contrast, “point-and-click” interfaces have only achieved about 40 characters per minute.
This system also provides a level of flexibility that is crucial to restoring communication. Some studies have gone as far as attempting direct thought-to-speech BCIs that, while promising, are currently limited by what is possible through recordings from the surface of the brain which averages responses across thousands of neurons.
“Right now, other investigators can achieve about a 50-word dictionary using machine learning methods when decoding speech,” said Dr. Shenoy. “By using handwriting to record from hundreds of individual neurons, we can write any letter and thus any word which provides a truly ‘open vocabulary’ that can be used in most any life situation.”
For individuals who are paralyzed or living with “locked-in syndrome” due to brainstem stroke or late-stage ALS, the ability to communicate is largely or even completely lost without technological intervention. While preliminary, the technologies being developed here offer the potential to help those who have completely lost the ability to write and speak. (National Institutes of Health)
YOU MAY ALSO LIKE
