Computer scientists during World War II were able to figure out how to turn a string of gibberish characters, known as the German Enigma code, into German — saving many lives and resulting in a huge victory for the Allies during the World War II. Fast forward to today, a team of researchers from the University of Pennsylvania, the Georgia Institute of Technology and Northwestern University have accomplished a similar feat using the same techniques. The team used cryptographic techniques to decode the activity of motor neurons in the brain. Their approach has allowed them to predict motions, from brain data and with only generic knowledge of typical movements, like which direction monkeys will move their arms.
Decoding structured movements with distribution alignment. Source: Eva Dyer, Northwestern University
This same cryptography-inspired technique could be used, eventually, to decode more complex patterns of muscle activation for use in prosthetics, speech and to aid people who have total paralysis.
Konrad Kording, a Penn Integrates Knowledge Professor, and Eva Dyer, a postdoctoral researcher at Kording’s lab, and now an assistant professor in the Department of Biomedical Engineering at the Georgia Institute of Technology and Emory University, led the research team. They also collaborated with Lee Miller, a professor of physiology at Northwestern University, and his team.
In an experiment with three rhesus macaques, the researchers took data from several hundred neurons associated with arm movement. As the monkeys completed tasks where they had to touch a target that appeared at different points around the main starting point, in-dwelling electrodes recorded spikes of electrical activity that corresponded with the movement of the monkey’s arm.
Current brain-computer interfaces can use this data to move robotic prosthesis, but they do this through “supervised learning.” The interface is trained to recognize how the patterns of neural firings correspond to each movement and can “reconstruct” movements based on the on-going neuronal signals recorded from the monkey’s brain.
Through the cryptography approach, there is not supervised learning.
"In cryptography, 'supervised learning' would be called a 'known plaintext attack,'" Kording said. "That is, we have both the encrypted and unencrypted message and just need to figure out the rules that turn one into the other. What we wanted to do in this study was to be able to decode the brain, using a movement model, from the encrypted message alone."
Starting with each neuron's firing partner, the team’s goal was to discover a consistent way of mathematically mapping those patterns to the monkeys’ real arm movements. This approach was made possible by finding statistical structures of movements. Similar insights about language structure, like frequency and distribution of vowels, were involved in breaking the Enigma code.
"Essentially, the algorithm tries a range of possible decoders until we get something where the output looks like typical movements," Kording said. "There are issues scaling this up — it's a hard computer science problem — but this is a proof-of-concept that cryptanalysis can work in the context of the neural activity."
A cryptanalysis approach to decoding neural activity is particularly attractive when it comes to brain-computer interfaces that control prosthetics. Robotic limbs that can directly interpret the user’s thoughts without having to be calibrated would be a huge improvement.
"These training periods can be long and annoying, and in some circumstances, they're truly problematic — say, if your arm is missing, rather than paralyzed," said Kording. "Willing movement is different from imagining moving a nonexistent arm."
The cryptanalysis approach could provide a way for brain-computer interfaces to achieve literal mind reading.
"You could ask a 'locked-in' patient to generate neural patterns associated with specific words, but the corpus of language is very large," Kording said. "Rather than having them generate a pattern that is associated with every word they want to say, we could build a decoder that transforms those patterns until it looks like language."
"I think we should be able to do this within the next decade," he said.
Working towards this goal is a consistent improvement in brain-recording technology with scientists currently working toward electrode arrays capable of simultaneously sampling data from a million neurons.
"As this capability grows, our technique becomes more useful," Kording said, "At the same time, I think we need to be very careful about the possibility of it being applied to people without their consent. What spy agencies could do with this technology is scary."
Even though it is at the preliminary stage, the researchers believe their cryptanalysis approach to decoding neural activity is a good match for this problem.
"The Germans were actively working against decryption and modern ciphertexts are basically impossible to break," Kording said. "We have it easier. The brain ended up with this encryption system through natural selection, so it's essentially making the same kind of 'mistakes' that allowed us to crack Enigma in the first place."
A paper on this research was published in Nature Biomedical Engineering.
