A tetraplegic (quadriplegic) man played rock, paper, scissors using a robotic arm – activating it purely by using his imagination, i.e. just by thinking about it. In a study he was also able to perform a fluid hand-shaking gesture and drink a beer “at my own pace.”
Erik G. Sorto, 34, who became paralyzed from the neck down (tetraplegic) after a gunshot wound when he was 21, is taking part in a clinical collaboration between the Rancho Los Amigos National Rehabilitation Center, California, Caltech (California Institute of Technology), and the Keck School of Medicine of the University of Southern California.
Mr. Sorto is the first person in the world to have a neural prosthetic device implanted in the brain, specifically in a region were intentions are made. It gives him the ability to send instructions to his robotic arm through his thoughts.
Mr. Sorto controls the robotic arm so that he can have a drink, just by thinking about it. (Image: Caltech)
Turning jerky movements into smooth ones
Neural prosthetic devices that are currently implanted into the motor cortex, the brain’s movement centre, can allow paralyzed patients to control the movement of a robotic arm or leg. The problem is movements are delayed and jerky – unlike the smooth and automatic gestures seen in natural limbs.
In this latest collaboration, the scientists implanted the neuroprosthetic device in a part of the brain that controls our intention to move rather than movement directly, which they say produces more natural and fluid motions.
The clinical trial was designed to test the safety and effectiveness of this new approach. It was led by principal investigator Professor Richard Andersen, who works at Caltech, Professor Charles Y. Liu, from USC, and Dr. Mindy Aisen, from Rancho Los Amigos.
Prof. Andersen and colleagues set out to improve the versatility of movement that a new neuroprosthetic could offer to patients by recording signals from the posterior parietal cortex (PPC), a high-level cognitive area.
Previous animal studies carried out at the Andersen lab found that it is in the PPC that the initial intent to make a movement is formed. These intentions are then sent to the motor cortex, through the spinal cord, and on to the limbs, which then move.
Prof. Andersen said:
“The PPC is earlier in the pathway, so signals there are more related to movement planning – what you actually intend to do – rather than the details of the movement execution.”
“When you move your arm, you really don’t think about which muscles to activate and the details of the movement – such as lift the arm, extend the arm, grasp the cup, close the hand around the cup, and so on. Instead, you think about the goal of the movement, for example, ‘I want to pick up that cup of water.’”
“So in this trial, we were successfully able to decode these actual intents, by asking the subject to simply imagine the movement as a whole, rather than breaking it down into a myriad of components. We expected that the signals from the PPC would be easier for patients to use, ultimately making the movement process more fluid.”
In April 2013, the device was surgically implanted in Mr. Sorto’s brain at Keck Hospital of USC. Since then, he has been training with Caltech scientists and Rancho Los Amigos staff to control a computer cursor and a robotic arm using his mind.
The researchers were delighted to see exactly what they were hoping for – the intuitive movement of the robotic arm.
Mr. Sorto, who was thrilled with the quick results, said:
“I was surprised at how easy it was [to control the robotic arm]. I remember just having this out-of-body experience, and I wanted to just run around and high-five everybody.”
The surgical procedure
On 17 April, 2013, the five-hour operation to implant the neuroprosthetic device was performed by a Keck Medicine team, led by Prof. Liu.
A pair of 4-by-4 millimeter electrode arrays were implanted in two parts of the PPC, one that controls grasp and another reach. Each array contains ninety-six active electrodes that record the activity of single neurons in the PPC.
An fMRI scan used for targeting the device implantation location. (Image: Caltech)
The arrays are connected by cable to a computer system that processes the signals, to decode Mr. Sorto’s bain’s intent and control output devices, such as the robotic arm and computer cursor.
Prof. Liu said:
“These arrays are very small so their placement has to be exceptionally precise, and it took a tremendous amount of planning, working with the Caltech team to make sure we got it right.”
“Because it was the first time anyone had implanted this part of the human brain, everything about the surgery was different: the location, the positioning and how you manage the hardware.”
“Keep in mind that what we’re able to do–the ability to record the brain’s signals and decode them to eventually move the robotic arm–is critically dependent on the functionality of these arrays, which is determined largely at the time of surgery.”
Neurologist Christianne Heck, associate professor of neurology at USC and co-director of the USC Neurorestoration Center, said:
“We are at a point in human research where we are making huge strides in overcoming a lot of neurologic disease. These very important early clinical trials could provide hope for patients with all sorts of neurologic problems that involve paralysis such as stroke, brain injury, ALS and even multiple sclerosis.”
The rehabilitation
Mr. Sorto’s training sessions started sixteen days after surgery. A computer was connected directly to the ports extending from his cranium, to communicate with his brain.
A team of specialised occupational therapists worked with Mr. Sorto and the Caltech team to help the patient visualize what it would be like to move his arm again.
Andersen and Sorto have shared many happy moments during this trial. (Image: Caltech)
Prof. Andersen said:
“It was a big surprise that the patient was able to control the limb on day one – the very first day he tried. This attests to how intuitive the control is when using PPC activity.”
After weeks of practice, Mr. Sorto was able to refine control of his robotic arm. He can now execute advanced tasks with his mind, such as moving the computer cursor, having a drink with virtually total control of the beverage container, making a hand-shaking gesture, and performing a range of other tasks.
Dr. Aisen, who led the rehabilitation team, said:
“We at Rancho are dedicated to advancing rehabilitation and to restoration of neurologic function through new technologies, which can be assistive or can promote recovery by capitalizing on the innate plasticity of the human nervous system.”
“This research is relevant to the role of robotics and brain-machine interfaces as assistive devices, but also speaks to the ability of the brain to learn to function in new ways. We have created a unique environment that can seamlessly bring together rehabilitation, medicine, and science as exemplified in this study.”
Mr. Sorto has agreed to continue participating in the projecting for a third year. He says the study has motivated him to pursue a master’s degree in social work.
Mr. Sorto said:
“This study has been very meaningful to me. As much as the project needed me, I needed the project. It gives me great pleasure to be part of the solution for improving paralyzed patients’ lives.”
“I joke around with the guys that I want to be able to drink my own beer–to be able to take a drink at my own pace, when I want to take a sip out of my beer and to not have to ask somebody to give it to me. I really miss that independence.”
“I think that if it were safe enough, I would really enjoy grooming myself – shaving, brushing my own teeth. That would be fantastic.”
Prof. Andersen said:
“The better understanding of the PPC will help the researchers improve the neuroprosthetic devices of the future. What we have here is a unique window into the workings of a complex high-level brain area, as we work collaboratively with our subjects to perfect their skill in controlling external devices.”
Citation: “Decoding motor imagery from the posterior parietal cortex of a tetraplegic human,” Tyson Aflalo, Richard A. Andersen, Spencer Kellis, Christian Klaes, Brian Lee, Charles Liu, Ying Shi, Kelsie Pejsa, Kathleen Shanfield, Christi Heck, Stephanie Hayes-Jackson and Mindy Aisen. Science. Published 22 May 2015. DOI: 10.1126/science.aaa5417.
Video – Controlling a robotic arm purely with thoughts