Mind Control Prosthesis
Turning thought to action
A new approach to myoelectric prosthetics gives amputees unprecedented, intuitive control of a robotic hand
Abstract
Losing a limb can be a devastating setback that science and technology struggle to fully offset, even after thousands of years of effort. Prosthetic devices in use today offer limited functionality or can be too cumbersome for amputees to use effectively.
While advanced robotic hands exist, amputees don’t have ways to intuitively control them. That lack of naturalistic control contributes, in many cases, to some abandoning their prostheses because they find life easier without them. [1]
A biological interface developed by clinicians and engineers at the University of Michigan is giving amputees new, intuitive control of the most advanced robotic hands on the market. Their new approach centers on the Regenerative Peripheral Nerve Interface (RPNI)—a small graft of muscle tissue surgically attached to the end of a severed nerve in an amputee’s arm.
While other neural interfaces are harmful to nerves, the RPNI promotes healthy nerve growth and acts as a bioamplifier, converting faint neural signals sent from the brain into large, recordable, muscle signals that remain stable for years. Combined with machine learning algorithms, these signals enable intuitive, real-time mind control of advanced robotic prosthetic hands.
The RPNI grafts also impede the growth of painful neuromas at the severed ends of nerves, reducing: the need for chronic pain medication, repeated surgical procedures, and an inability to use prosthetics due to pain levels.
This is a story about a research project at the University of Michigan.
The peer-reviewed paper was published on March 4, 2020 in the journal Science Translational Medicine and is titled, “A regenerative peripheral nerve interface allows real-time control of an artificial hand in upper limb amputees” DOI: 10.1126/scitranslmed.aay2857
Introduction
Connecting the mind to a robotic prosthetic hand
The human brain controls arm and hand movement using multiple peripheral nerves that run from the spinal cord to all of the muscles of the arm and hand. Impulses sent from the brain are carried down through the spinal cord and along the peripheral nerves to generate muscle contractions, which pull on the bones in the hand via attached tendons, generating movement.
When a surgeon amputates part of an upper limb, those peripheral nerves are severed.
These signals can be recorded and used to control a prosthetic hand. U-M’s approach is to attach small muscle grafts, called RPNIs, to the end of the severed nerves. [3] The grafts are stimulated by the neural impulses, generating an electric potential that an embedded electrode records and transmits to a robotic prosthesis. [4]
Preventing painful neuroma growth
The current state of prosthetic hands
The body-powered hook continues to be used by a large percentage of amputees due to ease of use and affordability. But its function is limited. It can only open and close and requires a harness that runs behind the back and over the opposite shoulder.
Myoelectric prostheses offer more functionality but the most common varieties available today fall short of an intuitive solution. These devices rely on electromyographic (EMG) signals from an amputee’s remaining muscles, typically recorded by electrodes placed on the skin of the arm. For example, a person amputated above the elbow could flex their biceps and triceps to toggle between and activate different hand functions, including open/close fist, rotate wrist, and pinch index finger and thumb.
New, intuitive prosthetic systems can tap into the brain, nerves or muscle tissue to let an amputee simply think about moving their hand and the prosthesis responds as intended. U-M’s approach uses grafted muscle tissue to harness these signals in a way that is less invasive and longer lasting that the other approaches.
Toward intuitive prostheses
Researchers around the world are searching for ways to access and amplify the body’s neural commands to give amputees a more natural experience with a prosthesis.
Targeted muscle reinnervation (TMR), an advanced myoelectric approach, involves rerouting severed peripheral nerves to large nearby muscle groups and picking up their signals with external electrodes placed on the skin.[7] With TMR, an amputee can more naturally control a prosthetic hand because nerves that originally ran to the phantom hand now run to the chest, for example, and generate prosthetic-controlling muscle contractions there. The electrodes that pick up these contractions can shift and encounter sweat, though, making TMR susceptible to signal disruption. It also requires an invasive surgical procedure and offers only a limited number of usable signals.
Epineural electrodes, often called “cuff” electrodes, wrap around the outside of a nerve and record the electric potential near its surface.[8] Without accessing the center of the nerve bundle, the full signal is not captured. In addition, nerves don’t like being touched by foreign material, leading to irritation, scar tissue and a signal-to-noise ratio that weakens over time.
Intraneural electrodes are tiny needles that penetrate the nerve itself, allowing for more selectivity in which signal is captured.[9] It’s an approach that has yielded encouraging results, but scar tissue can form, diminishing signal strength.
U-M’s RPNI
The evolution of the RPNI
Methods
The researchers partnered with upper-limb amputees and used ultrasound, electromyogram (EMG) recording and both virtual and physical prosthetic control tasks to validate whether human amputees can use RPNIs to naturally control an advanced robotic hand over a long-term period.
Three participants had elected to undergo RPNI surgery at U-M in order to treat pain, and they later agreed to join this prosthetics control study.
Details about Selected Study Participants
Creation of the Regenerative Peripheral Nerve Interface, or RPNI
To create each RPNI, surgeons harvested a 3 x 1.5 x 0.5 cm graft from a healthy muscle elsewhere on the patient’s body, often the vastus lateralis (large thigh muscle). Then they made a clean cut on the end of a peripheral nerve in the amputated limb to remove the neuroma. If multiple RPNIs were planned for that peripheral nerve, surgeons divided it into individual nerve bundles (fascicles). Then they wrapped the muscle graft around the end of the nerve and sutured it in place. They repeated the process until they created the desired number of RPNIs.
In the three months following surgery, the participant’s body helped the RPNI regenerate, revascularize and reinnervate.
Observing RPNI contractions via ultrasound imaging
Post-surgery, the team utilized ultrasound to visually measure the physical contractions of the RPNIs as each participant thought about moving their phantom hand in specific ways. The researchers quantified the strength of the RPNI contraction by measuring the amount of pixel change in an ultrasound video clip between an RPNI at rest and at maximum contraction.
Recording EMG signals from RPNI using indwelling electrodes
Their next step was recording EMG signals within the RPNIs as the participants thought about and generated specific phantom hand movements. An EMG signal is a measurement of the electric potential that muscle tissue generates when stimulated by a nerve.
Researchers first measured EMG signals by temporarily inserting, via a needle, thin wires through the skin and into the RPNIs. They removed the wires after each recording session. All three amputees participated.
Later, Hamilton and Sussex consented to surgeons implanting electrodes and wires for a specified length of time.
In both temporary and more permanent electrode configurations, researchers had the participants think about making specific postures or movements with their phantom hands. The team measured the strength of EMG signals from each electrode and how signals from different electrodes correlated with different hand gestures or finger movements.
Using RPNI to intuitively control a prosthetic hand
Once the team was able to record EMG signals from the RPNIs, they had participants try to intuitively control both virtual and physical prosthetic hands. Two varieties of machine learning algorithms built by the researchers took in recorded EMG signals and generated control parameters for the prosthetic hands.
The first algorithm used a naive-Bayes classifier
In one “discrete” task, the participant controlled a virtual or physical prosthetic hand in real time to mirror the posture of a separate virtual avatar hand. The researchers measured whether the participant was successful, and how quickly.
In another task, the participant controlled a physical prosthetic hand mounted to their amputated limb to perform simple tasks, like holding or moving objects of different shapes and sizes.
In a third exercise, researchers played a game of rock, paper, pliers with the participant. Pliers, or pinch, was a substitute for the traditional scissors, since a scissor motion could not be performed mechanically by the prosthesis.
The second type of algorithm used a Kalman filter
In one “continuous” task, called a center-out task, participants controlled the thumb of a virtual or physical prosthetic hand in real time to hit various targets. Researchers measured whether each participant was successful, and how quickly.
In another task, participants controlled a physical prosthetic hand attached to their amputated limbs to perform slightly more complex tasks, such as stacking small blocks, using a key, and using a zipper.
The differences between the algorithms
Results
RPNIs create natural prosthetic control with long-term stability
Using ultrasound imaging, EMG recording and prosthetic control tasks, researchers demonstrated that RPNIs attached to the arm’s peripheral nerves will physically contract when an amputee thinks about moving their phantom hand. This generates large EMG signals to serve as real-time control signals for a robotic prosthetic hand. The RPNIs remain stable for years and offer greater levels of control than other approaches.
Strong RPNI contractions observed using ultrasound
Ultrasound testing showed clear contractions of the RPNIs when participants thought about moving their phantom hand.
Ultrasound of RPNI Contractions
Live ultrasound of RPNI muscle grafts contracting.
Watch Video Video with audio descriptionsLarge signal-to-noise ratio recorded from RPNIs
The mean signal-to-noise (SNR) ratios during testing were 4.21 for Moran, 68.9 for Hamilton, and 21.0 for Sussex.
With other neural interface approaches, epineural electrodes and intraneural probes typically record SNRs between 4 and 15 [8] [9] [17] [18]. One intraneural probe, the Utah Slanted Electrode Array, has been shown to record larger signals, but it currently only remains stable for months. [19] RPNIs, however, have remained stable for years.
Stronger Signals
Live electromyographic (EMG) signals during a testing session
Watch Video Video with audio descriptionsQuick and dexterous control of prosthetic hand using RPNIs
Hamilton and Sussex successfully controlled a prosthetic hand to mirror gestures and movement of both a virtual and physical hand, as well as manipulated objects and played games like rock, paper, pliers.
In a posture-mirroring "discrete" task, the two participants controlled a virtual hand to intuitively generate and switch between useful hand postures like pinch, fist and point. They matched postures quickly with a success rate between 94% and 100%. The average time between an RPNI contraction and the team’s control algorithm generating a command was less than 0.3 seconds.
In the center-out "continuous" movement task, participants intuitively moved their thumb along two degrees of freedom—forward and back and side to side—giving them the ability to move their thumb in a circle. When testing only the forward and back motion, both participants hit between 96% and 100% of the targets on the first day. Without recalibrating the control algorithms, Hamilton still hit 100% on day 300. Sussex hit 96.4% on day 97, her last recorded day for this study.
Intuitive Control
Performing tasks with virtual and physical prostheses.
Watch Video Video with audio descriptionsConclusion
Advanced prosthetic control via RPNI and the way forward
The University of Michigan's RPNI approach shows great promise for improving quality of life for amputees. It can effectively treat pain and enable more intuitive, life-like use of prosthetic devices.
These small pieces of grafted muscle naturally interact with the body’s nervous system to amplify neural impulses, delivering strong, clear signals for naturalistic, real-time prosthetic control. And these signals remain stable over the long-term, beyond what other approaches can offer.
The team’s current FDA Investigational Device Exemption allows them to implant twelve indwelling electrodes in each of 10 participants. They are looking for more study participants to partner with, especially those with upper arm amputations.
Beyond this 10-participant study, the team is evaluating the use of finer electrodes, and more of them, for greater EMG signal resolution. Future enhancements could include adding a wireless capability and exploring the effectiveness of RPNIs for prosthetic leg and foot control. The U-M team has also seen evidence that the RPNI can send sensory information back to the brain—a focus of further study.
In addition, RPNIs generate control signals for finer movements of the hand than what most robotic prostheses can perform at present. The team has begun collaborating with prosthetic device companies to help future devices accommodate these finer movements.