Sensory stimulation enhances phantom limb perception and movement decoding

Objective. A major challenge for controlling a prosthetic arm is communication between the device and18 the user’s phantom limb. We show the ability to enhance amputees’ phantom limb perception and improve movement19 decoding through targeted transcutaneous electrical nerve stimulation (tTENS).20 Approach. Transcutaneous nerve stimulation experiments were performed with four amputee participants to map21 phantom limb perception. We measured myoelectric signals during phantom hand movements before and after22 amputees received sensory stimulation. Using electroencephalogram (EEG) monitoring, we measure the neural23 activity in sensorimotor regions during phantom movements and stimulation. In one participant, we also tracked24 sensory mapping over 2 years and movement decoding performance over 1 year.25 Main results. Results show improvements in the amputees’ ability to perceive and move the phantom hand as26 a result of sensory stimulation, which leads to improved movement decoding. In the extended study with one27 amputee, we found that sensory mapping remains stable over 2 years. Remarkably, sensory stimulation improves28 within-day movement decoding while performance remains stable over 1 year. From the EEG, we observed cortical29 correlates of sensorimotor integration and increased motor-related neural activity as a result of enhanced phantom30 limb perception.31 Significance. This work demonstrates that phantom limb perception influences prosthesis control and can benefit32 from targeted nerve stimulation. These findings have implications for improving prosthesis usability and function33 due to a heightened sense of the phantom hand.34 1. Introduction35 Sensory information, specifically touch and proprioception, are essential for palpating, exploring, and manipulating36 objects in our surroundings [1]. Through sensory feedback and errors in our sensory predictions, we develop so-37 phisticated internal models of sensorimotor integration [2], and we continue to update and strengthen our internal38 1 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice. sensorimotor models for controlling limb movement [3]. Recently, researchers showed that supplementary audi-39 tory feedback can help improve internal models and performance in myoelectric control of a virtual prosthesis by40 able-bodied subjects [4], further indicating the role of feedback in sensorimotor control loops.41 For upper limb amputees, the sensorimotor loop is severely disrupted as a result of limb loss; however, perception of42 the phantom limb persists for many [5]. Researchers made profound breakthroughs in providing naturalistic tactile43 sensations back to amputees by stimulating peripheral nerves, both invasively [6–9] and noninvasively [10–12], in44 the residual limb. Sensory feedback can provide perceptions of pressure [6,7], enable discrimination of textures [8],45 create perceptions of movement across the phantom hand [9], help in reducing phantom pain [13], and improve46 prosthesis use at home [14]. Biomimetic stimulation models can enhance naturalness of the tactile sensation [15],47 improve object manipulation [16], and be used to provide receptor specific information to enable sensations of48 pressure or pain [12]. Kinesthetic illusions of phantom hand movement have also been produced using skin vibration49 on amputees who had undergone targeted muscle reinnervation (TMR) surgery [17]. Despite these successes, there50 is an unanswered question about the effect enhancing phantom hand perception has on the internal sensorimotor51 models that control phantom hand movements. Specifically, it is unclear how phantom hand perception affects52 motor function and resulting activation of muscles in the residual limb. Pattern recognition techniques aim to create53 a natural and intuitive control strategy for upper limb amputees by decoding movement from electromyography54 (EMG) signals in the residual limb [18]. Recently, proportional control of multiple degrees of freedom was achieved55 with derived motor unit action potentials in TMR subjects [19] and direct control using surface EMG electrodes [20].56 We postulate that an important component of myoelectric decoding is the ability to perceive and move the phantom57 hand. Neural signals measured by EEG after TMR suggest that more natural cortical representations of the missing58 limb can occur in the motor cortex as a result of the surgery [21]. Additionally, recent results show somatosensory59 neural representation of the phantom limb exists even decades after amputation [22]. It is also known that movement60 representations persist in the motor cortex even when an amputee cannot generate voluntarily movements with the61 phantom hand, indicating that the lack of phantom control62 EMG Activity Transcutaneous Nerve Stimulation EMG electrodes Phantom Limb Control Phantom Limb Perception Movement Decoding Fig. 1. Phantom limb perception and control. Upper limb amputees often perceive their phantom limb. V oluntary movements of the phan- tom limb can be captured and decoded from electromyography (EMG) signals in the residual limb. We demonstrate the role of targeted tran- scutaneous electrical nerve stimulation (tTENS) to enhance phantom perception and improve movement decoding in a comprehensive in- vestigation with 4 amputees. is not equivalent to the loss of neural representation63 [23]. Furthermore, evidence suggests that despite clas-64 sical ideas of cortical reorginization after limb amputa-65 tion, phantom limb representation of motor commands66 and muscle synergies still persist in the primary mo-67 tor cortex [24]. Interestingly, activation of neural activ-68 ity from sensory feedback through electrical stimulation69 occurred in both somatosensory and premotor regions70 during evoked phantom limb sensations [25].71 In this work, we hypothesize that providing sensory72 stimulation to amputees can modulate the sensorimotor73 loop and enhance phantom limb perception, improving74 the ability to decode phantom hand movements using75 EMG pattern recognition (Fig. 1). Our study presents76 2 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint a number of important observations. Firstly, we demonstrate that sensory stimulation improves perception of the77 phantom hand. Secondly, we show that changes in phantom hand perception affect the ability to control phantom78 movements and a prosthesis. Finally, using EEG signals we show that increased activation of sensorimotor regions79 occur both during and after sensory stimulation and phantom limb activation.80 2. Methods81 Four male amputee participants with varying levels of prosthesis experience, ranging from none to over 8 yr, were82 recruited for this study and participated in at least one experiment (Table S1). Two amputee participants (A01 and83 A02) underwent elective amputations as a result of nerve injury, and three of the participants (A01, A03, and A04)84 have transhumeral amputations. Participant A02 has a transradial amputation. A03 also has a right arm transradial85 amputation but only uses a prosthesis on his left arm, which was the side used for the experiments in this study.86 Participants A01-A03 performed phantom hand movement tasks before and after receiving sensory stimulation to87 the phantom hand and took a user survey. A02 and A03 also participated in EEG recording experiments during88 phantom hand movements. A02 participated in a long-term study over 2 years to track changes in phantom sensory89 mapping and movement decoding performance. A04 participated in sensory mapping and the object movement task.90 Amputee participation is summarized in Table S1. All experiments were approved by the Johns Hopkins Medicine91 Institutional Review Boards. The amputees, who were recruited from previous studies or referrals, provided written92 informed consent to participate in the experiments.93 2.1. Sensory stimulation94 Sensory mapping was done with tTENS using a monopolar 1 mm beryllium copper (BeCu) probe connected to95 an isolated current stimulator (DS3, Digitimer Ltd., UK) to provide monophasic square wave pulses to underlying96 peripheral nerves, activating the phantom hand. This approach was validated in our previous work [12, 26]. An97 amplitude of 0.8 – 3.0 mA, frequency (f ) of 2 – 4 Hz, and pulse width (pw ) of 1 – 5 ms were used while mapping98 the phantom hand [12, 26]. Anatomical and ink markers were used, along with photographs of the amputee’s limbs,99 to map the areas of the residual limb to the phantom hand. For all other stimulation experiments, we used 5 mm100 disposable Ag/AgCl electrodes (Norotrode 20, Myotronics, USA) with pw = 1 ms and f = 45 Hz. Stimulation101 parameters were based on our previous work [12, 26] and were reliably detected by every participant.102 2.2. EMG recording and movement decoding103 For participants A01 and A02, 8 channels of raw EMG signals were measured using 13E200 Myobock amplifiers104 (Ottobock, Plymouth, MN) with bipolar Ag/AgCl electrodes placed uniformly around the circumference of the105 residual limb. No specific muscle groups were targeted for electrode placement. Signals were recorded by an NI106 USB-6009 (National Instruments, Austin, TX) at 1024 Hz with a 20 – 500 Hz digital bandpass and 60 Hz notch107 filters.108 Participant A03 used a custom socket manufactured by his prosthetist (Dankmeyer, Linthicum, MD). Eight bipolar109 Ag/AgCl electrodes (Infinite Biomedical Technologies, Baltimore, MD) were embedded within the socket. The110 bipolar electrodes in the custom socket were amplified and filtered with a 20 – 500 Hz digital bandpass and 60 Hz111 digital notch filters.112 3 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint EMG signal time domain features were extracted after 2 s of sustained movement using a 200 ms sliding window113 with new feature vectors computed every 50 ms. The features used were mean absolute value, waveform length,114 and variance (Supplementary Methods). Each movement cue was presented 3 times for 5 s and in a random order.115 Data from Day 187 of the long-term study was used for A03’s 9 class comparison because that was the first day116 he performed the movement decoding experiment before and after sensory stimulation. Movement decoding on117 Day 194 was done with 4 of the 8 electrodes in A03’s custom socket due to hardware failure. Data from 1 round118 of movements was discarded from each of A02’s visits due to hardware malfunction. For simultaneous tTENS119 with EMG recording from A02, grounding electrodes were placed on the residual limb to remove noise artifacts120 (Supplementary Methods, Fig. S1). Movements were decoded using the extracted EMG features with an LDA121 classifier. One-third of the EMG data was used as a holdout set from the training data for testing the classifier [18].122 The classifier was trained and tested on data from the same day.123 Participant A04 wore 2 EMG recording armbands (Myo, Thalmic Labs, CA), 8 stainless steel electrodes per band,124 on his residual limb, which was his typical setup for controlling the prosthesis used in this experiment during his125 daily activities. Filtered EMG data was collected from the armbands at 200 Hz. Training data was collected and126 movements were decoded using an LDA classifier on a custom controller embedded in the prosthesis [27].127 2.3. EEG recording and analysis128 Ag/AgCl EEG electrodes were used for recording neural activity at 500 Hz sampling frequency (64-ch, SynAmps2,129 Compumedics NeuroScan). Participant A02 and A03 took part in this experiment. Each participant was seated and130 tTENS electrodes were placed to activate the areas of the phantom hand corresponding to median, ulnar, and radial131 nerve innervation regions. Stimulation was for 2 s, followed by a 4 s delay with ± 25% time jitter before the next132 stimulation. The EEG data was band-pass filtered from 1 to 50 Hz and re-referenced to both mastoids. Automatic133 Artifact Removal (AAR) was used to remove the muscle (canonical correlation approach, 5 s window size) and134 ocular (blind source separation SOBI algorithm, 256 s window size) artifacts [28]. Independent component analysis135 (ICA) was used to remove additional artifacts. Continuous EEG data was epoched from 1 s before the start of each136 trial until 2 s after the stimulus presentation. All analysis was done using the EEGLAB toolbox in MATLAB [29].137 The epoched EEG data was band-pass filtered from 8 Hz to 12 Hz to obtain the alpha band. We further epoched138 the data from 450 – 850 ms after the stimulus presentation to remove early activation due to tTENS and visual139 stimulation from analysis and focused on the motor-related activity in the brain. We evaluated the alpha band power140 relative to the total power of all bands in all electrodes for each condition and each trial. For each participant,141 phantom hand stimulation conditions (thumb and wrist for A02; thumb, pinky, and wrist for A03) were included for142 the rest of the analysis.143 2.4. Experimental protocol144 Phantom Movements with Stimulation: We used a modified Virtual Integration Environment (VIE) (Johns Hopkins145 University Applied Physics Lab (JHU/APL), Laurel, USA) in MATLAB to display movement cues. The subjects146 were seated in front of a screen that displayed the movement classes. The skin of the residual limb was cleaned147 with an alcohol wipe before tTENS and EMG electrode placement. The electrodes were allowed to settle for up to148 10 min. After EMG data collection, the subject received tTENS. The sensory stimulation lasted between 30 – 60149 4 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint min with continuous site activation for up to 10 s at a time. After tTENS, the participants performed another round150 of EMG data collection. Anatomical markers and photographs were used to ensure the electrodes were positioned151 in approximately the same location for A01 and A02. A03 used a customized socket, which ensured consistent152 electrode placement. The experiment lasted up to 3 hr.153 For the long-term study, participant A03 performed periodic sensory mapping over 2 years (Day 1 - Day 738). A03154 also performed 3 different phases of EMG data collection starting on Day 128 (labeled as Week 1 of the long-term155 EMG data). Fourteen movement classes were used (Fig. 5C-F, including a rest class). During Phase I (Week 1-6),156 A03 came in for an EMG recording session on average once per week. For Phase II (Week 8-10), he came in for157 EMG recording sessions on 4 different days. There were 3 separate rounds of EMG data collection on each of158 those days. EMG signals during the Pre-Stim condition were recorded for each movement and repeated 3 times.159 Next, movement cues were shown while sensory stimulation was being given to the phantom hand (Fig. 5). A final160 EMG recording session was performed without stimulation. There was up to a 30 min break between each of the 3161 recording sessions. The total experiment lasted up to 3.5 hr each day. In Phase III (Week 12-48), A03 performed 4162 follow-up EMG recording sessions. All EMG experiments were offline and participants didn’t receive feedback on163 EMG activity or decoding performance to prevent bias across the testing conditions.164 Object Movement Task: Participant A04 used a modified VIE to interface with the pattern recognition software165 and prosthesis controller. The Modular Prosthetic Limb (MPL) [27], developed by JHU/APL, was mounted to the166 osseointegrated implant. A04 completed the object movement task before any sensory stimulation (Pre-Stim). A04167 underwent tTENS sensory mapping and phantom hand activation for approximately 1.5 hr before completing the168 object movement task again after the sensory stimulation (Post-Stim). A new LDA pattern recognition classifier169 was trained before performing the object movement task for both the Pre-Stim and Post-Stim conditions. Movement170 classes used were hand open, tripod grasp, elbow (flexion and extension), and wrist (pronation and supination).171 Each movement class contained up to 5 s of training data. Each trial consisted of 5 repetitions of grabbing the172 object, moving it approximately 60 cm, and then releasing it. Participant A04 performed 3 trials of the task in173 both the Pre-Stim and Post-Stim conditions. No tactile feedback or tTENS was provided to A04 during the object174 movement task. Time to complete the task was recorded for each trial. The participant successfully moved the object175 every trial without dropping it.176 Neural Recording: The participants were seated and shown visual movement cues with corresponding stimulation in177 median, ulnar, and radial regions, respectively (Fig. 5). Participant A02 was shown hand open and close. Participant178 A03 was shown tripod, index point, and wrist flexion. Baseline activity was recorded for up to 2 min. Pre-Stim: the179 participant mimicked movement cues with his phantom hand. Stim: the participant received tTENS to activate the180 phantom hand, but did not perform movements with his phantom hand. Stim-Move: the participant received sensory181 stimulation while performing movements with his phantom hand. Post-Stim: the participant performed phantom182 hand movements but with no sensory stimulation. For participant A02, each movement cue was presented 30 times183 for all conditions. For participant A03, each movement cue was presented 10 times for the Pre-Stim condition and184 20 times for all other conditions.185 For all experiments, results from data collected over multiple trials of the same experiment were averaged together.186 5 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint A01 A02 A03 A04 A01 A02 A03 A04 A01 A01 A02 A03 A03 A04 Ulnar Median Radial Coverage Region Pressure Buzzing/ Vibration Cold Electrical/ Tingling Sensation Type A B C DSensations Sensations Stimulation Sites Posterior PosteriorDistal DistalPosterior Distal Fig. 2. Sensory mapping of amputee participants. (A) Participant A01 reported sensations of general tactile activation, primarily buzzing or vibration, along with sensations of temperature changes on the palmar side of the middle and ring fingers. (B) Participant A02 reported sensations of pressure in the activated regions. The thumb and index finger, along with the ulnar and palmar sides of the hand, were the primary regions of activation. (C) Participant A03 perceived sensations of pressure and occasional tingling in the thumb, pinky, and wrist regions of his phantom hand. (D) Participant A04 perceived sensations as pressure in his phantom hand. For all phantom hand sensory maps, regions of strongest to faintest activation are indicated by a gradient of solid to faded color. In general, stimulation sites on the residual limb are <5 mm in diameter but are made larger here for illustration. Statistical p values were calculated using a two-tailed, two-sample t test and error bars represent the standard error187 of the mean, unless otherwise specified. All analysis was performed using MATLAB (MathWorks, Natick, USA).188 3. Results189 3.1. Sensory stimulation enhances phantom hand perception190 For each participant, we used sensory mapping to identify regions of phantom hand activation. Targeted transcuta-191 neous electrical nerve stimulation (tTENS) was used to activate underlying peripheral nerves in the residual limb,192 a method which was used in previous studies (Supplementary Discussion) [10, 12, 26]. Stimulation of the mapped193 regions on the residual limb resulted in perceived sensation in the phantom hand. Each amputee’s perception of194 their phantom limb is different and tTENS activated different phantom regions (Fig. 2). Sensations were reported195 primarily as tactile and included pressure, buzzing, vibration, and in the case of A01, a sensation of cold temperature196 on the palmar side of the middle and ring fingers (Fig. 2A).197 A user survey to gauge phantom hand perception, based on a previous study [30], was given to participants A01-A03198 at the end of the day after a testing session (Fig. 3). In general, participants felt as if something was touching the199 phantom hand during the sensory stimulation. Furthermore, all participants who took the survey felt as if they could200 better perceive and, more importantly, move their phantom hand as a result of the nerve stimulation (Fig. 3).201 Participant A01 took the survey once, A02 completed the survey twice in person and an additional time during a202 follow-up phone interview, and A03 completed the survey twice. The survey was meant to gauge user perception203 of the phantom hand and sensory stimulation. All users reported enhanced perception and control of the phantom204 limb compared to normal baseline as a result of sensory stimulation. The statements were modeled after surveys205 from a previous study [30]. Results from the survey targeted specifically at quantifying the enhanced perception206 of the phantom limb as a result of sensory stimulation are shown in Fig. 3B. In general, participants felt as if207 something was touching the phantom hand during the sensory stimulation (Fig. 3A). Furthermore, all participants208 6 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint Statement 2 10 11 12 0 1 2 3Response Phantom Hand Perception: Average Responses Survey Statements 1. It felt like something was touching my residual limb 2. It seemed like something was touching my phantom hand 3. It felt as if my phantom hand was intact 4. It felt like my residual limb was moving towards my phantom hand 5. It felt as if I had an extra arm 6. I could sense a touch somewhere between my residual limb and my phantom hand 7. My residual limb began to feel rubbery 8. It was as if I could feel my residual limb moving 9. My phantom hand started to change shape, color, and appearance in my mind so that it started to resemble my residual limb 10. I could better perceive my phantom hand 11. I could feel my phantom hand moving 12. I felt like I could better control my phantom hand Statement 1 2 3 4 5 6 7 8 9 10 11 12 -3 -2 -1 0 1 2 3 Response A01 A02 A03 Average (+3) Strongly agree (+2) Agree (+1) Somewhat agree (-1) Somewhat disagree (-2) Disagree (-3) Strong disagree (0) Neither agree nor disagree Likert Scale CB A D Fig. 3. Sensory stimulation improves phantom perception as reported by user surveys. (A) User survey aimed at understanding subjective perception of sensory stimulation. Mixed results for several statements across participants suggest the varying nature of perception due to sensory stimulation. However, all participants agreed that heightened perception were a result of sensory stimulation through electrical nerve stimulation. Participants A02 and A03 took the survey twice on different days after sensory stimulation and phantom hand movement matching experiments. Participant A02 completed the survey again during a follow-up phone interview. The results were averaged. A01 took the survey once. (B) Averaged user results from survey response specifically on phantom hand perception as a result of sensory stimulation. For all participants, sensory stimulation enhanced perception of the phantom hand and importantly also giving the perception of better control over phantom hand movements. (C) Statements from the user survey. (D) The survey was scored using a Likert Scale with answers to statements ranging from “Strongly agree” (+3) to “Strongly disagree” (-3). who took the survey felt as if they could better perceive and move (Q10 and Q12, respectively) their phantom209 hand as a result of the nerve stimulation (Fig. 3C). A04 did not take the survey, but he did verbally confirm that210 the sensory stimulation produced enhanced phantom hand perception. It should be noted that our survey does not211 capture changes in prosthesis embodiment or agency. The survey provides subjective responses from the participants212 to better understand their perceptions of phantom sensations.213 3.2. Sensory stimulation improves movement decoding214 Because sensory stimulation provides a heightened sense of the phantom hand (Fig. 3), we investigated the effect of215 this enhanced perception on the ability to make dexterous grasps with the phantom hand. Hand and wrist movements216 were visually presented to three of the participants (Fig. 4A, Fig. S2), who then attempted to mimic the movement217 with their phantom hand. Each amputee performed the hand and wrist movements before receiving any sensory218 stimulation (Pre-Stim). After EMG collection, regions of the phantom hand were activated via tTENS to provide219 general tactile sensation (see Methods).220 For participants A01-A03, the stimulation sites activated regions that covered the thumb, index, palm, and ulnar sides221 7 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint A01 A02 A03 Participant Accuracy Phantom Movement Decoding with Sensory Feedback A01: p=0.29 A02: p=0.22 A03: **p<0.01 Pre-Stim Post-Stim Participant 0 0.2 0.4 0.6 0.8 ** A01 A02 A03 %/uni0394 Accuracy Percentage Change n=3 per bar 0 50 100 150 Combined Participant Movement Decoding * Pre-Stim Post-Stim 0 0.2 0.4 0.6 0.8 1 Normalized Accuracy *p<0.05 n=9 per bar Visual cue EMG electrodes BA C D Fig. 4. EMG performance of amputee subjects. (A) Five hand movements (rest, open, close, tripod, index point) and four wrist movements (pronation, supination, flexion, extension) were presented, one at a time, to the amputee participant, who attempted to match the movement with his phantom hand. (B) EMG decoding accuracy from the 9 movement classes before (Pre-Stim) and after (Post-Stim) sensory stimula- tion. (C) Percentage change in performance accuracy for all participants (absolute changes in Fig. S2). Relative performance increased at least 35% (A01) and up to 95% (A02) from baseline as a result of enhanced phantom limb perception. (D) The combined performance of all participants, normalized to the maximum individual performance, increased increased after sensory stimulation. of the phantom hand (Fig. 2). The sensory stimulation session lasted up to 30 min and was followed by another222 round of EMG data collection (Post-Stim). The accuracy of the EMG movement classification is shown in Fig.223 4B-D. Linear discriminant analysis (LDA), a standard EMG pattern recognition algorithm [18], was used to classify224 the movements. Results indicate at least a 35% increase in baseline EMG pattern recognition performance for all225 three participants (Fig. 4B-C). Improvements occur for all participants, but p<0.05 only in the case of A03. Overall,226 the averaged normalized decoding performance across all participants increased as a result of sensory stimulation227 (Fig. 4D).228 3.3. Long-term sensory stimulation and EMG decoding229 To better understand the influence of enhanced sensory perception on EMG pattern recognition performance, partic-230 ipant A03 took part in an extended study over 2 years. The primary regions of perceived activation were the thumb231 and index finger, the ulnar side, and the wrist of the phantom hand. These regions remained stable; that is, they did232 not migrate over the course of the study (Fig. 5A, Fig. S3). The structural similarity (SSIM) index [31] was cal-233 culated for each region across all the days and shows good similarity (>0.75) in all cases (Fig. 5B, Supplementary234 Methods). With every stimulation session, the participant verbally indicated an enhanced perception of his phantom235 hand during sensory stimulation. This subjective response was based on the daily baseline phantom hand perception236 before any stimulation experiments began.237 We investigated the effects of sensory stimulation on movement decoding performance compared to long-term per-238 formance over 1 year. The participant identified different regions of activation that best corresponded to particular239 movements of his phantom hand (Fig. 5C-F). The combinations of sensation in targeted regions of the phantom hand240 with movement classes were made based on what the amputee determined as being relevant phantom hand regions241 during attempted phantom movements. For example, during the index point and precision close hand movements the242 participant said he moved the thumb and index fingers but his main focus was on closing his pinky and ring fingers.243 A custom prosthetic socket with embedded electrodes was used to ensure consistent electrode placement during244 each EMG recording session (Fig. S3). The long-term experiment was broken up into three phases. Phase I was 6245 weeks long (Week 1-6) to establish a baseline in performance. Phase II (Week 8-10) was a 3 week period of sensory246 stimulation with EMG recordings. Phase III (Week 12-48) was a 37 week follow-up set of sessions to evaluate247 8 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint 1 4 11 15 37 56 36729 327 738477 Median Ulnar Radial Coverage Region Day Open Close Key Radial Dev.Tripod Prec. Open Index Supinate Pronate Extension Flexion Prec. CloseUlnar Dev. B A C D E F G H Day Accuracy 187 193 194 196 198 0 0.2 0.4 0.6 Phase II: Movement Decoding with Sensory Feedback Pre-Stim Post-Stim ** ** p<0.01** n=3 per bar 128 328 428 528 Day Accuracy Performance Over Time 0 0.2 0.4 0.6 Phase I (Week 1-6) Phase III (Week 12-48) Phase II (Week 8-10) R2 <0.01 0.21 0.36 y = 0.0002x + 0.39 y = 0.0002x + 0.38 y = 0.0003x + 0.36 Trendlines Week 1-6: Week 1-48: Week 6-48: 228 Day 0.7 0.8 0.9 1 SSIM Sensory Map Similarity 1 400 600 800 Trendlines Median Ulnar Radial Fig. 5. Long-term sensory mapping and movements. (A) Sensory mapping from tTENS of the ulnar, median, and radial regions was performed on participant A03 over a 2 year period. Activation maps of his phantom hand remained stable over the duration of the study with the primary regions of sensation being on his thumb and index finger, pinky, and wrist. (B) Structural similarity (SSIM) indices of sensory maps for each region show high similarity (>0.75) across the extended study. C-F, The participant associated activation of certain regions of his phantom hand to different grasp patterns. (C) Activation in the median and ulnar regions of his phantom hand were most closely associated with opening, closing, and the lateral key grasp. (D) Thumb and index finger, (E) ulnar, and (F) wrist region activations were associated with corresponding hand and wrist movements. No stimulation was provided during the rest class. (G) EMG pattern recognition performance was measured over nearly 1 year. An initial set of baseline data was collected in Phase I (Week 1-6), followed by a 3 week period of sensory stimulation through tTENS (Phase II, Week 8-10). Phase III (Week 12-48) consisted of sessions over a 37 week period. The subject was experienced with pattern recognition and showed a fairly consistent level of performance with a non-significant increase over time likely a result of continued prosthesis use (p >0.05, Fig. S4). (H) The stimulation phase shows improvements in EMG movement decoding of the 14 classes as a result of enhanced phantom limb perception (individual classes in Fig. S4). EMG signal recordings were taken for each movement class before (Pre-Stim) and after (Post-Stim) stimulation. any lasting effects of the sensory feedback on the internal sensorimotor loop used by the amputee for moving his248 phantom hand (Fig. 5G). There were a total of 14 movement classes (Fig. 5C-F, 8 hand, including rest, and 6249 wrist movements). During Phase II, EMG signals were recorded during each movement class before (Pre-Stim) and250 after (Post-Stim) stimulation. The EMG pattern recognition accuracy remained fairly stable for the 6 week period251 of Phase I with slightly more variation during Phase III. The sensory information provided to the phantom hand252 resulted in within-day improved movement decoding in most cases during Phase II (Fig. 5H) and with significant253 improvements in tripod and radial deviation movements; however, the effect did not appear to persist into Phase III254 (Fig. S4). The long-term changes during Phase III match the overall trend from the beginning of Phase I, indicating255 that short-term improvements from sensory reinforcement did not translate beyond individual days (Fig. 5G).256 3.4. Movement decoding improves during sensory stimulation257 We also investigated the effects of sensory stimulation during active movement. Participant A02 identified thumb258 and wrist areas of his phantom hand that, when activated, corresponded with specific movements (Fig. 6A). Move-259 ment decoding was performed on EMG signals recorded before (Pre-Stim), during (Stim), and after (Post-Stim)260 stimulation. Results show an obvious improvement in classifying attempted phantom hand movements during tri-261 als with sensory activation, whereas only a slight improvement is observed for trials after stimulation (Fig. 6B).262 The stimulation noise artifact was removed from the myoelectric signal using a hardware grounding approach (Sup-263 plementary Methods). A classwise comparison shows improvement in some movements during and after sensory264 stimulation but a decrease in others (Fig. S5).265 9 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint Radial Deviation Supinate Pronate Extension Flexion Open A02: Activation During Movement Pre-Stim Stim Post-Stim Accuracy A02: Phantom Movement Decoding with Sensory Feedback n=3 per bar 0 0.2 0.4 0.6 0.8 ** * p<0.01 p<0.05 ** * A04: Object Movement Task Pre-Stim Post-Stim 0 10 20 30Completion Time (s) p=0.37 n=3 per bar A04: Object Movement Task Grab Move Release BA C D E A04: Osseointegrated Prosthesis Fig. 6. Movement decoding and prosthesis function improve after sensory stimulation. (A) A02 identified several wrist movements and corresponding phantom regions to receive sensation. (B) Movement decoding was done with EMG pattern recognition for trials before (Pre- Stim), during (Stim), and after (Post-Stim) stimulation to A02’s phantom hand. There was an increase in movement decoding during (Stim) and after (Post-Stim) stimulation, but the improvement is greater during the Stim condition. (C) Participant A04 used his osseointegrated prosthesis with EMG pattern recognition control for the functional task. (D) The object movement task consisted of grabbing, moving, and releasing an object using EMG pattern recognition prosthesis control . (E) The task completion time decreased after sensory activation of A04’s phantom hand (Post-Stim) as compared to the Pre-Stim task completion times. 3.5. Prosthesis control improves after sensory stimulation266 We also tested the functional difference of an object grasping task with a prosthesis before and after sensory ac-267 tivation of the phantom hand in a fourth amputee participant. A04, who has an osseointegrated implant [32] and268 TMR, controlled a prosthesis using EMG pattern recognition (Fig. 6C). The participant underwent sensory mapping269 (Fig. 2d) and performed the object movement task before and after sensory activation of his phantom hand. The270 participant grabbed, moved, and released a compact disc using a tripod grasp (Fig. 6D). The average task completion271 time decreased in the Post-Stim condition (p=0.37, Fig. 6E). A04 did not take the user survey or participate in the272 EMG movement decoding experiments; however, the participant did verbally confirm that the sensory stimulation273 produced enhanced phantom hand perception.274 3.6. Sensory stimulation increases EEG activity in sensorimotor regions275 EEG signals were recorded to capture the neural activity in sensorimotor regions during sensory stimulation and276 phantom hand movement in participants A02 and A03. The alpha band (8 – 12 Hz) is relevant for sensorimotor-277 related activity [33, 34] and was used to evaluate the influence of sensory stimulation on phantom hand movement278 related neural activity.279 The relative alpha power, the alpha power relative to the sum of power of all frequency bands, from the EEG was280 estimated for phantom hand movement before stimulation (Pre-Stim), during tTENS with no movement (Stim),281 during tTENS with movement (Stim-Move), and phantom hand movement after stimulation (Post-Stim) (Fig. 7A-282 10 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint Pre-Stim Stim-Move Post-Stim Stim p<0.001 p<0.01 p<0.05 *** ** * 0.34 0.30 0.26 0.18 0.22 A03 - Neural Activity: Relative Alpha Power Relative Power Pre-Stim: Movement Stim: Movement Post-Stim: Movement Stim: No Movement Pre-Stim: Movement Stim: Movement Post-Stim: Movement Stim: No Movement 0.15 0.2 0.25 0.3 Relative Power A02 - Neural Activity: Relative Alpha Power Participant A02 Participant A03 C3 CP2CP1 Cz C4 CPz CP4CP3 C2C1 C3 C1 Cz C2 C4 0.2 0.3 0.4 Electrodes Alpha Power ** * *** * **** *** * * ** * A03 EEG: Central Electrodes Electrodes CP3 CP1 CPz CP2 CP4 0.2 0.3 0.4 Alpha Power *** * *** * *** ** ***** ***** A03 EEG: Centro-Parietal Electrodes G H LK I J A B C D FE CP3 CP1 CPz CP2 CP4 Electrodes 0.15 0.25 0.35Alpha Power A02 EEG: Centro-Parietal Electrodes * * * C3 C1 Cz C2 C4 0.15 0.25 0.35 Electrodes Alpha Power A02 EEG: Central Electrodes * * * * Fig. 7 . Neural activity in sensorimotor regions. Participants A02 and A03 received visual cues and performed the corresponding hand movements during tTENS (Fig. S6). Sensory stimulation was given through tTENS and recordings for each condition (Pre-Stim, Stim, Stim-Move, Post-Stim) were performed in order with <10 min between conditions. (A-D) A02’s relative alpha power neural activation maps for movements before any sensory stimulation (Pre-Stim), stimulation with no phantom hand movements (Stim), movements with sensory stimulation (Stim-Move), and phantom hand movements (Post-Stim). The movements were hand open and close. Each grip corresponded to stimulation of different regions of the phantom hand (Fig. S7). (E-F) A02’s relative alpha power in the central and centro-parietal electrodes, respectively. For all conditions n = 60.(G-J) A03’s relative alpha power neural activation maps across the various conditions. The movements were tripod, index point, and wrist flexion. Each grip corresponded to stimulation of different regions of the phantom hand (Fig. S7). For Pre-Stim n=30 and for all other conditions n=60. (K-L) A03’s relative alpha power in the central and centro-parietal electrodes, respectively. There were noticeable changes in general neural (Fig. S8) and alpha band activity as a result of stimulation and these changes persisted during the Post-Stim condition. 11 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint D, G-J). Hand open and close were shown to participant A02 and tripod, index point, and wrist flexion movements283 were shown to participant A03. These classes were chosen by the participants because the classes were most closely284 associated with the tTENS phantom hand sensory mapping results. Furthermore, the classes and stimulation sites285 align with locations used by A03 in the long-term study (Fig. 5). Stimulation was applied to elicit activation of the286 phantom hand regions to correspond with the appropriate movement. There was higher activation in the central and287 centro-parietal regions during the Stim-Move condition compared to the Pre-Stim condition (Fig. 7C,I). In the Post-288 Stim condition, the effect of the stimulation persisted and changes in neural activity were observed in the central and289 the centro-parietal regions (Fig. 7D,J). We also compared the alpha power in individual central and centro-parietal290 electrodes across the conditions in both amputees (Fig. 7E-F,K-L). One-way ANOV A followed by post-hoc analysis291 was performed for each of the electrodes. In both participants, significant increases in the relative alpha power was292 observed for phantom hand movement during the Stim-Move and Post-Stim conditions compared to the Pre-Stim293 condition. Interestingly, the largest change in relative alpha power in the neural signal occurred in the ipsilateral294 hemisphere of A02, relative to the phantom hand. A03’s neural activity showed changes in both the contralateral295 and ipsilateral hemispheres.296 4. Discussion297 4.1. Sensory stimulation improves perception298 Our results show activation and enhanced perception of the phantom limb from tTENS (Fig. 2). Typically this299 technique is used to provide tactile sensations to the phantom hand [10, 12, 26]. Interestingly, the heightened sense300 of the phantom limb also seems to relate to changes in muscle activity during movements. The amputees felt as if301 the sensations were more or less natural, reported primarily as being a pressure or buzzing, and originating from302 their phantom hand. It is unclear if A01’s thermal sensation was a result of dominant thermal-specific afferents or a303 residual effect of the recent amputation. A01 described his phantom hand as a “foggy” and buzzing sensation as a304

of the recent amputation. In the survey, the amputees indicated stronger perception of the phantom hand as a305

of stimulation, which enabled a greater ability to move their phantom hand despite its absence (Fig. 3).306 Remarkably, over 2 years the stimulation sites and perceived activated regions in the phantom hand remained rela-307 tively stable for subject A03 (Fig. 5A-B, and Fig. S3). Despite an amputation over 7 years prior to the study, the308 sensory nerves in the residual limb still provided meaningful sensations of touch back to the user indicating corti-309 cal representation of the phantom hand as well as intact neural pathways. Although there are slight differences in310 activated sensory maps each day, the activated phantom regions themselves did not migrate and retained structural311 similarity over time (Fig. 5B), suggesting no major changes in the area of perceived activation. The fact that the312 sensory maps did not significantly change suggests that phantom limb representation remains many years after injury313 even without constant sensory stimulation.314 4.2. Phantom limb perception improves movement decoding315 Our results suggest that the internal sensorimotor pathway is affected by stimulation and enhanced phantom limb316 perception (Fig. 4). A crucial aspect of controlling the phantom hand, and in turn a prosthesis, is the internal317 perception of the phantom limb. Sensory feedback can be used to convey tactile information back to amputees318 [6–8, 12, 14–16]; however, we show that phantom hand perception is fundamentally linked to motor performance319 12 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint even in the absence of object manipulation.320 Our results suggest that sensory stimulation influences real-time myoelectric pattern recognition and that enhanced321 phantom perception temporarily improves movement decoding, regardless of experience level (Fig. 5 and 6). By322 working closely with patients A02 and A03, we identified the most relevant regions of the phantom hand to enhance323 perception during certain movements. A03 believed it would be difficult to achieve reliable control of more than 9324 movement classes as a non-TMR transhumeral prosthesis user. To see how much improvement was possible due to325 strengthening the internal sensorimotor control loop of the amputee, we expanded the number of classes to 14 (Fig.326 5C-F).327 Participant A03 had previous experience with myoelectric pattern recognition and did not show significant improve-328 ment as a result of additional training over Phase I of the long-term study (Fig. 5G); however, there were significant329 improvements during the sessions with sensory stimulation to the phantom hand (Phase II, Fig. 5H). These results in-330 dicate that the heightened sense of the phantom hand immediately strengthens the sensorimotor loop of the amputee,331 but this improvement does not extend across days if the stimulation does not persist. Periodic sensory reinforcement332 provides short-term benefit, but the ability to perform the movements was similar during both Phases I and III, with333 a slight upward trend throughout the study, indicating no long-term impact and likely a result of continued prosthesis334 use by A03 (Fig. 5G). Long-term improvements may be realized through continued sensory reinforcement over a335 longer period.336 The link between phantom perception and control is evident from the results, and it is supported by prior work that337 shows motor cortex excitability can increase with sensory activation [35]. According to participant A03, enhanced338 sensation of the phantom hand from tTENS can take up to several hours to subside after which the phantom hand per-339 ception returns to the baseline state. The temporary influence of tTENS aligns well with our observed improvements340 in motor control being limited to within a single day. The subsiding effect of the sensory stimulation and movement341 decoding improvements limited to a single day further supports the idea that recurring sensory stimulation sessions342 could help create more permanent improvement. It is possible that a more targeted prosthesis training paradigm,343 making use of combined phantom motor control and sensory stimulation, would lead to long-term improvements344 in prosthesis performance. It should be noted that we did not compare non-phantom hand sensory stimulation345 conditions to investigate if the improved movement decoding effect is also present during other types of sensory346 activation to the body. Regardless, the results suggest a relationship between phantom perception and movement347 decoding, which may prove valuable for improving prosthesis control. We also showed a slight improvement in348 task completion time after sensory activation of the phantom hand during prosthesis control in an object movement349 task (Fig. 6E), which further supports the idea of phantom hand perception playing an important role in functional350 prosthesis control.351 4.3. Sensory stimulation activates sensorimotor regions352 Our neurological studies based on EEG activation present evidence for sensory-motor integration in amputees. The353 central and centro-parietal electrodes cover the primary motor and somatosensory cortices, which are areas known to354 be activated during sensory processing and motor-related tasks [36, 37]. In stroke patients, median nerve activation355 using TENS is also known to increase motor cortex excitability and motor function [35]. Based on our results, we356 13 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint inferred that tTENS does not just act as a tactile stimulus but also improves the perceived control of the phantom357 hand by the amputee. This inference is supported by the improvements in movement decoding (Fig. 4, 5, 6) and358 aligns with previous results suggesting the role of central and parietal EEG activation in phantom limb vividness [38].359 Previous studies have also demonstrated central and somatosensory cortical region EEG activation [10, 12, 39] and360 enhanced connectivity [40] from noninvasive sensory feedback; however, here we also observed that the sensory361 stimulation effect persisted during the Post-Stim condition. It should be noted that the above observations were362 made for all the stimulation sites (median, ulnar and radial). We did not observe significant differences between363 them owing to the lower spatial resolution of EEG. Nevertheless, EEG-based classification of A03’s stimulation364 sites was possible with relatively high accuracy (Fig. S6).365 We believe that during the Post-Stim condition the activation of the primary somatosensory cortex showed tactile366 working memory aiding the amputee in better perception of the phantom hand movements even without the feedback367 stimulus. Prior work showed that the primary sensory cortex (both contralateral and ipsilateral) acts as a center for368 online sensory processing as well as a transient storage site for tactile information [41,42]. It’s possible that sensory369 stimulation to the phantom hand aids the amputees to make movement because the tactile working memory plays a370 valuable role in aiding the movement and perception both during and after the sensory stimulation.371 Interestingly, the most significant neural changes in A02 occurred in the ipsilateral hemisphere during tTENS (Fig.372 7B-D). While the traditional notion is that sensorimotor activity is in the contralateral hemisphere, previous work373 showed decoding motor commands from ipsilateral brain activity in the sensorimotor region [43]. The potential roles374 of ipsilateral activity suggested in the past include contributing to finger representation and voluntary execution of375 a movement [44] and maintaining an efference copy and muscle posture of the ipsilateral limb [45]. Another possi-376 bility of stronger ipsilateral activation could be to the absence of contralateral inhibition due to cortical adjustments377 after injury and amputation [46]. Because we know both hemispheres are used for tactile information storage, it’s378 likely that short-term tactile working memory was utilized [41, 42].379 Furthermore, the laterality differences observed in amputees A02 and A03 could be the result of time since ampu-380 tation, individual variability in amputation, and experiences in tTENS and myoelectric control. Participant A02 had381 15 years between paralysis from nerve injury and elective amputation (Table S1). Though the extent of peripheral382 sensory input loss effects in cortical behavior is not completely understood, cortical functional organizational dif-383 ferences could also contribute to the differences observed in A02 and A03. Participant A03 received an extended384 period of sensory mapping (Fig. 5A,B). Studying the long-term cortical effects of sensory stimulation can offer385 more insights to cortical behavior when sensory information is reintroduced to amputees. In targeted muscle and386 sensory reinnervation (TMSR) recipients, the strength in activation of primary motor and somatosensory regions387 shows similarity with that in intact limb controls [47].388 Despite disrupted sensorimotor pathways after limb amputation, there is bilateral activation during electrical sensory389 stimulation [39] and phantom movements [48]. Such cortical plasticity mechanisms are not completely understood,390 but deeper insight could be obtained from studying the causal interactions between the two hemispheres focusing on391 the sensorimotor loop. Our observations on enhanced phantom perception influencing control and neural activation392 support the idea that phantom hand representation in the cortex persists after amputation [22, 24]. More research393 14 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint is needed to develop methods sustain movement decoding improvement beyond a single day; however, our EEG394

offer insight on the role of sensory feedback in phantom hand perception and control.395 5. Conclusion396 We show that improving perception of the phantom hand in amputees can improve the ability to produce and control397 phantom hand movements. This improvement in phantom hand control can be captured and decoded through surface398 EMG. Sensory stimulation of the phantom hand appears to provide short-term (within a single day) improvements399 in movement decoding. Nerve stimulation can provide tactile feedback for amputees, but here we show that sensory400 stimulation of the phantom hand is also fundamentally linked to phantom hand control. Through EEG signals, we401 confirm that the sensory activation of the phantom hand influences relevant neural motor activity. Interestingly, the402 enhanced neural motor activity persists even after sensory stimulation is removed, which helps explain the movement403 decoding improvements after short phantom hand sensory stimulation sessions. When tracked over 2 years, we saw404 that the sensory regions of the phantom hand did not change, which demonstrates long-term stability in amputee405 sensory maps even many years after amputation. Movement decoding performance over 1 year did not substantially406 change; however, performance was affected on the same day as targeted phantom hand sensory stimulation. Our407 findings offer insight on how phantom hand perception can be modulate through sensory activation for improving408 motor control, prosthesis function, and rehabilitation after amputation.409 Acknowledgments410 This work was supported by Space@Hopkins, the National Institutes of Health (T32EB003383), the National Sci-411 ence Foundation (1849417), and the JHU/APL postdoctoral fellowship. The VIE was developed at JHU/APL under412 the Revolutionizing Prosthetics program (Defense Advanced Research Projects Agency, N66001-10-C-4056).413 Author contributions414 L.E.O., K.D., M.A.H., G.M.L., and C.L.H. developed hardware and software for the experiments. L.E.O., M.A.H.,415 K.D., and M.M.I. conducted experiments. L.E.O. and N.V .T. designed the experiments. L.E.O., M.A.H., R.B.,416 K.D., M.M.I., A.D., Z.T., A.B., and N.V .T. analyzed the data. N.V .T. supervised all experiments, data analysis, and417 interpretation of the results. All authors contributed to writing the paper.418 Competing interests419 N.V .T. is co-founder of Infinite Biomedical Technologies. This relationship has been disclosed and is managed420 by Johns Hopkins University. G.M.L. is an employee of Infinite Biomedical Technologies and was one of the ex-421 periment volunteers. He authorized the release of his name as a research volunteer through the JHMI IRB Health422 Insurance Portability and Accountability Act (HIPAA) privacy release form. He did not handle data, perform analy-423 sis, or interpret results from any experiment. All other authors declare no competing interests.424 ORCID iD425 Luke E. Osborn: 0000-0003-2985-6294426 Keqin Ding: 0000-0002-0680-673X427 Rohit Bose: 0000-0003-3966-4464428 15 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint Mark M. Iskarous: 0000-0003-4208-3943429 Andrei Dragomir: 0000-0003-2815-6958430 Zied Tayeb: 0000-0003-3257-0211431 Gorden Cheng: 0000-0003-0770-8717432 Robert S. Armiger: 0000-0002-3437-1884433 Anastasios Bezerianos: 0000-0002-8199-6000434 Matthew S. Fifer: 0000-0003-3425-7552435 Nitish V . Thakor: 0000-0002-9981-9395436 References437 [1] Johansson R S and Flanagan J R 2009 Coding and use of tactile signals from the fingertips in object manipula-438 tion tasks Nature Reviews Neuroscience 10 345–359439 [2] Wolpert D, Ghahramani Z and Jordan M 1995 An internal model for sensorimotor integration Science 269440 1880–1882441 [3] Shadmehr R, Smith M A and Krakauer J W 2010 Error correction, sensory prediction, and adaptation in motor442 control Annual Review of Neuroscience 33 89–108443 [4] Shehata A W, Engels L F, Controzzi M, Cipriani C, Scheme E J and Sensinger J W 2018 Improving internal444 model strength and performance of prosthetic hands using augmented feedback Journal of NeuroEngineering445 and Rehabilitation 15 70446 [5] Ramachandran V S and Hirstein W 1998 The perception of phantom limbs Brain 121 1603–1630447 [6] Raspopovic S et al 2014 Restoring natural sensory feedback in real-time bidirectional hand prostheses Science448 Translational Medicine 6 222ra19449 [7] Tan D W, Schiefer M A, Keith M W, Anderson J R, Tyler J and Tyler D J 2014 A neural interface provides450 long-term stable natural touch perception Science Translational Medicine 6 257ra138451 [8] Oddo C M et al 2016 Intraneural stimulation elicits discrimination of textural features by artificial fingertip in452 intact and amputee humans eLife 5 e09148453 [9] Wendelken S et al 2017 Restoration of motor control and proprioceptive and cutaneous sensation in humans454 with prior upper-limb amputation via multiple utah slanted electrode arrays (useas) implanted in residual pe-455 ripheral arm nerves Journal of Neuroengineering and Rehabilitation 14 121456 [10] D’Anna E et al 2017 A somatotopic bidirectional hand prosthesis with transcutaneous electrical nerve stimu-457 lation based sensory feedback Scientific Reports 7 10930458 [11] Shin H, Watkins Z, Huang H H, Zhu Y and Hu X 2018 Evoked haptic sensations in the hand via non-invasive459 proximal nerve stimulation Journal of Neural Engineering 15 046005460 [12] Osborn L E et al 2018 Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain Science461 16 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint Robotics 3 eaat3818462 [13] Page D M et al 2018 Motor control and sensory feedback enhance prosthesis embodiment and reduce phantom463 pain after long-term hand amputation Frontiers in Human Neuroscience12 352464 [14] Graczyk E L, Resnik L, Schiefer M A, Schmitt M S and Tyler D J 2018 Home use of a neural-connected sensory465 prosthesis provides the functional and psychosocial experience of having a hand againScientific Reports 8 9866466 [15] Valle G et al 2018 Biomimetic intraneural sensory feedback enhances sensation naturalness, tactile sensitivity,467 and manual dexterity in a bidirectional prosthesis Neuron 100 37 – 45.e7468 [16] George J A et al 2019 Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous469 use of a bionic hand Science Robotics 4470 [17] Marasco P D et al 2018 Illusory movement perception improves motor control for prosthetic hands Science471 Translational Medicine 10 eaao6990472 [18] Scheme E and Englehart K 2011 Electromyogram pattern recognition for control of powered upper-limb pros-473 theses: State of the art and challenges for clinical use Journal of Rehabilitation Research and Development 48474 643–659475 [19] Farina D et al 2017 Man/machine interface based on the discharge timings of spinal motor neurons after476 targeted muscle reinnervation Nature Biomedical Engineering 1 0025477 [20] Hahne J M, Schweisfurth M A, Koppe M and Farina D 2018 Simultaneous control of multiple functions of478 bionic hand prostheses: Performance and robustness in end users Science Robotics 3 eaat3630479 [21] Chen A, Yao J, Kuiken T A and Dewald J P A 2013 Cortical motor activity and reorganization following480 upper-limb amputation and subsequent targeted reinnervation NeuroImage: Clinical 3 498 – 506481 [22] Kikkert S et al 2016 Revealing the neural fingerprints of a missing handeLife 5 e15292482 [23] Mercier C, Reilly K T, Vargas C D, Aballea A and Sirigu A 2006 Mapping phantom movement representations483 in the motor cortex of amputees Brain 129 2202–2210484 [24] Reilly K T and Sirigu A 2008 The motor cortex and its role in phantom limb phenomena The Neuroscientist485 14 195–202486 [25] Andoh J, Diers M, Milde C, Frobel C, Kleinb ¨ohl D and Flor H 2017 Neural correlates of evoked phantom limb487 sensations Biological Psychology 126 89 – 97488 [26] Osborn L et al 2017 Targeted transcutaneous electrical nerve stimulation for phantom limb sensory feedback489 IEEE Biomedical Circuits and Systems (BioCAS) pp 1–4490 [27] Johannes M S et al 2020 The modular prosthetic limb Wearable Robotics ed Rosen J and Ferguson P W491 (Academic Press) chap 21, pp 393 – 444 1st ed492 [28] G ´omez-Herrero G 2007 Automatic artifact removal (aar) toolbox v1. 3 (release 09.12. 2007) for matlab URL493 17 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint http://germangh.github.io/aar/aardoc/aar.html494 [29] Delorme A and Makeig S 2004 Eeglab: an open source toolbox for analysis of single-trial eeg dynamics495 including independent component analysis Journal of Neuroscience Methods 134 9 – 21 ISSN 0165-0270496 [30] Schiefer M, Tan D, Sidek S M and Tyler D J 2016 Sensory feedback by peripheral nerve stimulation im-497 proves task performance in individuals with upper limb loss using a myoelectric prosthesis Journal of Neural498 Engineering 13 016001499 [31] Wang Z, Bovik A C, Sheikh H R and Simoncelli E P 2004 Image quality assessment: from error visibility to500 structural similarity IEEE Transactions on Image Processing 13 600–612501 [32] J ¨onsson S, Caine-Winterberger K and Br˚anemark R 2011 Osseointegration amputation prostheses on the upper502 limbs: methods, prosthetics and rehabilitation Prosthetics and Orthotics International 35 190–200503 [33] Neuper C, Scherer R, Reiner M and Pfurtscheller G 2005 Imagery of motor actions: Differential effects of504 kinesthetic and visual–motor mode of imagery in single-trial eeg Cognitive Brain Research 25 668 – 677505 [34] Neuper C, Scherer R, Wriessnegger S and Pfurtscheller G 2009 Motor imagery and action observation: Modu-506 lation of sensorimotor brain rhythms during mental control of a brain–computer interface Clinical Neurophys-507 iology 120 239 – 247 ISSN 1388-2457508 [35] Conforto A B et al 2010 Effects of somatosensory stimulation on motor function after subacute stroke Neu-509 rorehabilitation and Neural Repair 24 263–272510 [36] Porro C A et al 1996 Primary motor and sensory cortex activation during motor performance and motor im-511 agery: A functional magnetic resonance imaging study Journal of Neuroscience 16 7688–7698512 [37] Pfurtscheller G and Neuper C 1997 Motor imagery activates primary sensorimotor area in humans Neuro-513 science Letters 239 65 – 68514 [38] Lyu Y , Guo X, Bekrater-Bodmann R, Flor H and Tong S 2016 Phantom limb perception interferes with motor515 imagery after unilateral upper-limb amputation Scientific reports 6 21100516 [39] Yao J, Chen A, Kuiken T, Carmona C and Dewald J 2015 Sensory cortical re-mapping following upper-limb517 amputation and subsequent targeted reinnervation: A case report NeuroImage: Clinical 8 329 – 336 ISSN518 2213-1582519 [40] Ding K et al 2020 Towards machine to brain interfaces: Sensory stimulation enhances sensorimotor dynamic520 functional connectivity in upper limb amputees Journal of Neural Engineering521 [41] Harris J A, Miniussi C, Harris I M and Diamond M E 2002 Transient storage of a tactile memory trace in522 primary somatosensory cortex Journal of Neuroscience 22 8720–8725523 [42] Zhao D, Zhou Y D, Bodner M and Ku Y 2018 The causal role of the prefrontal cortex and somatosensory524 cortex in tactile working memory Cerebral Cortex 28 3468–3477525 18 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint [43] Hotson G et al 2014 Coarse electrocorticographic decoding of ipsilateral reach in patients with brain lesions526 PloS One 9 e115236527 [44] Berlot E, Prichard G, O’Reilly J, Ejaz N and Diedrichsen J 2019 Ipsilateral finger representations in the senso-528 rimotor cortex are driven by active movement processes, not passive sensory inputJournal of Neurophysiology529 121 418–426530 [45] Bundy D T and Leuthardt E C 2019 The cortical physiology of ipsilateral limb movements Trends in Neuro-531 sciences 42 825 – 839532 [46] Hordacre B, Bradnam L V , Barr C, Patritti B L and Crotty M 2015 Intracortical inhibition is modulated by533 phase of prosthetic rehabilitation in transtibial amputees Frontiers in Human Neuroscience9 276534 [47] Serino A et al 2017 Upper limb cortical maps in amputees with targeted muscle and sensory reinnervation535 Brain 140 2993–3011536 [48] Lundborg G, Waites A, Bj ¨orkman A, Ros´en B and Larsson E M 2006 Functional magnetic resonance imaging537 shows cortical activation on sensory stimulation of an osseointegrated prosthetic thumb.Scandinavian Journal538 of Plastic Surgery and Hand Surgery 40 234–239539 [49] Kuiken T A, Marasco P D, Lock B A, Harden R N and Dewald J P A 2007 Redirection of cutaneous sensation540 from the hand to the chest skin of human amputees with targeted reinnervation Proceedings of the National541 Academy of Sciences USA 104 20061–20066542 [50] Hebert J S et al 2014 Novel targeted sensory reinnervation technique to restore functional hand sensation after543 transhumeral amputation IEEE Transactions on Neural Systems and Rehabilitation Engineering 22 765–773544 [51] Rognini G et al 2018 Multisensory bionic limb to achieve prosthesis embodiment and reduce distorted phantom545 limb perceptions Journal of Neurology, Neurosurgery & Psychiatry0 1–3546 [52] Englehart K and Hudgins B 2003 A robust, real-time control scheme for multifunction myoelectric control547 IEEE Transactions on Biomedical Engineering 50 848–854548 [53] Betthauser J et al 2018 Limb position tolerant pattern recognition for myoelectric prosthesis control with549 adaptive sparse representations from extreme learning IEEE Transactions on Biomedical Engineering 65 770–550 778551 [54] Grosse-Wentrup M and Buss M 2008 Multiclass common spatial patterns and information theoretic feature552 extraction IEEE Transactions on Biomedical Engineering 55 1991–2000553 [55] Pudil P, Novovi ˇcov´a J and Kittler J 1994 Floating search methods in feature selection Pattern Recognition554 Letters 15 1119 – 1125 ISSN 0167-8655555 [56] Tayeb Z et al 2018 Gumpy: a python toolbox suitable for hybrid brain–computer interfaces Journal of Neural556 Engineering 15 065003557 19 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprintthis version posted May 25, 2020. ; https://doi.org/10.1101/2020.05.22.20109330doi: medRxiv preprint