Development of an Experimental Setup to Investigate Mirror Visual Feedback Effects on Cross-Education for an Intricate Finger Sequence Movement Task

Development of an Experimental Setup to Investigate Mirror Visual Feedback Effects on Cross-Education for an Intricate Finger Sequence Movement Task | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Development of an Experimental Setup to Investigate Mirror Visual Feedback Effects on Cross-Education for an Intricate Finger Sequence Movement Task Anurag Gupta, Varadhan SKM This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6957809/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 09 Apr, 2026 Read the published version in Scientific Reports → Version 1 posted 13 You are reading this latest preprint version Abstract Cross-education, a phenomenon where training one limb improves performance in the untrained limb, can be enhanced through mirror visual feedback (MVF), underpinning its potential for rehabilitating hemiparetic patients. While MVF-mediated enhancement is documented for simple motor tasks, its effectiveness in complex, fine finger movements remains underexplored. To address this gap, we developed a novel experimental setup to investigate MVF effects on cross-education for a unique typing task involving fine finger movements. The setup comprises three tightly synchronized systems: A novel typing device to implement the typing task, an IMU-based system for accurate real-time hand movement tracking, and a virtual reality (VR) system that provides normal or mirrored real-time replication of hand movements by virtual hands. Additionally, several methodologies and algorithms critical to the experimental task were developed: (1) Constructing participant-specific virtual hand models to accurately replicate thumb-to-phalanx touches. (2) An intuitive approach to manipulating quaternions for coordinate transformation and mirror animation. (3) Millisecond-level synchronization of movement and typing data. (4) Preventing false key press/release detections by filtering noise specific to the typing device. (5) Correcting reference frame misalignment between IMU sensors and their respective hand segments. Some of these methodologies contribute valuable tools to the hand biomechanics and VR research communities. Technical validation of the setup demonstrated robust real-time performance, millisecond-level data synchronization, and precise hand animation, confirming the system’s readiness for investigating MVF-based cross-education. Biological sciences/Neuroscience/Motor control Physical sciences/Engineering/Biomedical engineering intermanual transfer mirror visual feedback hand kinematics inertial measurement unit virtual reality hand animation Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 1. Introduction Cross-education (also known as intermanual transfer, interlimb transfer, bilateral transfer, etc.) is a phenomenon in which learning a task with one hand results in an improvement in the performance of the same or similar task in not only the trained hand but also in the untrained hand 1 – 3 . If cross-education can be enhanced, then it has the potential to benefit the rehabilitation of hemiparetic patients whose one side of the body is partially paralyzed. A few studies have explored whether cross-education can be enhanced by practicing a task with one hand while simultaneously observing the contralateral (i.e., opposite) hand performing the same task 4 – 9 . This can be achieved using either a mirror 10 or a Virtual Reality (VR)-based experimental setup 4 , 5 . For example, if a mirror is placed along the midsagittal plane of a person, with one hand placed on each side of the mirror, then performing a task with one hand while looking at its mirror reflection gives the illusion that the stationary contralateral (opposite) hand is performing the task. This type of visual feedback is known as Mirror Visual Feedback (MVF) 10 , 11 . VR setups have been developed to provide a more realistic experience of the movement illusion created by MVF, in which a virtual hand is animated to mirror the movements of the contralateral real hand 4 , 5 . Since the first documented evidence of cross-education over a century ago 1 , 2 , its characteristics and underlying neural mechanisms have been extensively investigated across various experimental paradigms 12 – 24 . Within the cross-education literature, several studies have investigated how the simultaneous observation of MVF while practicing a task affects cross-education 4 – 9 . For instance, Mukamel’s group developed one of the earliest VR-based setups to provide MVF during a motor sequence learning task in which participants flexed and extended individual fingers one at a time in a predefined sequence (e.g., index, ring, middle, little) 4 , 5 . In their studies using this setup, they provided evidence for the enhancement of cross-education due to MVF. Other studies have investigated the effect of MVF on cross-education for tasks such as a ball rotation task 6 , a ballistic wrist flexion task 7 , a shortening muscle contraction task 8 , and a stationary basketball dribbling task 9 , among others. While these studies have significantly advanced our understanding of MVF's effects on cross-education, many of the motor tasks explored have been relatively simple – typically involving gross movements, single-joint rotations, or limited fine motor coordination and multi-finger control. In the case of Mukamel's studies 4 , 5 , although the task employed was more complex as it incorporated finger individuation and sequence execution, the movements were still limited to single-joint flexion-extension actions. This leads us to some important unanswered questions: Could MVF also enhance cross-education for more complex tasks requiring fine motor control, such as those involving multi-finger coordination and spatially targeted finger movements executed in specific sequences? If yes, would such enhancement be bidirectional – i.e., from the right hand to the left hand and vice versa? Additionally, what would be the nature of such enhancement, i.e., would only certain performance metrics improve? Or would improvements of certain performance metrics be limited to specific directions of transfer? To address these questions, we built a novel experimental setup to investigate the effects of MVF on cross-education in a fine motor task. This task involves making sequences of intricate finger movements that require multi-finger coordination. Our group previously used a close variant of this task to study long-term motor sequence learning 25 – 27 . More specifically, the task requires participants to touch various finger phalanges one at a time with the tip of the thumb in specific sequences. To implement this task and investigate the potential MVF-mediated enhancement of cross-education for this task, we developed a novel VR-based experimental setup comprising three key components: (1) A Text Input System (TIS) – a custom-built typing device that consists of keys placed on the palmar surface of the finger phalanges to execute the finger sequence movement task; (2) A Hand Kinematics Measurement System (HKMS) – a real-time hand and finger movement tracking system based on BNO055 IMUs 28 ; and (3) A Virtual Reality (VR) headset to provide either normal or mirror feedback of the virtual hands and to present the task interface. For the task, participants are instructed to type words appearing in the VR world by touching the appropriate TIS keys with the tip of their thumb. This paper details the design and development of the novel VR-based experimental setup. Key aspects covered include: describing the unique typing task; developing the TIS; designing and refining a specialized stable period algorithm developed to accurately detect key press and release timestamps while eliminating false detections caused by mechanical and electrical fluctuations; synchronizing and integrating the TIS with the HKMS; synchronizing and integrating the VR system with both the HKMS and TIS; animating virtual hand models in real time using HKMS data; dynamically updating the word interface of the typing task using TIS data; and methods for precisely replicating thumb-to-phalanx touches in virtual hands. This replication is achieved by creating participant-specific virtual hand models and by implementing sensor-to-segment alignment to obtain accurate hand segment orientations. Additionally, the paper explains the quaternion transformations necessary for achieving normal and mirror virtual hand animation within the Unity software environment. Overall, this device and methods paper describes the design, development, and validation of a novel VR-based experimental setup for investigating the effects of mirror visual feedback on the cross-education of a fine finger movement task. This paper establishes a robust foundation for future studies using the same experimental paradigm. In addition to detailing the complete system architecture of the experimental setup, the paper details several methodologies and concepts that will benefit the broader hand biomechanics and VR research community. These include: methodology for correcting misalignment between sensor and segment reference frames; methodology for achieving accurate hand animation; techniques for system integration and millisecond-level data synchronization; and an intuitive approach for quaternion coordinate transformation. To validate the experimental setup's performance, a preliminary experiment was conducted with two participants. This pilot study demonstrated that the integrated setup functioned reliably and as intended, confirming its suitability for future MVF-based cross-education experiments. 2. Methodology Part 1: Experimental Setup Design and Development This section provides a detailed account of the experimental setup’s design and development. It includes the design and functioning of each individual component of the setup, the design and methods used to integrate and synchronize them, and the development and refinement of a critical algorithm associated with one of the components. A detailed description of the experimental task is also included. 2.1 Experimental Setup Overview A novel Virtual Reality (VR)-based experimental setup was developed to investigate the effect of mirror visual feedback on the enhancement of cross-education from the trained to the untrained hand for an intricate finger sequence movement task. This setup consists of three main components: Text Input System (TIS): A custom typing device used to implement the finger sequence movement task. This is mounted on the palmar surface (front side) of the hand. Hand Kinematics Measurement System (HKMS): A hand motion-sensing device used to track the motion of the hand and fingers in real time 28 . This is mounted on the dorsal surface of the hand. Virtual Reality (VR) headset: This presents a virtual environment containing a pair of virtual hands along with visual elements associated with the experimental task. Data from the HKMS is used to animate the virtual hands, either by replicating the real hand’s movements or by providing mirror visual feedback – wherein the movements of the real hand are mirrored by the contralateral (opposite) virtual hand. Additionally, data from the TIS is used to update a word interface presented as part of the task. The VR headset is worn over the participant's eyes to deliver the virtual environment. These three systems are fully integrated and operate in tight synchrony. Additionally, the hand movement data from the HKMS and the typing data from the TIS are timestamp-synchronized – an essential requirement for analyzing movement patterns in relation to typing activity. 2.2 The Task 2.2.1 Overview The intricate finger sequence movement task involves performing sequences of finger-thumb opposition movements. Specifically, the participant is required to touch the palmar surface of different finger phalanges with the tip of the thumb (one at a time) in prescribed sequences. To implement this, a key is mounted on the palmar surface of each finger phalanx, and the participant is instructed to type different words by pressing these phalanx-mounted keys with the tip of the thumb. 2.2.2 Detailed Task Description Upon wearing the VR headset, the participant is presented with a virtual environment containing a pair of virtual hands and other visual elements associated with the experimental task. The virtual hands accurately replicate the participant’s real hand movements in real-time. Characters are superimposed onto the palmar surfaces of the virtual finger phalanges, resembling tattoos (see Fig. 1 ). These characters represent the key-character mapping of the physical keys positioned on the participant’s real finger phalanges. To type a particular character, the participant must press the corresponding key with the tip of the thumb. In the virtual environment, this corresponds to touching the surface of the virtual phalanx containing the character to be typed with the tip of the virtual thumb (see Fig. 1 a). During each trial, 3D words are presented to the participant in the VR environment (see Fig. 1 ). The participant is instructed to type the presented words using a specified hand as quickly as possible and is informed of the hand to be used for typing before each trial. The 3D word to be typed is displayed just above the virtual hand performing the typing movements (see Fig. 1 ). If a letter is typed correctly, it turns green, signalling the participant to proceed to the next letter. If typed incorrectly, it turns red, signalling the participant to re-attempt typing the letter (see Fig. 1 ). The next letter cannot be typed until the current one is typed correctly. Once all letters of a word are typed correctly, the space character – labelled ‘SP’ on the virtual hand – must be typed to trigger the presentation of the next word. The accurate real-time replication of the participant’s hand movements by the virtual hands and real-time updating of the word interface of the typing task is achieved using data from the HKMS and TIS systems. Detailed description and working of these systems are provided in a subsequent section. 2.2.3 Scoring Scheme A simple scoring scheme was implemented to provide performance feedback to participants and encourage improvement across trials. In this scheme, participants are awarded two points for each correctly typed character, including the "space" character. No points are deducted for incorrectly typed characters, ensuring that the primary focus of the task remains on typing speed rather than accuracy. However, depending on the specific research question being investigated, both the task instructions and the associated scoring scheme can be modified accordingly. Real-time performance feedback is delivered by updating the participant’s score on a screen located just above the 3D word (see Fig. 1 ). This screen also displays a countdown timer indicating the trial time remaining. 2.2.4 Mirror Feedback There are two animation modes for the virtual hands: 1. Normal animation mode: The virtual hands accurately replicate the movements of the real hands. 2. Mirror animation mode: The movements of the typing hand are mirror-replicated by the opposite (i.e., contralateral) virtual hand. For e.g., if the typing hand is the right hand, then the movements of the right real hand are mirror-replicated by the left virtual hand. Mirror-replicates refers to the hand movements one would observe in the reflection of a mirror placed along the midsagittal plane, with each hand positioned on either side of the mirror. For instance, if the participant rotates the right hand clockwise, the left virtual hand rotates counterclockwise. Similarly, if the right hand rotates towards the body's midline, the left virtual hand also rotates towards the midline. Supplementary Video 1 shows a participant performing the typing task, first with the virtual hands in normal animation mode and then in mirror animation mode. In mirror feedback trials with the right hand as the typing hand, the participant has to perform the typing task with the right hand while looking at the left virtual hand for visual feedback . To prevent the participant from looking at the right virtual hand while typing, its fingers are frozen in a predefined posture. However, even in this fixed posture, the overall orientation of the right virtual hand continuously replicates the real hand’s orientation (see Supplementary Video 1), as measured by the dorsal hand sensor placed over the metacarpal region – details of this sensor are provided in a subsequent Section 2.7 : Hand Kinematics Measurement System (HKMS) . Additionally, since the participant is looking at the left virtual hand while typing, the 3D word to be typed and the screen displaying the trial score and trial time remaining are positioned above the left virtual hand for easy visual access. 2.3 Text Input System (TIS) Use The text input system (TIS) is a novel typing device that is used to implement the intricate finger sequence movement task. It consists of 10 keys placed on the palmar surface of the finger phalanges (see Fig. 1 for the key-phalanx mapping). To type a character, the participant must touch the corresponding key with the tip of the thumb’s distal phalanx by performing a finger-thumb opposition movement. Description The keys of the TIS are actually copper patches affixed to the palmar surface of the finger phalanges (see Fig. 2 ). Each copper patch is connected to a digital input pin of a microcontroller (Teensy 4.0) via a thin, lightweight wire (28 AWG, Teflon-coated, multistrand). Each digital input pin is internally connected to a pull-up resistor, maintaining the copper patch at 3.3V (HIGH) unless externally pulled to 0V (LOW). A copper patch is also affixed to the tip of the thumb and is connected to the ground pin (0V) of a microcontroller via a lightweight wire, maintaining it at 0V. Additionally, one more copper patch is connected to the ground pin (0V), and its conductive surface is placed in direct contact with the participant’s skin. In contrast, the conductive surfaces of the other TIS copper patches (including the thumb patch) are insulated from the skin by an adhesive layer, such that only the outward-facing surface (not in contact with the skin) is electrically active – either at 3.3V (HIGH) or at 0V (LOW) depending on the connection. Working During a finger-thumb opposition movement, when the thumb’s copper patch makes contact with a phalanx copper patch, the phalanx copper patch voltage transitions from 3.3V to 0V. Upon breaking this contact, the voltage transitions from 0V back to 3.3V. Each digital input pin connected to a phalanx copper patch is mapped to a specific TIS character, and the microcontroller continuously monitors the voltages of these pins. A transition from 3.3V to 0V is detected as a key press, and the corresponding timestamp (in milliseconds (ms)) is recorded. A transition from 0V to 3.3V is detected as a key release, and the corresponding timestamp (in ms) is recorded. The key press and release timestamps are sent from the microcontroller to the HKMS as they are detected, along with the character associated with the digital input pin. The HKMS, in turn, sends this data to a computer. Preventing False Detections Due to Ambient AC Electric Fields During the typing task, the skin – particularly from a bent finger segment – may come into contact with the conductive surface of a phalanx copper patch, occasionally resulting in false key press and release detections. These false detections occur because the skin’s voltage oscillates sinusoidally relative to the microcontroller’s ground, due to capacitive coupling between the body and ambient 50 Hz electric fields generated by nearby AC power lines. To suppress this effect, the conductive surface of a separate copper patch – connected to the microcontroller's ground – is placed in direct contact with the participant’s skin, thereby clamping the body’s potential to the system’s reference voltage. Additionally, participants wear rubber slippers to electrically isolate their bodies from the floor, further reducing the capacitive coupling. 2.4 Stable Period Algorithm The TIS uses an algorithm to accurately detect key press and release instances while avoiding false detections. This algorithm is referred to as the stable period algorithm. In this sub-section, the algorithm and the reasoning behind its use are explained. 2.4.1 Background In general, when a key is pressed, the key’s contact surfaces do not form a stable electrical connection immediately. Due to mechanical vibration, they bounce off each other and reconnect several times before a stable physical – and consequently electrical – connection is established. This results in multiple voltage transitions between HIGH (3.3V) and LOW (0V) in the key's output signal before stabilizing at LOW. This unstable transition period is referred to as the bounce period (see Fig. 3 ). Similarly, a bounce period occurs during key release, during which the voltage transitions multiple times before settling at HIGH (see Fig. 3 ). Transitions during the bounce period result in multiple false key press and release detections. A simple method often used to avoid false detections is to ignore any voltage transitions on the microcontroller's input pin for a fixed duration following a key press and key release detection. This fixed duration is called the lockout period and needs to be greater than the bounce period. Because different types of mechanical keys have different bounce durations – and because key press and release bounce durations may differ even for the same type of key – bounce periods are typically identified using simple tests, and corresponding lockout durations are hardcoded. 2.4.2 Need for an Alternate Approach As previously described, the TIS keys are copper patches. However, like the traditional mechanical keys, the TIS keys also have bounce periods associated with key press and release events for similar underlying reasons. The lockout period approach described in the preceding section was implemented for the TIS keys. This approach reduced but did not eliminate false key press and release detections. Upon investigation, the two chief reasons for false detections were identified: (1) variability in bounce durations (across different keys and also across time for the same keys) and (2) random voltage transitions occurring during the press period. A detailed account of the investigations into the lockout period approach is provided in the Supplementary Material Sections 1 and 2 . The bounce variability and random voltage transitions are due to a combination of various factors. Firstly, sticking the thin copper patches on the finger phalanges results in unique uneven surfaces for each copper patch. Secondly, the portion and surface area of the thumb copper patch making contact with the phalanx copper patch vary across different phalanx copper patches. Thirdly, while typing, the nature of the physical interaction between the contacting surfaces of the copper patches is different every time. For example, sometimes physical sliding occurs between the copper patches, while at other times, there is no sliding at all. When sliding does occur, the amount of slide can vary. Collectively, these factors contribute to variability in both the mechanical properties of the contacting surfaces and the physical nature of contact between the surfaces. This results in fluctuations in bounce periods and interruptions in stable electrical contact. Interruption in stable electrical contact, in turn, leads to random voltage transitions during the press duration. In conclusion, using a fixed lockout period is not the best approach to remove false detections in the TIS. 2.4.3 Stable Period Algorithm Working While there are several ways to solve the false detection problem, we decided to take the programmatic approach. To tackle the problem arising due to variability in the bounce periods, an algorithm called the stable period algorithm was implemented. This algorithm filters out the bounce period transitions without using fixed lockout periods. In this new approach, the algorithm monitors for a stable period of 15 ms following a key press. This stable period is a time span during which there are no transitions in the voltage of the copper patch. Once this key press stable period is detected, the algorithm starts monitoring for key release. Upon detection of the key release, the algorithm monitors for another stable period that lasts for 50 ms. Once this key release stable period is detected, the algorithm starts monitoring for the next key press, and the cycle continues. The stable period durations of 15 ms for key press and 50 ms for key release were determined empirically based on testing. The working of the stable period algorithm is illustrated in Fig. 3 . 2.4.4 Interpretation of Key Release Instance The onset of the key release bounce period indicates the initiation of movement to remove contact between the copper patches and is interpreted as the key release instance by the stable period algorithm (see Fig. 3 ). Alternatively, the final voltage transition at the end of the bounce period, which indicates complete separation of the copper patches, can also be interpreted as the key release instance. The choice between these two interpretations depends on the nature of the analysis. For example, if in the current experimental setup, synchronized EEG data is also collected, then the onset of the bounce period (reflecting movement initiation) may serve as a more meaningful key release marker, as it better corresponds to the timing of motor intention and cortical activation. 2.5 Stable Period Algorithm Refined Initial testing of the stable period algorithm revealed occasional false key press and release detections, which in some cases also caused the algorithm to fall out of sync with the actual typing events. This loss of synchronization, in turn, led to errors in detecting subsequent key press and release events. Upon investigation, two primary causes were identified: (1) Persistent voltage transitions throughout the entire key press duration, and (2) Random voltage fluctuations during the key press duration. A detailed explanation of how these two issues affect the functioning of the stable period algorithm is provided in the Supplementary Material Section 3 . The current section describes the refinements made to the algorithm to address these two issues. Refinement 1 – Handling Persistent Voltage Transitions When a key press stable period of 15 ms is detected, the algorithm simultaneously checks the voltage level of the corresponding copper patch. If the voltage level is LOW (0V), the algorithm continues functioning as usual and begins monitoring for the key release. However, if the voltage level is HIGH (3.3V), it indicates that the detected 15 ms stable period is actually a portion of the key release stable period, and that the actual key press stable period did not occur due to persistent voltage transitions throughout the key press duration. Consequently, the key release was also not detected. In such cases, the algorithm resets and begins monitoring for a key release stable period of 50 ms. Specifically, it monitors for an additional 35 ms of stable signal (to complete 50 ms), unless an intervening voltage transition occurs – in which case, monitoring for the full 50 ms stable period restarts from the new transition point. In cases where the entire key press duration is noisy, the timestamp of the final voltage transition during the key release bounce period is treated as the actual key release instance. Refinement 2 – Handling Random Voltage Fluctuations In the original version of the algorithm, once a key release was detected, the algorithm would begin monitoring for a key release stable period of 50 ms. In the refined version, a check is incorporated during this monitoring phase: the voltage level of the copper patch is read 2 ms into each candidate stable period. If the voltage level is HIGH (3.3V), the algorithm continues monitoring for a stable period of 50 ms. If the voltage level is LOW (0V), it indicates that the key is still pressed and that random noise was falsely detected as a key release. In such cases, the algorithm resets and resumes monitoring for a genuine key release. Fig. 4 illustrates the working of the refined algorithm, with the orange text highlighting the implemented refinement. As depicted in the figure, this updated approach effectively filters out both (1) isolated voltage spikes and (2) noisy segments where the voltage temporarily remains HIGH for a short duration before returning to LOW. Note that while the refined algorithm monitors for a key release stable period of 50 ms, it may encounter multiple shorter stable periods of varying durations. The voltage check 2 ms into the stable period is performed for each of these stable periods, as illustrated in Fig. 4. 2.6 Data Output Format of the TIS The TIS outputs various timing-related data for each key press–release event. This data includes: key press and release timestamps, press duration, bounce durations for key press and release, timing data related to key press and release stable periods, among other relevant data. All this data is output at different time instances, as and when detected by the stable period algorithm. A detailed account of the TIS data and its output format is provided in the Supplementary Material Section 4 . 2.7 Hand Kinematics Measurement System (HKMS) Use The Hand Kinematics Measurement System (HKMS) is used to track the motion of both hands in real time. Specifically, the HKMS records the orientation data – in the form of quaternions – of the hand and all the finger phalanges at 100 Hz. A detailed description of the design and validation of the HKMS is provided in our previously published work 28 . Description This device consists of 16 IMU sensors (BNO055, Bosch Sensortec) and five microcontrollers (Teensy 4.0; see Fig. 2 c). The IMU sensors (1.3 cm x 1 cm) are affixed to the dorsal surface of all the finger phalanges and the dorsal hand (i.e., over the metacarpal region). These sensors output orientation data in the form of quaternions. The three IMU sensors on each finger are connected in series using lightweight FFC cables (see Fig. 2 c). An additional fourth IMU is connected in series with the three IMUs on the middle finger and is placed on the dorsal hand (over the metacarpal region). The five microcontrollers are connected in the master–slave configuration on a single PCB. The master microcontroller is connected via an FFC cable to the four IMUs in series, located on the dorsal hand and middle finger. Each of the four slave microcontrollers is connected via an FFC cable to the three IMUs in series, located on a single finger. Working The HKMS operates on a synchronized data collection mechanism. Every 10 ms, the master microcontroller sends a synchronization signal simultaneously to all four slave microcontrollers. Upon receiving this signal, the four slave microcontrollers simultaneously start collecting quaternion orientation data from the IMUs to which they are connected (it takes approximately 2 ms for a microcontroller to get quaternion data from a single IMU). Meanwhile, after sending the synchronization signal, the master microcontroller starts collecting quaternion orientation data from the four IMUs connected to it and then waits for the data from the slaves to arrive. Once the data from all the slave microcontrollers has been received, the master sends the collected quaternion data from all 16 IMU sensors to a computer via USB. This entire process is completed within 10 ms, allowing the master to send the next synchronization signal on schedule, thereby maintaining a continuous data output rate of 100 Hz. The set of data collected during a single cycle is referred to as a data frame . Additionally, the timestamp at which the master sends the synchronization signal is recorded, and the orientation data from all 16 IMUs in that cycle is associated with this single timestamp (referred to as the data frame timestamp ). HKMS Operating Modes The HKMS operates in three distinct modes Orientation mode : In this mode, the HKMS outputs quaternion orientation data from all 16 IMU sensors at 100 Hz. The output also contains the data frame timestamp and the frame count (i.e., the total number of data frames output). Calibration mode : In this mode, the HKMS outputs the calibration status of all 16 IMU sensors at 100 Hz. Details regarding the calibration of the BNO055 IMUs and the calibration of the HKMS as a whole can be found in our previously published work 28 . Idle mode : In this mode, the HKMS does not transmit any data. The operating mode of the HKMS can be configured by sending it specific character commands from the computer: “O” for Orientation mode, “C” for Calibration mode, and “X” for Idle mode. These commands allow users to configure the HKMS output according to experimental requirements. Additionally, the timestamp at which the HKMS receives the “O” command is recorded. This timestamp serves as the new reference time (i.e., zero time) for all subsequent HKMS timing calculations. 2.8 Integration of HKMS and TIS To synchronize data from the HKMS and TIS, both systems were integrated together on a single printed circuit board (PCB) referred to as the HKMS-TIS board (see Fig. 5 ). A communication protocol was implemented for data exchange between the two systems. The HKMS-TIS board outputs the combined data of the HKMS and TIS as a single data frame every 10 ms. Additionally, the board contains screw terminals that are connected to the digital input pins of the TIS microcontroller. The copper patch wires of the TIS are connected to these screw terminals (see Fig. 5 ). 2.8.1 Communication Protocol The HKMS and TIS communicate with each other serially (UART protocol). During each HKMS data collection cycle, the master microcontroller of the HKMS, after receiving data from four slave microcontrollers, sends a data request to the TIS microcontroller (by transmitting the “ $ ” character). The TIS microcontroller, which continuously collects and concatenates typing data into a string, responds to the “ $ ” data request by sending to the master whatever typing data has accumulated since the previous request. All the timing data of the TIS is referenced relative to the TIS microcontroller's internal timer. The HKMS master, upon receiving data from the TIS microcontroller, merges both the HKMS and TIS data into a single data frame. This data frame is then transmitted to a computer via USB. This entire data collection and transmission process is completed within 10 ms. The format of the data frame output by the HKMS-TIS board is as follows: $ ” “HKMS Data” “&” “TIS Data” “*” “# Here, “ $ ” and “#” indicate the start and end of the combined HKMS-TIS data frame. “&” and “*” indicate the start and end of the TIS data within the HKMS-TIS data frame. If the TIS has no data to share, then it just sends “&*” to the HKMS master. These delimiting characters are used to parse the data frame at the computer's end. 2.8.2 Timestamp Synchronization The timestamp assigned to the HKMS data frame (i.e., the time at which the master sends the synchronization signal to the slaves) serves as the timestamp for the combined HKMS-TIS data frame. During post-processing, this timestamp can be used to link the TIS data to the corresponding HKMS data with which it was transmitted. For example, to extract kinematic data between two consecutive key presses, the timestamps of the HKMS-TIS data frames can be used to identify the HKMS data frames corresponding to the two key press events. Once these frames are located, all intermediate HKMS data frames can be extracted for further analysis. This data-linking method ensures precise temporal alignment of hand kinematic data with typing events, thereby enabling analysis of movement patterns in relation to typing activity. 2.8.3 Synchronizing the TIS with the HKMS Operating Modes The TIS activity is synchronized with the HKMS’s operating modes to ensure coordinated operation of both systems. This synchronization is implemented in the following way: whenever the HKMS receives a command to switch operating modes (“O,” “C,” or “X”), it immediately transmits the same command to the TIS. Based on the command received, the TIS adjusts its activity as follows: If the “O” command is received, the TIS monitors the digital input pins for typing events. Additionally, the timestamp at which the “O” command is received is recorded, and all variables associated with each key of the TIS are reset to their default values. This recorded timestamp serves as the new reference time (i.e., time zero) for all subsequent timing calculations by the TIS. If the “C” or “X” command is received, then the TIS remains idle. This mechanism ensures that both the TIS and HKMS operate in tandem. 2.9 Virtual Reality (VR) system Use : A VR system is used to implement the virtual environment described in Section 2.2 : The Task . Description The participant wears a VR headset (Meta Quest 2) during the experiment trials (see Fig. 6 ). The headset is connected to a computer via a high-speed and lightweight USB-C cable. The computer is running a PC-VR application that generates and manages the virtual environment that is displayed through the headset. Working The PC-VR application was built using the Unity game engine (version 2020.3.31f1). It accepts and processes data from the HKMS-TIS boards of both hands in real-time. The HKMS data is used to animate a pair of virtual hands, while the typing data is used to update the word interface of the typing task. Additionally, the application controls the operating modes of the HKMS and also synchronizes the data from both HKMS-TIS boards. 2.10 Unity Based PC-VR Application A PC-VR application was developed using the Unity software. This application simultaneously receives real-time data from the HKMS-TIS boards of both hands, parses this data (i.e., converts it into a structured format that can be easily used, analyzed, and stored), and uses it in real-time to animate virtual hand models and update the word interface during the typing task. Handling incoming 100 Hz data in real-time from both HKMS-TIS boards presents technical challenges in Unity. A detailed explanation of how background threads were employed to achieve real-time data reception and handling within the VR application is provided in Supplementary Material Section 5 . This section describes the different operating modes of the VR application and how they are synchronized with the corresponding operating modes of the HKMS-TIS boards to ensure coordinated functioning among the three components of the experimental setup. 2.10.1 Modes of Operation Similar to the HKMS, the VR application operates in three distinct modes: (1) Orientation Mode, (2) Calibration Mode, and (3) Idle Mode. Depending on the experimental requirements, the application can be set to operate in one of these modes. The following is a description of these modes: 1. Orientation Mode : In this mode, the application receives both orientation and typing data from the HKMS-TIS boards of both hands. If a trial is in progress, then the typing data from the hand designated for typing during that trial is used to update the word interface of the typing task in real-time. Concurrently, the orientation data from both hands is used to animate the virtual hand models in real-time. Depending on the experimental requirements, the virtual hand animation can be implemented in one of two ways: Normal Animation: Both virtual hands accurately replicate the movements of their corresponding real hands (see Supplementary Video 1). Mirror Animation: One virtual hand mirror-replicates the movements of the opposite real hand while the other virtual hand’s fingers are fixed in a pre-defined posture (for more details regarding mirror animation, refer to Section 2.2.4 : Mirror Feedback ). Mirror animation can be implemented in either of the following ways: (1) The left virtual hand mirror-replicates the movements of the right real hand while the right virtual hand remains fixed in a pre-defined posture (see Supplementary Video 1). (2) The right virtual hand mirror-replicates the movements of the left real hand while the left virtual hand remains fixed in a pre-defined posture. Note: During the typing task, the pre-defined posture prevents the participant from using the fixed virtual hand for feedback, thereby forcing them to focus on the virtual hand that mirror-replicates the movements of the typing hand. 2. Calibration Mode : In this mode, the application receives the calibration status from all the IMU sensors of both hands and displays it on the system monitor for the experimenter to review (see Supplementary Figure S9). This mode enables the experimenter to ensure that all IMU sensors are calibrated before the start of the experiment and to periodically check the calibration status of the sensors in between trials. 3. Idle Mode : In this mode, the application does not accept any data from the HKMS-TIS boards. 2.10.2 Synchronization The VR application controls the operating modes of the HKMS by sending it appropriate commands (“O,” “C,” or “X”). To synchronize the operating mode between the HKMSs of both hands, the application sends identical commands to both the HKMS-TIS boards in quick succession. This ensures that both the HKMSs switch their operating mode nearly simultaneously. Additionally, it is critical for the VR application to synchronize its operating mode with both the HKMSs to ensure correct data parsing. For example, if the application is in Orientation mode and it sends a “C” command to switch the HKMSs to Calibration mode, it must not switch its own mode immediately. If it does, any remaining unparsed Orientation data in the serial buffer will be incorrectly interpreted as Calibration data, leading to parsing errors. A simple synchronization mechanism was implemented to address this issue. Whenever the VR application sends a command to an HKMS-TIS board, the HKMS master immediately echoes this command back to the application before switching its own operating mode. For example, upon receiving a "C" command from the application, the HKMS-TIS board transmits "C" back to the application. At the application's end, parsing continues in the previous operating mode until this echoed confirmation is received. Once received, the application recognizes that subsequent incoming data frames correspond to the new operating mode and adjusts its parsing method accordingly. This synchronization process occurs independently for each HKMS-TIS board through its respective data collection threads (see Supplementary Material Section 5 for more on data collection using threads). Furthermore, at the beginning of each experimental trial, an "O" command is sent to both HKMS-TIS boards regardless of whether they are already in Orientation mode. Sending “O” resets internal timing variables within each board, and the subsequent data frames received after sending this command are used for the trial. 3. Methodology Part 2: Quaternion Transformations and Virtual Hand Calibration This section outlines the methodologies critical to implementing the experimental task. These include an intuitive approach for manipulating quaternions to achieve coordinate transformation and mirror animation; a method for correcting sensor-to-segment misalignment to ensure accurate hand segment orientations; and a procedure for constructing and refining participant-specific virtual hand models to accurately replicate the real hand’s thumb-to-phalanx touches. A detailed description of the virtual hand model and its animation method is also included. 3.1 Quaternion Transformations and Virtual Hand Model Structure This section presents an intuitive, axis-angle–based approach to manipulating quaternions for implementing (1) quaternion coordinate transformations and (2) mirror animation. It also describes the hierarchical structure of the virtual hand model and how segment-wise quaternion data is used to animate it in real time. 3.1.1 Quaternion Coordinate Transformation For an object in Unity’s 3D world to accurately replicate the BNO055 IMU’s real-world orientation, it is necessary to transform the quaternion from the IMU’s coordinate system to Unity’s coordinate system. To implement this transformation, we first need to understand the coordinate systems of the BNO055 IMU and Unity. The IMU operates in a right-hand coordinate system (see Fig. 7 (a)), and it outputs absolute orientation data with respect to the Earth's East-North-Up (ENU) frame of reference (while operating in the NDOF sensor fusion mode 29 ). In this system, the + X , +Y , and + Z coordinate axes align with the Earth's east, north, and up directions, respectively. In contrast, Unity uses a left-hand coordinate system where the + X , +Y , and + Z axes point to the right, up, and forward directions, respectively, within Unity’s 3D space (see Fig. 7 (b)). If the IMU is positioned such that the x- , y- , and z- axes of its local frame of reference are aligned with the Earth's east, north, and up directions, respectively, then the IMU is considered to be in its default orientation (see Fig. 7 (a)). Similarly, a Unity object in its default orientation is shown in Fig. 7 (b). The objective is to ensure that any movements of the IMU from its default orientation are accurately replicated by a Unity object that begins in its own default orientation. To achieve this, the IMU’s quaternion must be transformed from the IMU’s coordinate system to Unity’s coordinate system before being applied to a Unity object. To derive this transformation, it is first necessary to understand the axis-angle representation of quaternions. Consider a unit quaternion \(\:q=\left(x,\:y,\:z,\:w\right)\) . This quaternion represents an orientation that can be obtained by rotating through an angle \(\:\theta\:\) about a rotation axis defined by the vector \(\:P=\left(a,\:b,\:c\right)\) . Here, \(\:P\) and \(\:\theta\:\) can be derived from the quaternion \(\:q\) using the following relation 30 : $$\:q=\left(x,\:y,\:z,\:w\right)=(a\text{sin}\left(\frac{\theta\:}{2}\right),\:b\text{sin}\left(\frac{\theta\:}{2}\right),\:c\text{sin}\left(\frac{\theta\:}{2}\right),\:\text{cos}\left(\frac{\theta\:}{2}\right))$$ $$\:\theta\:=2co{s}^{-1}\left(w\right)\:$$ $$\:P=\left(a,\:b,\:c\right)=(\frac{x}{\text{sin}\left(\frac{\theta\:}{2}\right)},\:\frac{y}{\text{sin}\left(\frac{\theta\:}{2}\right)},\:\frac{z}{\text{sin}\left(\frac{\theta\:}{2}\right)})$$ We exploit this understanding of the axis-angle representation to derive the coordinate transformation. This is done in two steps: For the IMU unit quaternion \(\:{q}_{IMU}=\left(x,\:y,\:z,\:w\right)\) , the rotational axis \(\:P\) in the IMU’s frame needs to be represented in the Unity frame. This is done as follows: \(\:{P}_{Unity}\left(-b,\:c,\:a\right)=\:{P}_{IMU}\left(a,\:b,\:c\right)\) i.e., the vector \(\:P=\left(a,\:b,\:c\right)\) in the IMU frame can be represented as \(\:P=\left(-b,\:c,\:a\right)\) in the Unity frame (see Fig. 7 ). This is because Unity’s +X , +Y , and +Z coordinate axes are in the –Y , +Z , and +X directions of the IMU’s coordinate axes, respectively. In quaternion form, this corresponds to transforming \(\:{q}_{IMU}\) into \(\:{q}_{1}\) as follows: \(\:{q}_{1}=(-y,\:z,\:x,\:w)\) . The next step is to ensure that the rotation \(\:\theta\:\) about the rotational axis \(\:P\:\) is in the same direction in both the IMU and Unity coordinate systems. A positive rotation about \(\:P\:\) in the right-hand coordinate system of the IMU is equivalent to a negative rotation about \(\:P\:\) in the left-hand coordinate system of Unity. Therefore, to maintain consistency in rotation direction, the rotation angle \(\:\theta\:\) in \(\:{q}_{1}\:\) must be negated before applying it in Unity’s frame. $$\:{q}_{Unity}=\left(-b\text{sin}\left(\frac{-\theta\:}{2}\right),\:c\text{sin}\left(\frac{-\theta\:}{2}\right),\:a\text{sin}\left(\frac{-\theta\:}{2}\right),\:\text{cos}\left(\frac{-\theta\:}{2}\right)\right)$$ $$\:{q}_{Unity}=\left(b\text{sin}\left(\frac{\theta\:}{2}\right),\:-c\text{sin}\left(\frac{\theta\:}{2}\right),\:-a\text{sin}\left(\frac{\theta\:}{2}\right),\:\text{cos}\left(\frac{\theta\:}{2}\right)\right)$$ $$\:{q}_{Unity}=\left(y,\:-z,\:-x,\:w\right)$$ 1 $$\:{q}_{Unity}=conjugate\left({q}_{1}\right)$$ 2 The incoming IMU data from the HKMS, after coordinate conversion as defined in Eq. ( 1 ), can be directly applied to the virtual hand model for animation. However, a limitation of this approach is that the participant would need to physically face the Earth's magnetic north for the virtual hands to be oriented in the forward direction within Unity’s 3D space. This requirement is impractical in a typical experimental setting. To address this issue, a custom global frame of reference is defined within the experimental room. The raw IMU quaternion, which represents the sensor orientation with respect to the ENU frame of reference, is transformed to represent the sensor orientation with respect to this room-specific global frame of reference instead. The coordinate conversion defined in Eq. ( 1 ) is then applied to this transformed quaternion and used for animating the virtual hands. Now, for the virtual hands to be oriented in the forward direction in Unity’s 3D space, the participant simply needs to align themselves with the positive X -axis of the room’s global frame of reference, rather than aligning themselves to the Earth’s magnetic north. A detailed explanation of how the room-specific global frame of reference is established and how the raw IMU quaternion is transformed to represent the sensor orientation with respect to this global frame of reference is provided in Section 3.3.1 : Sensor-To-Segment Alignment . 3.1.2 Quaternion Transformation for Mirror Animation As previously described, mirror animation involves the movements of the typing hand being mirrored by the opposite (contralateral) virtual hand. For a more detailed explanation, refer to Section 2.2.4 : Mirror Feedback . The goal is to achieve mirror animation about the participant's midsagittal plane. Throughout the experiment, the participant will remain seated, with their forward view in the VR environment aligned along Unity’s + Z axis. Hence, to achieve mirror animation, the orientations must be mirrored about Unity’s YZ plane. To achieve this, we again exploit our understanding of the quaternion’s axis-angle representation to transform the quaternion appropriately. Let \(\:{q}_{Unity}=\left(x,\:y,\:z,\:w\right)\) be a unit quaternion after coordinate conversion using Eq. 1 . As per the axis-angle representation, \(\:{q}_{Unity}\) represents a rotation by \(\:\theta\:\) about a rotation axis defined by the vector \(\:P=\left(a,\:b,\:c\right)\) . The mirror orientation is achieved in two steps: Reflecting the Rotation Axis: The rotation axis \(\:P\) is reflected about the YZ plane by negating its x-component, resulting in \(\:P{\prime\:}=\left(-a,\:b,\:c\right)\) . In quaternion form, this corresponds to transforming \(\:{q}_{Unity}\) into \(\:{q}_{1}\) as follows: \(\:{q}_{1}=(-x,\:y,\:z,\:w)\) . Reversing the Rotation Direction: To achieve the mirror orientation, the rotation about the reflected axis \(\:P{\prime\:}\) must be reversed, that is, rotation by an angle of \(\:-\theta\:\) . The reason for doing this can be understood through the following example: if a hand is rotated clockwise, its image in a mirror kept along the midsagittal plane rotates counterclockwise. To implement this reversal of rotation direction mathematically, we take the quaternion conjugate of \(\:{q}_{1}\) : \(\:{q}_{mirror}=conjugate\left({q}_{1}\right)\) . For real-time mirror animation, the quaternion data from the incoming HKMS data frames of one hand are converted to \(\:{q}_{mirror}\) and then applied to the corresponding segments of the opposite (i.e., contralateral) virtual hand model. 3.2 Virtual Hand Model Structure and Real-Time Animation The virtual hand model adopts a hierarchical structure. At the root of this hierarchy is a cuboid (see Figs. 1 , 9 , and 10 ) that represents the four metacarpal bones and all the carpals as a single unit. This cuboid is located approximately 2 cm below the middle virtual finger and replicates the orientation of the “dorsal hand sensor” of the HKMS (affixed on the dorsal surface of the real hand, a few cm below the middle finger’s MCP joint). The four virtual fingers and the virtual thumb are positioned at anatomically appropriate locations around the cuboid but are not visually connected to it (see Figs. 1 , 9 , and 10 ). However, the bases of these virtual fingers and the virtual thumb are invisibly linked to the cuboid such that any change in the cuboid's orientation appropriately moves the spatial location of these bases. For each of the four virtual fingers, the base of the proximal phalanx is invisibly linked to the cuboid at a point corresponding to the metacarpophalangeal (MCP) joint, the middle phalanx is linked to the proximal phalanx at a point corresponding to the proximal interphalangeal (PIP) joint, and the distal phalanx is linked to the middle phalanx at a point corresponding to the distal interphalangeal (DIP) joint. Similarly, for the virtual thumb, the metacarpal bone is invisibly linked to the cuboid at a point corresponding to the carpometacarpal (CMC) joint, the proximal phalanx is linked to the metacarpal at a point corresponding to the MCP joint, and the distal phalanx is linked to the proximal phalanx at a point corresponding to the IP joint. Each segment’s local frame of reference originates at its associated joint (i.e., origins of the proximal, middle, and distal phalanges of a virtual finger are at the MCP, IP, and DIP joints, respectively). For the cuboid, the local frame of reference originates outside the cuboid at a point corresponding to the wrist joint. Because each segment’s local frame is anchored at its joint (hence acting as the pivot point), assigning an orientation to a segment rotates it about that joint. Unlike traditional hand models, which often constrain the joint rotation to specific axes, in the current hand model, since quaternions are directly applied to each hand segment, the axis of rotation at the joint is determined by the quaternion itself. Due to the hierarchical linkage of the virtual hand model, assigning an orientation to a base segment results in shifting the local origin of the subsequent segment. For example, consider the index finger: updating the cuboid’s orientation moves the MCP joint linked to it, along with the proximal phalanx whose origin is anchored at the MCP joint. Updating the proximal phalanx’s orientation rotates it about the MCP joint, thereby moving the PIP joint located at its distal end. The middle phalanx, whose origin is anchored at the PIP joint, moves with the PIP joint. This chain effect continues when updating the orientations of the middle and distal phalanges, and the same principle applies to all other fingers. The IMU sensors of the HKMS output the orientation of the hand segment to which they are attached. By assigning the quaternion data from an incoming HKMS data frame (after appropriate transformations) to the corresponding segments of the virtual hand model, the resulting virtual hand posture replicates the real hand posture. Continuously updating the virtual hand model’s posture with the incoming HKMS data frames in real-time results in the animation of the virtual hand model, replicating the participant's actual hand movements in real-time. 3.3 Obtaining Accurate Hand Segment Orientation and Precise Hand Animation The orientation data provided by the IMU sensors attached to the hand segments do not accurately represent the actual orientations of these segments. This inaccuracy arises due to a misalignment between each sensor's local frame of reference and the local frame of reference of the corresponding hand segment. Accurate hand segment orientations are required for both data analysis and precise hand animation. Additionally, accurate replication of thumb-to-phalanx touches during animation – which is critical for the current experimental task – cannot be achieved using a generic virtual hand model. This section describes the methodology and procedures used to correct sensor-to-segment misalignment and create participant-specific virtual hand models, thereby obtaining accurate hand segment orientations and achieving precise hand animation. The methods described in this section were tested on two participants. Note Details regarding the preparatory steps carried out before each experimental session, along with the materials and procedures used to affix the HKMS and TIS devices to the hand, are provided in Supplementary Material Sections 7 and 8. 3.3.1 Sensor-To-Segment Alignment While attaching IMU sensors to the hand, it is very difficult to manually place the sensors such that the local frame of reference of the sensor is aligned with the local frame of reference of the hand segment to which it is attached. As a result, the sensor's orientation data does not accurately represent the true orientation of the hand segment. To correct this discrepancy and obtain accurate segment orientation data, the sensor frame of reference needs to be aligned to the segment frame of reference through code. This process is referred to as sensor-to-segment alignment, and this sub-section details the methodology used to perform this alignment. As mentioned earlier, this methodology was validated on two participants. Step 1 – Establishing a custom global frame of reference A sheet with parallel lines was affixed to the experimental table. A BNO055 IMU sensor was placed flat on the table and manually aligned to one of the parallel lines as accurately as possible. Specifically, the sensor's x -axis was aligned along one of the parallel lines, its y -axis was pointed leftward, and its z -axis was perpendicular to the plane of the sheet, pointing out of the plane of the paper. The orientation of the sensor, in the form of a quaternion, was recorded in this position using a Teensy 4.0 microcontroller. This orientation value was used as the global frame of reference for all IMU orientation data collected throughout all the test sessions for the two participants. The X , Y , and Z axes of this custom global frame of reference are depicted in Fig. 8 (a). To ensure consistent sensor-to-segment alignment, it was crucial that the parallel-lined sheet always remained aligned with the global frame of reference just recorded. To maintain this alignment, the parallel-lined sheet was securely fixed to the table for all the test and experimental sessions, and the table itself was not moved or repositioned at any point during the test involving the 2 participants. Step 2 – Physically aligning the hand segments to the custom global frame of reference Before each experiment, participants placed their palms flat on the parallel-lined sheet, aligning their four fingers along the parallel lines (see Fig. 8 (a)). In this position, the local frames of reference of the finger phalanges and hand were physically aligned to the global frame of reference established in Step 1. The orientation data of the sensors in this position were recorded. For the thumb, additional alignment steps were required. The participant positioned their thumb at the edge of the table so that the proximal and distal phalanges rested on the table while the metacarpal extended beyond the edge (see Fig. 8 (b)). Care was taken to ensure that the metacarpal remained in the same horizontal plane as the table (see Fig. 8 (c)). Additionally, the alignment of the metacarpal to the parallel lines was challenging since the metacarpal bone’s outline was not clearly visible. To facilitate this alignment, a straight line was drawn on the dorsal surface of the thumb, running from its tip to the base of the metacarpal. This line was then aligned to one of the parallel lines. Once properly aligned, the orientation data of the thumb sensors were recorded. Note Drawing the straight line on the metacarpal was an involved process. First, the base of the participant’s thumb metacarpal was identified. Since the base was difficult to locate by sight alone, the participant was instructed to perform one of two movements upon request to aid in its identification through tactile sensation: (1) touch the proximal phalanx of the little finger with the tip of the thumb and (2) extend the thumb outward from this position. As the participant executed these movements when requested by the experimenter, the experimenter identified and marked the base of the thumb metacarpal through tactile examination. Once the base was marked, the experimenter used their own thumb and index finger to press along the sides of the participant’s thumb metacarpal, making the bone more visible. At the same time, the experimenter guided the participant’s thumb into a straightened posture, aligning the metacarpal with the two phalanges (see Fig. 8 (d)). With the participant’s thumb held in this position, a straight line was drawn centrally along the dorsal surface of the thumb, extending from the tip to the base of the metacarpal. Step 3 – Determining the error orientation between the sensor and the segment As mentioned previously, the raw quaternion values output by the IMU sensor represent the sensor’s orientation with respect to the Earth's East-North-Up (ENU) frame of reference. To express the sensor's orientation relative to the global frame of reference (established in Step 1) instead of the ENU frame, the raw quaternion values were transformed using the following equation $$\:{q}_{IMU\_G}=\:{q}_{G}^{conj}\:⨂\:\:{q}_{IMU\_ENU}$$ 3 here, \(\:{q}_{IMU\_ENU}\) is the IMU orientation with respect to the ENU frame of reference, \(\:{q}_{G}\) is the global frame of reference computed in step 1, \(\:{q}_{G}^{conj}\) is the conjugate of \(\:{q}_{G}\) and \(\:{q}_{IMU\_G}\) is the IMU sensor’s orientation with respect to the global frame of reference \(\:{q}_{G}.\) The data recorded in step 2 was converted from the ENU frame of reference to the global frame of reference using Eq. ( 3 ). These transformed values now represent the sensor orientation with respect to the segment's local frame of reference (since the segment's local frames of reference were aligned to the global frame of reference in step 2). In other words, these values represent the error orientation between the sensor frame of reference and the segment frame of reference and are referred to as \(\:{q}_{error}\) . Note In the context of this paper, a quaternion multiplication \(\:{q}_{3}=\:{q}_{1}\:⨂\:\:{q}_{2}\) can be interpreted in the following way: \(\:{q}_{3}\) represents an orientation that results from first rotating by \(\:{q}_{1}\) and then rotating by \(\:{q}_{2}\) relative to the reference frame resulting from \(\:{q}_{1}\) rotation. Step 4 – Implementing sensor-to-segment alignment To accurately compute a segment’s orientation, the attached sensor’s data was adjusted by applying an inverse rotation to it using the sensor's corresponding error quaternion \(\:{q}_{error}\) (which was pre-computed in Step 3) $$\:{q}_{seg\_G}=\:{q}_{IMU\_G}\:⨂\:\:{q}_{error\:}^{conj}$$ 4 Here, \(\:{q}_{seg\_G}\) is the error-corrected orientation of the hand segment (with respect to the global frame of reference), \(\:{q}_{IMU\_G}\) is the sensor orientation (with respect to the global frame of reference), and \(\:{q}_{error}\:\) is the error orientation between the segment and the sensor computed in step 3. The sensor-to-segment alignment correction (Eq. ( 4 )) is applied in real time to the data from all 16 IMU sensors within each HKMS data frame, resulting in accurate segment orientations that are then used to animate the virtual hand model in real time. Figure 9 illustrates the effect of sensor-to-segment alignment: without applying this correction, the virtual fingers appear crooked because the sensor orientations do not accurately represent the true segment orientations. 3.3.2 Achieving Accurate Virtual Hand Animation Given the nature of the typing task (refer to Section 2.2 : The Task for a recap), it is essential that the virtual hands accurately replicate the thumb-to-phalanx touches of the real hands. To achieve this, the spatial locations of the fingertips and joints need to be accurate during virtual hand animation. From the description of the virtual hand model in Section 3.2 : Virtual Hand Model Structure and Real-Time Animation , it can be inferred that the locations of the fingertips and joints during animation depend not only on the hand segment orientations but also on the hand segment lengths and relative finger locations. Therefore, to ensure accurate thumb-to-phalanx touch replication during animation, customized virtual hand models with the exact dimensions of the participant's real hands need to be built for each participant. To evaluate this approach, two participants were recruited, and personalized virtual hand models were created for each participant for both the left and right hands using the exact dimensions of their real hands. The models incorporated measurements such as the lengths of all finger phalanges, the length of the thumb metacarpal, the relative distances between the bases of the fingers, and the thickness (height) and width of each finger. These measurements were obtained using the Polhemus Patriot motion tracking system, as detailed in Supplementary Material Section 6: Measuring Hand Dimensions Using the Polhemus Patriot System . The recorded measurements were then used to build virtual hand models in Blender 3.5, a 3D creation software. Blender was also used to superimpose the characters represented by the keys of the TIS onto the corresponding phalanges of the virtual fingers (see Figs. 1 , 9 , and 10 ). While this approach significantly improved the replication accuracy of the virtual hands, there were still minor errors in the thumb-to-phalanx touch replication (see Fig. 10 ). This was due to slight inaccuracies in the exact relative positioning between the thumb and other fingers of the virtual hand. To resolve this and achieve optimal thumb-to-phalanx touch accuracy, manual adjustments to virtual finger positions were performed within Unity in the following way: Initially, the participants were instructed to touch their middle finger’s middle phalanx approximately at the phalanx centre using the tip of their thumb. While the participants held this posture without making any movements, the experimenter adjusted the virtual thumb’s position within Unity such that the thumb tip touched the surface of the virtual middle finger’s middle phalanx at approximately its centre. Following this adjustment, when the participant touched the other phalanges of the middle finger, it resulted in accurate replication of the thumb-to-phalanx touches by the virtual hand. Further minor adjustments were made to the virtual thumb and middle finger positions, if necessary, to refine the touch accuracy for the proximal and distal phalanges of the middle finger. The resulting position of the virtual thumb was fixed throughout the adjustment process. Next, the participants were instructed to touch any phalanx of one of the other fingers, and adjustments were made to the corresponding virtual finger’s position to achieve the desired touch accuracy. This process was repeated for all phalanges. These positional adjustments required approximately 10 minutes per hand for each participant. Figure 10 illustrates the thumb-to-phalanx touch accuracy before and after these adjustments. Additionally, a mirror version of each virtual hand model was created using Blender for use specifically during mirror animation. Hence, two pairs of virtual hand models were used for each participant: one pair with dimensions exactly matching those of the participant’s real hands and another “mirror” pair whose dimensions match the opposite (contralateral) real hand. For example, the "mirror" virtual left-hand model is created with dimensions identical to the participant's real right hand, effectively making it a mirror image of the virtual right hand. This approach was necessary because using a left virtual hand with the real left hand’s dimensions during mirror animation would result in inaccuracies in the thumb-to-phalanx touches. This is because the orientation data from the right hand – after appropriate transformations – is applied to the virtual left hand during mirror animation. 4. Results: Experimental Setup Validation To validate the functioning of the experimental setup, two participants were recruited to participate in an experiment spread out across four consecutive days. Ethical clearance for the experiment was obtained from the Institute Ethics Committee (IEC) of the Indian Institute of Technology Madras (Project No: IEC/2022-2/SKM/03/09). All experimental sessions were performed in accordance with the procedures approved by the Institute Ethics Committee of the Indian Institute of Technology Madras. Informed consent was obtained from all participants prior to their involvement. All methods were carried out in accordance with relevant guidelines and regulations. The experiment involved a typing task where the participants had to type words presented in the VR environment as quickly as possible using the instructed hand. A detailed description of the task is provided in Section 2.2 : The Task . On Day 1 of the experiment, participants performed a pre-practice trial using the left hand, followed by 12 practice trials using the right hand, and concluded with a post-practice trial using the left hand. On Days 2 through 4, the same protocol was followed, except that no pre-practice trial was conducted. During a trial, two points were awarded for each correctly typed character, while no points were deducted for incorrectly typed characters. The emphasis of the task was on speed, with the goal being to achieve the maximum score within the trial duration. Each trial was 2 minutes long. A fixed set of seven five-letter words (INSET, STAIR, SHINE, TETRA, SHIRT, TRAIN, TATER) was presented (one at a time) to the participant in a jumbled order for typing. Once all words were typed, the word set was reshuffled and presented again. This process was repeated until the trial ended. During the practice trials, one participant received mirror visual feedback (MVF), where the left virtual hand mirrored the movements of the right real hand, while the right virtual hand remained frozen in a pre-defined posture. As a result, the participant had to rely on the left virtual hand for visual feedback while typing with the right real hand (refer to Section 2.2.4 : Mirror Feedback for more details on MVF). The second participant received normal visual feedback (NVF), where both virtual hands accurately replicated the movements of their respective real hands. Note: The trial format shown in Supplementary Material Video 1 differs from that of the current experiment. In the video, a different word set was used, and participants received 5-second breaks after every 15 seconds of typing. Three performance variables were analysed for each trial: Correct Characters Typed (CCT) : Total number of characters typed correctly during a trial. Press Duration (PD) : Time between key press and key release events (in milliseconds). Movement Time (MT) : Time between the release of one key and the press of the next key (in milliseconds). Figure 11 depicts plots of these three variables for the left hand trials across the four days. The pre-practice trial on Day 1 was conducted to determine the participants' baseline performance level for the left hand. For both participants, the left hand performance improved after each right hand practice session over the first three days and plateaued on the fourth day. This improvement is indicated by increasing CCT values and decreasing MT and PD values across the first three days (see Fig. 11 ). Notably, the MVF participant exhibited greater left hand performance gains compared to the NVF participant after each practice session. While these preliminary results suggest that MVF may enhance cross-education, they should be interpreted with caution. The MVF participant reported occasional guitar playing, which could have contributed to the enhanced left-hand performance and thus confounded the results. To rigorously evaluate the effect of MVF on cross-education, a larger study is required. Participants should be divided into MVF and NVF groups, and individuals with experience in fine motor skill activities (e.g., playing musical instruments or video gaming) must be excluded from the study. Nonetheless, the current experiment successfully demonstrates the functionality and reliability of the experimental setup for investigating MVF effects on cross-education. All experimental sessions were completed without technical issues. To verify real-time data reception by the VR application, an additional feature was implemented: whenever the master microcontroller of the HKMS-TIS board sent a synchronization signal to the slave microcontrollers, it also transmitted the ‘ $ ’ character to the VR application. This marked the start of the data frame and also indicated to the application that the data collection cycle for that particular data frame had started at the HKM-TIS board's end. As described in Section 2.8.1 : Communication Protocol , the end of each data frame is marked by the character ‘#’. The timestamps of the reception of both ‘ $ ’ and ‘#’ were logged by the application. Visual inspection of these logged timestamps revealed that: Synchronization signals occurred every 10ms, as expected. Each data frame was received by the VR application within ~ 7–8 ms of initiation of the synchronization signal. No delays were observed in data frame reception over the full trial duration. Inspection of data files confirmed that there were no missing data frames, as verified by the data frame count value embedded in each data frame. Furthermore, performance variables (CCT, PD, MT) were successfully extracted from the TIS data for analysis. These findings collectively confirm that the system is ready for full-scale experimentation. 5. Discussion and Conclusion Research on the enhancement of cross-education through mirror visual feedback (MVF) has gained significant traction over the last two decades 4 – 9 due to its promising applications in the rehabilitation of hemiparetic patients 11 , 31 . Studies on healthy participants have yielded positive results supporting its use in such rehabilitation 4 – 9 . However, most of these studies have focused on relatively simple tasks – typically involving gross movements, single-joint rotations, or limited fine motor coordination and multi-finger control. To address this limitation, we built a novel experimental setup that enables the investigation of MVF-based cross-education for a more complex task involving sequenced fine finger movements and multi-finger coordination. This task requires participants to touch different finger phalanges with the tip of the thumb, one at a time, in various sequences. This experimental setup forms the foundation for our ongoing research on MVF-mediated cross-education and consists of three main components: (1) A novel typing device called the Text Input System (TIS), which is used to implement an intricate finger sequence movement task; (2) A Hand Kinematics Measurement System (HKMS) that accurately tracks hand and finger movements in real-time 28 ; and (3) A VR headset that provides normal and mirror visual feedback during the task in real time. These three systems are fully integrated and operate in synchrony (see Supplementary Video 1 for a demonstration of the experimental setup). The hand kinematics data from the HKMS and the typing data from the TIS are timestamp-synchronized – a critical requirement for analyzing movement patterns in relation to typing activity. To validate the functionality of the experimental setup, a preliminary experiment was conducted involving two participants. The setup performed as intended across all aspects of operation: all experimental sessions were completed without technical issues; data frames were received at the expected 100 Hz frequency, with no missing frames or transmission delays; and the key performance variables (CCT, PD, MT) were successfully extracted from the TIS data for analysis. These outcomes confirm the reliability of the setup and support its suitability for full-scale experiments investigating MVF effects on cross-education. In addition, key design and methodological innovations were implemented for the experimental setup. A stable period algorithm was developed for the TIS to precisely detect key press and release events. This algorithm filters out electrically and mechanically induced voltage fluctuations specific to the TIS that could otherwise lead to false key press or release detections. Participant-specific virtual hand models were built, and a methodology to refine these customized hand models was implemented, resulting in precise hand animation where the virtual hands accurately replicate the thumb-to-phalanx touches of the real hands. Given the nature of the experimental task, such precision in animation was required, which could otherwise not be achieved using a generic hand model. A clear methodology, along with the accompanying mathematical formulation, was developed to achieve sensor-to-segment alignment, which is necessary to obtain accurate orientation data of all the finger segments. This approach can be used by any application that uses IMUs and requires accurate hand kinematics data. An intuitive approach based on the axis-angle representation of quaternions was developed to transform quaternions from one coordinate system to another. This approach was used to transform the quaternion data of the HKMS from the BNO055 IMU coordinate system to the Unity coordinate system. Building on this, the approach was further extended to achieve mirror animation of the virtual hand model. Despite the various strengths, we recognize several limitations in the current experimental setup. A major limitation is the extensive and time-consuming pre-experimental setup preparations for each experimental session. This involves taking the hand measurements of each participant in a separate session, building customized hand models for each participant using the Blender software, preparing a fresh batch of copper patches soldered to wires for each experimental round, and cutting a sufficient quantity of double-sided tape in appropriate sizes for affixing the IMU sensors and copper patches. Additionally, it takes approximately 25–30 minutes per hand to affix the IMU sensors and the copper patches to the participant's hands, and approximately 10 minutes per hand to make adjustments to the virtual hand model to achieve accurate thumb-to-phalanx touch replication by the virtual hand models. Including the time required to perform the sensor-to-segment alignment procedures, it takes approximately 1.5 hours from the beginning of affixing the devices before the participant can actually start performing the task. In addition, the current design of the virtual hand model is such that the fingers of the hand model are separate elements (see Figs. 1 , 9 , and 10 ). This is necessary since it allows for adjustments to be made to the hand model, which is required for achieving accurate thumb-to-phalanx touch replication. Although this gives an unrealistic appearance to the hand model, while performing pilot experiments with the setup, the participants did not report this as a limitation. Instead, participants reported that the virtual hands felt like their own hands within the VR environment – likely due to the real-time and accurate replication of their hand movements, which created a strong sense of embodiment. As part of future work, instead of creating participant-specific virtual hand models manually, methods for automating this process can be explored. This could be achieved by developing software that takes hand dimensions as input and automatically generates a virtual hand model based on those dimensions. Additionally, after adjusting the hand model to achieve the desired thumb-to-phalanx touch accuracy, methods to automatically convert the refined hand model into a more realistic representation of the hand can be explored. Furthermore, the tedious preparation of a fresh batch of soldered copper patches and a fresh set of cut double-sided tapes before each experimental round and the long device mounting time make it impractical to investigate the long-term effects (spanning multiple days or weeks) of MVF on cross-education for this particular task. To overcome this limitation, the HKMS and TIS systems must be mounted on gloves that allow for easy adjustment of the IMU and copper patch placement. In summary, this paper presents the design, development, and validation of an experimental setup built to investigate the effects of mirror visual feedback on the enhancement of cross-education for a task at a level of complexity previously unexplored in the cross–education–MVF literature. The paper also details key methodologies for system integration, millisecond-level data synchronization, and precise real-time hand animation. An intuitive, axis-angle-based approach for quaternion transformation across coordinate systems is described, along with a method for achieving mirror animation of virtual hands. Furthermore, a detailed sensor-to-segment alignment methodology is outlined to obtain accurate finger segment orientations. Many of the methodologies detailed in the paper contribute valuable tools to the hand biomechanics and VR research community. The setup was validated through a preliminary experiment, which confirmed its reliability and operational integrity, thereby establishing its readiness for future MVF-based cross-education studies. Declarations Competing Interests The authors declare no competing interests. Funding This research was supported by the Department of Science and Technology (DST), Government of India, under the Cognitive Science Research Initiative (Grant No. DST/CSRI/2017/87, awarded to Varadhan SKM). It was also supported by the Ministry of Education, Government of India, through the Prime Minister’s Research Fellowship (PMRF ID: 2500902, awarded to Anurag Gupta). Author Contribution Experimental setup design and development — A.G.; Conceptualization of experimental task — V.S.K.M.; Methodologies and algorithm development — A.G.; Writing original draft — A.G.; Review and editing draft — V.S.K.M., A.G.; All authors reviewed and approved the final manuscript. Acknowledgement We gratefully acknowledge the Department of Science and Technology (DST), Government of India, for funding this work under the Cognitive Science Research Initiative (CSRI) (Grant No. DST/CSRI/2017/87, awarded to Varadhan SKM). We also thank the Ministry of Education, Government of India, for supporting this research through the Prime Minister’s Research Fellowship (PMRF) (PMRF ID: 2500902, awarded to Anurag Gupta). We are thankful to Thomas Jacob, a PhD scholar in our lab, for his encouragement, valuable suggestions, and assistance in testing the experimental setup throughout its development. We also acknowledge Kanva Aravind Kashyapa, a former research intern in the lab, for his help in testing the experimental setup. Data Availability The data collected for the current study are available from the corresponding author on reasonable request. References Volkmann, A. W. Über Den Einfluss Der Uebung Auf Das Erkennen R¨Aumlicher Distanzen. Berichte K. Sächs. Ges. Wiss. Math.-Phys. Cl. 10 , 38–69 (1858). Scripture, E. W., Smith, T. L. & Brown, E. M. On the education of muscular control and power. Stud. Yale Psycological Lab. 2 , (1894). Manca, A. et al. Contralateral Effects of Unilateral Strength and Skill Training: Modified Delphi Consensus to Establish Key Aspects of Cross-Education. Sports Med. Auckl. NZ 51 , 11–20 (2021). Ossmy, O. & Mukamel, R. Neural Network Underlying Intermanual Skill Transfer in Humans. Cell Rep. 17 , 2891–2900 (2016). Ossmy, O. & Mukamel, R. Using Virtual Reality to Transfer Motor Skill Knowledge from One Hand to Another. J. Vis. Exp. 55965 (2017) doi:10.3791/55965. Nojima, I. et al. Human Motor Plasticity Induced by Mirror Visual Feedback. J. Neurosci. 32 , 1293–1300 (2012). Ruddy, K. L. et al. Neural Adaptations Associated with Interlimb Transfer in a Ballistic Wrist Flexion Task. Front. Hum. Neurosci. 10 , 204 (2016). Zult, T. et al. Mirror Training Augments the Cross-education of Strength and Affects Inhibitory Paths. Med. Sci. Sports Exerc. 48 , 1001 (2016). Steinberg, F., Pixa, N. H. & Doppelmayr, M. Mirror Visual Feedback Training Improves Intermanual Transfer in a Sport-Specific Task: A Comparison between Different Skill Levels. Neural Plast. 2016 , 8628039 (2016). Ramachandran, V. & Rogers-Ramachandran, D. Synaesthesia in phantom limbs induced with mirrors. Proc. R. Soc. Lond. B Biol. Sci. 263 , 377–386 (1996). Altschuler, E. L. et al. Rehabilitation of hemiparesis after stroke with a mirror. The Lancet 353 , 2035–2036 (1999). Ruddy, K. L. & Carson, R. G. Neural pathways mediating cross education of motor function. Front. Hum. Neurosci. 7 , (2013). Ward, J. P., Alvis, G. R., Sanford, C. G., Dodson, D. L. & Pusakulich, R. L. Qualitative differences in tactuo-spatial motor learning by left-handers. Neuropsychologia 27 , 1091–1099 (1989). Stoddard, J. & Vaid, J. Asymmetries in intermanual transfer of maze learning in right- and left-handed adults. Neuropsychologia 34 , 605–608 (1996). Parlow, S. E. & Kinsbourne, M. Asymmetrical transfer of braille acquisition between hands. Brain Lang. 39 , 319–330 (1990). Criscimagna-Hemminger, S. E., Donchin, O., Gazzaniga, M. S. & Shadmehr, R. Learned dynamics of reaching movements generalize from dominant to nondominant arm. J. Neurophysiol. 89 , 168–176 (2003). Cunningham, H. A. & Welch, R. B. Multiple concurrent visual-motor mappings: Implications for models of adaptation. J. Exp. Psychol. Hum. Percept. Perform. 20 , 987–999 (1994). Redding, G. M. & and Wallace, B. Intermanual Transfer of Prism Adaptation. J. Mot. Behav. 40 , 246–264 (2008). Schulze, K., Lüders, E. & Jäncke, L. Intermanual Transfer in a Simple Motor Task. Cortex 38 , 805–815 (2002). Beg, R. A. et al. Intermanual Transfer Effects on Performance Gain Following Dominant Hand Training in Community-Dwelling Healthy Adults: A Preliminary Study. J. Multidiscip. Healthc. 14 , 1007–1016 (2021). Krishnan, C., Ranganathan, R. & Tetarbe, M. Interlimb Transfer of Motor Skill Learning during Walking: No evidence for asymmetric transfer. Gait Posture 56 , 24–30 (2017). Chase, C. & Seidler, R. Degree of handedness affects intermanual transfer of skill learning. Exp. Brain Res. 190 , 317–328 (2008). Green, L. A. & Gabriel, D. A. The cross education of strength and skill following unilateral strength training in the upper and lower limbs. J. Neurophysiol. 120 , 468–479 (2018). Lee, M., Hinder, M. R., Gandevia, S. C. & Carroll, T. J. The ipsilateral motor cortex contributes to cross-limb transfer of performance gains after ballistic motor practice. J. Physiol. 588 , 201–212 (2010). Rachaveti, D. & Skm, V. Motor sequence learning data collected continuously for fifteen days of practice using a novel glove-based typing device. Data Brief 29 , 105234 (2020). Rachaveti, D., Ranganathan, R. & Skm, V. Practice modifies the response to errors during a novel motor sequence learning task. 2020.10.09.334169 Preprint at https://doi.org/10.1101/2020.10.09.334169 (2020). Rachaveti, D. Studies of motor sequence learning using finger thumb opposition movements. (Indian Institute of Technology Madras, 2019). Shenoy, P., Gupta, A. & S.K.M., V. Design and Validation of an IMU Based Full Hand Kinematic Measurement System. IEEE Access 10 , 93812–93830 (2022). Sensortec, B. BNO055: Intelligent 9-Axis Absolute Orientation Sensor – Data Sheet . (2021). Challis, J. H. Quaternions as a solution to determining the angular kinematics of human movement. BMC Biomed. Eng. 2 , 5 (2020). Ramachandran, V. S. & Altschuler, E. L. The use of visual feedback, in particular mirror visual feedback, in restoring brain function. Brain 132 , 1693–1710 (2009). Additional Declarations No competing interests reported. Supplementary Files SupplementaryMaterialAnuragGupta.docx SupplementaryVideo1.mp4 SupplementaryVideoLegend.docx Cite Share Download PDF Status: Published Journal Publication published 09 Apr, 2026 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 17 Nov, 2025 Reviews received at journal 14 Nov, 2025 Reviewers agreed at journal 05 Nov, 2025 Reviewers agreed at journal 02 Nov, 2025 Reviews received at journal 29 Oct, 2025 Reviews received at journal 19 Oct, 2025 Reviewers agreed at journal 19 Oct, 2025 Reviewers agreed at journal 18 Oct, 2025 Reviewers invited by journal 24 Sep, 2025 Editor invited by journal 20 Aug, 2025 Editor assigned by journal 02 Jul, 2025 Submission checks completed at journal 25 Jun, 2025 First submitted to journal 25 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6957809","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":524785216,"identity":"cbd13e7a-fad0-4364-8cf8-9838c7a28d1d","order_by":0,"name":"Anurag Gupta","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/UlEQVRIiWNgGAWjYHACNjjrQEIFkGRmbsCrngdJC+ODB2dAWhiJ18Js+LANrBW/Fnv25mcPGGruJG5vP35NInFebTR/O1DLj4ptuG3hOWZuwHDsWeKcMzllEonbjufOOMzYwNhz5jZuLRI5bBIMbIcTZzDkpAG1HMttAGphZmzDo0X+DVDLP6AW/jdALXOO5c4nqEWCh02CsQ2oRSL9sEFiQ03uBoJazqSZSST2HTaeIfGG8UHCsQO5G4FaDuLzC3v74WcSH74dlp3Bn/7g4I+autx55w8ffPCjArcWMEiAWGgAJA6DmQfwq0dY+ABI1BGpeBSMglEwCkYSAAC/AV2Ui5k1hwAAAABJRU5ErkJggg==","orcid":"","institution":"Indian Institute of Technology Madras","correspondingAuthor":true,"prefix":"","firstName":"Anurag","middleName":"","lastName":"Gupta","suffix":""},{"id":524785217,"identity":"50d19ac9-ceca-4fb4-9bae-bf5892bf5916","order_by":1,"name":"Varadhan SKM","email":"","orcid":"","institution":"Indian Institute of Technology Madras","correspondingAuthor":false,"prefix":"","firstName":"Varadhan","middleName":"","lastName":"SKM","suffix":""}],"badges":[],"createdAt":"2025-06-23 14:38:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6957809/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6957809/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-026-41353-1","type":"published","date":"2026-04-09T15:58:57+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":93016564,"identity":"2b9d3723-7a76-4508-b39f-d452cd51bfc8","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10391610,"visible":true,"origin":"","legend":"","description":"","filename":"RevisedMainManuscriptAnuragGuptaVaradhanSKM.docx","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/0b33a56fd1c2c6e27b862b3b.docx"},{"id":93015627,"identity":"c1488342-465d-452f-b1d6-174a54696ff4","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5362,"visible":true,"origin":"","legend":"","description":"","filename":"b7e2ccce70384215bc04dc025a771e4d.json","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/f481b563bdae1f9e78ed56af.json"},{"id":93015642,"identity":"d936adbd-04e8-4261-9d7f-ca2e05e89cee","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":7852941,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryMaterialAnuragGupta.docx","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/ff1ed4b60775b13e591ac76c.docx"},{"id":93016572,"identity":"e639785f-eda2-4f52-972c-009e1dc91a86","added_by":"auto","created_at":"2025-10-08 08:06:43","extension":"mp4","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":29820747,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryVideo1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/cb533252d2af92f9c3596ea6.mp4"},{"id":93016565,"identity":"1832a0e0-c134-4f63-aea2-2e2b0f746051","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14638,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryVideoLegend.docx","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/2500b9bb1a651947052b7a13.docx"},{"id":93016567,"identity":"25da0f8b-0030-4d59-9053-e0f5935c1bdb","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":163948,"visible":true,"origin":"","legend":"","description":"","filename":"b7e2ccce70384215bc04dc025a771e4d1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/b039fe9b4c1be8283b87c86a.xml"},{"id":93015634,"identity":"85b7fa53-6575-4713-b91e-0c75f46f9e28","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"emf","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":6665684,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/ace9f66ed450d303386b0a72.emf"},{"id":93015651,"identity":"b350e44b-c2a2-499e-abb0-38fbf7f67f5a","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1333581,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/ded36c148bf035d275b04ad7.png"},{"id":93015635,"identity":"541bb56f-88fc-4682-a923-df49e90365d5","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"emf","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1060984,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage11.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/c13c14a62a3ff942fa12db3b.emf"},{"id":93015639,"identity":"8ab83053-bb91-42d0-819a-ec9d61618443","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":2500631,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/8fb6fe9a6d01eb5b00544874.png"},{"id":93016571,"identity":"bb095789-0304-4ce4-923a-1bc0f3f637e8","added_by":"auto","created_at":"2025-10-08 08:06:43","extension":"emf","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":83100,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/ffaa30b16554f9fcc7093545.emf"},{"id":93016573,"identity":"de31207f-d459-4aa3-813d-343f529331ba","added_by":"auto","created_at":"2025-10-08 08:06:43","extension":"emf","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":128340,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/d031bb54d84ba7236802a5d8.emf"},{"id":93017594,"identity":"8fc2b56c-9f90-48b6-8b0d-2f8dd804d548","added_by":"auto","created_at":"2025-10-08 08:14:43","extension":"emf","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1790608,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage5.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/0c4d5304f5a6d45e95a885ef.emf"},{"id":93018091,"identity":"f66026d0-cad3-493f-b6cc-6cf298cb6785","added_by":"auto","created_at":"2025-10-08 08:22:43","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1625979,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/8ea5bb29498325680ec43bce.png"},{"id":93015660,"identity":"d5246ec8-c600-4f2d-b43d-d260d291ab63","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"emf","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":217540,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage7.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/0540e3a77213d06d8fa813a8.emf"},{"id":93015655,"identity":"0c1f5186-cee8-471b-9379-b00950457ad7","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1894837,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/9a6d61f0f853ab94268228ca.png"},{"id":93015668,"identity":"8512a497-fb04-4815-aff7-a8c73c85661d","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"emf","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":3846060,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage9.emf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/175cef3d0629a7962c648759.emf"},{"id":93015648,"identity":"5ea9f4ac-b621-4b88-a6e2-976536ec63dd","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":336590,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/19906036ed5d752eecaea1d7.png"},{"id":93015657,"identity":"f5c4503e-f4e0-4885-81ef-e5c99acb5083","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":159588,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/8ff285b3388c96283294277c.png"},{"id":93017592,"identity":"05e00086-ac6e-4b4e-9673-e3b4c8cb86db","added_by":"auto","created_at":"2025-10-08 08:14:42","extension":"png","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":33919,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage11.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/a007f3cfded23d23ec25372b.png"},{"id":93017593,"identity":"6f01be24-f43e-4ef4-bcda-3011f9e51418","added_by":"auto","created_at":"2025-10-08 08:14:42","extension":"png","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":321974,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/96c78ad2291c3e1d00f76d38.png"},{"id":93016576,"identity":"9596d949-e390-4a09-9cde-2e8f857b3735","added_by":"auto","created_at":"2025-10-08 08:06:43","extension":"png","order_by":21,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":200139,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/e34e83337f10bc1713ec886b.png"},{"id":93015662,"identity":"11858182-698d-4e52-a3c3-9b4eb9efbf0b","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":22,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":282599,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/3c617aa1d14847c236c1a8cc.png"},{"id":93015652,"identity":"51be147b-d143-4a71-96af-1d0146fb8fee","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":23,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":328235,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/e6b1607cd0190c9e483b4074.png"},{"id":93015659,"identity":"51f7494e-074f-4d03-9ab5-32142a20c6a7","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":24,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":189027,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/f783345d95c730697fee3d67.png"},{"id":93015663,"identity":"43a6d5b5-7ab9-40a5-98ab-0e90c567f3c6","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":25,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":167118,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/9de052b0276562bdd31549f9.png"},{"id":93015665,"identity":"7bc17977-6a53-4ec7-84da-b51aede61144","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":26,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":226617,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/85bb25baa7d077852dfa8416.png"},{"id":93015661,"identity":"c8a58dc0-ccb5-4dd7-864c-231b6c9b71f2","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"png","order_by":27,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":301142,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/dd85d0b18af888aead76d4e4.png"},{"id":93016578,"identity":"33bc7090-9c8e-4e54-9607-4a904cd3d15d","added_by":"auto","created_at":"2025-10-08 08:06:43","extension":"xml","order_by":28,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":160410,"visible":true,"origin":"","legend":"","description":"","filename":"b7e2ccce70384215bc04dc025a771e4d1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/e99dc225027d36608e92a417.xml"},{"id":93016577,"identity":"e9fb4bb4-9428-46ae-9899-f70548a971fa","added_by":"auto","created_at":"2025-10-08 08:06:43","extension":"html","order_by":29,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":181610,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/f1064fd7bf6f628f174b1b07.html"},{"id":93015626,"identity":"c29a25d9-bb92-4664-a080-30de5406dea7","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":64955,"visible":true,"origin":"","legend":"\u003cp\u003eScreenshots depicting the VR environment during two different trials in which a participant is performing the typing task using the right hand. The superimposed characters on the surface of the virtual hand’s phalanges represent the key-character mapping of the physical keys placed on the phalanges of the participant’s real hand. The word to be typed appears above the right virtual hand as 3D letters. The “trial time remaining” and “trial score” are displayed and updated in real-time on a screen above the 3D word. \u003cstrong\u003e(a)\u003c/strong\u003eThe participant has correctly typed the letter ‘O’; thus, the corresponding 3D letter ‘O’ turns green. \u003cstrong\u003e(b)\u003c/strong\u003e The participant has incorrectly typed the letter ‘E’ instead of ‘O’; consequently, the 3D letter ‘O’ turns red, indicating the error and prompting retyping of the correct letter. Note: Upon successful completion of typing a word, the participant is required to type the “space” character (labelled ‘SP’ on the virtual hand), which triggers the presentation of a new word.\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/cdd1cbd17b944e36a3428a65.jpg"},{"id":93016562,"identity":"a537793a-d4e5-44a3-8956-36f63af1ed36","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":250921,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003e(a)\u003c/strong\u003e Palmar surface of a participant’s hand with copper patches affixed to the finger phalanges. These copper patches function as keys for the TIS. \u003cstrong\u003e(b)\u003c/strong\u003e A participant performing a key press by touching the copper patch on the distal phalanx of the index finger with the copper patch on the tip of the thumb. The TIS detects this contact as a key press. \u003cstrong\u003e(c)\u003c/strong\u003eDorsal surface of a participant’s hand with the BNO055 IMUs of the HKMS affixed to the finger phalanges and to the dorsal hand (i.e., over the metacarpal region) for tracking hand motion. The HKMS-TIS board, containing the five microcontrollers of the HKMS and one microcontroller of the TIS, is also shown. To manage the numerous copper patch wires conveniently, wires associated with a single finger are (1) routed together through a small green tube and (2) stuck to the back of the hand (over the metacarpal region), with sufficient slack to allow for finger movements without tugging on the copper patches.\u003c/p\u003e","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/43c586d30ab9dec59e7c0f36.jpg"},{"id":93015628,"identity":"fe8bd67b-f7bf-4bf1-8067-3d8c78004f19","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":75467,"visible":true,"origin":"","legend":"\u003cp\u003eIllustration of the voltage transitions of a phalanx copper patch when it is touched and released using the thumb copper patch. The functioning of the stable-period algorithm, which detects a stable period of 15 ms following key press and 50 ms following key release, is also illustrated. Additionally, the timing data associated with each key press-release event is transmitted in four parts; the instances at which these four parts are sent are indicated by the blue text in the figure (refer to Supplementary Material Section 4 for more details on TIS data output format).\u003c/p\u003e","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/e7ef6f852bb08ac2c1a6106c.jpg"},{"id":93015630,"identity":"ddd43940-b40f-4d20-ac93-c703f23bf86e","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":118149,"visible":true,"origin":"","legend":"\u003cp\u003eWorking of the refined stable period algorithm to prevent false detections due to noise during the key press duration. The refinement introduced is that after detecting a key release, when the algorithm is monitoring for a key release stable period of 50 ms, it reads the voltage level of the copper patch 2 ms into each candidate stable period. If the voltage is HIGH, the algorithm continues monitoring for a stable period lasting 50 ms. If the voltage is LOW, it indicates that the detected key release was false; hence, the algorithm resets and resumes monitoring for a genuine key release. The refinement introduced to the algorithm is highlighted in orange text. Additionally, the timing data associated with each key press-release event is transmitted in four parts; the instances at which these data parts are transmitted are indicated by the blue text (refer to Supplementary Material Section 4 for more details on TIS data output format).\u003c/p\u003e","description":"","filename":"4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/84036a76fbf14a616b111792.jpg"},{"id":93015636,"identity":"387089b3-412c-43e6-a544-1e2fe292902e","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":198618,"visible":true,"origin":"","legend":"\u003cp\u003eBlock diagram of the HKMS-TIS board, which consists of the HKMS and TIS integrated on a single PCB. The copper patch wires of the TIS are connected to the digital input pins of the TIS microcontroller via screw terminal connectors. The character represented by each copper patch key is also indicated. Two copper patches are connected to the GND pin of the TIS microcontroller – one for the thumb tip and one whose conductive surface will be affixed to the participant's skin. The copper patch wires are color-coded depending on the phalanx (i.e., proximal, middle, or distal) to which they will be attached. The IMU sensors are connected to the master and slave microcontrollers of the HKMS via FFC cables. The master microcontroller of the HKMS communicates with and synchronizes data collection across the TIS microcontroller and the four slave microcontrollers of the HKMS. Additionally, it collects quaternion orientation data from four IMU sensors placed on the dorsal hand (over the metacarpal region) and the middle finger. The master sends a data frame every 10 ms to a computer containing the typing data from the TIS and the quaternion orientation data from the 16 IMU sensors of the HKMS. \u003cstrong\u003eNote:\u003c/strong\u003e The labelling of the copper patches and the IMU sensors in this block diagram is done assuming that the HKMS and TIS will be mounted on the right hand.\u003c/p\u003e","description":"","filename":"5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/498b7ed2a3c36eb30483f748.jpg"},{"id":93015632,"identity":"80f9ffee-7d7b-4ef5-a3ee-ec9061224132","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":182010,"visible":true,"origin":"","legend":"\u003cp\u003eIllustration of the experimental setup, showing the Hand Kinematics Measurement System (HKMS) mounted on the dorsal surface and the Text Input System (TIS) mounted on the palmar surface of both hands of a participant. The participant is also wearing a VR headset. Through this headset, a virtual environment containing the experimental task elements – such as virtual hands and a typing-related interface – is presented.\u003c/p\u003e","description":"","filename":"6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/fec0b456404ac6adc1653ecb.jpg"},{"id":93015644,"identity":"78905b7e-fba2-46ea-ad0f-6f2d1a376362","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":101440,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003e(a)\u003c/strong\u003e Illustration of a vector \u003cem\u003eP\u003c/em\u003e in the right-hand coordinate system of the BNO055 IMU. A BNO055 IMU, along with its local frame of reference, is also shown, positioned in its default orientation. \u003cstrong\u003e(b)\u003c/strong\u003e Illustration of the same vector \u0026nbsp;\u003cem\u003eP\u003c/em\u003e in the left-hand coordinate system of Unity. Note the differences in the coordinate values of \u0026nbsp;\u0026nbsp;in comparison to its representation in the BNO055’s coordinate system. An object in Unity, along with its local frame of reference, is also shown. This object is positioned in its default orientation.\u003c/p\u003e","description":"","filename":"7.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/dd2b17f06ff4c868f95c8121.jpg"},{"id":93016570,"identity":"bc01c8e8-9ed1-472d-a698-af0499169fe5","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"jpg","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":199657,"visible":true,"origin":"","legend":"\u003cp\u003ePhysical alignment of a participant’s hand segments to a custom global frame of reference, as part of the sensor-to-segment alignment procedure. \u003cstrong\u003e(a)\u003c/strong\u003e The four fingers are aligned with the X-axis of the global frame of reference (depicted in the top left corner) by using the parallel-lined sheet as a reference. \u003cstrong\u003e(b) \u003c/strong\u003eThe thumb is aligned with the X-axis of the global frame of reference by using the parallel-lined sheet as a reference. \u003cstrong\u003e(c)\u003c/strong\u003e The side view of the thumb (from part (b)) at the table edge shows the metacarpal maintained in the same horizontal plane as the table surface. \u003cstrong\u003e(d)\u003c/strong\u003e The sides of the thumb’s metacarpal bone are pressed using the experimenter's thumb and index finger to make it visible, while simultaneously aligning the metacarpal and the two thumb phalanges into a straight line. With the thumb held in this position, a straight line was drawn along the center of the thumb, extending from the tip to the base. This line aids the participant in aligning the metacarpal bone of the thumb to the parallel-lined sheet.\u003c/p\u003e","description":"","filename":"8.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/7fe02aaf76e510a876fb8741.jpg"},{"id":93015643,"identity":"e929341d-b76f-4aea-ae7a-da6d105dc583","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"jpg","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":89819,"visible":true,"origin":"","legend":"\u003cp\u003eEffect of sensor-to-segment alignment on virtual hand posture. \u003cstrong\u003e(a)\u003c/strong\u003eVirtual hand posture without sensor-to-segment alignment: The virtual fingers appear crooked because the orientation of the IMU sensors does not accurately represent the orientation of the corresponding hand segments. \u003cstrong\u003e(b)\u003c/strong\u003eVirtual hand posture after applying sensor-to-segment alignment correction: The virtual hands accurately replicate the posture of the real hands, demonstrating the effectiveness of the alignment procedure.\u003c/p\u003e","description":"","filename":"9.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/3d7c49e054521de1547f82bb.jpg"},{"id":93015649,"identity":"ec917756-ace3-4e2a-94e3-977765a3da46","added_by":"auto","created_at":"2025-10-08 07:58:43","extension":"jpg","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":197640,"visible":true,"origin":"","legend":"\u003cp\u003eIllustration of virtual hand postures before and after adjustments to the hand model. The first row shows four postures of the real hand, with the thumb touching a different finger phalanx in each posture. The second row shows the corresponding virtual hand postures prior to model adjustments, where the thumb-to-phalanx touch is not accurately replicated. The third row shows the corresponding virtual hand postures after adjustments to the hand model, successfully replicating the precise thumb-to-phalanx touch seen in the real hand.\u003c/p\u003e","description":"","filename":"10.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/456e1fd859cd447c2195fd2a.jpg"},{"id":93016569,"identity":"8ad95af6-8268-4dcf-bbcc-7446a01dd647","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"jpg","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":205710,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance metrics for the left-hand trials across the four experimental days. The x-axis indicates the specific left hand trials for which the performance metric is plotted. The first plot displays a line graph of the total number of Correct Characters Typed (CCT) per trial. The second and third plots display bar graphs of the average Movement Time (MT) and average Press Duration (PD) per trial, respectively. The error bars represent the standard deviation of the corresponding performance metric within each trial.\u003c/p\u003e","description":"","filename":"11.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/be72073f80c0688c0bc66838.jpg"},{"id":106809040,"identity":"f78efbfc-9184-41b3-9976-480d28f266e6","added_by":"auto","created_at":"2026-04-13 16:05:57","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3284446,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/5057ce7f-040c-47e2-90fa-6ea1c1a16996.pdf"},{"id":93015646,"identity":"062f3ca3-382e-4ecd-b477-ca9d6ea832fa","added_by":"auto","created_at":"2025-10-08 07:58:42","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":7852941,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryMaterialAnuragGupta.docx","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/d6c324bf0195c8ea16805b1d.docx"},{"id":93015669,"identity":"c0ea91a1-9bba-4e64-91d2-2097243eeba1","added_by":"auto","created_at":"2025-10-08 07:58:44","extension":"mp4","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":29820747,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryVideo1.mp4","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/7e5f863e8015f040750ba4ce.mp4"},{"id":93016563,"identity":"2adbfb23-53e8-446e-88f9-e64b697495fa","added_by":"auto","created_at":"2025-10-08 08:06:42","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":14638,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryVideoLegend.docx","url":"https://assets-eu.researchsquare.com/files/rs-6957809/v1/c2b24cdaa8dff9bf73f69a85.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Development of an Experimental Setup to Investigate Mirror Visual Feedback Effects on Cross-Education for an Intricate Finger Sequence Movement Task","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eCross-education (also known as intermanual transfer, interlimb transfer, bilateral transfer, etc.) is a phenomenon in which learning a task with one hand results in an improvement in the performance of the same or similar task in not only the trained hand but also in the untrained hand \u003csup\u003e\u003cspan additionalcitationids=\"CR2\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e. If cross-education can be enhanced, then it has the potential to benefit the rehabilitation of hemiparetic patients whose one side of the body is partially paralyzed. A few studies have explored whether cross-education can be enhanced by practicing a task with one hand while simultaneously observing the contralateral (i.e., opposite) hand performing the same task \u003csup\u003e\u003cspan additionalcitationids=\"CR5 CR6 CR7 CR8\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e. This can be achieved using either a mirror \u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e or a Virtual Reality (VR)-based experimental setup \u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e,\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e. For example, if a mirror is placed along the midsagittal plane of a person, with one hand placed on each side of the mirror, then performing a task with one hand while looking at its mirror reflection gives the illusion that the stationary contralateral (opposite) hand is performing the task. This type of visual feedback is known as Mirror Visual Feedback (MVF) \u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e,\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e. VR setups have been developed to provide a more realistic experience of the movement illusion created by MVF, in which a virtual hand is animated to mirror the movements of the contralateral real hand \u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e,\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\u003cp\u003eSince the first documented evidence of cross-education over a century ago\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e,\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e, its characteristics and underlying neural mechanisms have been extensively investigated across various experimental paradigms\u003csup\u003e\u003cspan additionalcitationids=\"CR13 CR14 CR15 CR16 CR17 CR18 CR19 CR20 CR21 CR22 CR23\" citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e. Within the cross-education literature, several studies have investigated how the simultaneous observation of MVF while practicing a task affects cross-education \u003csup\u003e\u003cspan additionalcitationids=\"CR5 CR6 CR7 CR8\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e. For instance, Mukamel\u0026rsquo;s group developed one of the earliest VR-based setups to provide MVF during a motor sequence learning task in which participants flexed and extended individual fingers one at a time in a predefined sequence (e.g., index, ring, middle, little)\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e,\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e. In their studies using this setup, they provided evidence for the enhancement of cross-education due to MVF. Other studies have investigated the effect of MVF on cross-education for tasks such as a ball rotation task\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e, a ballistic wrist flexion task\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e, a shortening muscle contraction task\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e, and a stationary basketball dribbling task\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e, among others. While these studies have significantly advanced our understanding of MVF's effects on cross-education, many of the motor tasks explored have been relatively simple \u0026ndash; typically involving gross movements, single-joint rotations, or limited fine motor coordination and multi-finger control. In the case of Mukamel's studies\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e,\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e, although the task employed was more complex as it incorporated finger individuation and sequence execution, the movements were still limited to single-joint flexion-extension actions.\u003c/p\u003e\u003cp\u003eThis leads us to some important unanswered questions: Could MVF also enhance cross-education for more complex tasks requiring fine motor control, such as those involving multi-finger coordination and spatially targeted finger movements executed in specific sequences? If yes, would such enhancement be bidirectional \u0026ndash; i.e., from the right hand to the left hand and vice versa? Additionally, what would be the nature of such enhancement, i.e., would only certain performance metrics improve? Or would improvements of certain performance metrics be limited to specific directions of transfer? To address these questions, we built a novel experimental setup to investigate the effects of MVF on cross-education in a fine motor task. This task involves making sequences of intricate finger movements that require multi-finger coordination. Our group previously used a close variant of this task to study long-term motor sequence learning \u003csup\u003e\u003cspan additionalcitationids=\"CR26\" citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\u003cp\u003e More specifically, the task requires participants to touch various finger phalanges one at a time with the tip of the thumb in specific sequences. To implement this task and investigate the potential MVF-mediated enhancement of cross-education for this task, we developed a novel VR-based experimental setup comprising three key components: (1) A Text Input System (TIS) \u0026ndash; a custom-built typing device that consists of keys placed on the palmar surface of the finger phalanges to execute the finger sequence movement task; (2) A Hand Kinematics Measurement System (HKMS) \u0026ndash; a real-time hand and finger movement tracking system based on BNO055 IMUs\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e; and (3) A Virtual Reality (VR) headset to provide either normal or mirror feedback of the virtual hands and to present the task interface. For the task, participants are instructed to type words appearing in the VR world by touching the appropriate TIS keys with the tip of their thumb.\u003c/p\u003e\u003cp\u003eThis paper details the design and development of the novel VR-based experimental setup. Key aspects covered include: describing the unique typing task; developing the TIS; designing and refining a specialized stable period algorithm developed to accurately detect key press and release timestamps while eliminating false detections caused by mechanical and electrical fluctuations; synchronizing and integrating the TIS with the HKMS; synchronizing and integrating the VR system with both the HKMS and TIS; animating virtual hand models in real time using HKMS data; dynamically updating the word interface of the typing task using TIS data; and methods for precisely replicating thumb-to-phalanx touches in virtual hands. This replication is achieved by creating participant-specific virtual hand models and by implementing sensor-to-segment alignment to obtain accurate hand segment orientations. Additionally, the paper explains the quaternion transformations necessary for achieving normal and mirror virtual hand animation within the Unity software environment.\u003c/p\u003e\u003cp\u003eOverall, this device and methods paper describes the design, development, and validation of a novel VR-based experimental setup for investigating the effects of mirror visual feedback on the cross-education of a fine finger movement task. This paper establishes a robust foundation for future studies using the same experimental paradigm. In addition to detailing the complete system architecture of the experimental setup, the paper details several methodologies and concepts that will benefit the broader hand biomechanics and VR research community. These include: methodology for correcting misalignment between sensor and segment reference frames; methodology for achieving accurate hand animation; techniques for system integration and millisecond-level data synchronization; and an intuitive approach for quaternion coordinate transformation. To validate the experimental setup's performance, a preliminary experiment was conducted with two participants. This pilot study demonstrated that the integrated setup functioned reliably and as intended, confirming its suitability for future MVF-based cross-education experiments.\u003c/p\u003e"},{"header":"2. Methodology Part 1: Experimental Setup Design and Development","content":"\u003cp\u003eThis section provides a detailed account of the experimental setup\u0026rsquo;s design and development. It includes the design and functioning of each individual component of the setup, the design and methods used to integrate and synchronize them, and the development and refinement of a critical algorithm associated with one of the components. A detailed description of the experimental task is also included.\u003c/p\u003e\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003e2.1 Experimental Setup Overview\u003c/h2\u003e\u003cp\u003eA novel Virtual Reality (VR)-based experimental setup was developed to investigate the effect of mirror visual feedback on the enhancement of cross-education from the trained to the untrained hand for an intricate finger sequence movement task. This setup consists of three main components:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eText Input System (TIS): A custom typing device used to implement the finger sequence movement task. This is mounted on the palmar surface (front side) of the hand.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eHand Kinematics Measurement System (HKMS): A hand motion-sensing device used to track the motion of the hand and fingers in real time\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e. This is mounted on the dorsal surface of the hand.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eVirtual Reality (VR) headset: This presents a virtual environment containing a pair of virtual hands along with visual elements associated with the experimental task. Data from the HKMS is used to animate the virtual hands, either by replicating the real hand\u0026rsquo;s movements or by providing mirror visual feedback \u0026ndash; wherein the movements of the real hand are mirrored by the contralateral (opposite) virtual hand. Additionally, data from the TIS is used to update a word interface presented as part of the task. The VR headset is worn over the participant's eyes to deliver the virtual environment.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eThese three systems are fully integrated and operate in tight synchrony. Additionally, the hand movement data from the HKMS and the typing data from the TIS are timestamp-synchronized \u0026ndash; an essential requirement for analyzing movement patterns in relation to typing activity.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\u003ch2\u003e2.2 The Task\u003c/h2\u003e\u003cdiv id=\"Sec5\" class=\"Section3\"\u003e\u003ch2\u003e2.2.1 Overview\u003c/h2\u003e\u003cp\u003eThe intricate finger sequence movement task involves performing sequences of finger-thumb opposition movements. Specifically, the participant is required to touch the palmar surface of different finger phalanges with the tip of the thumb (one at a time) in prescribed sequences. To implement this, a key is mounted on the palmar surface of each finger phalanx, and the participant is instructed to type different words by pressing these phalanx-mounted keys with the tip of the thumb.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec6\" class=\"Section3\"\u003e\u003ch2\u003e2.2.2 Detailed Task Description\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eUpon wearing the VR headset, the participant is presented with a virtual environment containing a pair of virtual hands and other visual elements associated with the experimental task. The virtual hands accurately replicate the participant\u0026rsquo;s real hand movements in real-time. Characters are superimposed onto the palmar surfaces of the virtual finger phalanges, resembling tattoos (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). These characters represent the key-character mapping of the physical keys positioned on the participant\u0026rsquo;s real finger phalanges. To type a particular character, the participant must press the corresponding key with the tip of the thumb. In the virtual environment, this corresponds to touching the surface of the virtual phalanx containing the character to be typed with the tip of the virtual thumb (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea).\u003c/p\u003e\u003cp\u003eDuring each trial, 3D words are presented to the participant in the VR environment (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The participant is instructed to type the presented words using a specified hand as quickly as possible and is informed of the hand to be used for typing before each trial. The 3D word to be typed is displayed just above the virtual hand performing the typing movements (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). If a letter is typed correctly, it turns green, signalling the participant to proceed to the next letter. If typed incorrectly, it turns red, signalling the participant to re-attempt typing the letter (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The next letter cannot be typed until the current one is typed correctly. Once all letters of a word are typed correctly, the space character \u0026ndash; labelled \u0026lsquo;SP\u0026rsquo; on the virtual hand \u0026ndash; must be typed to trigger the presentation of the next word.\u003c/p\u003e\u003cp\u003e The accurate real-time replication of the participant\u0026rsquo;s hand movements by the virtual hands and real-time updating of the word interface of the typing task is achieved using data from the HKMS and TIS systems. Detailed description and working of these systems are provided in a subsequent section.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec7\" class=\"Section3\"\u003e\u003ch2\u003e2.2.3 Scoring Scheme\u003c/h2\u003e\u003cp\u003e A simple scoring scheme was implemented to provide performance feedback to participants and encourage improvement across trials. In this scheme, participants are awarded two points for each correctly typed character, including the \"space\" character. No points are deducted for incorrectly typed characters, ensuring that the primary focus of the task remains on typing speed rather than accuracy. However, depending on the specific research question being investigated, both the task instructions and the associated scoring scheme can be modified accordingly. Real-time performance feedback is delivered by updating the participant\u0026rsquo;s score on a screen located just above the 3D word (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). This screen also displays a countdown timer indicating the trial time remaining.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec8\" class=\"Section3\"\u003e\u003ch2\u003e2.2.4 Mirror Feedback\u003c/h2\u003e\u003cp\u003eThere are two animation modes for the virtual hands: 1. Normal animation mode: The virtual hands accurately replicate the movements of the real hands. 2. Mirror animation mode: The movements of the typing hand are mirror-replicated by the opposite (i.e., contralateral) virtual hand. For e.g., if the typing hand is the right hand, then the movements of the right real hand are mirror-replicated by the left virtual hand. Mirror-replicates refers to the hand movements one would observe in the reflection of a mirror placed along the midsagittal plane, with each hand positioned on either side of the mirror. For instance, if the participant rotates the right hand clockwise, the left virtual hand rotates counterclockwise. Similarly, if the right hand rotates towards the body's midline, the left virtual hand also rotates towards the midline. Supplementary Video 1 shows a participant performing the typing task, first with the virtual hands in normal animation mode and then in mirror animation mode.\u003c/p\u003e\u003cp\u003eIn mirror feedback trials with the right hand as the typing hand, the participant has \u003cem\u003eto perform the typing task with the right hand while looking at the left virtual hand for visual feedback\u003c/em\u003e. To prevent the participant from looking at the right virtual hand while typing, its fingers are frozen in a predefined posture. However, even in this fixed posture, the overall orientation of the right virtual hand continuously replicates the real hand\u0026rsquo;s orientation (see Supplementary Video 1), as measured by the dorsal hand sensor placed over the metacarpal region \u0026ndash; details of this sensor are provided in a subsequent Section \u003cspan refid=\"Sec17\" class=\"InternalRef\"\u003e2.7\u003c/span\u003e: \u003cem\u003eHand Kinematics Measurement System (HKMS)\u003c/em\u003e. Additionally, since the participant is looking at the left virtual hand while typing, the 3D word to be typed and the screen displaying the trial score and trial time remaining are positioned above the left virtual hand for easy visual access.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\u003ch2\u003e2.3 Text Input System (TIS)\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eUse\u003c/strong\u003e\u003cp\u003eThe text input system (TIS) is a novel typing device that is used to implement the intricate finger sequence movement task. It consists of 10 keys placed on the palmar surface of the finger phalanges (see Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e for the key-phalanx mapping). To type a character, the participant must touch the corresponding key with the tip of the thumb\u0026rsquo;s distal phalanx by performing a finger-thumb opposition movement.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eDescription\u003c/strong\u003e\u003cp\u003eThe keys of the TIS are actually copper patches affixed to the palmar surface of the finger phalanges (see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Each copper patch is connected to a digital input pin of a microcontroller (Teensy 4.0) via a thin, lightweight wire (28 AWG, Teflon-coated, multistrand). Each digital input pin is internally connected to a pull-up resistor, maintaining the copper patch at 3.3V (HIGH) unless externally pulled to 0V (LOW). A copper patch is also affixed to the tip of the thumb and is connected to the ground pin (0V) of a microcontroller via a lightweight wire, maintaining it at 0V. Additionally, one more copper patch is connected to the ground pin (0V), and its conductive surface is placed in direct contact with the participant\u0026rsquo;s skin. In contrast, the conductive surfaces of the other TIS copper patches (including the thumb patch) are insulated from the skin by an adhesive layer, such that only the outward-facing surface (not in contact with the skin) is electrically active \u0026ndash; either at 3.3V (HIGH) or at 0V (LOW) depending on the connection.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eWorking\u003c/strong\u003e\u003cp\u003eDuring a finger-thumb opposition movement, when the thumb\u0026rsquo;s copper patch makes contact with a phalanx copper patch, the phalanx copper patch voltage transitions from 3.3V to 0V. Upon breaking this contact, the voltage transitions from 0V back to 3.3V. Each digital input pin connected to a phalanx copper patch is mapped to a specific TIS character, and the microcontroller continuously monitors the voltages of these pins. A transition from 3.3V to 0V is detected as a key press, and the corresponding timestamp (in milliseconds (ms)) is recorded. A transition from 0V to 3.3V is detected as a key release, and the corresponding timestamp (in ms) is recorded. The key press and release timestamps are sent from the microcontroller to the HKMS as they are detected, along with the character associated with the digital input pin. The HKMS, in turn, sends this data to a computer.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003ePreventing False Detections Due to Ambient AC Electric Fields\u003c/strong\u003e\u003cp\u003eDuring the typing task, the skin \u0026ndash; particularly from a bent finger segment \u0026ndash; may come into contact with the conductive surface of a phalanx copper patch, occasionally resulting in false key press and release detections. These false detections occur because the skin\u0026rsquo;s voltage oscillates sinusoidally relative to the microcontroller\u0026rsquo;s ground, due to capacitive coupling between the body and ambient 50 Hz electric fields generated by nearby AC power lines. To suppress this effect, the conductive surface of a separate copper patch \u0026ndash; connected to the microcontroller's ground \u0026ndash; is placed in direct contact with the participant\u0026rsquo;s skin, thereby clamping the body\u0026rsquo;s potential to the system\u0026rsquo;s reference voltage. Additionally, participants wear rubber slippers to electrically isolate their bodies from the floor, further reducing the capacitive coupling.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e\u003ch2\u003e2.4 Stable Period Algorithm\u003c/h2\u003e\u003cp\u003eThe TIS uses an algorithm to accurately detect key press and release instances while avoiding false detections. This algorithm is referred to as the stable period algorithm. In this sub-section, the algorithm and the reasoning behind its use are explained.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cdiv id=\"Sec11\" class=\"Section3\"\u003e\u003ch2\u003e2.4.1 Background\u003c/h2\u003e\u003cp\u003eIn general, when a key is pressed, the key\u0026rsquo;s contact surfaces do not form a stable electrical connection immediately. Due to mechanical vibration, they bounce off each other and reconnect several times before a stable physical \u0026ndash; and consequently electrical \u0026ndash; connection is established. This results in multiple voltage transitions between HIGH (3.3V) and LOW (0V) in the key's output signal before stabilizing at LOW. This unstable transition period is referred to as the bounce period (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Similarly, a bounce period occurs during key release, during which the voltage transitions multiple times before settling at HIGH (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eTransitions during the bounce period result in multiple false key press and release detections. A simple method often used to avoid false detections is to ignore any voltage transitions on the microcontroller's input pin for a fixed duration following a key press and key release detection. This fixed duration is called the lockout period and needs to be greater than the bounce period. Because different types of mechanical keys have different bounce durations \u0026ndash; and because key press and release bounce durations may differ even for the same type of key \u0026ndash; bounce periods are typically identified using simple tests, and corresponding lockout durations are hardcoded.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section3\"\u003e\u003ch2\u003e2.4.2 Need for an Alternate Approach\u003c/h2\u003e\u003cp\u003eAs previously described, the TIS keys are copper patches. However, like the traditional mechanical keys, the TIS keys also have bounce periods associated with key press and release events for similar underlying reasons. The lockout period approach described in the preceding section was implemented for the TIS keys. This approach reduced but did not eliminate false key press and release detections. Upon investigation, the two chief reasons for false detections were identified: (1) variability in bounce durations (across different keys and also across time for the same keys) and (2) random voltage transitions occurring during the press period. A detailed account of the investigations into the lockout period approach is provided in the Supplementary Material Sections \u003cspan refid=\"Sec1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\u003cp\u003eThe bounce variability and random voltage transitions are due to a combination of various factors. Firstly, sticking the thin copper patches on the finger phalanges results in unique uneven surfaces for each copper patch. Secondly, the portion and surface area of the thumb copper patch making contact with the phalanx copper patch vary across different phalanx copper patches. Thirdly, while typing, the nature of the physical interaction between the contacting surfaces of the copper patches is different every time. For example, sometimes physical sliding occurs between the copper patches, while at other times, there is no sliding at all. When sliding does occur, the amount of slide can vary. Collectively, these factors contribute to variability in both the mechanical properties of the contacting surfaces and the physical nature of contact between the surfaces. This results in fluctuations in bounce periods and interruptions in stable electrical contact. Interruption in stable electrical contact, in turn, leads to random voltage transitions during the press duration. In conclusion, using a fixed lockout period is not the best approach to remove false detections in the TIS.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec13\" class=\"Section3\"\u003e\u003ch2\u003e2.4.3 Stable Period Algorithm Working\u003c/h2\u003e\u003cp\u003eWhile there are several ways to solve the false detection problem, we decided to take the programmatic approach. To tackle the problem arising due to variability in the bounce periods, an algorithm called the stable period algorithm was implemented. This algorithm filters out the bounce period transitions without using fixed lockout periods. In this new approach, the algorithm monitors for a stable period of 15 ms following a key press. This stable period is a time span during which there are no transitions in the voltage of the copper patch. Once this key press stable period is detected, the algorithm starts monitoring for key release. Upon detection of the key release, the algorithm monitors for another stable period that lasts for 50 ms. Once this key release stable period is detected, the algorithm starts monitoring for the next key press, and the cycle continues. The stable period durations of 15 ms for key press and 50 ms for key release were determined empirically based on testing. The working of the stable period algorithm is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec14\" class=\"Section3\"\u003e\u003ch2\u003e2.4.4 Interpretation of Key Release Instance\u003c/h2\u003e\u003cp\u003eThe onset of the key release bounce period indicates the initiation of movement to remove contact between the copper patches and is interpreted as the key release instance by the stable period algorithm (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Alternatively, the final voltage transition at the end of the bounce period, which indicates complete separation of the copper patches, can also be interpreted as the key release instance. The choice between these two interpretations depends on the nature of the analysis. For example, if in the current experimental setup, synchronized EEG data is also collected, then the onset of the bounce period (reflecting movement initiation) may serve as a more meaningful key release marker, as it better corresponds to the timing of motor intention and cortical activation.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e\u003ch2\u003e2.5 Stable Period Algorithm Refined\u003c/h2\u003e\u003cp\u003eInitial testing of the stable period algorithm revealed occasional false key press and release detections, which in some cases also caused the algorithm to fall out of sync with the actual typing events. This loss of synchronization, in turn, led to errors in detecting subsequent key press and release events. Upon investigation, two primary causes were identified: (1) Persistent voltage transitions throughout the entire key press duration, and (2) Random voltage fluctuations during the key press duration. A detailed explanation of how these two issues affect the functioning of the stable period algorithm is provided in the Supplementary Material Section \u003cspan refid=\"Sec26\" class=\"InternalRef\"\u003e3\u003c/span\u003e. The current section describes the refinements made to the algorithm to address these two issues.\u003c/p\u003e\u003cp\u003e\u003cb\u003eRefinement 1 \u0026ndash; Handling Persistent Voltage Transitions\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWhen a key press stable period of 15 ms is detected, the algorithm simultaneously checks the voltage level of the corresponding copper patch. If the voltage level is LOW (0V), the algorithm continues functioning as usual and begins monitoring for the key release. However, if the voltage level is HIGH (3.3V), it indicates that the detected 15 ms stable period is actually a portion of the key release stable period, and that the actual key press stable period did not occur due to persistent voltage transitions throughout the key press duration. Consequently, the key release was also not detected.\u003c/p\u003e\u003cp\u003eIn such cases, the algorithm resets and begins monitoring for a key release stable period of 50 ms. Specifically, it monitors for an additional 35 ms of stable signal (to complete 50 ms), unless an intervening voltage transition occurs \u0026ndash; in which case, monitoring for the full 50 ms stable period restarts from the new transition point. In cases where the entire key press duration is noisy, the timestamp of the final voltage transition during the key release bounce period is treated as the actual key release instance.\u003c/p\u003e\u003cp\u003e\u003cb\u003eRefinement 2 \u0026ndash; Handling Random Voltage Fluctuations\u003c/b\u003e\u003c/p\u003e\u003cp\u003eIn the original version of the algorithm, once a key release was detected, the algorithm would begin monitoring for a key release stable period of 50 ms. In the refined version, a check is incorporated during this monitoring phase: the voltage level of the copper patch is read 2 ms into each candidate stable period. If the voltage level is HIGH (3.3V), the algorithm continues monitoring for a stable period of 50 ms. If the voltage level is LOW (0V), it indicates that the key is still pressed and that random noise was falsely detected as a key release. In such cases, the algorithm resets and resumes monitoring for a genuine key release.\u003c/p\u003e\n\u003cp\u003eFig. 4 illustrates the working of the refined algorithm, with the orange text highlighting the implemented refinement. As depicted in the figure, this updated approach effectively filters out both (1) isolated voltage spikes and (2) noisy segments where the voltage temporarily remains HIGH for a short duration before returning to LOW. Note that while the refined algorithm monitors for a key release stable period of 50 ms, it may encounter multiple shorter stable periods of varying durations. The voltage check 2 ms into the stable period is performed for each of these stable periods, as illustrated in Fig. 4.\u0026nbsp;\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\u003ch2\u003e2.6 Data Output Format of the TIS\u003c/h2\u003e\u003cp\u003eThe TIS outputs various timing-related data for each key press\u0026ndash;release event. This data includes: key press and release timestamps, press duration, bounce durations for key press and release, timing data related to key press and release stable periods, among other relevant data. All this data is output at different time instances, as and when detected by the stable period algorithm. A detailed account of the TIS data and its output format is provided in the Supplementary Material Section \u003cspan refid=\"Sec34\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\u003ch2\u003e2.7 Hand Kinematics Measurement System (HKMS)\u003c/h2\u003e\u003cp\u003e\u003cstrong\u003eUse\u003c/strong\u003e\u003cp\u003eThe Hand Kinematics Measurement System (HKMS) is used to track the motion of both hands in real time. Specifically, the HKMS records the orientation data \u0026ndash; in the form of quaternions \u0026ndash; of the hand and all the finger phalanges at 100 Hz. A detailed description of the design and validation of the HKMS is provided in our previously published work\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eDescription\u003c/strong\u003e\u003cp\u003eThis device consists of 16 IMU sensors (BNO055, Bosch Sensortec) and five microcontrollers (Teensy 4.0; see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ec). The IMU sensors (1.3 cm x 1 cm) are affixed to the dorsal surface of all the finger phalanges and the dorsal hand (i.e., over the metacarpal region). These sensors output orientation data in the form of quaternions. The three IMU sensors on each finger are connected in series using lightweight FFC cables (see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003ec). An additional fourth IMU is connected in series with the three IMUs on the middle finger and is placed on the dorsal hand (over the metacarpal region). The five microcontrollers are connected in the master\u0026ndash;slave configuration on a single PCB. The master microcontroller is connected via an FFC cable to the four IMUs in series, located on the dorsal hand and middle finger. Each of the four slave microcontrollers is connected via an FFC cable to the three IMUs in series, located on a single finger.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eWorking\u003c/strong\u003e\u003cp\u003eThe HKMS operates on a synchronized data collection mechanism. Every 10 ms, the master microcontroller sends a synchronization signal simultaneously to all four slave microcontrollers. Upon receiving this signal, the four slave microcontrollers simultaneously start collecting quaternion orientation data from the IMUs to which they are connected (it takes approximately 2 ms for a microcontroller to get quaternion data from a single IMU). Meanwhile, after sending the synchronization signal, the master microcontroller starts collecting quaternion orientation data from the four IMUs connected to it and then waits for the data from the slaves to arrive. Once the data from all the slave microcontrollers has been received, the master sends the collected quaternion data from all 16 IMU sensors to a computer via USB. This entire process is completed within 10 ms, allowing the master to send the next synchronization signal on schedule, thereby maintaining a continuous data output rate of 100 Hz. The set of data collected during a single cycle is referred to as a \u003cb\u003edata frame\u003c/b\u003e. Additionally, the timestamp at which the master sends the synchronization signal is recorded, and the orientation data from all 16 IMUs in that cycle is associated with this single timestamp (referred to as the \u003cb\u003edata frame timestamp\u003c/b\u003e).\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eHKMS Operating Modes\u003c/strong\u003e\u003cp\u003eThe HKMS operates in three distinct modes\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003col style=\"list-style-type:lower-roman;\"\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eOrientation mode\u003c/b\u003e: In this mode, the HKMS outputs quaternion orientation data from all 16 IMU sensors at 100 Hz. The output also contains the data frame timestamp and the frame count (i.e., the total number of data frames output).\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eCalibration mode\u003c/b\u003e: In this mode, the HKMS outputs the calibration status of all 16 IMU sensors at 100 Hz. Details regarding the calibration of the BNO055 IMUs and the calibration of the HKMS as a whole can be found in our previously published work\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eIdle mode\u003c/b\u003e: In this mode, the HKMS does not transmit any data.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eThe operating mode of the HKMS can be configured by sending it specific character commands from the computer: \u0026ldquo;O\u0026rdquo; for Orientation mode, \u0026ldquo;C\u0026rdquo; for Calibration mode, and \u0026ldquo;X\u0026rdquo; for Idle mode. These commands allow users to configure the HKMS output according to experimental requirements. Additionally, the timestamp at which the HKMS receives the \u0026ldquo;O\u0026rdquo; command is recorded. This timestamp serves as the new reference time (i.e., zero time) for all subsequent HKMS timing calculations.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e\u003ch2\u003e2.8 Integration of HKMS and TIS\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eTo synchronize data from the HKMS and TIS, both systems were integrated together on a single printed circuit board (PCB) referred to as the HKMS-TIS board (see Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). A communication protocol was implemented for data exchange between the two systems. The HKMS-TIS board outputs the combined data of the HKMS and TIS as a single data frame every 10 ms. Additionally, the board contains screw terminals that are connected to the digital input pins of the TIS microcontroller. The copper patch wires of the TIS are connected to these screw terminals (see Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e).\u003c/p\u003e\u003cdiv id=\"Sec19\" class=\"Section3\"\u003e\u003ch2\u003e2.8.1 Communication Protocol\u003c/h2\u003e\u003cp\u003eThe HKMS and TIS communicate with each other serially (UART protocol). During each HKMS data collection cycle, the master microcontroller of the HKMS, after receiving data from four slave microcontrollers, sends a data request to the TIS microcontroller (by transmitting the \u0026ldquo;\u003cspan\u003e$\u003c/span\u003e\u0026rdquo; character). The TIS microcontroller, which continuously collects and concatenates typing data into a string, responds to the \u0026ldquo;\u003cspan\u003e$\u003c/span\u003e\u0026rdquo; data request by sending to the master whatever typing data has accumulated since the previous request. All the timing data of the TIS is referenced relative to the TIS microcontroller's internal timer. The HKMS master, upon receiving data from the TIS microcontroller, merges both the HKMS and TIS data into a single data frame. This data frame is then transmitted to a computer via USB. This entire data collection and transmission process is completed within 10 ms.\u003c/p\u003e\u003cp\u003eThe format of the data frame output by the HKMS-TIS board is as follows:\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cspan\u003e$\u003c/span\u003e\u0026rdquo; \u0026ldquo;HKMS Data\u0026rdquo; \u0026ldquo;\u0026amp;\u0026rdquo; \u0026ldquo;TIS Data\u0026rdquo; \u0026ldquo;*\u0026rdquo; \u0026ldquo;#\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eHere, \u0026ldquo;\u003cspan\u003e$\u003c/span\u003e\u0026rdquo; and \u0026ldquo;#\u0026rdquo; indicate the start and end of the combined HKMS-TIS data frame. \u0026ldquo;\u0026amp;\u0026rdquo; and \u0026ldquo;*\u0026rdquo; indicate the start and end of the TIS data within the HKMS-TIS data frame. If the TIS has no data to share, then it just sends \u0026ldquo;\u0026amp;*\u0026rdquo; to the HKMS master. These delimiting characters are used to parse the data frame at the computer's end.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec20\" class=\"Section3\"\u003e\u003ch2\u003e2.8.2 Timestamp Synchronization\u003c/h2\u003e\u003cp\u003eThe timestamp assigned to the HKMS data frame (i.e., the time at which the master sends the synchronization signal to the slaves) serves as the timestamp for the combined HKMS-TIS data frame. During post-processing, this timestamp can be used to link the TIS data to the corresponding HKMS data with which it was transmitted. For example, to extract kinematic data between two consecutive key presses, the timestamps of the HKMS-TIS data frames can be used to identify the HKMS data frames corresponding to the two key press events. Once these frames are located, all intermediate HKMS data frames can be extracted for further analysis. This data-linking method ensures precise temporal alignment of hand kinematic data with typing events, thereby enabling analysis of movement patterns in relation to typing activity.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec21\" class=\"Section3\"\u003e\u003ch2\u003e2.8.3 Synchronizing the TIS with the HKMS Operating Modes\u003c/h2\u003e\u003cp\u003eThe TIS activity is synchronized with the HKMS\u0026rsquo;s operating modes to ensure coordinated operation of both systems. This synchronization is implemented in the following way: whenever the HKMS receives a command to switch operating modes (\u0026ldquo;O,\u0026rdquo; \u0026ldquo;C,\u0026rdquo; or \u0026ldquo;X\u0026rdquo;), it immediately transmits the same command to the TIS. Based on the command received, the TIS adjusts its activity as follows:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eIf the \u0026ldquo;O\u0026rdquo; command is received, the TIS monitors the digital input pins for typing events. Additionally, the timestamp at which the \u0026ldquo;O\u0026rdquo; command is received is recorded, and all variables associated with each key of the TIS are reset to their default values. This recorded timestamp serves as the new reference time (i.e., time zero) for all subsequent timing calculations by the TIS.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eIf the \u0026ldquo;C\u0026rdquo; or \u0026ldquo;X\u0026rdquo; command is received, then the TIS remains idle.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eThis mechanism ensures that both the TIS and HKMS operate in tandem.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec22\" class=\"Section2\"\u003e\u003ch2\u003e2.9 Virtual Reality (VR) system\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eUse\u003c/b\u003e: A VR system is used to implement the virtual environment described in Section \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e: \u003cem\u003eThe Task\u003c/em\u003e.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eDescription\u003c/strong\u003e\u003cp\u003eThe participant wears a VR headset (Meta Quest 2) during the experiment trials (see Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e). The headset is connected to a computer via a high-speed and lightweight USB-C cable. The computer is running a PC-VR application that generates and manages the virtual environment that is displayed through the headset.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eWorking\u003c/strong\u003e\u003cp\u003eThe PC-VR application was built using the Unity game engine (version 2020.3.31f1). It accepts and processes data from the HKMS-TIS boards of both hands in real-time. The HKMS data is used to animate a pair of virtual hands, while the typing data is used to update the word interface of the typing task. Additionally, the application controls the operating modes of the HKMS and also synchronizes the data from both HKMS-TIS boards.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec23\" class=\"Section2\"\u003e\u003ch2\u003e2.10 Unity Based PC-VR Application\u003c/h2\u003e\u003cp\u003eA PC-VR application was developed using the Unity software. This application simultaneously receives real-time data from the HKMS-TIS boards of both hands, parses this data (i.e., converts it into a structured format that can be easily used, analyzed, and stored), and uses it in real-time to animate virtual hand models and update the word interface during the typing task. Handling incoming 100 Hz data in real-time from both HKMS-TIS boards presents technical challenges in Unity. A detailed explanation of how background threads were employed to achieve real-time data reception and handling within the VR application is provided in Supplementary Material Section \u003cspan refid=\"Sec35\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e\u003cp\u003eThis section describes the different operating modes of the VR application and how they are synchronized with the corresponding operating modes of the HKMS-TIS boards to ensure coordinated functioning among the three components of the experimental setup.\u003c/p\u003e\u003cdiv id=\"Sec24\" class=\"Section3\"\u003e\u003ch2\u003e2.10.1 Modes of Operation\u003c/h2\u003e\u003cp\u003eSimilar to the HKMS, the VR application operates in three distinct modes: (1) Orientation Mode, (2) Calibration Mode, and (3) Idle Mode. Depending on the experimental requirements, the application can be set to operate in one of these modes. The following is a description of these modes:\u003c/p\u003e\u003cp\u003e\u003cb\u003e1. Orientation Mode\u003c/b\u003e: In this mode, the application receives both orientation and typing data from the HKMS-TIS boards of both hands. If a trial is in progress, then the typing data from the hand designated for typing during that trial is used to update the word interface of the typing task in real-time. Concurrently, the orientation data from both hands is used to animate the virtual hand models in real-time. Depending on the experimental requirements, the virtual hand animation can be implemented in one of two ways:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eNormal Animation: Both virtual hands accurately replicate the movements of their corresponding real hands (see Supplementary Video 1).\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMirror Animation: One virtual hand mirror-replicates the movements of the opposite real hand while the other virtual hand\u0026rsquo;s fingers are fixed in a pre-defined posture (for more details regarding mirror animation, refer to Section \u003cspan refid=\"Sec8\" class=\"InternalRef\"\u003e2.2.4\u003c/span\u003e: \u003cem\u003eMirror Feedback\u003c/em\u003e). Mirror animation can be implemented in either of the following ways: (1) The left virtual hand mirror-replicates the movements of the right real hand while the right virtual hand remains fixed in a pre-defined posture (see Supplementary Video 1). (2) The right virtual hand mirror-replicates the movements of the left real hand while the left virtual hand remains fixed in a pre-defined posture. Note: During the typing task, the pre-defined posture prevents the participant from using the fixed virtual hand for feedback, thereby forcing them to focus on the virtual hand that mirror-replicates the movements of the typing hand.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003e2. Calibration Mode\u003c/b\u003e: In this mode, the application receives the calibration status from all the IMU sensors of both hands and displays it on the system monitor for the experimenter to review (see Supplementary Figure S9). This mode enables the experimenter to ensure that all IMU sensors are calibrated before the start of the experiment and to periodically check the calibration status of the sensors in between trials.\u003c/p\u003e\u003cp\u003e\u003cb\u003e3. Idle Mode\u003c/b\u003e: In this mode, the application does not accept any data from the HKMS-TIS boards.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec25\" class=\"Section3\"\u003e\u003ch2\u003e2.10.2 Synchronization\u003c/h2\u003e\u003cp\u003eThe VR application controls the operating modes of the HKMS by sending it appropriate commands (\u0026ldquo;O,\u0026rdquo; \u0026ldquo;C,\u0026rdquo; or \u0026ldquo;X\u0026rdquo;). To synchronize the operating mode between the HKMSs of both hands, the application sends identical commands to both the HKMS-TIS boards in quick succession. This ensures that both the HKMSs switch their operating mode nearly simultaneously.\u003c/p\u003e\u003cp\u003eAdditionally, it is critical for the VR application to synchronize its operating mode with both the HKMSs to ensure correct data parsing. For example, if the application is in Orientation mode and it sends a \u0026ldquo;C\u0026rdquo; command to switch the HKMSs to Calibration mode, it must not switch its own mode immediately. If it does, any remaining unparsed Orientation data in the serial buffer will be incorrectly interpreted as Calibration data, leading to parsing errors.\u003c/p\u003e\u003cp\u003eA simple synchronization mechanism was implemented to address this issue. Whenever the VR application sends a command to an HKMS-TIS board, the HKMS master immediately echoes this command back to the application before switching its own operating mode. For example, upon receiving a \"C\" command from the application, the HKMS-TIS board transmits \"C\" back to the application. At the application's end, parsing continues in the previous operating mode until this echoed confirmation is received. Once received, the application recognizes that subsequent incoming data frames correspond to the new operating mode and adjusts its parsing method accordingly. This synchronization process occurs independently for each HKMS-TIS board through its respective data collection threads (see Supplementary Material Section \u003cspan refid=\"Sec35\" class=\"InternalRef\"\u003e5\u003c/span\u003e for more on data collection using threads).\u003c/p\u003e\u003cp\u003eFurthermore, at the beginning of each experimental trial, an \"O\" command is sent to both HKMS-TIS boards regardless of whether they are already in Orientation mode. Sending \u0026ldquo;O\u0026rdquo; resets internal timing variables within each board, and the subsequent data frames received after sending this command are used for the trial.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"3. Methodology Part 2: Quaternion Transformations and Virtual Hand Calibration","content":"\u003cp\u003eThis section outlines the methodologies critical to implementing the experimental task. These include an intuitive approach for manipulating quaternions to achieve coordinate transformation and mirror animation; a method for correcting sensor-to-segment misalignment to ensure accurate hand segment orientations; and a procedure for constructing and refining participant-specific virtual hand models to accurately replicate the real hand\u0026rsquo;s thumb-to-phalanx touches. A detailed description of the virtual hand model and its animation method is also included.\u003c/p\u003e\u003cdiv id=\"Sec27\" class=\"Section2\"\u003e\u003ch2\u003e3.1 Quaternion Transformations and Virtual Hand Model Structure\u003c/h2\u003e\u003cp\u003eThis section presents an intuitive, axis-angle\u0026ndash;based approach to manipulating quaternions for implementing (1) quaternion coordinate transformations and (2) mirror animation. It also describes the hierarchical structure of the virtual hand model and how segment-wise quaternion data is used to animate it in real time.\u003c/p\u003e\u003cdiv id=\"Sec28\" class=\"Section3\"\u003e\u003ch2\u003e3.1.1 Quaternion Coordinate Transformation\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eFor an object in Unity\u0026rsquo;s 3D world to accurately replicate the BNO055 IMU\u0026rsquo;s real-world orientation, it is necessary to transform the quaternion from the IMU\u0026rsquo;s coordinate system to Unity\u0026rsquo;s coordinate system. To implement this transformation, we first need to understand the coordinate systems of the BNO055 IMU and Unity. The IMU operates in a right-hand coordinate system (see Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e (a)), and it outputs absolute orientation data with respect to the Earth's East-North-Up (ENU) frame of reference (while operating in the NDOF sensor fusion mode\u003csup\u003e\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u003c/sup\u003e). In this system, the \u003cem\u003e+\u0026thinsp;X\u003c/em\u003e, \u003cem\u003e+Y\u003c/em\u003e, and \u003cem\u003e+\u0026thinsp;Z\u003c/em\u003e coordinate axes align with the Earth's east, north, and up directions, respectively. In contrast, Unity uses a left-hand coordinate system where the \u003cem\u003e+\u0026thinsp;X\u003c/em\u003e, \u003cem\u003e+Y\u003c/em\u003e, and \u003cem\u003e+\u0026thinsp;Z\u003c/em\u003e axes point to the right, up, and forward directions, respectively, within Unity\u0026rsquo;s 3D space (see Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e (b)).\u003c/p\u003e\u003cp\u003eIf the IMU is positioned such that the \u003cem\u003ex-\u003c/em\u003e, \u003cem\u003ey-\u003c/em\u003e, and \u003cem\u003ez-\u003c/em\u003eaxes of its local frame of reference are aligned with the Earth's east, north, and up directions, respectively, then the IMU is considered to be in its default orientation (see Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e (a)). Similarly, a Unity object in its default orientation is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e (b). The objective is to ensure that any movements of the IMU from its default orientation are accurately replicated by a Unity object that begins in its own default orientation. To achieve this, the IMU\u0026rsquo;s quaternion must be transformed from the IMU\u0026rsquo;s coordinate system to Unity\u0026rsquo;s coordinate system before being applied to a Unity object.\u003c/p\u003e\u003cp\u003eTo derive this transformation, it is first necessary to understand the axis-angle representation of quaternions. Consider a unit quaternion \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:q=\\left(x,\\:y,\\:z,\\:w\\right)\\)\u003c/span\u003e\u003c/span\u003e. This quaternion represents an orientation that can be obtained by rotating through an angle \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\theta\\:\\)\u003c/span\u003e\u003c/span\u003e about a rotation axis defined by the vector \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P=\\left(a,\\:b,\\:c\\right)\\)\u003c/span\u003e\u003c/span\u003e. Here, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\theta\\:\\)\u003c/span\u003e\u003c/span\u003e can be derived from the quaternion \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:q\\)\u003c/span\u003e\u003c/span\u003e using the following relation\u003csup\u003e\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e:\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\:q=\\left(x,\\:y,\\:z,\\:w\\right)=(a\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right),\\:b\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right),\\:c\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right),\\:\\text{cos}\\left(\\frac{\\theta\\:}{2}\\right))$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equb\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$$\\:\\theta\\:=2co{s}^{-1}\\left(w\\right)\\:$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equc\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$$\\:P=\\left(a,\\:b,\\:c\\right)=(\\frac{x}{\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right)},\\:\\frac{y}{\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right)},\\:\\frac{z}{\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right)})$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eWe exploit this understanding of the axis-angle representation to derive the coordinate transformation. This is done in two steps:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eFor the IMU unit quaternion \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{IMU}=\\left(x,\\:y,\\:z,\\:w\\right)\\)\u003c/span\u003e\u003c/span\u003e, the rotational axis \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\)\u003c/span\u003e\u003c/span\u003e in the IMU\u0026rsquo;s frame needs to be represented in the Unity frame. This is done as follows: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{P}_{Unity}\\left(-b,\\:c,\\:a\\right)=\\:{P}_{IMU}\\left(a,\\:b,\\:c\\right)\\)\u003c/span\u003e\u003c/span\u003e i.e., the vector \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P=\\left(a,\\:b,\\:c\\right)\\)\u003c/span\u003e\u003c/span\u003e in the IMU frame can be represented as \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P=\\left(-b,\\:c,\\:a\\right)\\)\u003c/span\u003e\u003c/span\u003e in the Unity frame (see Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e). This is because Unity\u0026rsquo;s \u003cem\u003e+X\u003c/em\u003e, \u003cem\u003e+Y\u003c/em\u003e, and \u003cem\u003e+Z\u003c/em\u003e coordinate axes are in the \u003cem\u003e\u0026ndash;Y\u003c/em\u003e, \u003cem\u003e+Z\u003c/em\u003e, and \u003cem\u003e+X\u003c/em\u003e directions of the IMU\u0026rsquo;s coordinate axes, respectively. In quaternion form, this corresponds to transforming \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{IMU}\\)\u003c/span\u003e\u003c/span\u003e into \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}\\)\u003c/span\u003e\u003c/span\u003e as follows: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}=(-y,\\:z,\\:x,\\:w)\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eThe next step is to ensure that the rotation \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\theta\\:\\)\u003c/span\u003e\u003c/span\u003e about the rotational axis \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\:\\)\u003c/span\u003e\u003c/span\u003eis in the same direction in both the IMU and Unity coordinate systems. A positive rotation about \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\:\\)\u003c/span\u003e\u003c/span\u003ein the right-hand coordinate system of the IMU is equivalent to a negative rotation about \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\:\\)\u003c/span\u003e\u003c/span\u003ein the left-hand coordinate system of Unity. Therefore, to maintain consistency in rotation direction, the rotation angle \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\theta\\:\\)\u003c/span\u003e\u003c/span\u003e in \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}\\:\\)\u003c/span\u003e\u003c/span\u003emust be negated before applying it in Unity\u0026rsquo;s frame.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003cdiv id=\"Equd\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e\n$$\\:{q}_{Unity}=\\left(-b\\text{sin}\\left(\\frac{-\\theta\\:}{2}\\right),\\:c\\text{sin}\\left(\\frac{-\\theta\\:}{2}\\right),\\:a\\text{sin}\\left(\\frac{-\\theta\\:}{2}\\right),\\:\\text{cos}\\left(\\frac{-\\theta\\:}{2}\\right)\\right)$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Eque\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Eque\" name=\"EquationSource\"\u003e\n$$\\:{q}_{Unity}=\\left(b\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right),\\:-c\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right),\\:-a\\text{sin}\\left(\\frac{\\theta\\:}{2}\\right),\\:\\text{cos}\\left(\\frac{\\theta\\:}{2}\\right)\\right)$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\:{q}_{Unity}=\\left(y,\\:-z,\\:-x,\\:w\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\:{q}_{Unity}=conjugate\\left({q}_{1}\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eThe incoming IMU data from the HKMS, after coordinate conversion as defined in Eq.\u0026nbsp;(\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e), can be directly applied to the virtual hand model for animation. However, a limitation of this approach is that the participant would need to physically face the Earth's magnetic north for the virtual hands to be oriented in the forward direction within Unity\u0026rsquo;s 3D space. This requirement is impractical in a typical experimental setting. To address this issue, a custom global frame of reference is defined within the experimental room. The raw IMU quaternion, which represents the sensor orientation with respect to the ENU frame of reference, is transformed to represent the sensor orientation with respect to this room-specific global frame of reference instead. The coordinate conversion defined in Eq.\u0026nbsp;(\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) is then applied to this transformed quaternion and used for animating the virtual hands. Now, for the virtual hands to be oriented in the forward direction in Unity\u0026rsquo;s 3D space, the participant simply needs to align themselves with the positive \u003cem\u003eX\u003c/em\u003e-axis of the room\u0026rsquo;s global frame of reference, rather than aligning themselves to the Earth\u0026rsquo;s magnetic north. A detailed explanation of how the room-specific global frame of reference is established and how the raw IMU quaternion is transformed to represent the sensor orientation with respect to this global frame of reference is provided in Section \u003cspan refid=\"Sec32\" class=\"InternalRef\"\u003e3.3.1\u003c/span\u003e: \u003cem\u003eSensor-To-Segment Alignment\u003c/em\u003e.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec29\" class=\"Section3\"\u003e\u003ch2\u003e3.1.2 Quaternion Transformation for Mirror Animation\u003c/h2\u003e\u003cp\u003eAs previously described, mirror animation involves the movements of the typing hand being mirrored by the opposite (contralateral) virtual hand. For a more detailed explanation, refer to Section \u003cspan refid=\"Sec8\" class=\"InternalRef\"\u003e2.2.4\u003c/span\u003e: \u003cem\u003eMirror Feedback\u003c/em\u003e. The goal is to achieve mirror animation about the participant's midsagittal plane. Throughout the experiment, the participant will remain seated, with their forward view in the VR environment aligned along Unity\u0026rsquo;s\u0026thinsp;\u003cem\u003e+\u0026thinsp;Z\u003c/em\u003e axis. Hence, to achieve mirror animation, the orientations must be mirrored about Unity\u0026rsquo;s \u003cem\u003eYZ\u003c/em\u003e plane. To achieve this, we again exploit our understanding of the quaternion\u0026rsquo;s axis-angle representation to transform the quaternion appropriately.\u003c/p\u003e\u003cp\u003eLet \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{Unity}=\\left(x,\\:y,\\:z,\\:w\\right)\\)\u003c/span\u003e\u003c/span\u003e be a unit quaternion after coordinate conversion using Eq.\u0026nbsp;\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. As per the axis-angle representation, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{Unity}\\)\u003c/span\u003e\u003c/span\u003e represents a rotation by \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\theta\\:\\)\u003c/span\u003e\u003c/span\u003e about a rotation axis defined by the vector \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P=\\left(a,\\:b,\\:c\\right)\\)\u003c/span\u003e\u003c/span\u003e. The mirror orientation is achieved in two steps:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eReflecting the Rotation Axis: The rotation axis \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P\\)\u003c/span\u003e\u003c/span\u003e is reflected about the \u003cem\u003eYZ\u003c/em\u003e plane by negating its x-component, resulting in \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P{\\prime\\:}=\\left(-a,\\:b,\\:c\\right)\\)\u003c/span\u003e\u003c/span\u003e. In quaternion form, this corresponds to transforming \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{Unity}\\)\u003c/span\u003e\u003c/span\u003e into \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}\\)\u003c/span\u003e\u003c/span\u003e as follows: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}=(-x,\\:y,\\:z,\\:w)\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eReversing the Rotation Direction: To achieve the mirror orientation, the rotation about the reflected axis \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:P{\\prime\\:}\\)\u003c/span\u003e\u003c/span\u003e must be reversed, that is, rotation by an angle of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:-\\theta\\:\\)\u003c/span\u003e\u003c/span\u003e. The reason for doing this can be understood through the following example: if a hand is rotated clockwise, its image in a mirror kept along the midsagittal plane rotates counterclockwise. To implement this reversal of rotation direction mathematically, we take the quaternion conjugate of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}\\)\u003c/span\u003e\u003c/span\u003e: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{mirror}=conjugate\\left({q}_{1}\\right)\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eFor real-time mirror animation, the quaternion data from the incoming HKMS data frames of one hand are converted to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{mirror}\\)\u003c/span\u003e\u003c/span\u003e and then applied to the corresponding segments of the opposite (i.e., contralateral) virtual hand model.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec30\" class=\"Section2\"\u003e\u003ch2\u003e3.2 Virtual Hand Model Structure and Real-Time Animation\u003c/h2\u003e\u003cp\u003eThe virtual hand model adopts a hierarchical structure. At the root of this hierarchy is a cuboid (see Figs.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, \u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e, and \u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e) that represents the four metacarpal bones and all the carpals as a single unit. This cuboid is located approximately 2 cm below the middle virtual finger and replicates the orientation of the \u0026ldquo;dorsal hand sensor\u0026rdquo; of the HKMS (affixed on the dorsal surface of the real hand, a few cm below the middle finger\u0026rsquo;s MCP joint). The four virtual fingers and the virtual thumb are positioned at anatomically appropriate locations around the cuboid but are not visually connected to it (see Figs.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, \u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e, and \u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e). However, the bases of these virtual fingers and the virtual thumb are invisibly linked to the cuboid such that any change in the cuboid's orientation appropriately moves the spatial location of these bases.\u003c/p\u003e\u003cp\u003eFor each of the four virtual fingers, the base of the proximal phalanx is invisibly linked to the cuboid at a point corresponding to the metacarpophalangeal (MCP) joint, the middle phalanx is linked to the proximal phalanx at a point corresponding to the proximal interphalangeal (PIP) joint, and the distal phalanx is linked to the middle phalanx at a point corresponding to the distal interphalangeal (DIP) joint. Similarly, for the virtual thumb, the metacarpal bone is invisibly linked to the cuboid at a point corresponding to the carpometacarpal (CMC) joint, the proximal phalanx is linked to the metacarpal at a point corresponding to the MCP joint, and the distal phalanx is linked to the proximal phalanx at a point corresponding to the IP joint.\u003c/p\u003e\u003cp\u003eEach segment\u0026rsquo;s local frame of reference originates at its associated joint (i.e., origins of the proximal, middle, and distal phalanges of a virtual finger are at the MCP, IP, and DIP joints, respectively). For the cuboid, the local frame of reference originates outside the cuboid at a point corresponding to the wrist joint. Because each segment\u0026rsquo;s local frame is anchored at its joint (hence acting as the pivot point), assigning an orientation to a segment rotates it about that joint. Unlike traditional hand models, which often constrain the joint rotation to specific axes, in the current hand model, since quaternions are directly applied to each hand segment, the axis of rotation at the joint is determined by the quaternion itself.\u003c/p\u003e\u003cp\u003eDue to the hierarchical linkage of the virtual hand model, assigning an orientation to a base segment results in shifting the local origin of the subsequent segment. For example, consider the index finger: updating the cuboid\u0026rsquo;s orientation moves the MCP joint linked to it, along with the proximal phalanx whose origin is anchored at the MCP joint. Updating the proximal phalanx\u0026rsquo;s orientation rotates it about the MCP joint, thereby moving the PIP joint located at its distal end. The middle phalanx, whose origin is anchored at the PIP joint, moves with the PIP joint. This chain effect continues when updating the orientations of the middle and distal phalanges, and the same principle applies to all other fingers.\u003c/p\u003e\u003cp\u003eThe IMU sensors of the HKMS output the orientation of the hand segment to which they are attached. By assigning the quaternion data from an incoming HKMS data frame (after appropriate transformations) to the corresponding segments of the virtual hand model, the resulting virtual hand posture replicates the real hand posture. Continuously updating the virtual hand model\u0026rsquo;s posture with the incoming HKMS data frames in real-time results in the animation of the virtual hand model, replicating the participant's actual hand movements in real-time.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec31\" class=\"Section2\"\u003e\u003ch2\u003e3.3 Obtaining Accurate Hand Segment Orientation and Precise Hand Animation\u003c/h2\u003e\u003cp\u003eThe orientation data provided by the IMU sensors attached to the hand segments do not accurately represent the actual orientations of these segments. This inaccuracy arises due to a misalignment between each sensor's local frame of reference and the local frame of reference of the corresponding hand segment. Accurate hand segment orientations are required for both data analysis and precise hand animation. Additionally, accurate replication of thumb-to-phalanx touches during animation \u0026ndash; which is critical for the current experimental task \u0026ndash; cannot be achieved using a generic virtual hand model. This section describes the methodology and procedures used to correct sensor-to-segment misalignment and create participant-specific virtual hand models, thereby obtaining accurate hand segment orientations and achieving precise hand animation. The methods described in this section were tested on two participants.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e\u003cp\u003eDetails regarding the preparatory steps carried out before each experimental session, along with the materials and procedures used to affix the HKMS and TIS devices to the hand, are provided in Supplementary Material Sections 7 and 8.\u003c/p\u003e\u003c/p\u003e\u003cdiv id=\"Sec32\" class=\"Section3\"\u003e\u003ch2\u003e3.3.1 Sensor-To-Segment Alignment\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eWhile attaching IMU sensors to the hand, it is very difficult to manually place the sensors such that the local frame of reference of the sensor is aligned with the local frame of reference of the hand segment to which it is attached. As a result, the sensor's orientation data does not accurately represent the true orientation of the hand segment. To correct this discrepancy and obtain accurate segment orientation data, the sensor frame of reference needs to be aligned to the segment frame of reference through code. This process is referred to as sensor-to-segment alignment, and this sub-section details the methodology used to perform this alignment. As mentioned earlier, this methodology was validated on two participants.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eStep 1 \u0026ndash; Establishing a custom global frame of reference\u003c/strong\u003e\u003cp\u003eA sheet with parallel lines was affixed to the experimental table. A BNO055 IMU sensor was placed flat on the table and manually aligned to one of the parallel lines as accurately as possible. Specifically, the sensor's \u003cem\u003ex\u003c/em\u003e-axis was aligned along one of the parallel lines, its \u003cem\u003ey\u003c/em\u003e-axis was pointed leftward, and its \u003cem\u003ez\u003c/em\u003e-axis was perpendicular to the plane of the sheet, pointing out of the plane of the paper. The orientation of the sensor, in the form of a quaternion, was recorded in this position using a Teensy 4.0 microcontroller. This orientation value was used as the global frame of reference for all IMU orientation data collected throughout all the test sessions for the two participants. The \u003cem\u003eX\u003c/em\u003e, \u003cem\u003eY\u003c/em\u003e, and \u003cem\u003eZ\u003c/em\u003e axes of this custom global frame of reference are depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e(a). To ensure consistent sensor-to-segment alignment, it was crucial that the parallel-lined sheet always remained aligned with the global frame of reference just recorded. To maintain this alignment, the parallel-lined sheet was securely fixed to the table for all the test and experimental sessions, and the table itself was not moved or repositioned at any point during the test involving the 2 participants.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eStep 2 \u0026ndash; Physically aligning the hand segments to the custom global frame of reference\u003c/strong\u003e\u003cp\u003eBefore each experiment, participants placed their palms flat on the parallel-lined sheet, aligning their four fingers along the parallel lines (see Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e(a)). In this position, the local frames of reference of the finger phalanges and hand were physically aligned to the global frame of reference established in Step 1. The orientation data of the sensors in this position were recorded. For the thumb, additional alignment steps were required. The participant positioned their thumb at the edge of the table so that the proximal and distal phalanges rested on the table while the metacarpal extended beyond the edge (see Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e(b)). Care was taken to ensure that the metacarpal remained in the same horizontal plane as the table (see Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e(c)). Additionally, the alignment of the metacarpal to the parallel lines was challenging since the metacarpal bone\u0026rsquo;s outline was not clearly visible. To facilitate this alignment, a straight line was drawn on the dorsal surface of the thumb, running from its tip to the base of the metacarpal. This line was then aligned to one of the parallel lines. Once properly aligned, the orientation data of the thumb sensors were recorded.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e\u003cp\u003eDrawing the straight line on the metacarpal was an involved process. First, the base of the participant\u0026rsquo;s thumb metacarpal was identified. Since the base was difficult to locate by sight alone, the participant was instructed to perform one of two movements upon request to aid in its identification through tactile sensation: (1) touch the proximal phalanx of the little finger with the tip of the thumb and (2) extend the thumb outward from this position. As the participant executed these movements when requested by the experimenter, the experimenter identified and marked the base of the thumb metacarpal through tactile examination. Once the base was marked, the experimenter used their own thumb and index finger to press along the sides of the participant\u0026rsquo;s thumb metacarpal, making the bone more visible. At the same time, the experimenter guided the participant\u0026rsquo;s thumb into a straightened posture, aligning the metacarpal with the two phalanges (see Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e(d)). With the participant\u0026rsquo;s thumb held in this position, a straight line was drawn centrally along the dorsal surface of the thumb, extending from the tip to the base of the metacarpal.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eStep 3 \u0026ndash; Determining the error orientation between the sensor and the segment\u003c/strong\u003e\u003cp\u003eAs mentioned previously, the raw quaternion values output by the IMU sensor represent the sensor\u0026rsquo;s orientation with respect to the Earth's East-North-Up (ENU) frame of reference. To express the sensor's orientation relative to the global frame of reference (established in Step 1) instead of the ENU frame, the raw quaternion values were transformed using the following equation\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ3\" name=\"EquationSource\"\u003e\n$$\\:{q}_{IMU\\_G}=\\:{q}_{G}^{conj}\\:⨂\\:\\:{q}_{IMU\\_ENU}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003ehere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{IMU\\_ENU}\\)\u003c/span\u003e\u003c/span\u003e is the IMU orientation with respect to the ENU frame of reference, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{G}\\)\u003c/span\u003e\u003c/span\u003eis the global frame of reference computed in step 1, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{G}^{conj}\\)\u003c/span\u003e\u003c/span\u003e is the conjugate of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{G}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{IMU\\_G}\\)\u003c/span\u003e\u003c/span\u003e is the IMU sensor\u0026rsquo;s orientation with respect to the global frame of reference \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{G}.\\)\u003c/span\u003e\u003c/span\u003e The data recorded in step 2 was converted from the ENU frame of reference to the global frame of reference using Eq.\u0026nbsp;(\u003cspan refid=\"Equ3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). These transformed values now represent the sensor orientation with respect to the segment's local frame of reference (since the segment's local frames of reference were aligned to the global frame of reference in step 2). In other words, these values represent the error orientation between the sensor frame of reference and the segment frame of reference and are referred to as \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{error}\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eNote\u003c/strong\u003e\u003cp\u003eIn the context of this paper, a quaternion multiplication \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{3}=\\:{q}_{1}\\:⨂\\:\\:{q}_{2}\\)\u003c/span\u003e\u003c/span\u003e can be interpreted in the following way: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{3}\\)\u003c/span\u003e\u003c/span\u003e represents an orientation that results from first rotating by \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}\\)\u003c/span\u003e\u003c/span\u003e and then rotating by \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{2}\\)\u003c/span\u003e\u003c/span\u003e relative to the reference frame resulting from \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{1}\\)\u003c/span\u003e\u003c/span\u003e rotation.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eStep 4 \u0026ndash; Implementing sensor-to-segment alignment\u003c/strong\u003e\u003cp\u003eTo accurately compute a segment\u0026rsquo;s orientation, the attached sensor\u0026rsquo;s data was adjusted by applying an inverse rotation to it using the sensor's corresponding error quaternion \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{error}\\)\u003c/span\u003e\u003c/span\u003e (which was pre-computed in Step 3)\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ4\" name=\"EquationSource\"\u003e\n$$\\:{q}_{seg\\_G}=\\:{q}_{IMU\\_G}\\:⨂\\:\\:{q}_{error\\:}^{conj}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eHere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{seg\\_G}\\)\u003c/span\u003e\u003c/span\u003e is the error-corrected orientation of the hand segment (with respect to the global frame of reference), \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{IMU\\_G}\\)\u003c/span\u003e\u003c/span\u003e is the sensor orientation (with respect to the global frame of reference), and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{q}_{error}\\:\\)\u003c/span\u003e\u003c/span\u003eis the error orientation between the segment and the sensor computed in step 3. The sensor-to-segment alignment correction (Eq.\u0026nbsp;(\u003cspan refid=\"Equ4\" class=\"InternalRef\"\u003e4\u003c/span\u003e)) is applied in real time to the data from all 16 IMU sensors within each HKMS data frame, resulting in accurate segment orientations that are then used to animate the virtual hand model in real time. Figure\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e illustrates the effect of sensor-to-segment alignment: without applying this correction, the virtual fingers appear crooked because the sensor orientations do not accurately represent the true segment orientations.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec33\" class=\"Section3\"\u003e\u003ch2\u003e3.3.2 Achieving Accurate Virtual Hand Animation\u003c/h2\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eGiven the nature of the typing task (refer to Section \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e: \u003cem\u003eThe Task\u003c/em\u003e for a recap), it is essential that the virtual hands accurately replicate the thumb-to-phalanx touches of the real hands. To achieve this, the spatial locations of the fingertips and joints need to be accurate during virtual hand animation. From the description of the virtual hand model in Section \u003cspan refid=\"Sec30\" class=\"InternalRef\"\u003e3.2\u003c/span\u003e: \u003cem\u003eVirtual Hand Model Structure and Real-Time Animation\u003c/em\u003e, it can be inferred that the locations of the fingertips and joints during animation depend not only on the hand segment orientations but also on the hand segment lengths and relative finger locations. Therefore, to ensure accurate thumb-to-phalanx touch replication during animation, customized virtual hand models with the exact dimensions of the participant's real hands need to be built for each participant.\u003c/p\u003e\u003cp\u003e To evaluate this approach, two participants were recruited, and personalized virtual hand models were created for each participant for both the left and right hands using the exact dimensions of their real hands. The models incorporated measurements such as the lengths of all finger phalanges, the length of the thumb metacarpal, the relative distances between the bases of the fingers, and the thickness (height) and width of each finger. These measurements were obtained using the Polhemus Patriot motion tracking system, as detailed in Supplementary Material Section 6: \u003cem\u003eMeasuring Hand Dimensions Using the Polhemus Patriot System\u003c/em\u003e. The recorded measurements were then used to build virtual hand models in Blender 3.5, a 3D creation software. Blender was also used to superimpose the characters represented by the keys of the TIS onto the corresponding phalanges of the virtual fingers (see Figs.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, \u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e, and \u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eWhile this approach significantly improved the replication accuracy of the virtual hands, there were still minor errors in the thumb-to-phalanx touch replication (see Fig.\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e). This was due to slight inaccuracies in the exact relative positioning between the thumb and other fingers of the virtual hand. To resolve this and achieve optimal thumb-to-phalanx touch accuracy, manual adjustments to virtual finger positions were performed within Unity in the following way: Initially, the participants were instructed to touch their middle finger\u0026rsquo;s middle phalanx approximately at the phalanx centre using the tip of their thumb. While the participants held this posture without making any movements, the experimenter adjusted the virtual thumb\u0026rsquo;s position within Unity such that the thumb tip touched the surface of the virtual middle finger\u0026rsquo;s middle phalanx at approximately its centre. Following this adjustment, when the participant touched the other phalanges of the middle finger, it resulted in accurate replication of the thumb-to-phalanx touches by the virtual hand. Further minor adjustments were made to the virtual thumb and middle finger positions, if necessary, to refine the touch accuracy for the proximal and distal phalanges of the middle finger. The resulting position of the virtual thumb was fixed throughout the adjustment process. Next, the participants were instructed to touch any phalanx of one of the other fingers, and adjustments were made to the corresponding virtual finger\u0026rsquo;s position to achieve the desired touch accuracy. This process was repeated for all phalanges. These positional adjustments required approximately 10 minutes per hand for each participant. Figure\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e illustrates the thumb-to-phalanx touch accuracy before and after these adjustments.\u003c/p\u003e\u003cp\u003eAdditionally, a mirror version of each virtual hand model was created using Blender for use specifically during mirror animation. Hence, two pairs of virtual hand models were used for each participant: one pair with dimensions exactly matching those of the participant\u0026rsquo;s real hands and another \u0026ldquo;mirror\u0026rdquo; pair whose dimensions match the opposite (contralateral) real hand. For example, the \"mirror\" virtual left-hand model is created with dimensions identical to the participant's real right hand, effectively making it a mirror image of the virtual right hand. This approach was necessary because using a left virtual hand with the real left hand\u0026rsquo;s dimensions during mirror animation would result in inaccuracies in the thumb-to-phalanx touches. This is because the orientation data from the right hand \u0026ndash; after appropriate transformations \u0026ndash; is applied to the virtual left hand during mirror animation.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"4. Results: Experimental Setup Validation","content":"\u003cp\u003eTo validate the functioning of the experimental setup, two participants were recruited to participate in an experiment spread out across four consecutive days. Ethical clearance for the experiment was obtained from the Institute Ethics Committee (IEC) of the Indian Institute of Technology Madras (Project No: IEC/2022-2/SKM/03/09). All experimental sessions were performed in accordance with the procedures approved by the Institute Ethics Committee of the Indian Institute of Technology Madras. Informed consent was obtained from all participants prior to their involvement. All methods were carried out in accordance with relevant guidelines and regulations. The experiment involved a typing task where the participants had to type words presented in the VR environment as quickly as possible using the instructed hand. A detailed description of the task is provided in Section \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e: \u003cem\u003eThe Task\u003c/em\u003e. On Day 1 of the experiment, participants performed a pre-practice trial using the left hand, followed by 12 practice trials using the right hand, and concluded with a post-practice trial using the left hand. On Days 2 through 4, the same protocol was followed, except that no pre-practice trial was conducted.\u003c/p\u003e\u003cp\u003eDuring a trial, two points were awarded for each correctly typed character, while no points were deducted for incorrectly typed characters. The emphasis of the task was on speed, with the goal being to achieve the maximum score within the trial duration. Each trial was 2 minutes long. A fixed set of seven five-letter words (INSET, STAIR, SHINE, TETRA, SHIRT, TRAIN, TATER) was presented (one at a time) to the participant in a jumbled order for typing. Once all words were typed, the word set was reshuffled and presented again. This process was repeated until the trial ended.\u003c/p\u003e\u003cp\u003eDuring the practice trials, one participant received mirror visual feedback (MVF), where the left virtual hand mirrored the movements of the right real hand, while the right virtual hand remained frozen in a pre-defined posture. As a result, the participant had to rely on the left virtual hand for visual feedback while typing with the right real hand (refer to Section \u003cspan refid=\"Sec8\" class=\"InternalRef\"\u003e2.2.4\u003c/span\u003e: \u003cem\u003eMirror Feedback\u003c/em\u003e for more details on MVF). The second participant received normal visual feedback (NVF), where both virtual hands accurately replicated the movements of their respective real hands. Note: The trial format shown in Supplementary Material Video 1 differs from that of the current experiment. In the video, a different word set was used, and participants received 5-second breaks after every 15 seconds of typing.\u003c/p\u003e\u003cp\u003eThree performance variables were analysed for each trial:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eCorrect Characters Typed (CCT)\u003c/b\u003e: Total number of characters typed correctly during a trial.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003ePress Duration (PD)\u003c/b\u003e: Time between key press and key release events (in milliseconds).\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eMovement Time (MT)\u003c/b\u003e: Time between the release of one key and the press of the next key (in milliseconds).\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eFigure\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e depicts plots of these three variables for the left hand trials across the four days. The pre-practice trial on Day 1 was conducted to determine the participants' baseline performance level for the left hand. For both participants, the left hand performance improved after each right hand practice session over the first three days and plateaued on the fourth day. This improvement is indicated by increasing CCT values and decreasing MT and PD values across the first three days (see Fig.\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e). Notably, the MVF participant exhibited greater left hand performance gains compared to the NVF participant after each practice session.\u003c/p\u003e\u003cp\u003eWhile these preliminary results suggest that MVF may enhance cross-education, they should be interpreted with caution. The MVF participant reported occasional guitar playing, which could have contributed to the enhanced left-hand performance and thus confounded the results. To rigorously evaluate the effect of MVF on cross-education, a larger study is required. Participants should be divided into MVF and NVF groups, and individuals with experience in fine motor skill activities (e.g., playing musical instruments or video gaming) must be excluded from the study.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eNonetheless, the current experiment successfully demonstrates the functionality and reliability of the experimental setup for investigating MVF effects on cross-education. All experimental sessions were completed without technical issues. To verify real-time data reception by the VR application, an additional feature was implemented: whenever the master microcontroller of the HKMS-TIS board sent a synchronization signal to the slave microcontrollers, it also transmitted the \u0026lsquo;\u003cspan\u003e$\u003c/span\u003e\u0026rsquo; character to the VR application. This marked the start of the data frame and also indicated to the application that the data collection cycle for that particular data frame had started at the HKM-TIS board's end. As described in Section \u003cspan refid=\"Sec19\" class=\"InternalRef\"\u003e2.8.1\u003c/span\u003e: \u003cem\u003eCommunication Protocol\u003c/em\u003e, the end of each data frame is marked by the character \u0026lsquo;#\u0026rsquo;. The timestamps of the reception of both \u0026lsquo;\u003cspan\u003e$\u003c/span\u003e\u0026rsquo; and \u0026lsquo;#\u0026rsquo; were logged by the application. Visual inspection of these logged timestamps revealed that:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eSynchronization signals occurred every 10ms, as expected.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eEach data frame was received by the VR application within ~\u0026thinsp;7\u0026ndash;8 ms of initiation of the synchronization signal.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eNo delays were observed in data frame reception over the full trial duration.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eInspection of data files confirmed that there were no missing data frames, as verified by the data frame count value embedded in each data frame. Furthermore, performance variables (CCT, PD, MT) were successfully extracted from the TIS data for analysis. These findings collectively confirm that the system is ready for full-scale experimentation.\u003c/p\u003e"},{"header":"5. Discussion and Conclusion","content":"\u003cp\u003eResearch on the enhancement of cross-education through mirror visual feedback (MVF) has gained significant traction over the last two decades\u003csup\u003e\u003cspan additionalcitationids=\"CR5 CR6 CR7 CR8\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e due to its promising applications in the rehabilitation of hemiparetic patients\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e,\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. Studies on healthy participants have yielded positive results supporting its use in such rehabilitation\u003csup\u003e\u003cspan additionalcitationids=\"CR5 CR6 CR7 CR8\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e. However, most of these studies have focused on relatively simple tasks \u0026ndash; typically involving gross movements, single-joint rotations, or limited fine motor coordination and multi-finger control. To address this limitation, we built a novel experimental setup that enables the investigation of MVF-based cross-education for a more complex task involving sequenced fine finger movements and multi-finger coordination. This task requires participants to touch different finger phalanges with the tip of the thumb, one at a time, in various sequences.\u003c/p\u003e\u003cp\u003eThis experimental setup forms the foundation for our ongoing research on MVF-mediated cross-education and consists of three main components: (1) A novel typing device called the Text Input System (TIS), which is used to implement an intricate finger sequence movement task; (2) A Hand Kinematics Measurement System (HKMS) that accurately tracks hand and finger movements in real-time\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e; and (3) A VR headset that provides normal and mirror visual feedback during the task in real time. These three systems are fully integrated and operate in synchrony (see Supplementary Video 1 for a demonstration of the experimental setup). The hand kinematics data from the HKMS and the typing data from the TIS are timestamp-synchronized \u0026ndash; a critical requirement for analyzing movement patterns in relation to typing activity.\u003c/p\u003e\u003cp\u003eTo validate the functionality of the experimental setup, a preliminary experiment was conducted involving two participants. The setup performed as intended across all aspects of operation: all experimental sessions were completed without technical issues; data frames were received at the expected 100 Hz frequency, with no missing frames or transmission delays; and the key performance variables (CCT, PD, MT) were successfully extracted from the TIS data for analysis. These outcomes confirm the reliability of the setup and support its suitability for full-scale experiments investigating MVF effects on cross-education.\u003c/p\u003e\u003cp\u003eIn addition, key design and methodological innovations were implemented for the experimental setup. A stable period algorithm was developed for the TIS to precisely detect key press and release events. This algorithm filters out electrically and mechanically induced voltage fluctuations specific to the TIS that could otherwise lead to false key press or release detections. Participant-specific virtual hand models were built, and a methodology to refine these customized hand models was implemented, resulting in precise hand animation where the virtual hands accurately replicate the thumb-to-phalanx touches of the real hands. Given the nature of the experimental task, such precision in animation was required, which could otherwise not be achieved using a generic hand model. A clear methodology, along with the accompanying mathematical formulation, was developed to achieve sensor-to-segment alignment, which is necessary to obtain accurate orientation data of all the finger segments. This approach can be used by any application that uses IMUs and requires accurate hand kinematics data. An intuitive approach based on the axis-angle representation of quaternions was developed to transform quaternions from one coordinate system to another. This approach was used to transform the quaternion data of the HKMS from the BNO055 IMU coordinate system to the Unity coordinate system. Building on this, the approach was further extended to achieve mirror animation of the virtual hand model.\u003c/p\u003e\u003cp\u003eDespite the various strengths, we recognize several limitations in the current experimental setup. A major limitation is the extensive and time-consuming pre-experimental setup preparations for each experimental session. This involves taking the hand measurements of each participant in a separate session, building customized hand models for each participant using the Blender software, preparing a fresh batch of copper patches soldered to wires for each experimental round, and cutting a sufficient quantity of double-sided tape in appropriate sizes for affixing the IMU sensors and copper patches. Additionally, it takes approximately 25\u0026ndash;30 minutes per hand to affix the IMU sensors and the copper patches to the participant's hands, and approximately 10 minutes per hand to make adjustments to the virtual hand model to achieve accurate thumb-to-phalanx touch replication by the virtual hand models. Including the time required to perform the sensor-to-segment alignment procedures, it takes approximately 1.5 hours from the beginning of affixing the devices before the participant can actually start performing the task.\u003c/p\u003e\u003cp\u003eIn addition, the current design of the virtual hand model is such that the fingers of the hand model are separate elements (see Figs.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, \u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e, and \u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e). This is necessary since it allows for adjustments to be made to the hand model, which is required for achieving accurate thumb-to-phalanx touch replication. Although this gives an unrealistic appearance to the hand model, while performing pilot experiments with the setup, the participants did not report this as a limitation. Instead, participants reported that the virtual hands felt like their own hands within the VR environment \u0026ndash; likely due to the real-time and accurate replication of their hand movements, which created a strong sense of embodiment.\u003c/p\u003e\u003cp\u003eAs part of future work, instead of creating participant-specific virtual hand models manually, methods for automating this process can be explored. This could be achieved by developing software that takes hand dimensions as input and automatically generates a virtual hand model based on those dimensions. Additionally, after adjusting the hand model to achieve the desired thumb-to-phalanx touch accuracy, methods to automatically convert the refined hand model into a more realistic representation of the hand can be explored. Furthermore, the tedious preparation of a fresh batch of soldered copper patches and a fresh set of cut double-sided tapes before each experimental round and the long device mounting time make it impractical to investigate the long-term effects (spanning multiple days or weeks) of MVF on cross-education for this particular task. To overcome this limitation, the HKMS and TIS systems must be mounted on gloves that allow for easy adjustment of the IMU and copper patch placement.\u003c/p\u003e\u003cp\u003eIn summary, this paper presents the design, development, and validation of an experimental setup built to investigate the effects of mirror visual feedback on the enhancement of cross-education for a task at a level of complexity previously unexplored in the cross\u0026ndash;education\u0026ndash;MVF literature. The paper also details key methodologies for system integration, millisecond-level data synchronization, and precise real-time hand animation. An intuitive, axis-angle-based approach for quaternion transformation across coordinate systems is described, along with a method for achieving mirror animation of virtual hands. Furthermore, a detailed sensor-to-segment alignment methodology is outlined to obtain accurate finger segment orientations. Many of the methodologies detailed in the paper contribute valuable tools to the hand biomechanics and VR research community. The setup was validated through a preliminary experiment, which confirmed its reliability and operational integrity, thereby establishing its readiness for future MVF-based cross-education studies.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003ch2\u003eCompeting Interests\u003c/h2\u003e\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e\u003c/p\u003e\u003ch2\u003eFunding\u003c/h2\u003e\u003cp\u003eThis research was supported by the Department of Science and Technology (DST), Government of India, under the Cognitive Science Research Initiative (Grant No. DST/CSRI/2017/87, awarded to Varadhan SKM). It was also supported by the Ministry of Education, Government of India, through the Prime Minister\u0026rsquo;s Research Fellowship (PMRF ID: 2500902, awarded to Anurag Gupta).\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eExperimental setup design and development \u0026mdash; A.G.; Conceptualization of experimental task \u0026mdash; V.S.K.M.; Methodologies and algorithm development \u0026mdash; A.G.; Writing original draft \u0026mdash; A.G.; Review and editing draft \u0026mdash; V.S.K.M., A.G.; All authors reviewed and approved the final manuscript.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eWe gratefully acknowledge the Department of Science and Technology (DST), Government of India, for funding this work under the Cognitive Science Research Initiative (CSRI) (Grant No. DST/CSRI/2017/87, awarded to Varadhan SKM). We also thank the Ministry of Education, Government of India, for supporting this research through the Prime Minister\u0026rsquo;s Research Fellowship (PMRF) (PMRF ID: 2500902, awarded to Anurag Gupta). We are thankful to Thomas Jacob, a PhD scholar in our lab, for his encouragement, valuable suggestions, and assistance in testing the experimental setup throughout its development. We also acknowledge Kanva Aravind Kashyapa, a former research intern in the lab, for his help in testing the experimental setup.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe data collected for the current study are available from the corresponding author on reasonable request.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eVolkmann, A. W. \u0026Uuml;ber Den Einfluss Der Uebung Auf Das Erkennen R\u0026uml;Aumlicher Distanzen. \u003cem\u003eBerichte K. S\u0026auml;chs. Ges. Wiss. Math.-Phys. Cl.\u003c/em\u003e \u003cstrong\u003e10\u003c/strong\u003e, 38\u0026ndash;69 (1858).\u003c/li\u003e\n\u003cli\u003eScripture, E. W., Smith, T. L. \u0026amp; Brown, E. M. On the education of muscular control and power. \u003cem\u003eStud. Yale Psycological Lab.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, (1894).\u003c/li\u003e\n\u003cli\u003eManca, A. \u003cem\u003eet al.\u003c/em\u003e Contralateral Effects of Unilateral Strength and Skill Training: Modified Delphi Consensus to Establish Key Aspects of Cross-Education. \u003cem\u003eSports Med. Auckl. NZ\u003c/em\u003e \u003cstrong\u003e51\u003c/strong\u003e, 11\u0026ndash;20 (2021).\u003c/li\u003e\n\u003cli\u003eOssmy, O. \u0026amp; Mukamel, R. Neural Network Underlying Intermanual Skill Transfer in Humans. \u003cem\u003eCell Rep.\u003c/em\u003e \u003cstrong\u003e17\u003c/strong\u003e, 2891\u0026ndash;2900 (2016).\u003c/li\u003e\n\u003cli\u003eOssmy, O. \u0026amp; Mukamel, R. Using Virtual Reality to Transfer Motor Skill Knowledge from One Hand to Another. \u003cem\u003eJ. Vis. Exp.\u003c/em\u003e 55965 (2017) doi:10.3791/55965.\u003c/li\u003e\n\u003cli\u003eNojima, I. \u003cem\u003eet al.\u003c/em\u003e Human Motor Plasticity Induced by Mirror Visual Feedback. \u003cem\u003eJ. Neurosci.\u003c/em\u003e \u003cstrong\u003e32\u003c/strong\u003e, 1293\u0026ndash;1300 (2012).\u003c/li\u003e\n\u003cli\u003eRuddy, K. L. \u003cem\u003eet al.\u003c/em\u003e Neural Adaptations Associated with Interlimb Transfer in a Ballistic Wrist Flexion Task. \u003cem\u003eFront. Hum. Neurosci.\u003c/em\u003e \u003cstrong\u003e10\u003c/strong\u003e, 204 (2016).\u003c/li\u003e\n\u003cli\u003eZult, T. \u003cem\u003eet al.\u003c/em\u003e Mirror Training Augments the Cross-education of Strength and Affects Inhibitory Paths. \u003cem\u003eMed. Sci. Sports Exerc.\u003c/em\u003e \u003cstrong\u003e48\u003c/strong\u003e, 1001 (2016).\u003c/li\u003e\n\u003cli\u003eSteinberg, F., Pixa, N. H. \u0026amp; Doppelmayr, M. Mirror Visual Feedback Training Improves Intermanual Transfer in a Sport-Specific Task: A Comparison between Different Skill Levels. \u003cem\u003eNeural Plast.\u003c/em\u003e \u003cstrong\u003e2016\u003c/strong\u003e, 8628039 (2016).\u003c/li\u003e\n\u003cli\u003eRamachandran, V. \u0026amp; Rogers-Ramachandran, D. Synaesthesia in phantom limbs induced with mirrors. \u003cem\u003eProc. R. Soc. Lond. B Biol. Sci.\u003c/em\u003e \u003cstrong\u003e263\u003c/strong\u003e, 377\u0026ndash;386 (1996).\u003c/li\u003e\n\u003cli\u003eAltschuler, E. L. \u003cem\u003eet al.\u003c/em\u003e Rehabilitation of hemiparesis after stroke with a mirror. \u003cem\u003eThe Lancet\u003c/em\u003e \u003cstrong\u003e353\u003c/strong\u003e, 2035\u0026ndash;2036 (1999).\u003c/li\u003e\n\u003cli\u003eRuddy, K. L. \u0026amp; Carson, R. G. Neural pathways mediating cross education of motor function. \u003cem\u003eFront. Hum. Neurosci.\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, (2013).\u003c/li\u003e\n\u003cli\u003eWard, J. P., Alvis, G. R., Sanford, C. G., Dodson, D. L. \u0026amp; Pusakulich, R. L. Qualitative differences in tactuo-spatial motor learning by left-handers. \u003cem\u003eNeuropsychologia\u003c/em\u003e \u003cstrong\u003e27\u003c/strong\u003e, 1091\u0026ndash;1099 (1989).\u003c/li\u003e\n\u003cli\u003eStoddard, J. \u0026amp; Vaid, J. Asymmetries in intermanual transfer of maze learning in right- and left-handed adults. \u003cem\u003eNeuropsychologia\u003c/em\u003e \u003cstrong\u003e34\u003c/strong\u003e, 605\u0026ndash;608 (1996).\u003c/li\u003e\n\u003cli\u003eParlow, S. E. \u0026amp; Kinsbourne, M. Asymmetrical transfer of braille acquisition between hands. \u003cem\u003eBrain Lang.\u003c/em\u003e \u003cstrong\u003e39\u003c/strong\u003e, 319\u0026ndash;330 (1990).\u003c/li\u003e\n\u003cli\u003eCriscimagna-Hemminger, S. E., Donchin, O., Gazzaniga, M. S. \u0026amp; Shadmehr, R. Learned dynamics of reaching movements generalize from dominant to nondominant arm. \u003cem\u003eJ. Neurophysiol.\u003c/em\u003e \u003cstrong\u003e89\u003c/strong\u003e, 168\u0026ndash;176 (2003).\u003c/li\u003e\n\u003cli\u003eCunningham, H. A. \u0026amp; Welch, R. B. Multiple concurrent visual-motor mappings: Implications for models of adaptation. \u003cem\u003eJ. Exp. Psychol. Hum. Percept. Perform.\u003c/em\u003e \u003cstrong\u003e20\u003c/strong\u003e, 987\u0026ndash;999 (1994).\u003c/li\u003e\n\u003cli\u003eRedding, G. M. \u0026amp; and Wallace, B. Intermanual Transfer of Prism Adaptation. \u003cem\u003eJ. Mot. Behav.\u003c/em\u003e \u003cstrong\u003e40\u003c/strong\u003e, 246\u0026ndash;264 (2008).\u003c/li\u003e\n\u003cli\u003eSchulze, K., L\u0026uuml;ders, E. \u0026amp; J\u0026auml;ncke, L. Intermanual Transfer in a Simple Motor Task. \u003cem\u003eCortex\u003c/em\u003e \u003cstrong\u003e38\u003c/strong\u003e, 805\u0026ndash;815 (2002).\u003c/li\u003e\n\u003cli\u003eBeg, R. A. \u003cem\u003eet al.\u003c/em\u003e Intermanual Transfer Effects on Performance Gain Following Dominant Hand Training in Community-Dwelling Healthy Adults: A Preliminary Study. \u003cem\u003eJ. Multidiscip. Healthc.\u003c/em\u003e \u003cstrong\u003e14\u003c/strong\u003e, 1007\u0026ndash;1016 (2021).\u003c/li\u003e\n\u003cli\u003eKrishnan, C., Ranganathan, R. \u0026amp; Tetarbe, M. Interlimb Transfer of Motor Skill Learning during Walking: No evidence for asymmetric transfer. \u003cem\u003eGait Posture\u003c/em\u003e \u003cstrong\u003e56\u003c/strong\u003e, 24\u0026ndash;30 (2017).\u003c/li\u003e\n\u003cli\u003eChase, C. \u0026amp; Seidler, R. Degree of handedness affects intermanual transfer of skill learning. \u003cem\u003eExp. Brain Res.\u003c/em\u003e \u003cstrong\u003e190\u003c/strong\u003e, 317\u0026ndash;328 (2008).\u003c/li\u003e\n\u003cli\u003eGreen, L. A. \u0026amp; Gabriel, D. A. The cross education of strength and skill following unilateral strength training in the upper and lower limbs. \u003cem\u003eJ. Neurophysiol.\u003c/em\u003e \u003cstrong\u003e120\u003c/strong\u003e, 468\u0026ndash;479 (2018).\u003c/li\u003e\n\u003cli\u003eLee, M., Hinder, M. R., Gandevia, S. C. \u0026amp; Carroll, T. J. The ipsilateral motor cortex contributes to cross-limb transfer of performance gains after ballistic motor practice. \u003cem\u003eJ. Physiol.\u003c/em\u003e \u003cstrong\u003e588\u003c/strong\u003e, 201\u0026ndash;212 (2010).\u003c/li\u003e\n\u003cli\u003eRachaveti, D. \u0026amp; Skm, V. Motor sequence learning data collected continuously for fifteen days of practice using a novel glove-based typing device. \u003cem\u003eData Brief\u003c/em\u003e \u003cstrong\u003e29\u003c/strong\u003e, 105234 (2020).\u003c/li\u003e\n\u003cli\u003eRachaveti, D., Ranganathan, R. \u0026amp; Skm, V. Practice modifies the response to errors during a novel motor sequence learning task. 2020.10.09.334169 Preprint at https://doi.org/10.1101/2020.10.09.334169 (2020).\u003c/li\u003e\n\u003cli\u003eRachaveti, D. Studies of motor sequence learning using finger thumb opposition movements. (Indian Institute of Technology Madras, 2019).\u003c/li\u003e\n\u003cli\u003eShenoy, P., Gupta, A. \u0026amp; S.K.M., V. Design and Validation of an IMU Based Full Hand Kinematic Measurement System. \u003cem\u003eIEEE Access\u003c/em\u003e \u003cstrong\u003e10\u003c/strong\u003e, 93812\u0026ndash;93830 (2022).\u003c/li\u003e\n\u003cli\u003eSensortec, B. \u003cem\u003eBNO055: Intelligent 9-Axis Absolute Orientation Sensor \u0026ndash; Data Sheet\u003c/em\u003e. (2021).\u003c/li\u003e\n\u003cli\u003eChallis, J. H. Quaternions as a solution to determining the angular kinematics of human movement. \u003cem\u003eBMC Biomed. Eng.\u003c/em\u003e \u003cstrong\u003e2\u003c/strong\u003e, 5 (2020).\u003c/li\u003e\n\u003cli\u003eRamachandran, V. S. \u0026amp; Altschuler, E. L. The use of visual feedback, in particular mirror visual feedback, in restoring brain function. \u003cem\u003eBrain\u003c/em\u003e \u003cstrong\u003e132\u003c/strong\u003e, 1693\u0026ndash;1710 (2009).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"intermanual transfer, mirror visual feedback, hand kinematics, inertial measurement unit, virtual reality, hand animation","lastPublishedDoi":"10.21203/rs.3.rs-6957809/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6957809/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eCross-education, a phenomenon where training one limb improves performance in the untrained limb, can be enhanced through mirror visual feedback (MVF), underpinning its potential for rehabilitating hemiparetic patients. While MVF-mediated enhancement is documented for simple motor tasks, its effectiveness in complex, fine finger movements remains underexplored. To address this gap, we developed a novel experimental setup to investigate MVF effects on cross-education for a unique typing task involving fine finger movements. The setup comprises three tightly synchronized systems: A novel typing device to implement the typing task, an IMU-based system for accurate real-time hand movement tracking, and a virtual reality (VR) system that provides normal or mirrored real-time replication of hand movements by virtual hands. Additionally, several methodologies and algorithms critical to the experimental task were developed: (1) Constructing participant-specific virtual hand models to accurately replicate thumb-to-phalanx touches. (2) An intuitive approach to manipulating quaternions for coordinate transformation and mirror animation. (3) Millisecond-level synchronization of movement and typing data. (4) Preventing false key press/release detections by filtering noise specific to the typing device. (5) Correcting reference frame misalignment between IMU sensors and their respective hand segments. Some of these methodologies contribute valuable tools to the hand biomechanics and VR research communities. Technical validation of the setup demonstrated robust real-time performance, millisecond-level data synchronization, and precise hand animation, confirming the system\u0026rsquo;s readiness for investigating MVF-based cross-education.\u003c/p\u003e","manuscriptTitle":"Development of an Experimental Setup to Investigate Mirror Visual Feedback Effects on Cross-Education for an Intricate Finger Sequence Movement Task","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-08 07:58:37","doi":"10.21203/rs.3.rs-6957809/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-11-18T04:39:33+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-14T18:46:29+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"333607595564910998818820379098616388461","date":"2025-11-06T04:07:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"116240300431796429043074357831601724320","date":"2025-11-02T13:31:28+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-29T13:34:51+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-20T00:36:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"326382362128124105986222191455858216690","date":"2025-10-19T08:35:43+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"190636202785830329383179133377765706945","date":"2025-10-18T08:00:47+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-09-25T02:06:59+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-08-20T09:21:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-02T10:32:34+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-25T16:25:08+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-06-25T16:20:32+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"9c4df093-f557-42ed-96db-9a26fd643404","owner":[],"postedDate":"October 8th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":55778611,"name":"Biological sciences/Neuroscience/Motor control"},{"id":55778612,"name":"Physical sciences/Engineering/Biomedical engineering"}],"tags":[],"updatedAt":"2026-04-13T16:02:23+00:00","versionOfRecord":{"articleIdentity":"rs-6957809","link":"https://doi.org/10.1038/s41598-026-41353-1","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2026-04-09 15:58:57","publishedOnDateReadable":"April 9th, 2026"},"versionCreatedAt":"2025-10-08 07:58:37","video":"","vorDoi":"10.1038/s41598-026-41353-1","vorDoiUrl":"https://doi.org/10.1038/s41598-026-41353-1","workflowStages":[]},"version":"v1","identity":"rs-6957809","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6957809","identity":"rs-6957809","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}