A novel tactile technology enables sound source identification by hearing-impaired individuals in a complex 3D audio environment

A novel tactile technology enables sound source identification by hearing-impaired individuals in a complex 3D audio environment | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A novel tactile technology enables sound source identification by hearing-impaired individuals in a complex 3D audio environment Adi Snir, Katazyna Ciesla, Amir Amedi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7418108/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Congenitally hearing-impaired individuals have reduced auditory localization capabilities and often find identification of sources to be challenging in noisy environments. The congenital nature of their hearing-impairment and the fact that hearing is the only modality capable of representing the full three-dimensional surrounding, mean this population have never experienced a proper representation of their spatial surrounding. We use an inhouse tactile device, which performs level weighting to four vibration actuators to the fingers to reproduce spatial positions and Higher-order Ambisonics to test congenitally hearing-impaired and typically-hearing individuals on their ability to pair between localized tactile information and audio sources within a complex three-dimensional audio environment. Participants of both groups show accuracy significantly higher than chance, with the typically hearing performing better than the hearing impaired. We further see rapid improvement in the task with no training. We discuss the importance of our findings within the discourse of sensory binding and assistive development towards rehabilitation. Cognitive Neuroscience Psychology Spatial Perception Sensory Binding Sensory Integration Auditory Scene Analysis Hearing Impaired Audio-Tactile Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction Our environment is complex in nature, containing multiple stimuli constantly changing positions, while moving in and out of the field associated with each sensory modality. The auditory system nonetheless, is the only modality which enables a simultaneous representation of the surrounding three-dimensional environment (considering that vision is frontally oriented, while touch is typically limited to only the peripersonal space, i.e. the space within reach; Cléry et al., 2015). Audio localization can pose a challenge for hearing impaired individuals, who very often show reduced capabilities, including for sound source segregation abilities, particularly when multiple sources are present (Glyde et al., 2013). Various research has been dedicated to investigating how the auditory system is able to function when confronted by complex auditory stimuli (also coined Auditory Scene Analysis ; Bregman 1990). These include considerations of how one contends with multiple sound streams, each with its own spectral, spatial and temporal characteristics. Further research investigates the formation of “auditory-objects”, i.e. how a representation of each object emitting acoustical information is formed (Bizley, 2010). Attention through task manipulation has shown to be key in impacting such capabilities as well (Bregman, 2015; Sussman, 2017). In the real world nevertheless, auditory information is usually experienced as part of a multisensory construct, and indeed research on multisensory processing has accumulated evidence showing the influence of non-auditory stimulation on auditory capabilities and representations classically attributed to this modality (King, Hammond-Kenny & Nodal, 2019; ref). Multisensory illusions are often used to demonstrate the potential effects of sensory binding on spatial awareness (Zeller et al, 2015; Innes-Brown & Crewther, 2009; ref). Such illusions are caused when binding information from two sensory modalities, due to temporal matching, while the sources are not spatially matching (Chen & Vroomen, 2013). One such instance is the well known ventriloquism effect, in which the perceived auditory source’s position shifts towards the visual stimulus (face; Alais & Burr, 2004; Bonath et al, 2014; 2007; Bruns, 2019). Similar effects have been found using somatosensory inputs, showing that congruent tactile actuation can have a biasing effect on the perceived location of auditory stimuli, coined as the “capture” of sound (Caclin, 2002; Bruns & Röder, 2010; Spence & Zampini, 2009; Driver & Spence, 1998). In fact, some researchers claim the effects of audio-tactile integration are almost automatic, due to the similarities among the two senses (for instance, both are able of encoding frequencies within a partially overlapping frequency range) and their adjacent structures in the brain (Schürman et al, 2006; Landelle et al, 2023; Commett et al, 2018, Ciesla x 2) Another line of research has shown that proper development of auditory spatial understanding may be dependent on information arriving from another sensory modality (e.g. vision). It was for example demonstrated that already at the age of four months old, infants respond differently to complex auditory stimuli paired with visuals, depending on whether the multisensory information is presented in a congruent or incongruent manner (Smith et al., 2017). Other studies have found impaired auditory spatial capabilities in congenitally blind individuals, suggesting that a proper construction of auditory spatial representation may depend on visual experience, with vision “calibrating” the less precise auditory localization (Gori et al, 2014a). Meanwhile, some authors showed enhanced auditory processing in the periphery of blind individuals, but the findings are mixed (Röder et al, 1999). Interestingly, spatial perception has been shown to improve in congenitally blind individuals who trained using a dedicated tactile device (Gori et al, 2014b). This while training with a row of tactile actuators along the arm, each representing one of the positions in a row of loudspeakers. Research on aided congenitally hearing-impaired individuals has also shown improved capabilities of auditory localization by integrating dedicated tactile feedback in the form of vibrotactile vibrations on the body (Snir & Ciesla, 2024b). Similar findings have been shown in non-congenital cochlear implant users as well (Fletcher, 2020). It is still, however, not clear whether individuals with congenital hearing impairments would retain the ability to integrate spatially constructed tactile information within a complex audio environment containing multiple moving sources. Hearing impairments and hearing assistive devices impact perception of both the sound content (due to reduced perceived levels and frequency representation; Karimi-Boroujeni et al, 2023) and its location (in part due to the sound correction algorithms, including gain adjustments and signal compression; Johnson, Xu & Cox, 2017; Akeroyd, 2015; Zheng et. al, 2022). Specifically, they have reduced performance in stream segregation tasks (intended to test one’s ability to separate sources from within an interleaved audio stream; Shearer et al., 2018; Oxenham, 2008; Paredes-Gallardo et al., 2019). Hearing impairments have also been found to be associated with reduced performance in non-auditory spatial cognitive tasks (Taljaard et al, 2015; Uchida et al, 2016). Vast progress has been made over the past decades in advancing hearing assistive technologies such as cochlear implants (CI) and hearing aids (HA). New developments include reduction of noise in the environment (Henry et al, 2023) and consideration of spatial cues (Gajecki & Nogueira, 2021); yet most new features aim at improving speech comprehension, as does the training received by the users in the clinic (Carlyon & Goehring, 2021; Lu et al, 2019). Furthermore, many individuals who perform speech comprehension tasks well in clinical setting, continue to be challenged in real-world environments, which are inherently more complex due to both degraded signal-to-noise (SNR) as well as a multitude of constantly moving sound sources (Lewis, 2023; see further regarding the cocktail party effect and its relation to auditory spatial abilities: Arons, 1992; Ebata, 2003). Multisensory based solutions for sensory enhancement have been the subject of research towards rehabilitation of multiple sensory impaired populations, amongst these are hearing-impaired individuals (Ramones & Guera, 2023; Sorgini et al, 2018). Solutions towards enhancing auditory capabilities in hearing-impaired population in particular have shown various levels of promise. Authors have for instance shown improved speech perception (Cieśla et al., 2022; Fletcher et al., 2019; Huang et al., 2017; Drullman & Bronkhorst, 2004) and improved auditory localization (Gori et al., 2014; Occelli, Spence & Zampini, 2011; Snir & Ciesla, 2024a) capabilities. Stream segregation abilities were also found to improve when using audio-tactile integration (Slater & Marozeau, 2016). These effects may be the result of similarities between the auditory and tactile senses (Merchel & Altinsoy, 2020), both capable of perceiving vibrations within a partially overlapping frequency range (between ~ 20Hz and ~ 700Hz), leading to almost automatic audio-tactile integration, even when perceiving unfamiliar, or novel , sensory information (Landelle et al, 2023; Crommett, Madala & Yau, 2019; Bernard et al., 2022; Schürmann et al., 2006; Gick & Derrick, 2009; Hayward et al 2016; Snir et al, 2024b). With hearing being the only natural sensory modality which can represent one’s entire spatial field, in the current study, we set out to explore whether congenitally hearing-impaired individuals, who have demonstrated deficits in auditory spatial capabilities, would be able to bind tactile information to sources within a complex 3D audio environment in order to identify auditory sources. Integration of sensory information has often been defined in terms of temporal and spatial congruency (see: spatial and temporal rule of multisensory integration; e.g. Holmes 2007), leading to the assumption of a common source (see: causal inference ;Stevenson 2012. This process of combining sensory streams into a single representation is otherwise known as sensory binding (SB; Quintero et al, 2022). Binding becomes much harder in a complex real world environment, which often contains multiple interfering sources with overlapping parameters to the target source (i.e. frequency range, position, etc; Noppeney, 2021). For this research, we utilize the touch-motion algorithm (TMA), an in-house solution developed by one of the authors (AS). TMA provides weighted vibrotactile cues to four fingertips to represent sound source positions in the surrounding 360º space, while preserving sound content. Our prior research shows that congenitally hearing impaired individuals have highly compromised localization capabilities for spatially moving audio sources (Snir & Ciesla, 2024b). Nevertheless performance improves quickly with the use of assistive touch (via TMA). In the current study, we further investigate the feasibility of using touch in a a dynamic 3D auditory scene, using Higher-Order Ambisonic (HOA) environment. We hypothesize that, since the auditory and vibrotactile stimuli are designed to represent the same content and spatial location, they should bind easily in the typically-hearing individuals (rules of multisensory integration). We hypothesize that the hearing impairments will affect this ability, due to both reduced auditory spatial capabilities as well as stream segregation (as compared to typical hearing). Nevertheless, in the hearing-impaired, we hypothesize that if they perform the task with high accuracy (i.e. correctly identify the sounds paired with the tactile vibrations), this will indicate successful binding despite reduced auditory capabilities. Subjective questionnaires are also used to inquire regarding the experience of the participants during their experience of auditory-tactile stimulation. Material & Methods The experiment was approved by the Institutional Review Board (IRB; Ethical clearance number: P_re3_2022123) of the Reichman University, Herzliya, Israel and was in accordance with the regulations of the Declaration of Helsinki 2013. All participants gave informed consent via signature, and were either paid or received student credits for their participation. Participants were recruited through social media, the university’s student credit system and word of mouth. Participants Participants included 20 congenitally hearing-impaired subjects: 10 bilateral cochlear-implant (biCI) users (8 females, 2 males; age 25.7+/-4.85) and 10 other cochlear-implant (CI) and/or hearing-aid (HA) users (8 females, 2 males; age 30.9+/-14.67); and 59 typically-hearing subjects (30 females, 29 male; age 23.5+/-2.7). For further information on hearing-impairments and assistive devices see Table 1 . Table 1 Details of the hearing-impaired participants. Initials (code) Age Age at HL diagnosis Age at RE CI/HA Age at LE CI/HA Etiology Device RE Device LE Preferred ear Gender DS (RCILDO00) 43 Birth 29 - Unknown Med-El Concerto - R M TT (RCILCI004) 33 Birth 3 14 Waardenburg syndrome CI, Cochlear Nucleus 7 CI, Cochlear Nucleus 7 R F OAS (RCILCI005) 26 Birth 2 15 Unknown CI, AB Harmony CI, AB Harmony R M SWC (RCILCI007) 29 Birth 2 14 Meningitis CI, Cochlear Nucleus 24 CI, Cochlear, Nucleus 24 R M DT (RCILCI009) 33 Birth 3* 14 Waardenburg syndrome CI, Cochlear Kanso 1 CI, Cochlear Kanso 1 R F EY (RHALCI010) 19 Birth 1 6 Connexin 26 HA, Pure 13 CI, Medel, Sonnet R F HE (RCILCI011) 23 Birth 2 9 Mother’s illness during pregnancy CI, Cochlear, Nucleus 24 CI, Cochlear, Freedom R F SZ (RCILDO012) 22 Birth 2 9*** Connexin 26 CI, Nucleus 24 - R F EG (RBCLBC013) 21 Birth 15 15 Turner Syndrome HA (BC) BAHA 6 max HA (BC) BAHA 6 max R F YL (RCILHA014) 20 5 15 5 Turner Syndrome CI, Kanso 1 HA, NaidaV50 R F CS (RCILCI015) 22 Birth 2 17 Mother’s illness during pregnancy CI, Cochlear Nucleus 7 CI, Cochlear, Nucleus 6 R F SE (RDOLCI016) 22 Birth - 0.5 Genetic (TMC1) - CI Med-el Combi 40 L F SB (RCILCI017) 25 Birth 2 11 Unknown CI, Cochlear, Nucleus 7 CI, Cochlear, Nucleus 7 R F NH (RCILDO018) 30 Birth 1 - Genetic CI, Nucleus 6 - R F NT (RCILDO019) 22 Birth 3 12**** Unknown CI, Nucleus 7 - R F NR (RCILCI020) 18 Birth 1 10 Unknown CI, Cochlear, Nucleus 7 CI, Cochlear, Nucleus 7 R F RS (RHALHA021) 59 1.5 4 4 Measles vaccine HA C&G SP 7X BG HA, C&G SP 7XBG L M HH (RCILCI022) 22 Birth 1 6.5 Connexin 26 CI, Cochlear, Nucleus 6 CI, Cochlear, Nucleus 6 R F HL (RCILCI023) 26 Birth 3** 12 Genetic CI, AB, Nadia CI, AB, Nadia R F TM (RHALCI024) 51 Birth 14 2 Mother’s illness during pregnancy (rheumatism) HA, Signia CI, Med-el Opus None F F- female, M - male; HL - hearing loss; CI - cochlear implant; HA - hearing aid; RE - right ear, LE - left ear; *replaced at the age of 21; ** replaced at age 25; *** not using it at all, came to the test without; **** not using it at all, came to the test without; BC - bone conduction hearing aid Audio and Tactile Stimuli The experiment involved an audio-tactile source pairing task in a complex auditory environment (a synthetized Higher-Order Ambisonics scene). The scene consisted of eight “forest” sounds, six of which were single sounds and remained in constant motion, and two were ambient static tracks on the right and left side of the scene. The moving sounds could appear anywhere on the 360º surrounding horizontal plane, with an elevation between − 30° and 30° (with the participant’s head at 0º). The moving sounds were all circling around the participants at a rate of 40–75 seconds per full encirclement. The four target sounds included owl hoots, a howling wolf, thunder rumbling, and cricket chirps (for spectrograms see Fig. 2). The other two moving sources were distractors and included sounds of rain and multiple types of bird chirps. The tactile vibrations on fingertips of two hands corresponded to a single target auditory source in both content and location (i.e. moving synchronously). The paradigm The experiment consisted of twenty trials, designed as a four alternative forced choice (4AFC) paradigm. Participants received a sheet containing four possible answers: 1) “owl”, 2) “wolf, 3) “thunder”, 4) “cricket”, and were asked to respond as to which source was paired with the simultaneous tactile feedback. Target sources were chosen using a pseudo-random order, with each participant experiencing 20 trials in total (each of the 4 target sounds was selected 5 times in pseudorandom order). Each trial began with a 3D audio scene (in HOA) playing for a duration of 2–10 seconds, then tactile vibration was delivered (while the audio continued), matching one of the four target sources. The audio-tactile scene was experienced for 10 seconds, after which the scene was stopped and participants were prompted to verbally indicate which of the four possible sounds was paired with the vibrotactile input (see Fig. 1 ). Following the experiment, participants were given a brief questionnaire to assess their subjective experience of the task, on a VAS scale 1 to 10 (Table 2 ). Apparatus The experiment was conducted in a cube-shaped (4 x 4 x 4 meter) dimly lit room at the The Baruch Ivcher Institute For Brain, Cognition & Technology at Reichman University. The room is acoustically sealed from the outside and controlled for reflections on the inside using mineral wool soundproofing, with a reverberation time of < 0.3 s. There are 97 loudspeakers placed on the ceiling and in three rows on the walls. The participant was seated on a chair with their head at the level of the middle loudspeaker row in the center of the room. The researcher was located in a separate control room communicating with the participant using a studio talkback system. Tactile stimulators (each consisting of two discrete channels) were placed on two small tables to each side of the participant (see Fig. 1 ). Apparatus: Audio Ninety-seven loudspeakers (JBL Control 23-1L, powered by 13 Dante-enabled amplifiers, Crown Audio DCi 8|600DA) were placed in three rings on the walls (72 total, each ring containing 24 speakers) and ceiling (25 speakers in a 5x5 equidistant grid). A uniform azimuthal angle difference of 15° (in relation to the center point) defined the distance between the adjacent wall speakers. The heights of the three speaker rings were: 48cm, 148cm and 248cm from the ground. The participant’s head was placed at the center of the room, at the height of the middle speaker ring. The loudspeaker outputs were measured from the participant’s head position (center position of the space) and corrected for the frequency response and the delay, to create a uniform audio response (measurements and corrections were configured using the Audio Architect software by HiQnet). The All-Round Ambisonic Decoding (AllRAD) method along with High-Order Ambisonics (HOA) encoding was used to synthesize audio positioning (configured to 12th order Ambisonics). Ambisonic encoding and decoding utilized the Spat ~ library (version 5.2) developed at the Institut de Recherche et Coordination Acoustique/Musique (IRCAM). All code was programmed within the Max MSP environment. Apparatus: Tactile Tactile stimulation during the experiments was emitted using two identical Vibrating Auditory Stimulators (VAS; https://www.neurodevice.pl/en/ ‎‎ ), each consisting of two discrete channels and piezoelectric plates. VAS boxes were placed on two small tables to the sides of the participant. The participants placed the fingertips of two fingers (index and middle) of both hands through two holes on the side of each VAS on the vibrating plates. The openings for the fingers in each tactile device had silicone covers with slits and the boxes were internally padded with a foam layer. Sound level measurements To ensure no perceivable sound was emitted by the tactile devices during the experiment, a measurement microphone (MiniDSP UMIK-1, frequency response 8Hz-20kHz) was used to measure sound levels at the participants’ ear position with and without tactile sources present. Sound levels were measured four times (30 seconds each) one time for each target sound, the tactile devices were either activated (ON) or left deactivated (OFF), while the sound was always ON. The results for the measurements were as follows : a) Target 1 (Owl) - tactile device OFF/ON: LZSmax 67.2dB/67dB; b) Target 2 (Wolf) - tactile devices OFF/ON: LZSmax 67dB/67.1dB; c) Target 3 (Thunder) - tactile devices OFF/ON: LZSmax. 67dB/66.9dB; d) Target 4 (Crickets) - tactile device OFF/ON: LZSmax. 67dB/69.5dB. Touch Motion Algorithm (TMA) To represent spatial locations of the stimulus through touch, we used the azimuthal component of the touch motion algorithm (TMA; for further details see Snir & Ciesla 2024a, 2024b and Amedi & Snir, 2024) developed by the author of the current manuscript (A.S.). TMA compares the angle of the intended virtual source’s position to the tactile actuator’s virtual angle. This enables the representation of locations in 3D space on the horizontal plane by weighting vibration level differences among four discrete vibration actuators (on four fingertips of two hands). Smooth changes of the levels reproduce motion. The gain scaling of each actuator follows a logarithmic amplitude scale, in a manner similar to vector based amplitude modulation (Pulkki, 1997). In the current setup, each of the 4 fingers represents a diagonal orientation of the surrounding space (i.e. azimuthal 45°, 135°, 225° and 315°; the azimuthal front at 0°). As an example, if the index fingers of both hands received vibrations of equal intensity, this reproduced a source in front-center (0° azimuth); vibrations delivered on only the middle finger of the right hand indicated a stimulus in the back right position (135°). The vibration content of the tactile feed corresponded directly to the paired sound, i.e. no transformation of the signal was performed aside from gain weighting among the four actuators. For further details, please contact the corresponding author (A.S.). Data Analysis Trials in which subjects correctly identified the paired stimulus were designated as a ‘success’ ( 1 ) and those in which subjects failed to correctly identify the stimulus were designated as ‘failure’ (0). The performance of each group (typically-hearing and hearing-impaired) was then compared to chance (25%) using a Brunner-Munzel (BM) test. The BM test was done against the one-sided alternative that the probability of success is greater than 25%. This data was then submitted to a mixed-effect logistic regression analysis. Backward selection was performed on the following fixed effects: group belonging (typically-hearing vs hearing-impaired), stimulus type (four target sounds), and trial number ( 1 – 20 ). The pairwise interactions between stimulus type and group, and group and trial number were also tested. To test improvement throughout the task, a mixed-effect logistic regression was fit to the data and backwards elimination was performed. No variables were removed during the backwards selection process. Results Both typically hearing and hearing impaired participants can identify the sounds paired spatially with tactile stimuli with accuracy significantly above chance Two Brunner-Munzel tests were conducted in order to compare the performance of each group (typically-hearing [TH] and hearing-impaired [HI]) in correctly identifying the stimulus type against chance (25%). Both typically-hearing and hearing-impaired groups performed better than chance according to the tests (TH: Brunner-Munzel Test Statistic = -59, df = 59, p < 0.001, 95% CI: -0.008, 0.025; HI: Brunner-Munzel Test Statistic = -5.812, df = 19, p < 0.001, 95% CI: -0.044, 0.244). See Fig. 3 . Hearing impaired individuals and typically hearing improve significantly throughout the task Using mixed-effects logistic regression, among others, the results indicate that the hearing-impaired and the typically-hearing controls performed differently (p = 0.001). The results also indicate that subjects improved throughout the experiment (p < 0.001). The different stimuli were also found to effect performance. (See the results of the mixed-effect logistic regression summarized below, in Table 2 ) The interactions between: a) trial number and group belonging, and b) stimulus type and group belonging were also tested. Both interactions were insignificant [trial num X group: \(\:{X}^{2}\) (1) = 2.419, p = 0.120, stimulus X group: \(\:{X}^{2}\) (3) = 4.220, p = 0.239]. Figure 4 shows the mixed-effects model considering the total scores (all stimuli together) and separated by the different target sources. Table 2 Fixed Effects of Mixed-Effect Logistic Regression Model for Identification of Stimulus Type (∗∗∗ p < 0.001, ∗∗ p < 0.01, ∗ p |z|) Intercept + Group HI + Stimulus 1 1.120 0.432 2.596 0.009** Stimulus 2 - Stimulus 1 -1.016 0.254 -4.003 < 0.001*** Stimulus 3 - Stimulus 1 -0.761 0.259 -2.938 0.003** Stimlus 4 - Stimulus 1 -0.568 0.265 -2.142 0.032* Group TH - Group HI 1.347 0.420 3.212 0.001** Trial Num 0.095 0.013 7.044 < 0.001*** Table 2 depicts the questions and responses of the hearing impaired individuals to several questions following the task. Table 2 Subjective questionnaire and responses following Task 1 (“Forest”). No Question Responses in CI/HA (N = 20) [mean +/- SD] 1 How easy/difficult was it for you to localize/follow the sound? [1–10, “very difficult” to “very easy”] 7.9+/-2.2 2 How confident did you feel with your answers? [1–10, “very unconfident” to “very confident”] 7.3+/-2.8 3 Were the sounds and the tactile inputs connected? [1–10, “not connected” to “very connected”] 7.4+/-2.9 4 How easy/hard was it to combine the information that came from auditory and tactile inputs ? [1–10, “very difficult” to “very easy”] 8.05+/-2.03 Discussion Rapid multisensory binding between tactile and audio sources in a complex environment The purpose of the study was to assess the potential for binding of a spatially oriented tactile stimulus to an auditory source in a complex environment and using it to identify the latter. Our results show that both the typically-hearing and hearing-impaired individuals performed the audio-tactile pairing with accuracy significantly above chance. This was found with regards to all of the target sounds. In fact, many of the hearing-impaired participants performed in a manner comparable to the typically-hearing population (above 75%; see Fig. 3 ). However, the hearing-impaired population, on average, performed the task with lower accuracy in comparison to the typically-hearing population. Although the task was a multisensory binding task, and not an auditory task per se (the task could not be performed using auditory information alone), auditory capabilities seem to affect the performance. The results may for instance be impacted by the hearing-impaired group’s reduced ability to depend on auditory spatial cues. Poor localization due to a hearing deficit can affect identification of environmental sounds (as opposed to speech), as they can easily be mistaken for irrelevant background noise or noises caused by the hearing assistive devices (Shafiro 2015). An even more challenging task is to detect and separate multiple competing simultaneous sources (Oxenham, 2008). Performing source separation is thought to be somewhat dependent on localization capabilities, as well as on the ability to detect variation in other features, such as timbre and temporal characteristics (Eramudugolla, 2008; Middlebrooks, 2020). Similar reduced abilities are also demonstrated through tasks intended to test stream segregation using iterated rippled noises (Shearer et al, 2018). Characteristics shown to contribute to confusion between audio sources with similar characteristics may have similar impact within complex environments as well, these include similar fundamental frequencies, particularly on the mid-low range of the auditory spectrum (around 250Hz) and similarities in spectral characteristics. These effects become visible in the comparison of responses for each of the alternative forced choice audio sources. While the owl seems to be the most highly recognized source in both groups, the wolf which happens to be within a similar frequency range, is also most commonly confused with the owl. The wolf is thus found to be the lowest scoring source overall (see Fig. 3 ). The other two sources which contained higher and lower frequencies were found to be confused less often. Other such characteristics which could have impacted the results are the ordered overtone structure of the Owl and Wolf sound, which unlike the Thunder and Cricket which have less ordered spectral characteristics, create a clear pitch profile. Thus, it seems that the range of frequencies plays an important role in the participants’ ability to match among auditory and tactile stimuli. Further research is needed in order to test each parameter in the complex environment separately, to assess the contribution of the specific characteristics of the stimulus (frequency, timber, onset/offset) to successful audio-tactile binding. Participants rapidly improve in the binding task without any dedicated training A trial by trial analysis of the results indicates that participants of both populations improved in the task throughout the study, and such improvement can be seen for every one of the 4AFC target sounds. Such rapid improvement indicates potential implicit learning of the multisensory task. This finding is crucial, since improving integration between the auditory and tactile modality through untrained means would be paramount to the potential use of a multisensory based device for rehabilitative purposes. The immediate learning curve also indicates that accuracy can be enhanced through exposure, considering no training or feedback whatsoever was provided throughout the experimental paradigm (Lauzon, et al, 2022; Shams & Seitz, 2008). Although the typically-hearing group had higher audio-tactile identification capabilities than the hearing-impaired to begin with, this group improved throughout the experimental paradigm as well. The hearing-impaired furthermore came much closer to the typically-hearing group at the end of the experiment. The binding task was new for both populations, considering none of them have ever experienced the spatialized touch of TMA before. Nevertheless, one must consider that in real world situations there are opportunities to experience an integrated perception of auditory and tactile stimuli, including when touching our own body while speaking, singing or otherwise vocalizing, phone ringing and vibrating. However, the improvement in the task may have also been the result of enhanced familiarity with the audio scene, and more likely even the tactile characteristics of the target stimuli and their correspondence to the various auditory streams. Further research is needed to better evaluate the generalizability of the improved audio-tactile abilities to unfamiliar environments. Hearing impaired individuals respond positively to the device Subjective responses indicate that the participants were able to easily locate the target sounds, and confident about their choices in the task. They further indicate that they felt the auditory and tactile information was “connected” and that combining this information was rather easy. These responses align well with their objective results, showing a rapid capability to perform the task. Indeed the hearing-impaired participants may be relying on an already familiar use of tactile vibration, which this population is known to use in order to enhance auditory capability and learnability in the natural world (Sorgini, 2017). In combination with our prior work showing rapid spatial localization capabilities using TMA (Snir & Ciesla, 2024b), we believe that the current findings serve as further proof that hearing-impaired individuals could quickly learn to experience particular tactile sources and their locations produced by the device. Implications of our research to technology development The current work is an initial stepping stone towards the consideration of using multisensory based sensory enhancement tools by hearing-impaired within naturalistic environments. It shows that a dedicated tactile device can indeed serve as a sensory enhancement solution which is implicitly learned and rapidly usable. Such findings are important for clinical use, as current clinical setups for training are rarely dedicated to the consideration of such naturalistic environments. Declarations Availability of Data and Materials The datasets used and/or analysed during the current study available from the corresponding author on reasonable request. References Cléry J, Guipponi O, Wardak C, Ben Hamed S (2015) Neuronal bases of peripersonal and extrapersonal spaces, their plasticity and their dynamics: knowns and unknowns. Neuropsychologia. ;70:313 – 26. 10.1016/j.neuropsychologia.2014.10.022 . Epub 2014 Oct 24. PMID: 25447371 Glyde H, Cameron S, Dillon H, Hickson L, Seeto M (2013) The Effects of Hearing Impairment and Aging on Spatial Processing. Ear Hear 34(1):15. https://doi.org/10.1097/AUD.0b013e3182617f94 Gfeller K, Driscoll V, Schwalje A (2019) Adult cochlear implant recipients’ perspectives on experiences with music in everyday life: A multifaceted and dynamic phenomenon. Front NeuroSci 13:1229 Auditory Scene AnalysisThe Perceptual Organization of Sound | Books Gateway | MIT Press. (n.d.). Retrieved December 25 (2024) from https://direct.mit.edu/books/monograph/3887/Auditory-Scene-AnalysisThe-Perceptual-Organization Progress in Understanding Auditory Scene Analysis | Music Perception | University of California Press. (n.d.). Retrieved December 25 (2024) from https://online-ucpress-edu.eu1.proxy.openathens.net/mp/article-abstract/33/1/12/62665/Progress-in-Understanding-Auditory-Scene-Analysis?redirectedFrom=PDF Bizley JK, Cohen YE (2013) The what, where and how of auditory-object perception. Nat Rev Neurosci 14(10):693–707. https://doi.org/10.1038/nrn3565 Bregman AS (2015) Progress in understanding auditory scene analysis. Music Percept 33(1):12–19. https://doi.org/10.1525/mp.2015.33.1.12 Sussman ES (2017) Auditory Scene Analysis: An Attention Perspective. J Speech Lang Hear Res 60(10):2989–3000. https://doi.org/10.1044/2017_JSLHR-H-17-0041 King AJ, Hammond-Kenny A, Nodal FR (2019) Multisensory Processing in the Auditory Cortex. In A. K. C. Lee, M. T. Wallace, A. B. Coffin, A. N. Popper, & R. R. Fay (Eds.), Multisensory Processes: The Auditory Perspective (pp. 105–133). Springer International Publishing. https://doi.org/10.1007/978-3-030-10461-0_6 Serafin S (2023) Audio in Multisensory Interactions: From Experiments to Experiences. In M. Geronazzo & S. Serafin (Eds.), Sonic Interactions in Virtual Environments (pp. 305–318). Springer International Publishing. https://doi.org/10.1007/978-3-031-04021-4_10 von Kriegstein K (2012) A Multisensory Perspective on Human Auditory Communication. The Neural Bases of Multisensory Processes. CRC Press/Taylor & Francis, Boca Raton (FL), p 22593871 Innes-Brown H, Crewther D (2009) The Impact of Spatial Incongruence on an Auditory-Visual Illusion. PLoS ONE 4(7):e6450. https://doi.org/10.1371/journal.pone.0006450 Bonath etal et al (2007) - Neural Basis of the Ventriloquist Illusion: https://www.cell.com/AJHG/fulltext/S0960-9822(07)01906-9 Bonath etal et al (2014) - Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex - https://www.sciencedirect.com/science/article/abs/pii/S1053811914003577 Chen L, Vroomen J (2013) Intersensory binding across space and time: A tutorial review. Atten Percept Psychophys 75:790–811. https://doi.org/10.3758/s13414-013-0475-4 Alais D, Burr D (2004) The Ventriloquist Effect Results from Near-Optimal Bimodal Integration. Curr Biol 14(3):257–262. https://doi.org/10.1016/j.cub.2004.01.029 Bruns P (2019) The Ventriloquist Illusion as a Tool to Study Multisensory Processing: An Update. Front Integr Nuerosci 13. https://doi.org/10.3389/fnint.2019.00051 Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile capture of audition. Percept Psychophys 64(4):616–630. https://doi.org/10.3758/BF03194730 Occelli V, Spence C, Zampini M (2009) Compatibility effects between sound frequency and tactile elevation. NeuroReport 20(8):793–797. https://doi.org/10.1097/WNR.0b013e32832b8069 Driver J, Spence C (1998) Cross-modal links in spatial attention. Philosophical Trans Royal Soc B: Biol Sci 353(1373):1319–1331 Schürmann M, Caetano G, Hlushchuk Y, Jousmäki V, Hari R (2006) Touch activates human auditory cortex. NeuroImage 30(4):1325–1331. 10.1016/j.neuroimage.2005.11.020 Epub 2006 Feb 17. PMID: 16488157 Röder B, Teder-Sälejärvi W, Sterr A, Rösler F, Hillyard SA, Neville HJ (1999) Improved auditory spatial tuning in blind humans. Nature 400(6740):162–166. https://doi.org/10.1038/22106 Landelle C, Caron-Guyon J, Nazarian B, Anton JL, Sein J, Pruvost L, Amberg M, Giraud F, Félician O, Danna J, Kavounoudias A (2023) Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement. iScience 26(10):107965 PMID: 37810223; PMCID: PMC10551894 Crommett LE, Madala D, Yau JM (2019) Multisensory perceptual interactions between higher-order temporal frequency signals. J Exp Psychol Gen 148(7):1124–1137. 10.1037/xge0000513 Epub 2018 Oct 18. PMID: 30335446; PMCID: PMC6472995 Smith (2017) https://www.sciencedirect.com/science/article/pii/S0010027717300240 Gori M, Sandini G, Martinoli C, Burr DC (2014) Impairment of auditory spatial localization in congenitally blind human subjects. Brain 137(Pt 1):288–293. 10.1093/brain/awt311 Epub 2013 Nov 21. PMID: 24271326; PMCID: PMC3891446 Gori M, Sandini G, Martinoli C, Burr DC (2014) Impairment of auditory spatial localization in congenitally blind human subjects. Brain 137(Pt 1):288–293. 10.1093/brain/awt311 Epub 2013 Nov 21. PMID: 24271326; PMCID: PMC3891446 Snir A, Cieśla K, Vekslar R, Amedi A (2024) Highly compromised auditory spatial perception in aided congenitally hearing-impaired and rapid improvement with tactile technology. IScience, 27(9) Snir A et al Highly compromised auditory spatial perception in aided congenitally hearing-impaired and rapid improvement with tactile technology iScience, 27, Issue 9, 110808 Fletcher MD, Cunningham RO, Mills SR (2020) Electro-Haptic Enhancement of Spatial Hearing in Cochlear Implant Users. Sci Rep 10(1):1621. https://doi.org/10.1038/s41598-020-58503-8 Karimi-Boroujeni M, Dajani HR, Giguère C (2023) Perception of Prosody in Hearing-Impaired Individuals and Users of Hearing Assistive Devices: An Overview of Recent Advances. J Speech Lang Hear Res 66(2):775–789. 10.1044/2022_JSLHR-22-00125 Epub 2023 Jan 18. PMID: 36652704 Johnson JA, Xu J, Cox RM Impact of Hearing Aid Technology on Outcomes in Daily Life III: Localization. Ear Hear. 2017 Nov/Dec ;38(6):746–759. 10.1097/AUD.0000000000000473 . PMID: 28700447; PMCID: PMC5659949. Akeroyd MA (2014) An Overview of the Major Phenomena of the Localization of Sound Sources by Normal-Hearing, Hearing-Impaired, and Aided Listeners. Trends Hear 18. 10.1177/2331216514560442 Zheng Y, Swanson J, Koehnke J, Guan J (2022) Sound Localization of Listeners With Normal Hearing, Impaired Hearing, Hearing Aids, Bone-Anchored Hearing Instruments, and Cochlear Implants: A Review. Am J Audiol 31(3):819–834. 10.1044/2022_AJA-22-00006 Epub 2022 Aug 2. PMID: 35917460 Shaerer DE, Molis MR, Bennett KO, Leek MR (2018) Auditory stream segregation of iterated rippled noises by normal-hearing and hearing-impaired listeners. J Acoust Soc Am 143(1):378–387. https://doi.org/10.1121/1.5021333 Oxenham AJ (2008) Pitch Perception and Auditory Stream Segregation: Implications for Hearing Loss and Cochlear Implants. Trends Amplif 12(4):316–331. https://doi.org/10.1177/1084713808325881 Paredes-Gallardo A, Dau T, Marozeau J (2019) Auditory Stream Segregation Can Be Modeled by Neural Competition in Cochlear Implant Listeners. Front Comput Neurosci 13. https://doi.org/10.3389/fncom.2019.00042 Taljaard Ds, Olaithe M, Brennan-Jones Cg, Eikelboom Rh, Bucks Rs (2016) The relationship between hearing impairment and cognitive function: A meta-analysis in adults. Clin Otolaryngol 41(6):718–729. https://doi.org/10.1111/coa.12607 Uchida Y, Nishita Y, Tange C, Sugiura S, Otsuka R, Ueda H, Nakashima T, Ando F, Shimokata H (2016) The Longitudinal Impact of Hearing Impairment on Cognition Differs According to Cognitive Domain. Front Aging Neurosci 8. https://doi.org/10.3389/fnagi.2016.00201 Henry F, Glavin M, Jones E (2023) Noise Reduction in Cochlear Implant Signal Processing: A Review and Recent Developments. IEEE Rev Biomed Eng. ;16:319–331. 10.1109/RBME.2021.3095428 . Epub 2023 Jan 5. PMID: 34232892 Gajecki T, Nogueira W (2021) Enhancement of interaural level differences for bilateral cochlear implant users. Hear Res 409:108313. https://doi.org/10.1016/j.heares.2021.108313 Flores Ramones A, del-Rio-Guerra MS (2023) Recent Developments in Haptic Devices Designed for Hearing-Impaired People: A Literature Review. Sensors 23(6):6. https://doi.org/10.3390/s23062968 Sorgini F, Caliò R, Carrozza MC, Oddo CM (2018) Haptic-assistive technologies for audition and vision sensory disabilities. Disabil Rehabilitation: Assist Technol 13(4):394–421. https://doi.org/10.1080/17483107.2017.1385100 Cieśla K, Wolak T, Lorens A, Mentzel M, Skarżyński H, Amedi A (2022) Effects of training and using an audio-tactile sensory substitution device on speech-in-noise understanding. Sci Rep 12(1):3206. https://doi.org/10.1038/s41598-022-06855-8 Fletcher MD, Hadeedi A, Goehring T, Mills SR (2019) Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Sci Rep 9(1):1. https://doi.org/10.1038/s41598-019-47718-z Huang J, Sheffield B, Lin P, Zeng F-G (2017) Electro-Tactile Stimulation Enhances Cochlear Implant Speech Recognition in Noise. Sci Rep 7(1):2196. https://doi.org/10.1038/s41598-017-02429-1 Drullman R, Bronkhorst AW (2004) Speech perception and talker segregation: Effects of level, pitch, and tactile support with multiple simultaneous talkers. J Acoust Soc Am 116(5):3090–3098. https://doi.org/10.1121/1.1802535 Occelli V, Spence C, Zampini M (2011) Audiotactile interactions in front and rear space. Neurosci Biobehav Rev 35(3):589–598. https://doi.org/10.1016/j.neubiorev.2010.07.004 Slater KD, Marozeau J (2016) The effect of tactile cues on auditory stream segregation ability of musicians and nonmusicians. Psychomusicology: Music Mind Brain 26(2):162–166. https://doi.org/10.1037/pmu0000143 Merchel S, Altinsoy ME (2020) Psychophysical comparison of the auditory and tactile perception: A survey. J Multimodal User Interfaces 14(3):271–283. https://doi.org/10.1007/s12193-020-00333-z Bernard C, Kronland-Martinet R, Fery M, Ystad S, Thoret E (2022) Tactile perception of auditory roughness. JASA Express Lett. https://doi.org/10.1121/10.0016603 Holmes NP (2007) The law of inverse effectiveness in neurons and behaviour: multisensory integration versus normal variability. Neuropsychologia 45(14):3340–3345 Noppeney U (2021) Perceptual Inference, Learning, and Attention in a Multisensory World. Annual Review of Neuroscience, 44(Volume 44, 2021), 449–473. https://doi.org/10.1146/annurev-neuro-100120-085519 Stevenson RA, Fister JK, Barnett ZP, Nidiffer AR, Wallace MT (2012) Interactions between the spatial and temporal stimulus factors that influence multisensory integration in human performance. Exp Brain Res 219:121–137 Quintero SI, Shams L, Kamal K (2022) Changing the Tendency to Integrate the Senses. Brain Sci 12:1384. https://doi.org/10.3390/brainsci12101384 Amedi A, Snir A, Wald I, Ciesla K (2024) Tactile Representation of Location Characteristics and Content in 3d (20240419252). https://www.freepatentsonline.com/y2024/0419252.html#google_vignette Pulkki V (1997) Virtual Sound Source Positioning Using Vector Base Amplitude Panning. J Audio Eng Soc 45(6):456–466 Eramudugolla R, McAnally KI, Martin RL, Irvine DRF, Mattingley JB (2008) The role of spatial location in auditory search. Hear Res 238(1):139–146. https://doi.org/10.1016/j.heares.2007.10.004 Middlebrooks JC, Waters MF (2020) Spatial Mechanisms for Segregation of Competing Sounds, and a Breakdown in Spatial Hearing. Front Neurosci. ;14:571095. 10.3389/fnins.2020.571095 . PMID: 33041763; PMCID: PMC7525094 Oxenham AJ (2008) Pitch Perception and Auditory Stream Segregation: Implications for Hearing Loss and Cochlear Implants. Trends Amplif 12(4):316–331 Lauzon SA, Abraham AE, Curcin K, Butler BE, Stevenson RA (2022) The relationship between multisensory associative learning and multisensory integration. Neuropsychologia 174:108336. https://doi.org/10.1016/j.neuropsychologia.2022.108336 Shams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12(11):411–417. https://doi.org/10.1016/j.tics.2008.07.006 Sorgini F, Caliò R, Carrozza MC, Oddo CM (2017) Haptic-assistive technologies for audition and vision sensory disabilities. Disabil Rehabilitation: Assist Technol 13(4):394–421. https://doi.org/10.1080/17483107.2017.1385100 Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7418108","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":503178499,"identity":"01e55823-b0c5-4f9f-80f9-0e5b65389ffc","order_by":0,"name":"Adi Snir","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7UlEQVRIiWNgGAWjYFCCxAYwZQAiPgAxGzspWhhngLQwE9SSwADXwswDYhHSwt+e3PzhZ5tNnjl778HHNr+2yfMxMzB++JiDW4vEmYdtkj1n0oote84lG+f23TZsY2Zglpy5DY81NxLbGHgqDiduuJFjJp3bc5sRqIWNmRePFvkbic0f/xiAtZj/tuy5bU9Qi8GNxAZpmC3MDD9uJxLUYgj0i7QM0C8GZ84YS/Y23E5uY2ZsxusXuePpjz++BYaYwfEeww8//ty2nd/efPDDR3zeh4IEMMnYBiYbCKuHa2H4Q5TiUTAKRsEoGGEAAAk4Vf+huWYkAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-0335-1362","institution":"Reichman University","correspondingAuthor":true,"prefix":"","firstName":"Adi","middleName":"","lastName":"Snir","suffix":""},{"id":503178500,"identity":"fe4cd772-711f-42a1-bb80-7614b4f37329","order_by":1,"name":"Katazyna Ciesla","email":"","orcid":"https://orcid.org/0000-0001-7783-9988","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Katazyna","middleName":"","lastName":"Ciesla","suffix":""},{"id":503178501,"identity":"b5d0cbb2-664d-433c-9d2a-2cc52234d4cb","order_by":2,"name":"Amir Amedi","email":"","orcid":"https://orcid.org/0000-0001-5042-2533","institution":"Reichman University","correspondingAuthor":false,"prefix":"","firstName":"Amir","middleName":"","lastName":"Amedi","suffix":""}],"badges":[],"createdAt":"2025-08-20 13:48:34","currentVersionCode":1,"declarations":{"humanSubjects":true,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":true,"humanSubjectConsent":true,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7418108/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7418108/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":89642588,"identity":"1ee18aa6-055e-4c2b-aa9c-2e9c34b9eff7","added_by":"auto","created_at":"2025-08-22 08:21:41","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":329247,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ea) Participants were seated in the center of a Higher-Order Ambisonic environment containing 97 loudspeakers on the walls and ceiling. Their hands were placed out to the sides with their fingers inserted into two tactile boxes. b) The test group included twenty congenitally hearing-impaired individuals, ten of which are bilateral cochlear implant users. Participant index and middle fingers were rested on separate multi-frequency tactile vibration actuators. Weighted differences among the actuators were calculated using the touch-motion algorithm (TMA) in order to reproduce target source positions and motion. Participants were asked to select the sound source in the complex audio scene which matched the tactile stimulus as part of a 4AFC paradigm.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7418108/v1/71c9f1e940b97bf1ec29d206.png"},{"id":89642236,"identity":"508a7918-2428-4c99-91c7-710d130c6169","added_by":"auto","created_at":"2025-08-22 08:13:41","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":1678109,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSpectrograms of the target sounds used within the 4AFC paradigm. Window size of 2048 samples for spectrograms and 131072 for spectrum energy depiction. Audio sample rate was 48kHz. Created using Sonic Visualizer version 4.4 developed by Chris Cannam. Spectrograms: X-axis - timeline (milliseconds); Y-axis - frequency.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-7418108/v1/0c5460a7b18c1de72322df27.png"},{"id":89642239,"identity":"4f0b171a-387a-49f0-a7fe-fac3004400b4","added_by":"auto","created_at":"2025-08-22 08:13:41","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":200885,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eParticipants’ scores. Participants were asked to select the audio source which was paired with spatialized tactile input within a complex 3D audio environment (Higher-Order Ambisonics) in which all sources were in motion. The paradigm included four possible choices (4AFC). a) Overall scores of individuals within each group. Each participant received 20 trials. Each individual’s score was calculated by dividing the amount of correct responses by 20. HI - hearing impaired (N=20), TH - typically hearing (N=59). b)\u003c/strong\u003e \u003cstrong\u003eAccuracy in selecting the correct source. TH - typically hearing (N=59); HI - hearing impaired (N=20). All sources in both groups were selected with an accuracy above chance (Brunner-Munzel test; p \u0026lt; 0.001).\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-7418108/v1/b3b4cefd0a10e0ac3c59df56.png"},{"id":89642244,"identity":"7e7cf2e0-3501-43f0-966e-3c390d9f0c98","added_by":"auto","created_at":"2025-08-22 08:13:41","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":250650,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRate of improvement per trial (mixed-effects model). a) Average probability of score per trial, considering all sound sources. b) Probability of score per trial considering each sound source separately. HI - hearing impaired (N=20); TH - typically hearing (N=59).\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-7418108/v1/9c26dc06696280db8b220a07.png"},{"id":89643735,"identity":"26b4cf9a-a0bf-4d2c-bdf0-b4e87428272a","added_by":"auto","created_at":"2025-08-22 08:37:43","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3810657,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7418108/v1/81564e71-f3ff-4740-8cc4-2ec879773ab4.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eA novel tactile technology enables sound source identification by hearing-impaired individuals in a complex 3D audio environment\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eOur environment is complex in nature, containing multiple stimuli constantly changing positions, while moving in and out of the field associated with each sensory modality. The auditory system nonetheless, is the only modality which enables a simultaneous representation of the surrounding three-dimensional environment (considering that vision is frontally oriented, while touch is typically limited to only the peripersonal space, i.e. the space within reach; Cléry et al., 2015). Audio localization can pose a challenge for hearing impaired individuals, who very often show reduced capabilities, including for sound source segregation abilities, particularly when multiple sources are present (Glyde et al., 2013).\u003c/p\u003e\u003cp\u003eVarious research has been dedicated to investigating how the auditory system is able to function when confronted by complex auditory stimuli (also coined \u003cem\u003eAuditory Scene Analysis\u003c/em\u003e; Bregman 1990). These include considerations of how one contends with multiple sound streams, each with its own spectral, spatial and temporal characteristics. Further research investigates the formation of “auditory-objects”, i.e. how a representation of each object emitting acoustical information is formed (Bizley, 2010). Attention through task manipulation has shown to be key in impacting such capabilities as well (Bregman, 2015; Sussman, 2017).\u003c/p\u003e\u003cp\u003eIn the real world nevertheless, auditory information is usually experienced as part of a multisensory construct, and indeed research on multisensory processing has accumulated evidence showing the influence of non-auditory stimulation on auditory capabilities and representations classically attributed to this modality (King, Hammond-Kenny \u0026amp; Nodal, 2019; ref).\u003c/p\u003e\u003cp\u003eMultisensory illusions are often used to demonstrate the potential effects of sensory binding on spatial awareness (Zeller et al, 2015; Innes-Brown \u0026amp; Crewther, 2009; ref). Such illusions are caused when binding information from two sensory modalities, due to temporal matching, while the sources are not spatially matching (Chen \u0026amp; Vroomen, 2013). One such instance is the well known ventriloquism effect, in which the perceived auditory source’s position shifts towards the visual stimulus (face; Alais \u0026amp; Burr, 2004; Bonath et al, 2014; 2007; Bruns, 2019). Similar effects have been found using somatosensory inputs, showing that congruent tactile actuation can have a biasing effect on the perceived location of auditory stimuli, coined as the “capture” of sound (Caclin, 2002; Bruns \u0026amp; Röder, 2010; Spence \u0026amp; Zampini, 2009; Driver \u0026amp; Spence, 1998). In fact, some researchers claim the effects of audio-tactile integration are almost automatic, due to the similarities among the two senses (for instance, both are able of encoding frequencies within a partially overlapping frequency range) and their adjacent structures in the brain (Schürman et al, 2006; Landelle et al, 2023; Commett et al, 2018, Ciesla x 2)\u003c/p\u003e\u003cp\u003eAnother line of research has shown that proper development of auditory spatial understanding may be dependent on information arriving from another sensory modality (e.g. vision). It was for example demonstrated that already at the age of four months old, infants respond differently to complex auditory stimuli paired with visuals, depending on whether the multisensory information is presented in a congruent or incongruent manner (Smith et al., 2017).\u003c/p\u003e\u003cp\u003eOther studies have found impaired auditory spatial capabilities in congenitally blind individuals, suggesting that a proper construction of auditory spatial representation may depend on visual experience, with vision “calibrating” the less precise auditory localization (Gori et al, 2014a). Meanwhile, some authors showed enhanced auditory processing in the periphery of blind individuals, but the findings are mixed (Röder et al, 1999). Interestingly, spatial perception has been shown to improve in congenitally blind individuals who trained using a dedicated tactile device (Gori et al, 2014b). This while training with a row of tactile actuators along the arm, each representing one of the positions in a row of loudspeakers.\u003c/p\u003e\u003cp\u003eResearch on aided congenitally hearing-impaired individuals has also shown improved capabilities of auditory localization by integrating dedicated tactile feedback in the form of vibrotactile vibrations on the body (Snir \u0026amp; Ciesla, 2024b). Similar findings have been shown in non-congenital cochlear implant users as well (Fletcher, 2020). It is still, however, not clear whether individuals with congenital hearing impairments would retain the ability to integrate spatially constructed tactile information within a complex audio environment containing multiple moving sources.\u003c/p\u003e\u003cp\u003eHearing impairments and hearing assistive devices impact perception of both the sound content (due to reduced perceived levels and frequency representation; Karimi-Boroujeni et al, 2023) and its location (in part due to the sound correction algorithms, including gain adjustments and signal compression; Johnson, Xu \u0026amp; Cox, 2017; Akeroyd, 2015; Zheng et. al, 2022). Specifically, they have reduced performance in \u003cem\u003estream segregation\u003c/em\u003e tasks (intended to test one’s ability to separate sources from within an interleaved audio stream; Shearer et al., 2018; Oxenham, 2008; Paredes-Gallardo et al., 2019).\u003c/p\u003e\u003cp\u003eHearing impairments have also been found to be associated with reduced performance in non-auditory spatial cognitive tasks (Taljaard et al, 2015; Uchida et al, 2016).\u003c/p\u003e\u003cp\u003eVast progress has been made over the past decades in advancing hearing assistive technologies such as cochlear implants (CI) and hearing aids (HA). New developments include reduction of noise in the environment (Henry et al, 2023) and consideration of spatial cues (Gajecki \u0026amp; Nogueira, 2021); yet most new features aim at improving speech comprehension, as does the training received by the users in the clinic (Carlyon \u0026amp; Goehring, 2021; Lu et al, 2019). Furthermore, many individuals who perform speech comprehension tasks well in clinical setting, continue to be challenged in real-world environments, which are inherently more complex due to both degraded signal-to-noise (SNR) as well as a multitude of constantly moving sound sources (Lewis, 2023; see further regarding the cocktail party effect and its relation to auditory spatial abilities: Arons, 1992; Ebata, 2003).\u003c/p\u003e\u003cp\u003eMultisensory based solutions for sensory enhancement have been the subject of research towards rehabilitation of multiple sensory impaired populations, amongst these are hearing-impaired individuals (Ramones \u0026amp; Guera, 2023; Sorgini et al, 2018). Solutions towards enhancing auditory capabilities in hearing-impaired population in particular have shown various levels of promise. Authors have for instance shown improved speech perception (Cieśla et al., 2022; Fletcher et al., 2019; Huang et al., 2017; Drullman \u0026amp; Bronkhorst, 2004) and improved auditory localization (Gori et al., 2014; Occelli, Spence \u0026amp; Zampini, 2011; Snir \u0026amp; Ciesla, 2024a) capabilities. Stream segregation abilities were also found to improve when using audio-tactile integration (Slater \u0026amp; Marozeau, 2016). These effects may be the result of similarities between the auditory and tactile senses (Merchel \u0026amp; Altinsoy, 2020), both capable of perceiving vibrations within a partially overlapping frequency range (between ~ 20Hz and ~ 700Hz), leading to almost automatic audio-tactile integration, even when perceiving unfamiliar, or \u003cem\u003enovel\u003c/em\u003e, sensory information (Landelle et al, 2023; Crommett, Madala \u0026amp; Yau, 2019; Bernard et al., 2022; Schürmann et al., 2006; Gick \u0026amp; Derrick, 2009; Hayward et al 2016; Snir et al, 2024b).\u003c/p\u003e\u003cp\u003eWith hearing being the only natural sensory modality which can represent one’s entire spatial field, in the current study, we set out to explore whether congenitally hearing-impaired individuals, who have demonstrated deficits in auditory spatial capabilities, would be able to bind tactile information to sources within a complex 3D audio environment in order to identify auditory sources. Integration of sensory information has often been defined in terms of temporal and spatial congruency (see: spatial and temporal rule of multisensory integration; e.g. Holmes 2007), leading to the assumption of a common source (see: \u003cem\u003ecausal inference\u003c/em\u003e;Stevenson 2012. This process of combining sensory streams into a single representation is otherwise known as sensory binding (SB; Quintero et al, 2022). Binding becomes much harder in a complex real world environment, which often contains multiple interfering sources with overlapping parameters to the target source (i.e. frequency range, position, etc; Noppeney, 2021).\u003c/p\u003e\u003cp\u003eFor this research, we utilize the touch-motion algorithm (TMA), an in-house solution developed by one of the authors (AS). TMA provides weighted vibrotactile cues to four fingertips to represent sound source positions in the surrounding 360º space, while preserving sound content. Our prior research shows that congenitally hearing impaired individuals have highly compromised localization capabilities for spatially moving audio sources (Snir \u0026amp; Ciesla, 2024b). Nevertheless performance improves quickly with the use of assistive touch (via TMA). In the current study, we further investigate the feasibility of using touch in a a dynamic 3D auditory scene, using Higher-Order Ambisonic (HOA) environment.\u003c/p\u003e\u003cp\u003eWe hypothesize that, since the auditory and vibrotactile stimuli are designed to represent the same content and spatial location, they should bind easily in the typically-hearing individuals (rules of multisensory integration).\u003c/p\u003e\u003cp\u003eWe hypothesize that the hearing impairments will affect this ability, due to both reduced auditory spatial capabilities as well as stream segregation (as compared to typical hearing). Nevertheless, in the hearing-impaired, we hypothesize that if they perform the task with high accuracy (i.e. correctly identify the sounds paired with the tactile vibrations), this will indicate successful binding despite reduced auditory capabilities. Subjective questionnaires are also used to inquire regarding the experience of the participants during their experience of auditory-tactile stimulation.\u003c/p\u003e"},{"header":"Material \u0026 Methods","content":"\u003cp\u003eThe experiment was approved by the Institutional Review Board (IRB; Ethical clearance number: P_re3_2022123) of the Reichman University, Herzliya, Israel and was in accordance with the regulations of the Declaration of Helsinki 2013. All participants gave informed consent via signature, and were either paid or received student credits for their participation. Participants were recruited through social media, the university\u0026rsquo;s student credit system and word of mouth.\u003c/p\u003e\n\u003cp\u003eParticipants\u003c/p\u003e\n\u003cp\u003eParticipants included 20 congenitally hearing-impaired subjects: 10 bilateral cochlear-implant (biCI) users (8 females, 2 males; age 25.7+/-4.85) and 10 other cochlear-implant (CI) and/or hearing-aid (HA) users (8 females, 2 males; age 30.9+/-14.67); and 59 typically-hearing subjects (30 females, 29 male; age 23.5+/-2.7). For further information on hearing-impairments and assistive devices see Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eDetails of the hearing-impaired participants.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\u003ccolgroup\u003e\u003c/colgroup\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eInitials\u003c/p\u003e\n\u003cp\u003e(code)\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eAge\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eAge at HL diagnosis\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eAge at RE CI/HA\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eAge at LE CI/HA\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eEtiology\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eDevice RE\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eDevice LE\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003ePreferred ear\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eGender\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDS (RCILDO00)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e43\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e29\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnknown\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMed-El Concerto\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eM\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTT\u003c/p\u003e\n\u003cp\u003e(RCILCI004)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e33\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e14\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eWaardenburg syndrome\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eOAS\u003c/p\u003e\n\u003cp\u003e(RCILCI005)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e26\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnknown\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, AB Harmony\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI,\u003c/p\u003e\n\u003cp\u003eAB Harmony\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eM\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eSWC\u003c/p\u003e\n\u003cp\u003e(RCILCI007)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e29\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e14\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMeningitis\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear\u003c/p\u003e\n\u003cp\u003eNucleus 24\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 24\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eM\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDT\u003c/p\u003e\n\u003cp\u003e(RCILCI009)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e33\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3*\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e14\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eWaardenburg syndrome\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear Kanso 1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear Kanso 1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eEY\u003c/p\u003e\n\u003cp\u003e(RHALCI010)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e19\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eConnexin 26\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA,\u003c/p\u003e\n\u003cp\u003ePure 13\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Medel, Sonnet\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHE\u003c/p\u003e\n\u003cp\u003e(RCILCI011)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e23\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e9\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMother\u0026rsquo;s illness during pregnancy\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 24\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Freedom\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eSZ\u003c/p\u003e\n\u003cp\u003e(RCILDO012)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e22\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e9***\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eConnexin 26\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI,\u003c/p\u003e\n\u003cp\u003eNucleus 24\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eEG (RBCLBC013)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e21\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTurner Syndrome\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA (BC)\u003c/p\u003e\n\u003cp\u003eBAHA 6 max\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA (BC)\u003c/p\u003e\n\u003cp\u003eBAHA 6 max\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eYL\u003c/p\u003e\n\u003cp\u003e(RCILHA014)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e20\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTurner Syndrome\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI,\u003c/p\u003e\n\u003cp\u003eKanso 1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA,\u003c/p\u003e\n\u003cp\u003eNaidaV50\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCS\u003c/p\u003e\n\u003cp\u003e(RCILCI015)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e22\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e17\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMother\u0026rsquo;s illness during pregnancy\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eSE\u003c/p\u003e\n\u003cp\u003e(RDOLCI016)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e22\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eGenetic (TMC1)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI\u003c/p\u003e\n\u003cp\u003eMed-el Combi 40\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eL\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eSB\u003c/p\u003e\n\u003cp\u003e(RCILCI017)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e25\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e11\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnknown\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNH (RCILDO018)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e30\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eGenetic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Nucleus 6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNT (RCILDO019)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e22\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e12****\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnknown\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e-\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNR\u003c/p\u003e\n\u003cp\u003e(RCILCI020)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e18\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e10\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eUnknown\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eRS (RHALHA021)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e59\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMeasles vaccine\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA C\u0026amp;G SP 7X BG\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA,\u003c/p\u003e\n\u003cp\u003eC\u0026amp;G SP 7XBG\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eL\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eM\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHH\u003c/p\u003e\n\u003cp\u003e(RCILCI022)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e22\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6.5\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eConnexin 26\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Cochlear, Nucleus 6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHL\u003c/p\u003e\n\u003cp\u003e(RCILCI023)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e26\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e3**\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e12\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eGenetic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, AB, Nadia\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, AB, Nadia\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTM\u003c/p\u003e\n\u003cp\u003e(RHALCI024)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e51\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBirth\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e14\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eMother\u0026rsquo;s illness during pregnancy (rheumatism)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eHA, Signia\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eCI, Med-el Opus\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eF\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"10\" align=\"left\"\u003e\n\u003cp\u003eF- female, M - male; HL - hearing loss; CI - cochlear implant; HA - hearing aid; RE - right ear, LE - left ear; *replaced at the age of 21; ** replaced at age 25; *** not using it at all, came to the test without; **** not using it at all, came to the test without; BC - bone conduction hearing aid\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAudio and Tactile Stimuli\u003c/p\u003e\n\u003cp\u003eThe experiment involved an audio-tactile source pairing task in a complex auditory environment (a synthetized Higher-Order Ambisonics scene). The scene consisted of eight \u0026ldquo;forest\u0026rdquo; sounds, six of which were single sounds and remained in constant motion, and two were ambient static tracks on the right and left side of the scene. The moving sounds could appear anywhere on the 360\u0026ordm; surrounding horizontal plane, with an elevation between \u0026minus;\u0026thinsp;30\u0026deg; and 30\u0026deg; (with the participant\u0026rsquo;s head at 0\u0026ordm;). The moving sounds were all circling around the participants at a rate of 40\u0026ndash;75 seconds per full encirclement. The four target sounds included owl hoots, a howling wolf, thunder rumbling, and cricket chirps (for spectrograms see Fig.\u0026nbsp;2). The other two moving sources were distractors and included sounds of rain and multiple types of bird chirps.\u003c/p\u003e\n\u003cp\u003eThe tactile vibrations on fingertips of two hands corresponded to a single target auditory source in both content and location (i.e. moving synchronously).\u003c/p\u003e\n\u003cp\u003eThe paradigm\u003c/p\u003e\n\u003cp\u003eThe experiment consisted of twenty trials, designed as a four alternative forced choice (4AFC) paradigm. Participants received a sheet containing four possible answers: 1) \u0026ldquo;owl\u0026rdquo;, 2) \u0026ldquo;wolf, 3) \u0026ldquo;thunder\u0026rdquo;, 4) \u0026ldquo;cricket\u0026rdquo;, and were asked to respond as to which source was paired with the simultaneous tactile feedback. Target sources were chosen using a pseudo-random order, with each participant experiencing 20 trials in total (each of the 4 target sounds was selected 5 times in pseudorandom order).\u003c/p\u003e\n\u003cp\u003eEach trial began with a 3D audio scene (in HOA) playing for a duration of 2\u0026ndash;10 seconds, then tactile vibration was delivered (while the audio continued), matching one of the four target sources. The audio-tactile scene was experienced for 10 seconds, after which the scene was stopped and participants were prompted to verbally indicate which of the four possible sounds was paired with the vibrotactile input (see Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eFollowing the experiment, participants were given a brief questionnaire to assess their subjective experience of the task, on a VAS scale 1 to 10 (Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eApparatus\u003c/p\u003e\n\u003cp\u003eThe experiment was conducted in a cube-shaped (4 x 4 x 4 meter) dimly lit room at the The Baruch Ivcher Institute For Brain, Cognition \u0026amp; Technology at Reichman University. The room is acoustically sealed from the outside and controlled for reflections on the inside using mineral wool soundproofing, with a reverberation time of \u0026lt;\u0026thinsp;0.3 s. There are 97 loudspeakers placed on the ceiling and in three rows on the walls. The participant was seated on a chair with their head at the level of the middle loudspeaker row in the center of the room. The researcher was located in a separate control room communicating with the participant using a studio talkback system. Tactile stimulators (each consisting of two discrete channels) were placed on two small tables to each side of the participant (see Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eApparatus: Audio\u003c/p\u003e\n\u003cp\u003eNinety-seven loudspeakers (JBL Control 23-1L, powered by 13 Dante-enabled amplifiers, Crown Audio DCi 8|600DA) were placed in three rings on the walls (72 total, each ring containing 24 speakers) and ceiling (25 speakers in a 5x5 equidistant grid). A uniform azimuthal angle difference of 15\u0026deg; (in relation to the center point) defined the distance between the adjacent wall speakers. The heights of the three speaker rings were: 48cm, 148cm and 248cm from the ground. The participant\u0026rsquo;s head was placed at the center of the room, at the height of the middle speaker ring.\u003c/p\u003e\n\u003cp\u003eThe loudspeaker outputs were measured from the participant\u0026rsquo;s head position (center position of the space) and corrected for the frequency response and the delay, to create a uniform audio response (measurements and corrections were configured using the Audio Architect software by HiQnet). The All-Round Ambisonic Decoding (AllRAD) method along with High-Order Ambisonics (HOA) encoding was used to synthesize audio positioning (configured to 12th order Ambisonics). Ambisonic encoding and decoding utilized the Spat\u0026thinsp;~\u0026thinsp;library (version 5.2) developed at the Institut de Recherche et Coordination Acoustique/Musique (IRCAM). All code was programmed within the Max MSP environment.\u003c/p\u003e\n\u003cp\u003eApparatus: Tactile\u003c/p\u003e\n\u003cp\u003eTactile stimulation during the experiments was emitted using two identical Vibrating Auditory Stimulators (VAS; \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.neurodevice.pl/en/\u003c/span\u003e\u003c/span\u003e\u003csup\u003e\u0026lrm;\u0026lrm;\u003c/sup\u003e), each consisting of two discrete channels and piezoelectric plates. VAS boxes were placed on two small tables to the sides of the participant. The participants placed the fingertips of two fingers (index and middle) of both hands through two holes on the side of each VAS on the vibrating plates. The openings for the fingers in each tactile device had silicone covers with slits and the boxes were internally padded with a foam layer.\u003c/p\u003e\n\u003ch3\u003e\u0026nbsp;\u003c/h3\u003e\n\u003ch3\u003eSound level measurements\u003c/h3\u003e\n\u003cp\u003eTo ensure no perceivable sound was emitted by the tactile devices during the experiment, a measurement microphone (MiniDSP UMIK-1, frequency response 8Hz-20kHz) was used to measure sound levels at the participants\u0026rsquo; ear position with and without tactile sources present. Sound levels were measured four times (30 seconds each) one time for each target sound, the tactile devices were either activated (ON) or left deactivated (OFF), while the sound was always ON. The results for the measurements were as follows : a) Target 1 (Owl) - tactile device OFF/ON: LZSmax 67.2dB/67dB; b) Target 2 (Wolf) - tactile devices OFF/ON: LZSmax 67dB/67.1dB; c) Target 3 (Thunder) - tactile devices OFF/ON: LZSmax. 67dB/66.9dB; d) Target 4 (Crickets) - tactile device OFF/ON: LZSmax. 67dB/69.5dB.\u003c/p\u003e\n\u003cp\u003eTouch Motion Algorithm (TMA)\u003c/p\u003e\n\u003cp\u003eTo represent spatial locations of the stimulus through touch, we used the azimuthal component of the touch motion algorithm (TMA; for further details see Snir \u0026amp; Ciesla 2024a, 2024b and Amedi \u0026amp; Snir, 2024) developed by the author of the current manuscript (A.S.). TMA compares the angle of the intended virtual source\u0026rsquo;s position to the tactile actuator\u0026rsquo;s virtual angle. This enables the representation of locations in 3D space on the horizontal plane by weighting vibration level differences among four discrete vibration actuators (on four fingertips of two hands). Smooth changes of the levels reproduce motion. The gain scaling of each actuator follows a logarithmic amplitude scale, in a manner similar to vector based amplitude modulation (Pulkki, 1997).\u003c/p\u003e\n\u003cp\u003eIn the current setup, each of the 4 fingers represents a diagonal orientation of the surrounding space (i.e. azimuthal 45\u0026deg;, 135\u0026deg;, 225\u0026deg; and 315\u0026deg;; the azimuthal front at 0\u0026deg;). As an example, if the index fingers of both hands received vibrations of equal intensity, this reproduced a source in front-center (0\u0026deg; azimuth); vibrations delivered on only the middle finger of the right hand indicated a stimulus in the back right position (135\u0026deg;). The vibration content of the tactile feed corresponded directly to the paired sound, i.e. no transformation of the signal was performed aside from gain weighting among the four actuators. For further details, please contact the corresponding author (A.S.).\u003c/p\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n\u003ch2\u003eData Analysis\u003c/h2\u003e\n\u003cp\u003eTrials in which subjects correctly identified the paired stimulus were designated as a \u0026lsquo;success\u0026rsquo; (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e) and those in which subjects failed to correctly identify the stimulus were designated as \u0026lsquo;failure\u0026rsquo; (0). The performance of each group (typically-hearing and hearing-impaired) was then compared to chance (25%) using a Brunner-Munzel (BM) test. The BM test was done against the one-sided alternative that the probability of success is greater than 25%.\u003c/p\u003e\n\u003cp\u003eThis data was then submitted to a mixed-effect logistic regression analysis. Backward selection was performed on the following fixed effects: group belonging (typically-hearing vs hearing-impaired), stimulus type (four target sounds), and trial number (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e). The pairwise interactions between stimulus type and group, and group and trial number were also tested.\u003c/p\u003e\n\u003cp\u003eTo test improvement throughout the task, a mixed-effect logistic regression was fit to the data and backwards elimination was performed. No variables were removed during the backwards selection process.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cem\u003e\u0026nbsp;Both typically hearing and hearing impaired participants can identify the sounds paired spatially with tactile stimuli with accuracy significantly above chance\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eTwo Brunner-Munzel tests were conducted in order to compare the performance of each group (typically-hearing [TH] and hearing-impaired [HI]) in correctly identifying the stimulus type against chance (25%). Both typically-hearing and hearing-impaired groups performed better than chance according to the tests (TH: Brunner-Munzel Test Statistic = -59, df\u0026thinsp;=\u0026thinsp;59, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001, 95% CI: -0.008, 0.025; HI: Brunner-Munzel Test Statistic = -5.812, df\u0026thinsp;=\u0026thinsp;19, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001, 95% CI: -0.044, 0.244). See Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003eHearing impaired individuals and typically hearing improve significantly throughout the task\u003c/h2\u003e\n \u003cp\u003eUsing mixed-effects logistic regression, among others, the results indicate that the hearing-impaired and the typically-hearing controls performed differently (p\u0026thinsp;=\u0026thinsp;0.001). The results also indicate that subjects improved throughout the experiment (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001). The different stimuli were also found to effect performance. (See the results of the mixed-effect logistic regression summarized below, in Table \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e) The interactions between: a) trial number and group belonging, and b) stimulus type and group belonging were also tested. Both interactions were insignificant [trial num X group: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{X}^{2}\\)\u003c/span\u003e\u003c/span\u003e(1)\u0026thinsp;=\u0026thinsp;2.419, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.120, stimulus X group: \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{X}^{2}\\)\u003c/span\u003e\u003c/span\u003e(3)\u0026thinsp;=\u0026thinsp;4.220, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.239]. Figure 4 shows the mixed-effects model considering the total scores (all stimuli together) and separated by the different target sources.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eFixed Effects of Mixed-Effect Logistic Regression Model for Identification of Stimulus Type (\u0026lowast;\u0026lowast;\u0026lowast;\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001, \u0026lowast;\u0026lowast;\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.01, \u0026lowast;\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.05). Significance threshold was \u0026alpha;\u0026thinsp;=\u0026thinsp;0.05. TH - typically hearing (N\u0026thinsp;=\u0026thinsp;59); HI - hearing-impaired (N\u0026thinsp;=\u0026thinsp;20).\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eParameter\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eEstimate\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eStd.Error\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003ez\u003c/em\u003e-value\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003ep\u003c/em\u003e-value (Pr(\u0026gt;|z|)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIntercept\u0026thinsp;+\u0026thinsp;Group HI\u0026thinsp;+\u0026thinsp;Stimulus 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.120\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.432\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.596\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.009**\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStimulus 2 - Stimulus 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-1.016\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.254\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-4.003\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u0026lt;\u0026thinsp;0.001***\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStimulus 3 - Stimulus 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-0.761\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.259\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-2.938\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.003**\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStimlus 4 - Stimulus 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-0.568\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.265\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-2.142\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.032*\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGroup TH - Group HI\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.347\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.420\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.212\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.001**\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTrial Num\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.095\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.013\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7.044\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u0026lt;\u0026thinsp;0.001***\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003ch3\u003e\u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eTable\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e depicts the questions and responses of the hearing impaired individuals to several questions following the task.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003ctable id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eSubjective questionnaire and responses following Task 1 (\u0026ldquo;Forest\u0026rdquo;).\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eQuestion\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eResponses in CI/HA (N\u0026thinsp;=\u0026thinsp;20)\u003c/p\u003e\n \u003cp\u003e[mean +/- SD]\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow easy/difficult was it for you to localize/follow the sound?\u003c/p\u003e\n \u003cp\u003e[1\u0026ndash;10, \u0026ldquo;very difficult\u0026rdquo; to \u0026ldquo;very easy\u0026rdquo;]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e7.9+/-2.2\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow confident did you feel with your answers?\u003c/p\u003e\n \u003cp\u003e[1\u0026ndash;10, \u0026ldquo;very unconfident\u0026rdquo; to \u0026ldquo;very confident\u0026rdquo;]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e7.3+/-2.8\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWere the sounds and the tactile inputs connected?\u003c/p\u003e\n \u003cp\u003e[1\u0026ndash;10, \u0026ldquo;not connected\u0026rdquo; to \u0026ldquo;very connected\u0026rdquo;]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e7.4+/-2.9\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow easy/hard was it to combine the information that came from auditory and tactile inputs ?\u003c/p\u003e\n \u003cp\u003e[1\u0026ndash;10, \u0026ldquo;very difficult\u0026rdquo; to \u0026ldquo;very easy\u0026rdquo;]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e8.05+/-2.03\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003eRapid multisensory binding between tactile and audio sources in a complex environment\u003c/h2\u003e\u003cp\u003eThe purpose of the study was to assess the potential for binding of a spatially oriented tactile stimulus to an auditory source in a complex environment and using it to identify the latter. Our results show that both the typically-hearing and hearing-impaired individuals performed the audio-tactile pairing with accuracy significantly above chance. This was found with regards to all of the target sounds. In fact, many of the hearing-impaired participants performed in a manner comparable to the typically-hearing population (above 75%; see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eHowever, the hearing-impaired population, on average, performed the task with lower accuracy in comparison to the typically-hearing population. Although the task was a multisensory binding task, and not an auditory task per se (the task could not be performed using auditory information alone), auditory capabilities seem to affect the performance. The results may for instance be impacted by the hearing-impaired group\u0026rsquo;s reduced ability to depend on auditory spatial cues. Poor localization due to a hearing deficit can affect identification of environmental sounds (as opposed to speech), as they can easily be mistaken for irrelevant background noise or noises caused by the hearing assistive devices (Shafiro 2015). An even more challenging task is to detect and separate multiple competing simultaneous sources (Oxenham, 2008). Performing source separation is thought to be somewhat dependent on localization capabilities, as well as on the ability to detect variation in other features, such as timbre and temporal characteristics (Eramudugolla, 2008; Middlebrooks, 2020). Similar reduced abilities are also demonstrated through tasks intended to test stream segregation using iterated rippled noises (Shearer et al, 2018). Characteristics shown to contribute to confusion between audio sources with similar characteristics may have similar impact within complex environments as well, these include similar fundamental frequencies, particularly on the mid-low range of the auditory spectrum (around 250Hz) and similarities in spectral characteristics.\u003c/p\u003e\u003cp\u003eThese effects become visible in the comparison of responses for each of the alternative forced choice audio sources. While the \u003cem\u003eowl\u003c/em\u003e seems to be the most highly recognized source in both groups, the \u003cem\u003ewolf\u003c/em\u003e which happens to be within a similar frequency range, is also most commonly confused with the owl. The wolf is thus found to be the lowest scoring source overall (see Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e3\u003c/span\u003e). The other two sources which contained higher and lower frequencies were found to be confused less often. Other such characteristics which could have impacted the results are the ordered overtone structure of the Owl and Wolf sound, which unlike the Thunder and Cricket which have less ordered spectral characteristics, create a clear pitch profile. Thus, it seems that the range of frequencies plays an important role in the participants\u0026rsquo; ability to match among auditory and tactile stimuli. Further research is needed in order to test each parameter in the complex environment separately, to assess the contribution of the specific characteristics of the stimulus (frequency, timber, onset/offset) to successful audio-tactile binding.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\u003ch2\u003eParticipants rapidly improve in the binding task without any dedicated training\u003c/h2\u003e\u003cp\u003e A trial by trial analysis of the results indicates that participants of both populations improved in the task throughout the study, and such improvement can be seen for every one of the 4AFC target sounds. Such rapid improvement indicates potential implicit learning of the multisensory task. This finding is crucial, since improving integration between the auditory and tactile modality through untrained means would be paramount to the potential use of a multisensory based device for rehabilitative purposes. The immediate learning curve also indicates that accuracy can be enhanced through exposure, considering no training or feedback whatsoever was provided throughout the experimental paradigm (Lauzon, et al, 2022; Shams \u0026amp; Seitz, 2008).\u003c/p\u003e\u003cp\u003eAlthough the typically-hearing group had higher audio-tactile identification capabilities than the hearing-impaired to begin with, this group improved throughout the experimental paradigm as well. The hearing-impaired furthermore came much closer to the typically-hearing group at the end of the experiment. The binding task was new for both populations, considering none of them have ever experienced the spatialized touch of TMA before. Nevertheless, one must consider that in real world situations there are opportunities to experience an integrated perception of auditory and tactile stimuli, including when touching our own body while speaking, singing or otherwise vocalizing, phone ringing and vibrating. However, the improvement in the task may have also been the result of enhanced familiarity with the audio scene, and more likely even the tactile characteristics of the target stimuli and their correspondence to the various auditory streams. Further research is needed to better evaluate the generalizability of the improved audio-tactile abilities to unfamiliar environments.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e\u003ch2\u003eHearing impaired individuals respond positively to the device\u003c/h2\u003e\u003cp\u003e Subjective responses indicate that the participants were able to easily locate the target sounds, and confident about their choices in the task. They further indicate that they felt the auditory and tactile information was \u0026ldquo;connected\u0026rdquo; and that combining this information was rather easy. These responses align well with their objective results, showing a rapid capability to perform the task. Indeed the hearing-impaired participants may be relying on an already familiar use of tactile vibration, which this population is known to use in order to enhance auditory capability and learnability in the natural world (Sorgini, 2017). In combination with our prior work showing rapid spatial localization capabilities using TMA (Snir \u0026amp; Ciesla, 2024b), we believe that the current findings serve as further proof that hearing-impaired individuals could quickly learn to experience particular tactile sources and their locations produced by the device.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\u003ch2\u003eImplications of our research to technology development\u003c/h2\u003e\u003cp\u003eThe current work is an initial stepping stone towards the consideration of using multisensory based sensory enhancement tools by hearing-impaired within naturalistic environments. It shows that a dedicated tactile device can indeed serve as a sensory enhancement solution which is implicitly learned and rapidly usable. Such findings are important for clinical use, as current clinical setups for training are rarely dedicated to the consideration of such naturalistic environments.\u003c/p\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAvailability of Data and Materials\u003c/h2\u003e\u003cp\u003eThe datasets used and/or analysed during the current study available from the corresponding author on reasonable request.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eCl\u0026eacute;ry J, Guipponi O, Wardak C, Ben Hamed S (2015) Neuronal bases of peripersonal and extrapersonal spaces, their plasticity and their dynamics: knowns and unknowns. Neuropsychologia. ;70:313\u0026thinsp;\u0026ndash;\u0026thinsp;26. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.neuropsychologia.2014.10.022\u003c/span\u003e\u003cspan address=\"10.1016/j.neuropsychologia.2014.10.022\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 2014 Oct 24. PMID: 25447371\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGlyde H, Cameron S, Dillon H, Hickson L, Seeto M (2013) The Effects of Hearing Impairment and Aging on Spatial Processing. Ear Hear 34(1):15. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1097/AUD.0b013e3182617f94\u003c/span\u003e\u003cspan address=\"10.1097/AUD.0b013e3182617f94\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGfeller K, Driscoll V, Schwalje A (2019) Adult cochlear implant recipients\u0026rsquo; perspectives on experiences with music in everyday life: A multifaceted and dynamic phenomenon. Front NeuroSci 13:1229\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAuditory Scene AnalysisThe Perceptual Organization of Sound | Books Gateway | MIT Press. (n.d.). Retrieved December 25 (2024) from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://direct.mit.edu/books/monograph/3887/Auditory-Scene-AnalysisThe-Perceptual-Organization\u003c/span\u003e\u003cspan address=\"https://direct.mit.edu/books/monograph/3887/Auditory-Scene-AnalysisThe-Perceptual-Organization\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eProgress in Understanding Auditory Scene Analysis | Music Perception | University of California Press. (n.d.). Retrieved December 25 (2024) from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://online-ucpress-edu.eu1.proxy.openathens.net/mp/article-abstract/33/1/12/62665/Progress-in-Understanding-Auditory-Scene-Analysis?redirectedFrom=PDF\u003c/span\u003e\u003cspan address=\"https://online-ucpress-edu.eu1.proxy.openathens.net/mp/article-abstract/33/1/12/62665/Progress-in-Understanding-Auditory-Scene-Analysis?redirectedFrom=PDF\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBizley JK, Cohen YE (2013) The what, where and how of auditory-object perception. Nat Rev Neurosci 14(10):693\u0026ndash;707. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/nrn3565\u003c/span\u003e\u003cspan address=\"10.1038/nrn3565\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBregman AS (2015) Progress in understanding auditory scene analysis. Music Percept 33(1):12\u0026ndash;19. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1525/mp.2015.33.1.12\u003c/span\u003e\u003cspan address=\"10.1525/mp.2015.33.1.12\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSussman ES (2017) Auditory Scene Analysis: An Attention Perspective. J Speech Lang Hear Res 60(10):2989\u0026ndash;3000. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1044/2017_JSLHR-H-17-0041\u003c/span\u003e\u003cspan address=\"10.1044/2017_JSLHR-H-17-0041\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKing AJ, Hammond-Kenny A, Nodal FR (2019) Multisensory Processing in the Auditory Cortex. In A. K. C. Lee, M. T. Wallace, A. B. Coffin, A. N. Popper, \u0026amp; R. R. Fay (Eds.), Multisensory Processes: The Auditory Perspective (pp. 105\u0026ndash;133). Springer International Publishing. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/978-3-030-10461-0_6\u003c/span\u003e\u003cspan address=\"10.1007/978-3-030-10461-0_6\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSerafin S (2023) Audio in Multisensory Interactions: From Experiments to Experiences. In M. Geronazzo \u0026amp; S. Serafin (Eds.), Sonic Interactions in Virtual Environments (pp. 305\u0026ndash;318). Springer International Publishing. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/978-3-031-04021-4_10\u003c/span\u003e\u003cspan address=\"10.1007/978-3-031-04021-4_10\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003evon Kriegstein K (2012) A Multisensory Perspective on Human Auditory Communication. The Neural Bases of Multisensory Processes. CRC Press/Taylor \u0026amp; Francis, Boca Raton (FL), p 22593871\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eInnes-Brown H, Crewther D (2009) The Impact of Spatial Incongruence on an Auditory-Visual Illusion. PLoS ONE 4(7):e6450. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1371/journal.pone.0006450\u003c/span\u003e\u003cspan address=\"10.1371/journal.pone.0006450\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBonath etal et al (2007) - Neural Basis of the Ventriloquist Illusion: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cell.com/AJHG/fulltext/S0960-9822(07)01906-9\u003c/span\u003e\u003cspan address=\"https://www.cell.com/AJHG/fulltext/S0960-9822(07)01906-9\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBonath etal et al (2014) - Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex - \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.sciencedirect.com/science/article/abs/pii/S1053811914003577\u003c/span\u003e\u003cspan address=\"https://www.sciencedirect.com/science/article/abs/pii/S1053811914003577\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChen L, Vroomen J (2013) Intersensory binding across space and time: A tutorial review. Atten Percept Psychophys 75:790\u0026ndash;811. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3758/s13414-013-0475-4\u003c/span\u003e\u003cspan address=\"10.3758/s13414-013-0475-4\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAlais D, Burr D (2004) The Ventriloquist Effect Results from Near-Optimal Bimodal Integration. Curr Biol 14(3):257\u0026ndash;262. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.cub.2004.01.029\u003c/span\u003e\u003cspan address=\"10.1016/j.cub.2004.01.029\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBruns P (2019) The Ventriloquist Illusion as a Tool to Study Multisensory Processing: An Update. Front Integr Nuerosci 13. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3389/fnint.2019.00051\u003c/span\u003e\u003cspan address=\"10.3389/fnint.2019.00051\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCaclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile capture of audition. Percept Psychophys 64(4):616\u0026ndash;630. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3758/BF03194730\u003c/span\u003e\u003cspan address=\"10.3758/BF03194730\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOccelli V, Spence C, Zampini M (2009) Compatibility effects between sound frequency and tactile elevation. NeuroReport 20(8):793\u0026ndash;797. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1097/WNR.0b013e32832b8069\u003c/span\u003e\u003cspan address=\"10.1097/WNR.0b013e32832b8069\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDriver J, Spence C (1998) Cross-modal links in spatial attention. Philosophical Trans Royal Soc B: Biol Sci 353(1373):1319\u0026ndash;1331\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSch\u0026uuml;rmann M, Caetano G, Hlushchuk Y, Jousm\u0026auml;ki V, Hari R (2006) Touch activates human auditory cortex. NeuroImage 30(4):1325\u0026ndash;1331. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.neuroimage.2005.11.020\u003c/span\u003e\u003cspan address=\"10.1016/j.neuroimage.2005.11.020\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003eEpub 2006 Feb 17. PMID: 16488157\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eR\u0026ouml;der B, Teder-S\u0026auml;lej\u0026auml;rvi W, Sterr A, R\u0026ouml;sler F, Hillyard SA, Neville HJ (1999) Improved auditory spatial tuning in blind humans. Nature 400(6740):162\u0026ndash;166. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/22106\u003c/span\u003e\u003cspan address=\"10.1038/22106\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLandelle C, Caron-Guyon J, Nazarian B, Anton JL, Sein J, Pruvost L, Amberg M, Giraud F, F\u0026eacute;lician O, Danna J, Kavounoudias A (2023) Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement. iScience 26(10):107965 PMID: 37810223; PMCID: PMC10551894\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCrommett LE, Madala D, Yau JM (2019) Multisensory perceptual interactions between higher-order temporal frequency signals. J Exp Psychol Gen 148(7):1124\u0026ndash;1137. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1037/xge0000513\u003c/span\u003e\u003cspan address=\"10.1037/xge0000513\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003eEpub 2018 Oct 18. PMID: 30335446; PMCID: PMC6472995\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSmith (2017) \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.sciencedirect.com/science/article/pii/S0010027717300240\u003c/span\u003e\u003cspan address=\"https://www.sciencedirect.com/science/article/pii/S0010027717300240\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGori M, Sandini G, Martinoli C, Burr DC (2014) Impairment of auditory spatial localization in congenitally blind human subjects. Brain 137(Pt 1):288\u0026ndash;293. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/brain/awt311\u003c/span\u003e\u003cspan address=\"10.1093/brain/awt311\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003eEpub 2013 Nov 21. PMID: 24271326; PMCID: PMC3891446\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGori M, Sandini G, Martinoli C, Burr DC (2014) Impairment of auditory spatial localization in congenitally blind human subjects. Brain 137(Pt 1):288\u0026ndash;293. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1093/brain/awt311\u003c/span\u003e\u003cspan address=\"10.1093/brain/awt311\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003eEpub 2013 Nov 21. PMID: 24271326; PMCID: PMC3891446\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSnir A, Cieśla K, Vekslar R, Amedi A (2024) Highly compromised auditory spatial perception in aided congenitally hearing-impaired and rapid improvement with tactile technology. IScience, 27(9)\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSnir A et al Highly compromised auditory spatial perception in aided congenitally hearing-impaired and rapid improvement with tactile technology iScience, 27, Issue 9, 110808\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFletcher MD, Cunningham RO, Mills SR (2020) Electro-Haptic Enhancement of Spatial Hearing in Cochlear Implant Users. Sci Rep 10(1):1621. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-020-58503-8\u003c/span\u003e\u003cspan address=\"10.1038/s41598-020-58503-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKarimi-Boroujeni M, Dajani HR, Gigu\u0026egrave;re C (2023) Perception of Prosody in Hearing-Impaired Individuals and Users of Hearing Assistive Devices: An Overview of Recent Advances. J Speech Lang Hear Res 66(2):775\u0026ndash;789. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1044/2022_JSLHR-22-00125\u003c/span\u003e\u003cspan address=\"10.1044/2022_JSLHR-22-00125\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003eEpub 2023 Jan 18. PMID: 36652704\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJohnson JA, Xu J, Cox RM Impact of Hearing Aid Technology on Outcomes in Daily Life III: Localization. Ear Hear. 2017 Nov/Dec ;38(6):746\u0026ndash;759. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/AUD.0000000000000473\u003c/span\u003e\u003cspan address=\"10.1097/AUD.0000000000000473\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PMID: 28700447; PMCID: PMC5659949.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAkeroyd MA (2014) An Overview of the Major Phenomena of the Localization of Sound Sources by Normal-Hearing, Hearing-Impaired, and Aided Listeners. Trends Hear 18. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/2331216514560442\u003c/span\u003e\u003cspan address=\"10.1177/2331216514560442\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eZheng Y, Swanson J, Koehnke J, Guan J (2022) Sound Localization of Listeners With Normal Hearing, Impaired Hearing, Hearing Aids, Bone-Anchored Hearing Instruments, and Cochlear Implants: A Review. Am J Audiol 31(3):819\u0026ndash;834. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1044/2022_AJA-22-00006\u003c/span\u003e\u003cspan address=\"10.1044/2022_AJA-22-00006\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003eEpub 2022 Aug 2. PMID: 35917460\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShaerer DE, Molis MR, Bennett KO, Leek MR (2018) Auditory stream segregation of iterated rippled noises by normal-hearing and hearing-impaired listeners. J Acoust Soc Am 143(1):378\u0026ndash;387. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1121/1.5021333\u003c/span\u003e\u003cspan address=\"10.1121/1.5021333\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOxenham AJ (2008) Pitch Perception and Auditory Stream Segregation: Implications for Hearing Loss and Cochlear Implants. Trends Amplif 12(4):316\u0026ndash;331. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/1084713808325881\u003c/span\u003e\u003cspan address=\"10.1177/1084713808325881\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eParedes-Gallardo A, Dau T, Marozeau J (2019) Auditory Stream Segregation Can Be Modeled by Neural Competition in Cochlear Implant Listeners. Front Comput Neurosci 13. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3389/fncom.2019.00042\u003c/span\u003e\u003cspan address=\"10.3389/fncom.2019.00042\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTaljaard Ds, Olaithe M, Brennan-Jones Cg, Eikelboom Rh, Bucks Rs (2016) The relationship between hearing impairment and cognitive function: A meta-analysis in adults. Clin Otolaryngol 41(6):718\u0026ndash;729. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/coa.12607\u003c/span\u003e\u003cspan address=\"10.1111/coa.12607\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eUchida Y, Nishita Y, Tange C, Sugiura S, Otsuka R, Ueda H, Nakashima T, Ando F, Shimokata H (2016) The Longitudinal Impact of Hearing Impairment on Cognition Differs According to Cognitive Domain. Front Aging Neurosci 8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3389/fnagi.2016.00201\u003c/span\u003e\u003cspan address=\"10.3389/fnagi.2016.00201\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHenry F, Glavin M, Jones E (2023) Noise Reduction in Cochlear Implant Signal Processing: A Review and Recent Developments. IEEE Rev Biomed Eng. ;16:319\u0026ndash;331. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/RBME.2021.3095428\u003c/span\u003e\u003cspan address=\"10.1109/RBME.2021.3095428\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Epub 2023 Jan 5. PMID: 34232892\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGajecki T, Nogueira W (2021) Enhancement of interaural level differences for bilateral cochlear implant users. Hear Res 409:108313. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.heares.2021.108313\u003c/span\u003e\u003cspan address=\"10.1016/j.heares.2021.108313\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFlores Ramones A, del-Rio-Guerra MS (2023) Recent Developments in Haptic Devices Designed for Hearing-Impaired People: A Literature Review. Sensors 23(6):6. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/s23062968\u003c/span\u003e\u003cspan address=\"10.3390/s23062968\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSorgini F, Cali\u0026ograve; R, Carrozza MC, Oddo CM (2018) Haptic-assistive technologies for audition and vision sensory disabilities. Disabil Rehabilitation: Assist Technol 13(4):394\u0026ndash;421. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/17483107.2017.1385100\u003c/span\u003e\u003cspan address=\"10.1080/17483107.2017.1385100\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCieśla K, Wolak T, Lorens A, Mentzel M, Skarżyński H, Amedi A (2022) Effects of training and using an audio-tactile sensory substitution device on speech-in-noise understanding. Sci Rep 12(1):3206. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-022-06855-8\u003c/span\u003e\u003cspan address=\"10.1038/s41598-022-06855-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFletcher MD, Hadeedi A, Goehring T, Mills SR (2019) Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Sci Rep 9(1):1. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-019-47718-z\u003c/span\u003e\u003cspan address=\"10.1038/s41598-019-47718-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHuang J, Sheffield B, Lin P, Zeng F-G (2017) Electro-Tactile Stimulation Enhances Cochlear Implant Speech Recognition in Noise. Sci Rep 7(1):2196. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-017-02429-1\u003c/span\u003e\u003cspan address=\"10.1038/s41598-017-02429-1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDrullman R, Bronkhorst AW (2004) Speech perception and talker segregation: Effects of level, pitch, and tactile support with multiple simultaneous talkers. J Acoust Soc Am 116(5):3090\u0026ndash;3098. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1121/1.1802535\u003c/span\u003e\u003cspan address=\"10.1121/1.1802535\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOccelli V, Spence C, Zampini M (2011) Audiotactile interactions in front and rear space. Neurosci Biobehav Rev 35(3):589\u0026ndash;598. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.neubiorev.2010.07.004\u003c/span\u003e\u003cspan address=\"10.1016/j.neubiorev.2010.07.004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSlater KD, Marozeau J (2016) The effect of tactile cues on auditory stream segregation ability of musicians and nonmusicians. Psychomusicology: Music Mind Brain 26(2):162\u0026ndash;166. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1037/pmu0000143\u003c/span\u003e\u003cspan address=\"10.1037/pmu0000143\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMerchel S, Altinsoy ME (2020) Psychophysical comparison of the auditory and tactile perception: A survey. J Multimodal User Interfaces 14(3):271\u0026ndash;283. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s12193-020-00333-z\u003c/span\u003e\u003cspan address=\"10.1007/s12193-020-00333-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBernard C, Kronland-Martinet R, Fery M, Ystad S, Thoret E (2022) Tactile perception of auditory roughness. JASA Express Lett. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1121/10.0016603\u003c/span\u003e\u003cspan address=\"10.1121/10.0016603\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHolmes NP (2007) The law of inverse effectiveness in neurons and behaviour: multisensory integration versus normal variability. Neuropsychologia 45(14):3340\u0026ndash;3345\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNoppeney U (2021) Perceptual Inference, Learning, and Attention in a Multisensory World. Annual Review of Neuroscience, 44(Volume 44, 2021), 449\u0026ndash;473. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1146/annurev-neuro-100120-085519\u003c/span\u003e\u003cspan address=\"10.1146/annurev-neuro-100120-085519\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eStevenson RA, Fister JK, Barnett ZP, Nidiffer AR, Wallace MT (2012) Interactions between the spatial and temporal stimulus factors that influence multisensory integration in human performance. Exp Brain Res 219:121\u0026ndash;137\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eQuintero SI, Shams L, Kamal K (2022) Changing the Tendency to Integrate the Senses. Brain Sci 12:1384. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.3390/brainsci12101384\u003c/span\u003e\u003cspan address=\"10.3390/brainsci12101384\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAmedi A, Snir A, Wald I, Ciesla K (2024) Tactile Representation of Location Characteristics and Content in 3d (20240419252). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.freepatentsonline.com/y2024/0419252.html#google_vignette\u003c/span\u003e\u003cspan address=\"https://www.freepatentsonline.com/y2024/0419252.html#google_vignette\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePulkki V (1997) Virtual Sound Source Positioning Using Vector Base Amplitude Panning. J Audio Eng Soc 45(6):456\u0026ndash;466\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eEramudugolla R, McAnally KI, Martin RL, Irvine DRF, Mattingley JB (2008) The role of spatial location in auditory search. Hear Res 238(1):139\u0026ndash;146. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.heares.2007.10.004\u003c/span\u003e\u003cspan address=\"10.1016/j.heares.2007.10.004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMiddlebrooks JC, Waters MF (2020) Spatial Mechanisms for Segregation of Competing Sounds, and a Breakdown in Spatial Hearing. Front Neurosci. ;14:571095. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3389/fnins.2020.571095\u003c/span\u003e\u003cspan address=\"10.3389/fnins.2020.571095\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. PMID: 33041763; PMCID: PMC7525094\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOxenham AJ (2008) Pitch Perception and Auditory Stream Segregation: Implications for Hearing Loss and Cochlear Implants. Trends Amplif 12(4):316\u0026ndash;331\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLauzon SA, Abraham AE, Curcin K, Butler BE, Stevenson RA (2022) The relationship between multisensory associative learning and multisensory integration. Neuropsychologia 174:108336. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.neuropsychologia.2022.108336\u003c/span\u003e\u003cspan address=\"10.1016/j.neuropsychologia.2022.108336\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShams L, Seitz AR (2008) Benefits of multisensory learning. Trends Cogn Sci 12(11):411\u0026ndash;417. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.tics.2008.07.006\u003c/span\u003e\u003cspan address=\"10.1016/j.tics.2008.07.006\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSorgini F, Cali\u0026ograve; R, Carrozza MC, Oddo CM (2017) Haptic-assistive technologies for audition and vision sensory disabilities. Disabil Rehabilitation: Assist Technol 13(4):394\u0026ndash;421. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/17483107.2017.1385100\u003c/span\u003e\u003cspan address=\"10.1080/17483107.2017.1385100\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[{"identity":"14201f93-7d8a-4604-84a6-b922049ab69d","identifier":"10.13039/100010663","name":"H2020 European Research Council","awardNumber":"101017884","order_by":0},{"identity":"7c7dc175-f097-44fe-b65c-6b932f237e69","identifier":"10.13039/501100000781","name":"European Research Council","awardNumber":"TouchingSpace360","order_by":1},{"identity":"fb1508c1-0d03-4475-95da-3c82a9ebb94d","identifier":"10.13039/501100003977","name":"Israel Science Foundation","awardNumber":"370924","order_by":2}],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Reichman University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Spatial Perception; Sensory Binding; Sensory Integration; Auditory Scene Analysis; Hearing Impaired; Audio-Tactile","lastPublishedDoi":"10.21203/rs.3.rs-7418108/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7418108/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e Congenitally hearing-impaired individuals have reduced auditory localization capabilities and often find identification of sources to be challenging in noisy environments. The congenital nature of their hearing-impairment and the fact that hearing is the only modality capable of representing the full three-dimensional surrounding, mean this population have never experienced a proper representation of their spatial surrounding. We use an inhouse tactile device, which performs level weighting to four vibration actuators to the fingers to reproduce spatial positions and Higher-order Ambisonics to test congenitally hearing-impaired and typically-hearing individuals on their ability to pair between localized tactile information and audio sources within a complex three-dimensional audio environment. Participants of both groups show accuracy significantly higher than chance, with the typically hearing performing better than the hearing impaired. We further see rapid improvement in the task with no training. We discuss the importance of our findings within the discourse of sensory binding and assistive development towards rehabilitation.\u003c/p\u003e","manuscriptTitle":"A novel tactile technology enables sound source identification by hearing-impaired individuals in a complex 3D audio environment","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-22 08:13:36","doi":"10.21203/rs.3.rs-7418108/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f49c6d1e-d3e3-4534-90d7-e7cbf8db6fb7","owner":[],"postedDate":"August 22nd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":53454039,"name":"Cognitive Neuroscience"},{"id":53454040,"name":"Psychology"}],"tags":[],"updatedAt":"2025-08-22T08:13:36+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-22 08:13:36","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7418108","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7418108","identity":"rs-7418108","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}