The Effect of Training on Localizing Hololens-generated 3D Sound Sources

methods Visual Guide 1, Visual Guide 2, and Sound Guide, in improving sound localization skills in Augmented Reality (AR) environments using the HoloLens2 device. Sound localization is a critical aspect of human auditory perception, and AR technology offers an immersive platform for training individuals in this skill. The HoloLens2, which employs Head-Related Transfer Function (HRTF) technology, provides a more realistic sound experience in AR by simulating sounds from various directions and distances. In this study, we explore how sound source localization training can help individuals adapt to the general HRTF implemented in AR devices despite that the individual HRTFs are quite different from the general HRTF. To that end, 12 participants were divided into groups of four for each training method, and trained six times every other day for two weeks. The training methods combined visual, auditory, and kinesthetic feedback to enhance the participants’ sound localization accuracy in terms of elevation, azimuth, and distance. The experiments were conducted in a controlled testbed environment with minimal external noise interference. The results revealed that Visual Guide 2 provided the most significant training effect across all measured aspects, followed by Sound Guide, which also demonstrated considerable improvement. These findings suggest that training methods incorporating kinesthetic feedback into the visual feedback, as well as relative error guidance into the audio feedback, such as Visual Guide 2 and Sound Guide, are more effective than traditional visual training for sound localization in AR environments. Additionally, Sound Guide showed potential for training visually impaired individuals, as it relies on auditory feedback, rather than visual cues. This study underscores the importance of integrating auditory and kinesthetic feedback in AR-based sound localization training for enhanced outcomes, and highlights the potential applications of such training for various user groups, including those with visual impairments.