Spatial Haptics in VR for Deaf and Hard of Hearing

We have developed a head-based haptic wearable prototype designed for individuals who are deaf or hard of hearing (DHH) in Virtual Reality (VR), with the primary objective of enhancing sound perception, specifically in terms of sound localization and pattern identification. The prototype is equipped with multiple vibration devices embedded into the headset strap to effectively spatialize sound sources, providing users with an immersive surround experience. The prototype includes multiple Voice Coil Actuator which convert a wide range of sounds (such as a dog bark, ball kick, waterfall, laughter, gunshots, etc.) into distinct vibration patterns. This approach helps us to provide more adaptive vibration patterns based on each unique sound patterns, thereby enhancing the overall immersion in VR gameplay. We are currently conducting a user study with our prototype, and the preliminary results are promising. This innovative technology has the potential to significantly improve the tactile experience for individuals with hearing impairments in virtual environments. For the exhibit, the visitor will experience our prototype in a pre built VR scene where they feel the sounds around them in the virtual environment through vibration. Along with that, they can interact with objects that generate sound which they can feel through vibrations. We are expecting students and researchers who are interested in topics such as accessibility in VR, multi-modal interaction in VR, VR based wearables and the DHH community.

Iterations of Prototype

Participant performing tasks