Superhuman Hearing - Virtual Prototyping of Artificial Hearing: A Case Study on Interactions and Acoustic Beamforming

Michele Geronazzo, Luis S. Vieira, Niels Christian Nilsson, Jesper Udesen, Stefania Serafin

Research output: Contribution to journalJournal articleResearchpeer-review

1 Citation (Scopus)

Abstract

Directivity and gain in microphone array systems for hearing aids or hearable devices allow users to acoustically enhance the information of a source of interest. This source is usually positioned directly in front. This feature is called acoustic beamforming. The current study aimed to improve users' interactions with beamforming via a virtual prototyping approach in immersive virtual environments (VEs). Eighteen participants took part in experimental sessions composed of a calibration procedure and a selective auditory attention voice-pairing task. Eight concurrent speakers were placed in an anechoic environment in two virtual reality (VR) scenarios. The scenarios were a purely virtual scenario and a realistic 360° audio-visual recording. Participants were asked to find an individual optimal parameterization for three different virtual beamformers: (i) head-guided, (ii) eye gaze-guided, and (iii) a novel interaction technique called dual beamformer, where head-guided is combined with an additional hand-guided beamformer. None of the participants were able to complete the task without a virtual beamformer (i.e., in normal hearing condition) due to the high complexity introduced by the experimental design. However, participants were able to correctly pair all speakers using all three proposed interaction metaphors. Providing superhuman hearing abilities in the form of a dual acoustic beamformer guided by head and hand movements resulted in statistically significant improvements in terms of pairing time, suggesting the task-relevance of interacting with multiple points of interests.
Original languageEnglish
Article number8998401
JournalIEEE Transactions on Visualization and Computer Graphics
Volume26
Issue number5
Pages (from-to)1912-1922
Number of pages11
ISSN1077-2626
DOIs
Publication statusPublished - May 2020

Keywords

  • Acoustic beamforming
  • Artificial hearing
  • Multi-speaker scenario
  • Sonic interactions
  • Virtual prototyping
  • Virtual reality

Fingerprint Dive into the research topics of 'Superhuman Hearing - Virtual Prototyping of Artificial Hearing: A Case Study on Interactions and Acoustic Beamforming'. Together they form a unique fingerprint.

Cite this