Audio system using binaural synthesis for multimodal telepresence applications

An audio system was developed as part of a multimodal system aiming to go beyond current state of the art in telepresence.This paper provides an overview of how the audio was implemented and documents measurements that were performed on the audio system. The measurements include equalization of microphones, headphones and loudspeakers as well as measurements of network latency and bandwidth requirements of the system. Furthermore, measurements were made to determine whether the level of echo and
cross talk cause any issues. The overall system employs multiple modalities to virtually transport a person (the visitor) to a different physical location (the destination). The goal is that both the visitor and the people physically at the destination (the locals) should be provided with a sensation that the visitor is really there. Both the general multimodal system and the auditory part are implemented in a distributed manner. Body-tracking of all participants is provided through the system for the purpose of using binaural synthesis for directional sound. Head-worn microphones are used to capture sound, and the visitor is provided with directional sound through headphones. The visitor is represented to the locals through a physical representation (eg. a
robot) with a loudspeaker.