Setup for demonstrating interactive binaural synthesis for telepresence applications

Esben Madsen, Søren Krarup Olesen, Milos Markovic, Pablo F. Hoffmann, Dorte Hammershøi

Publikation: Bidrag til tidsskriftKonferenceabstrakt i tidsskriftForskningpeer review

643 Downloads (Pure)

Resumé

In the telepresence research project BEAMING, a prototype system has been set up to demonstrate basic audio and video interaction between two distant locations: the Destination, where 2 Locals are present and the Visitor Site where 1 Visitor is present. This paper describes the auditory parts of this system as well as interfaces to relevant parts of the complete system, including tracking and network streaming.

In the demonstration, the Visitor is wearing headphones and a microphone. At the Destination, the two Locals are both wearing a microphone, while the Visitor is represented using a fixed position Totem with a single loudspeaker. The Position and movements of participants, particularly the head, are tracked and from this sound is rendered to include binaural cues so the Visitor is able to move around in a limited space while perceiving Destination sound as "stationary".

This setup includes 3 main tasks:
Tracking coordinates are combined to calculate directions. This is handled by sharing "global" coordinates across the sites and adding local changes with a low latency, ending with a "direction of sound" for each source. Audio is recorded and transmitted over network. Here bandwidth, latency and transmission reliablity must be adjusted to obtain the best compromise. Bandwidth use and reliablity can be improved at the cost of latency. Finally the binaural synthesis for each source is processed at the listener's site (here Visitor) to have a minimum latency on responding to movement. The combined system is evaluated by the experience of the test users.
OriginalsprogEngelsk
TidsskriftAcustica United with Acta Acustica
Vol/bind97
Udgave nummerSupplement 1
Sider (fra-til)S 90
ISSN1610-1928
StatusUdgivet - 2011
BegivenhedForum Acusticum 2011 - Aalborg, Danmark
Varighed: 27 jun. 20111 jul. 2011
Konferencens nummer: 6

Konference

KonferenceForum Acusticum 2011
Nummer6
LandDanmark
ByAalborg
Periode27/06/201101/07/2011

Fingerprint

microphones
acoustics
synthesis
bandwidth
research projects
loudspeakers
cues
prototypes
interactions
Latency
Sound

Citer dette

Madsen, E., Olesen, S. K., Markovic, M., Hoffmann, P. F., & Hammershøi, D. (2011). Setup for demonstrating interactive binaural synthesis for telepresence applications. Acustica United with Acta Acustica, 97(Supplement 1), S 90.
Madsen, Esben ; Olesen, Søren Krarup ; Markovic, Milos ; Hoffmann, Pablo F. ; Hammershøi, Dorte. / Setup for demonstrating interactive binaural synthesis for telepresence applications. I: Acustica United with Acta Acustica. 2011 ; Bind 97, Nr. Supplement 1. s. S 90.
@article{5f3d0c1d763b4a709d2f7c7e89352119,
title = "Setup for demonstrating interactive binaural synthesis for telepresence applications",
abstract = "In the telepresence research project BEAMING, a prototype system has been set up to demonstrate basic audio and video interaction between two distant locations: the Destination, where 2 Locals are present and the Visitor Site where 1 Visitor is present. This paper describes the auditory parts of this system as well as interfaces to relevant parts of the complete system, including tracking and network streaming. In the demonstration, the Visitor is wearing headphones and a microphone. At the Destination, the two Locals are both wearing a microphone, while the Visitor is represented using a fixed position Totem with a single loudspeaker. The Position and movements of participants, particularly the head, are tracked and from this sound is rendered to include binaural cues so the Visitor is able to move around in a limited space while perceiving Destination sound as {"}stationary{"}. This setup includes 3 main tasks: Tracking coordinates are combined to calculate directions. This is handled by sharing {"}global{"} coordinates across the sites and adding local changes with a low latency, ending with a {"}direction of sound{"} for each source. Audio is recorded and transmitted over network. Here bandwidth, latency and transmission reliablity must be adjusted to obtain the best compromise. Bandwidth use and reliablity can be improved at the cost of latency. Finally the binaural synthesis for each source is processed at the listener's site (here Visitor) to have a minimum latency on responding to movement. The combined system is evaluated by the experience of the test users.",
author = "Esben Madsen and Olesen, {S{\o}ren Krarup} and Milos Markovic and Hoffmann, {Pablo F.} and Dorte Hammersh{\o}i",
year = "2011",
language = "English",
volume = "97",
pages = "S 90",
journal = "Acustica United with Acta Acustica",
issn = "1610-1928",
publisher = "S.Hirzel Verlag",
number = "Supplement 1",

}

Madsen, E, Olesen, SK, Markovic, M, Hoffmann, PF & Hammershøi, D 2011, 'Setup for demonstrating interactive binaural synthesis for telepresence applications', Acustica United with Acta Acustica, bind 97, nr. Supplement 1, s. S 90.

Setup for demonstrating interactive binaural synthesis for telepresence applications. / Madsen, Esben; Olesen, Søren Krarup; Markovic, Milos; Hoffmann, Pablo F.; Hammershøi, Dorte.

I: Acustica United with Acta Acustica, Bind 97, Nr. Supplement 1, 2011, s. S 90.

Publikation: Bidrag til tidsskriftKonferenceabstrakt i tidsskriftForskningpeer review

TY - ABST

T1 - Setup for demonstrating interactive binaural synthesis for telepresence applications

AU - Madsen, Esben

AU - Olesen, Søren Krarup

AU - Markovic, Milos

AU - Hoffmann, Pablo F.

AU - Hammershøi, Dorte

PY - 2011

Y1 - 2011

N2 - In the telepresence research project BEAMING, a prototype system has been set up to demonstrate basic audio and video interaction between two distant locations: the Destination, where 2 Locals are present and the Visitor Site where 1 Visitor is present. This paper describes the auditory parts of this system as well as interfaces to relevant parts of the complete system, including tracking and network streaming. In the demonstration, the Visitor is wearing headphones and a microphone. At the Destination, the two Locals are both wearing a microphone, while the Visitor is represented using a fixed position Totem with a single loudspeaker. The Position and movements of participants, particularly the head, are tracked and from this sound is rendered to include binaural cues so the Visitor is able to move around in a limited space while perceiving Destination sound as "stationary". This setup includes 3 main tasks: Tracking coordinates are combined to calculate directions. This is handled by sharing "global" coordinates across the sites and adding local changes with a low latency, ending with a "direction of sound" for each source. Audio is recorded and transmitted over network. Here bandwidth, latency and transmission reliablity must be adjusted to obtain the best compromise. Bandwidth use and reliablity can be improved at the cost of latency. Finally the binaural synthesis for each source is processed at the listener's site (here Visitor) to have a minimum latency on responding to movement. The combined system is evaluated by the experience of the test users.

AB - In the telepresence research project BEAMING, a prototype system has been set up to demonstrate basic audio and video interaction between two distant locations: the Destination, where 2 Locals are present and the Visitor Site where 1 Visitor is present. This paper describes the auditory parts of this system as well as interfaces to relevant parts of the complete system, including tracking and network streaming. In the demonstration, the Visitor is wearing headphones and a microphone. At the Destination, the two Locals are both wearing a microphone, while the Visitor is represented using a fixed position Totem with a single loudspeaker. The Position and movements of participants, particularly the head, are tracked and from this sound is rendered to include binaural cues so the Visitor is able to move around in a limited space while perceiving Destination sound as "stationary". This setup includes 3 main tasks: Tracking coordinates are combined to calculate directions. This is handled by sharing "global" coordinates across the sites and adding local changes with a low latency, ending with a "direction of sound" for each source. Audio is recorded and transmitted over network. Here bandwidth, latency and transmission reliablity must be adjusted to obtain the best compromise. Bandwidth use and reliablity can be improved at the cost of latency. Finally the binaural synthesis for each source is processed at the listener's site (here Visitor) to have a minimum latency on responding to movement. The combined system is evaluated by the experience of the test users.

M3 - Conference abstract in journal

VL - 97

SP - S 90

JO - Acustica United with Acta Acustica

JF - Acustica United with Acta Acustica

SN - 1610-1928

IS - Supplement 1

ER -