Abstract
This paper provides initial efforts in developing and evaluating a real-time movement sonification framework for physical activity practice and learning. Reactive Video provides an interactive, vision-based, adaptive video playback with auditory feedback on users' performance to better support when learning and practicing new physical skills. We implement the sonification for auditory feedback design by extending the Web Audio API framework. The current application focuses on Tai-Chi performance and provides two main audio cues to users for several Tai Chi exercises. We provide our design approach, implementation, and sound generation and mapping, specifically for interactive systems with direct video manipulation. Our observations reveal the relationship between the movement-to-sound mapping and characteristics of the physical activity.
Originalsprog | Engelsk |
---|---|
Bogserie | Proceedings of the International Conference on New Interfaces for Musical Expression |
ISSN | 2220-4792 |
DOI | |
Status | Udgivet - 2021 |
Begivenhed | International Conference on New Interfaces for Musical Expression, 2021 - Shanghai, Kina Varighed: 14 jun. 2021 → 18 jun. 2021 |
Konference
Konference | International Conference on New Interfaces for Musical Expression, 2021 |
---|---|
Land/Område | Kina |
By | Shanghai |
Periode | 14/06/2021 → 18/06/2021 |
Bibliografisk note
Publisher Copyright:© 2021, Steering Committee of the International Conference on New Interfaces for Musical Expression. All rights reserved.