Audio-Visual Feedback for Self-monitoring Posture in Ballet Training

Esben Winther Knudsen, Malte Lindholm Hølledig, Sebastian Siem Bach-Nielsen, Rikke Katrine Petersen, Bogdan-Constantin Zanescu, Mads Juel Nielsen, Kim Helweg, Daniel Overholt, Hendrik Purwins

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

111 Downloads (Pure)

Abstract

An application for ballet training is presented that monitors the posture position (straightness of the spine and rotation of the pelvis) deviation from the ideal position in real-time. The human skeletal data is acquired through a Microsoft Kinect v2. The movement of the student is mirrored through an abstract skeletal figure and instructions are provided through a virtual teacher. Posture deviation is measured as torso misalignment, via comparing hip center joint, shoulder center joint and neck joint position with an ideal posture position retrieved through initial calibration, and pelvis deviation, expressed as the xz-rotation of the hipcenter joint. The posture deviation is sonified via a varying cut-off frequency of a high-pass filter applied to floating water sound. The posture deviation is visualized via a curve and a rigged skeleton in which the misaligned torso parts are color-coded. In an experiment with 9-12 year-old dance students from a ballet school, comparing the audio-visual feedback modality with no feedback leads to an increase in posture accuracy (p < 0.001, Cohen’s d = 1.047). Reaction card feedback and expert interviews indicate that the feedback is considered fun and useful for training independently from the teacher.
Original languageEnglish
Title of host publicationNIME 2017 Papers and Posters Proceedings
Place of PublicationCopenhagen
PublisherNew Interfaces for Musical Expression
Publication date15 May 2017
Pages71-76
Publication statusPublished - 15 May 2017
EventNew Interfaces for Musical Expression 2017 - AAU Sydhavn, Copenhagen, Denmark
Duration: 14 May 201718 May 2017
http://www.nime17.org

Conference

ConferenceNew Interfaces for Musical Expression 2017
LocationAAU Sydhavn
CountryDenmark
CityCopenhagen
Period14/05/201718/05/2017
Internet address
SeriesNIME Proceedings
Volume2017
ISSN2220-4806

Fingerprint

Feedback
High pass filters
Monitoring
Calibration
Students
Color
Experiments

Cite this

Knudsen, E. W., Hølledig, M. L., Bach-Nielsen, S. S., Petersen, R. K., Zanescu, B-C., Nielsen, M. J., ... Purwins, H. (2017). Audio-Visual Feedback for Self-monitoring Posture in Ballet Training. In NIME 2017 Papers and Posters Proceedings (pp. 71-76). Copenhagen: New Interfaces for Musical Expression. NIME Proceedings, Vol.. 2017
Knudsen, Esben Winther ; Hølledig, Malte Lindholm ; Bach-Nielsen, Sebastian Siem ; Petersen, Rikke Katrine ; Zanescu, Bogdan-Constantin ; Nielsen, Mads Juel ; Helweg, Kim ; Overholt, Daniel ; Purwins, Hendrik. / Audio-Visual Feedback for Self-monitoring Posture in Ballet Training. NIME 2017 Papers and Posters Proceedings. Copenhagen : New Interfaces for Musical Expression, 2017. pp. 71-76 (NIME Proceedings, Vol. 2017).
@inproceedings{0aa389fee56b4f87b5f8277412e40bf1,
title = "Audio-Visual Feedback for Self-monitoring Posture in Ballet Training",
abstract = "An application for ballet training is presented that monitors the posture position (straightness of the spine and rotation of the pelvis) deviation from the ideal position in real-time. The human skeletal data is acquired through a Microsoft Kinect v2. The movement of the student is mirrored through an abstract skeletal figure and instructions are provided through a virtual teacher. Posture deviation is measured as torso misalignment, via comparing hip center joint, shoulder center joint and neck joint position with an ideal posture position retrieved through initial calibration, and pelvis deviation, expressed as the xz-rotation of the hipcenter joint. The posture deviation is sonified via a varying cut-off frequency of a high-pass filter applied to floating water sound. The posture deviation is visualized via a curve and a rigged skeleton in which the misaligned torso parts are color-coded. In an experiment with 9-12 year-old dance students from a ballet school, comparing the audio-visual feedback modality with no feedback leads to an increase in posture accuracy (p < 0.001, Cohen’s d = 1.047). Reaction card feedback and expert interviews indicate that the feedback is considered fun and useful for training independently from the teacher.",
author = "Knudsen, {Esben Winther} and H{\o}lledig, {Malte Lindholm} and Bach-Nielsen, {Sebastian Siem} and Petersen, {Rikke Katrine} and Bogdan-Constantin Zanescu and Nielsen, {Mads Juel} and Kim Helweg and Daniel Overholt and Hendrik Purwins",
year = "2017",
month = "5",
day = "15",
language = "English",
series = "NIME Proceedings",
pages = "71--76",
booktitle = "NIME 2017 Papers and Posters Proceedings",
publisher = "New Interfaces for Musical Expression",

}

Knudsen, EW, Hølledig, ML, Bach-Nielsen, SS, Petersen, RK, Zanescu, B-C, Nielsen, MJ, Helweg, K, Overholt, D & Purwins, H 2017, Audio-Visual Feedback for Self-monitoring Posture in Ballet Training. in NIME 2017 Papers and Posters Proceedings. New Interfaces for Musical Expression, Copenhagen, NIME Proceedings, vol. 2017, pp. 71-76, New Interfaces for Musical Expression 2017, Copenhagen, Denmark, 14/05/2017.

Audio-Visual Feedback for Self-monitoring Posture in Ballet Training. / Knudsen, Esben Winther; Hølledig, Malte Lindholm; Bach-Nielsen, Sebastian Siem; Petersen, Rikke Katrine; Zanescu, Bogdan-Constantin; Nielsen, Mads Juel; Helweg, Kim; Overholt, Daniel; Purwins, Hendrik.

NIME 2017 Papers and Posters Proceedings. Copenhagen : New Interfaces for Musical Expression, 2017. p. 71-76 (NIME Proceedings, Vol. 2017).

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - Audio-Visual Feedback for Self-monitoring Posture in Ballet Training

AU - Knudsen, Esben Winther

AU - Hølledig, Malte Lindholm

AU - Bach-Nielsen, Sebastian Siem

AU - Petersen, Rikke Katrine

AU - Zanescu, Bogdan-Constantin

AU - Nielsen, Mads Juel

AU - Helweg, Kim

AU - Overholt, Daniel

AU - Purwins, Hendrik

PY - 2017/5/15

Y1 - 2017/5/15

N2 - An application for ballet training is presented that monitors the posture position (straightness of the spine and rotation of the pelvis) deviation from the ideal position in real-time. The human skeletal data is acquired through a Microsoft Kinect v2. The movement of the student is mirrored through an abstract skeletal figure and instructions are provided through a virtual teacher. Posture deviation is measured as torso misalignment, via comparing hip center joint, shoulder center joint and neck joint position with an ideal posture position retrieved through initial calibration, and pelvis deviation, expressed as the xz-rotation of the hipcenter joint. The posture deviation is sonified via a varying cut-off frequency of a high-pass filter applied to floating water sound. The posture deviation is visualized via a curve and a rigged skeleton in which the misaligned torso parts are color-coded. In an experiment with 9-12 year-old dance students from a ballet school, comparing the audio-visual feedback modality with no feedback leads to an increase in posture accuracy (p < 0.001, Cohen’s d = 1.047). Reaction card feedback and expert interviews indicate that the feedback is considered fun and useful for training independently from the teacher.

AB - An application for ballet training is presented that monitors the posture position (straightness of the spine and rotation of the pelvis) deviation from the ideal position in real-time. The human skeletal data is acquired through a Microsoft Kinect v2. The movement of the student is mirrored through an abstract skeletal figure and instructions are provided through a virtual teacher. Posture deviation is measured as torso misalignment, via comparing hip center joint, shoulder center joint and neck joint position with an ideal posture position retrieved through initial calibration, and pelvis deviation, expressed as the xz-rotation of the hipcenter joint. The posture deviation is sonified via a varying cut-off frequency of a high-pass filter applied to floating water sound. The posture deviation is visualized via a curve and a rigged skeleton in which the misaligned torso parts are color-coded. In an experiment with 9-12 year-old dance students from a ballet school, comparing the audio-visual feedback modality with no feedback leads to an increase in posture accuracy (p < 0.001, Cohen’s d = 1.047). Reaction card feedback and expert interviews indicate that the feedback is considered fun and useful for training independently from the teacher.

UR - http://www.nime.org

UR - http://www.nime.org/archives/

M3 - Article in proceeding

T3 - NIME Proceedings

SP - 71

EP - 76

BT - NIME 2017 Papers and Posters Proceedings

PB - New Interfaces for Musical Expression

CY - Copenhagen

ER -

Knudsen EW, Hølledig ML, Bach-Nielsen SS, Petersen RK, Zanescu B-C, Nielsen MJ et al. Audio-Visual Feedback for Self-monitoring Posture in Ballet Training. In NIME 2017 Papers and Posters Proceedings. Copenhagen: New Interfaces for Musical Expression. 2017. p. 71-76. (NIME Proceedings, Vol. 2017).