ADEPT: Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

18 Downloads (Pure)

Abstract

One of the biggest challenges in learning how to play a musical instrument is learning how to move one's body with a nuanced physicality. Technology can expand available forms of physical interactions to help cue specific movements and postures. This cueing can reinforce new sensorimotor couplings to enhance motor learning and performance. Using Mixed Reality (MR), we present a system that allows students to share a first-person audiovisual perspective with a piano teacher. Students place their hands into the virtual gloves of a teacher. Motor learning and audio-motor associations are reinforced through motion feedback and spatialized audio. The Augmented Design to Embody a Piano Teacher (ADEPT) application is an early design prototype of this piano training system.
Original languageEnglish
Title of host publicationProceedings of the 16th Sound & Music Computing Conference SMC 2019
Number of pages9
Volume1
PublisherSound and Music Computing Network
Publication date2019
Pages241-249
ISBN (Electronic)978-84-09-08518-7
DOIs
Publication statusPublished - 2019
Event16th Sound and music computing conference - University of Malaga (UMA). , Malaga, Spain
Duration: 28 May 201931 May 2019
http://smc2019.uma.es/index.html

Conference

Conference16th Sound and music computing conference
LocationUniversity of Malaga (UMA).
CountrySpain
CityMalaga
Period28/05/201931/05/2019
Internet address
SeriesProceedings of the Sound and Music Computing Conference
ISSN2518-3672

Fingerprint

Students
Musical instruments
Feedback

Keywords

  • augmented reality
  • mixed reality
  • embodied cognition
  • technology-enhanced learning
  • piano training applications
  • motion capture
  • audio spatialization

Cite this

Gerry, L., Dahl, S., & Serafin, S. (2019). ADEPT: Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training. In Proceedings of the 16th Sound & Music Computing Conference SMC 2019 (Vol. 1, pp. 241-249). Sound and Music Computing Network. Proceedings of the Sound and Music Computing Conference https://doi.org/10.5281/zenodo.3249333
Gerry, Lynda ; Dahl, Sofia ; Serafin, Stefania. / ADEPT : Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training. Proceedings of the 16th Sound & Music Computing Conference SMC 2019. Vol. 1 Sound and Music Computing Network, 2019. pp. 241-249 (Proceedings of the Sound and Music Computing Conference).
@inproceedings{cdc1eff704284585bc6e6d3e6481846e,
title = "ADEPT: Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training",
abstract = "One of the biggest challenges in learning how to play a musical instrument is learning how to move one's body with a nuanced physicality. Technology can expand available forms of physical interactions to help cue specific movements and postures. This cueing can reinforce new sensorimotor couplings to enhance motor learning and performance. Using Mixed Reality (MR), we present a system that allows students to share a first-person audiovisual perspective with a piano teacher. Students place their hands into the virtual gloves of a teacher. Motor learning and audio-motor associations are reinforced through motion feedback and spatialized audio. The Augmented Design to Embody a Piano Teacher (ADEPT) application is an early design prototype of this piano training system.",
keywords = "augmented reality, mixed reality, embodied cognition, technology-enhanced learning, piano training applications, motion capture, audio spatialization",
author = "Lynda Gerry and Sofia Dahl and Stefania Serafin",
year = "2019",
doi = "10.5281/zenodo.3249333",
language = "English",
volume = "1",
series = "Proceedings of the Sound and Music Computing Conference",
publisher = "Sound and Music Computing Network",
pages = "241--249",
booktitle = "Proceedings of the 16th Sound & Music Computing Conference SMC 2019",
note = "16th Sound and music computing conference, SMC2019 ; Conference date: 28-05-2019 Through 31-05-2019",
url = "http://smc2019.uma.es/index.html",

}

Gerry, L, Dahl, S & Serafin, S 2019, ADEPT: Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training. in Proceedings of the 16th Sound & Music Computing Conference SMC 2019. vol. 1, Sound and Music Computing Network, Proceedings of the Sound and Music Computing Conference, pp. 241-249, 16th Sound and music computing conference, Malaga, Spain, 28/05/2019. https://doi.org/10.5281/zenodo.3249333

ADEPT : Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training. / Gerry, Lynda; Dahl, Sofia; Serafin, Stefania.

Proceedings of the 16th Sound & Music Computing Conference SMC 2019. Vol. 1 Sound and Music Computing Network, 2019. p. 241-249 (Proceedings of the Sound and Music Computing Conference).

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - ADEPT

T2 - 16th Sound and music computing conference

AU - Gerry, Lynda

AU - Dahl, Sofia

AU - Serafin, Stefania

PY - 2019

Y1 - 2019

N2 - One of the biggest challenges in learning how to play a musical instrument is learning how to move one's body with a nuanced physicality. Technology can expand available forms of physical interactions to help cue specific movements and postures. This cueing can reinforce new sensorimotor couplings to enhance motor learning and performance. Using Mixed Reality (MR), we present a system that allows students to share a first-person audiovisual perspective with a piano teacher. Students place their hands into the virtual gloves of a teacher. Motor learning and audio-motor associations are reinforced through motion feedback and spatialized audio. The Augmented Design to Embody a Piano Teacher (ADEPT) application is an early design prototype of this piano training system.

AB - One of the biggest challenges in learning how to play a musical instrument is learning how to move one's body with a nuanced physicality. Technology can expand available forms of physical interactions to help cue specific movements and postures. This cueing can reinforce new sensorimotor couplings to enhance motor learning and performance. Using Mixed Reality (MR), we present a system that allows students to share a first-person audiovisual perspective with a piano teacher. Students place their hands into the virtual gloves of a teacher. Motor learning and audio-motor associations are reinforced through motion feedback and spatialized audio. The Augmented Design to Embody a Piano Teacher (ADEPT) application is an early design prototype of this piano training system.

KW - augmented reality

KW - mixed reality

KW - embodied cognition

KW - technology-enhanced learning

KW - piano training applications

KW - motion capture

KW - audio spatialization

UR - http://smc2019.uma.es/articles/P2/P2_05_SMC2019_paper.pdf

UR - http://smc2019.uma.es/

U2 - 10.5281/zenodo.3249333

DO - 10.5281/zenodo.3249333

M3 - Article in proceeding

VL - 1

T3 - Proceedings of the Sound and Music Computing Conference

SP - 241

EP - 249

BT - Proceedings of the 16th Sound & Music Computing Conference SMC 2019

PB - Sound and Music Computing Network

Y2 - 28 May 2019 through 31 May 2019

ER -

Gerry L, Dahl S, Serafin S. ADEPT: Exploring the Design, Pedagogy, and Analysis of a Mixed Reality Application for Piano Training. In Proceedings of the 16th Sound & Music Computing Conference SMC 2019. Vol. 1. Sound and Music Computing Network. 2019. p. 241-249. (Proceedings of the Sound and Music Computing Conference). https://doi.org/10.5281/zenodo.3249333