Multimodal Analysis of Piano Performances Portraying Different Emotions

This paper discusses the role of gestural vs auditive components of a piano performance when the performer is prompted to portray a specific emo- tion. Pianist William Westney was asked to perform a short passage from a spe- cific piece of music 6 times, 3 times without making any deliberate changes, and 3 times where the music was intended to portray the emotions happy, sad and angry, respectively. Motion-capture data from all of the performances was recorded alongside the audio. We analyze differences in the data for the differ- ent emotions, both with respect to the size and shape of the pianist’s movements and with respect to the sonic qualities of the performances. We discuss probable explanations of these differences. Although differences are found in both the gestural and auditive components of the performance, we argue that the ges- tural components are of particular importance to the performer’s shaping of a musical expression.