Flip, Freeze, Flow: Exploring Interactive Sonification of Breakdancing

Whilst movement sonification has been applied within sports training,
motor rehabilitation, and several modern dance forms, it has
not yet been researched in the context of the dance style of breaking
(breakdancing in common parlance) despite significant potential
to enhance dancer expressivity and audience engagement.
As part of a student project, we designed and developed a realtime
camera-based sonification system able to respond to various
aspects of the dancer’s movement in a musical and intuitively
movement-congruent manner using a sample-based audio architecture
in Max/MSP. Parameter mappings were grounded in six
conceptual movement-sound interactions informed by the fundamental
movement vocabulary specific to breaking. Through reallife
tests and interviews with two professional dancers, we found
that the movement-sound interactions based on vertical motion
(e.g. head height on the Y-axis) were the most intuitive and engaging,
although the unintended activation of certain other effects
caused frustration. In addition, the perceptually subtle mappings
were often overlooked or ignored, highlighting the importance of
salient sonic changes to the dancer experience. Nevertheless, we
believe that our technological framework and mapping design philosophy
can inform future research on the sonification of breaking.