Improvisation and co-expression in explorative digital music systems

During the last two decades a new genre of “casual games,” defined by Juul 2010, has become popular among players of all age groups. Players who are physically present play these games in a social setting. The game play is simple and provides fairly short and intense play. This dissertation has investigated a new genre of casual games that allows players to both collaborate and be creative in their expression. A series of music-oriented games and play environments was designed, implemented and investigated according to how two players, who are not musicians, coordinate musical expression amongst each other when they are given a number of creative restrictions in the sonic/musical material that they interact with. The benefit with digital musical instruments is that non-musicians and novices can get access to limited musical material that they are immediately able to master without any musical training. The creative restrictions that the games provided were inspired by methods in music therapy and play patterns in Central African folk music. Furthermore the instrument designs concerned the distribution of musical roles that could possibly encourage players to establish specific musical relationships. The benefit of the digitally networked electronic musical instruments is that particular patterns of co-expression can be found and mediated by the music system (that also contains all individual instruments) in ways that make players aware of their mutual play and perhaps will encourage players to obtain certain kinds of co-expression. Thus the aim of the PhD study was to present players to simple musical frameworks and through these find and describe musical collaboration and co-expression in such a way that pattern recognition algorithms and mediation of shared musical expression can be developed in future designs.
The Wacom® pen tablet, a simple drawing interface, was turned into an array of digital musical instruments in order to investigate the benefit of networked musical instruments in the context of the genre of casual games. Through qualitative and quantitative studies of player action, collaboration and musical expression it was possible to narrow down the interesting moments where co-expression happens in music improvisation.
The qualitative video microanalysis of player communication and ongoing negotiation of musical expression informed the quantitative analysis of logged interaction data from both players. Results from the different music systems investigated show that players engage in a musical collaboration that is similar to verbal conversation and that they should be challenged by musical games and play environments to engage in musical dialogues that contain simultaneous and contrasting play forms.
However, results from the quantitative analysis also show that players applied their social skills to the musical context: they were able to adapt quickly to each others’ changes in tempo and they were very flexible in terms of the distribution of musical roles. Duets were most successful in their engagement in musical relationships when they introduced each other to short, repeated and slightly varied phrases. Furthermore results from the qualitative analysis show that players were very creative in their improvisation of musical content. Most duets managed to communicate musical motifs to each other, without having to plan ahead through the use of talk. Instead players used gaze, gestures, postures, and single word statements to guide each other. The process through which players collaborated musically started with imitation as the main tool to gain a shared musical understanding: the creation of a musical ground. Then later variation was used to explore a found musical theme further. In some duets new musical relationship was discovered–the solo and accompaniment relationship. Here one player typically supported the other player with a repeated rhythm or chord progression that functioned as a base for a possible musical theme. In very few cases teasing was discovered as a way of creating contrasting play.
The combination of qualitative and quantitative analysis of player action and collaboration was used to shine light upon the process of collaboration in music improvisation in an ensemble situation and the very details of how the collaboration came about. The results from each study served to formulate new forms of level design in open-ended music oriented games, as well as design opportunities and “guidelines” for the future development of new digital and interlinked musical instruments for novices. This especially concerns which pattern recognition algorithms that should be applied in different types of music systems that challenge and mediate collaboration in music improvisation.