AVAI: A tool for expressive music visualization based on autoencoders and constant Q transformation

This paper describes an approach using machine learning to extract sound expressiveness and translate it into
light. An autoencoder was trained on several musical examples, thereby learning how to compress a constant Q
transformed audio input, and then the latent space representation was used to generate visuals, in real time. A single interactive control parameter, the interpolation speed
between new and current values, was made for customisation. The expressiveness of the visuals was tested through
a variety of musical textures and genres rated by participants. Results indicate that participants found the system’s
translations to be visually expressive, and reacted very positively to the experience. The control parameter was tested
for customization potential and found to be a good tool
for allowing the participant to adjust the visual expression.
The method for expressive visualization used in the system
shows promise for further development.