Abstract
This paper describes an approach using machine learning to extract sound expressiveness and translate it into
light. An autoencoder was trained on several musical examples, thereby learning how to compress a constant Q
transformed audio input, and then the latent space representation was used to generate visuals, in real time. A single interactive control parameter, the interpolation speed
between new and current values, was made for customisation. The expressiveness of the visuals was tested through
a variety of musical textures and genres rated by participants. Results indicate that participants found the system’s
translations to be visually expressive, and reacted very positively to the experience. The control parameter was tested
for customization potential and found to be a good tool
for allowing the participant to adjust the visual expression.
The method for expressive visualization used in the system
shows promise for further development.
light. An autoencoder was trained on several musical examples, thereby learning how to compress a constant Q
transformed audio input, and then the latent space representation was used to generate visuals, in real time. A single interactive control parameter, the interpolation speed
between new and current values, was made for customisation. The expressiveness of the visuals was tested through
a variety of musical textures and genres rated by participants. Results indicate that participants found the system’s
translations to be visually expressive, and reacted very positively to the experience. The control parameter was tested
for customization potential and found to be a good tool
for allowing the participant to adjust the visual expression.
The method for expressive visualization used in the system
shows promise for further development.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 17th Sound and Music Computing Conference |
| Editors | S. Spagnol, A. Valle |
| Number of pages | 8 |
| Publisher | Axea sas/SMC Network |
| Publication date | 20 Jun 2020 |
| Pages | 378-385 |
| ISBN (Electronic) | 978-88-945415-0-2 |
| DOIs | |
| Publication status | Published - 20 Jun 2020 |
| Event | 17th Sound and Music Computing Conference - Torino, Italy Duration: 24 Jun 2020 → 26 Jun 2020 Conference number: 17 https://smc2020torino.it/uk/ |
Conference
| Conference | 17th Sound and Music Computing Conference |
|---|---|
| Number | 17 |
| Country/Territory | Italy |
| City | Torino |
| Period | 24/06/2020 → 26/06/2020 |
| Internet address |
| Series | Proceedings of the Sound and Music Computing Conference |
|---|---|
| ISSN | 2518-3672 |