Cortical linear encoding and decoding of sounds: Similarities and differences between naturalistic speech and music listening

Adèle Simon*, Søren Bech, Gérard Loquet, Jan Østergaard

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

Linear models are becoming increasingly popular to investigate brain activity in response to continuous and naturalistic stimuli. In the context of auditory perception, these predictive models can be ‘encoding’, when stimulus features are used to reconstruct brain activity, or ‘decoding’ when neural features are used to reconstruct the audio stimuli. These linear models are a central component of some brain–computer interfaces that can be integrated into hearing assistive devices (e.g., hearing aids). Such advanced neurotechnologies have been widely investigated when listening to speech stimuli but rarely when listening to music. Recent attempts at neural tracking of music show that the reconstruction performances are reduced compared with speech decoding. The present study investigates the performance of stimuli reconstruction and electroencephalogram prediction (decoding and encoding models) based on the cortical entrainment of temporal variations of the audio stimuli for both music and speech listening. Three hypotheses that may explain differences between speech and music stimuli reconstruction were tested to assess the importance of the speech-specific acoustic and linguistic factors. While the results obtained with encoding models suggest different underlying cortical processing between speech and music listening, no differences were found in terms of reconstruction of the stimuli or the cortical data. The results suggest that envelope-based linear modelling can be used to study both speech and music listening, despite the differences in the underlying cortical mechanisms.

Original languageEnglish
JournalEuropean Journal of Neuroscience
Volume59
Issue number8
Pages (from-to)2059-2074
Number of pages16
ISSN0953-816X
DOIs
Publication statusPublished - Apr 2024

Bibliographical note

Publisher Copyright:
© 2024 The Authors. European Journal of Neuroscience published by Federation of European Neuroscience Societies and John Wiley & Sons Ltd.

Keywords

  • EEG
  • linear models
  • music listening
  • stimulus reconstruction
  • temporal response functions

Fingerprint

Dive into the research topics of 'Cortical linear encoding and decoding of sounds: Similarities and differences between naturalistic speech and music listening'. Together they form a unique fingerprint.

Cite this