TY - JOUR
T1 - Cortical linear encoding and decoding of sounds
T2 - Similarities and differences between naturalistic speech and music listening
AU - Simon, Adèle
AU - Bech, Søren
AU - Loquet, Gérard
AU - Østergaard, Jan
N1 - Publisher Copyright:
© 2024 The Authors. European Journal of Neuroscience published by Federation of European Neuroscience Societies and John Wiley & Sons Ltd.
PY - 2024/4
Y1 - 2024/4
N2 - Linear models are becoming increasingly popular to investigate brain activity in response to continuous and naturalistic stimuli. In the context of auditory perception, these predictive models can be ‘encoding’, when stimulus features are used to reconstruct brain activity, or ‘decoding’ when neural features are used to reconstruct the audio stimuli. These linear models are a central component of some brain–computer interfaces that can be integrated into hearing assistive devices (e.g., hearing aids). Such advanced neurotechnologies have been widely investigated when listening to speech stimuli but rarely when listening to music. Recent attempts at neural tracking of music show that the reconstruction performances are reduced compared with speech decoding. The present study investigates the performance of stimuli reconstruction and electroencephalogram prediction (decoding and encoding models) based on the cortical entrainment of temporal variations of the audio stimuli for both music and speech listening. Three hypotheses that may explain differences between speech and music stimuli reconstruction were tested to assess the importance of the speech-specific acoustic and linguistic factors. While the results obtained with encoding models suggest different underlying cortical processing between speech and music listening, no differences were found in terms of reconstruction of the stimuli or the cortical data. The results suggest that envelope-based linear modelling can be used to study both speech and music listening, despite the differences in the underlying cortical mechanisms.
AB - Linear models are becoming increasingly popular to investigate brain activity in response to continuous and naturalistic stimuli. In the context of auditory perception, these predictive models can be ‘encoding’, when stimulus features are used to reconstruct brain activity, or ‘decoding’ when neural features are used to reconstruct the audio stimuli. These linear models are a central component of some brain–computer interfaces that can be integrated into hearing assistive devices (e.g., hearing aids). Such advanced neurotechnologies have been widely investigated when listening to speech stimuli but rarely when listening to music. Recent attempts at neural tracking of music show that the reconstruction performances are reduced compared with speech decoding. The present study investigates the performance of stimuli reconstruction and electroencephalogram prediction (decoding and encoding models) based on the cortical entrainment of temporal variations of the audio stimuli for both music and speech listening. Three hypotheses that may explain differences between speech and music stimuli reconstruction were tested to assess the importance of the speech-specific acoustic and linguistic factors. While the results obtained with encoding models suggest different underlying cortical processing between speech and music listening, no differences were found in terms of reconstruction of the stimuli or the cortical data. The results suggest that envelope-based linear modelling can be used to study both speech and music listening, despite the differences in the underlying cortical mechanisms.
KW - EEG
KW - linear models
KW - music listening
KW - stimulus reconstruction
KW - temporal response functions
UR - http://www.scopus.com/inward/record.url?scp=85184267441&partnerID=8YFLogxK
U2 - 10.1109/TNSRE.2023.3291239
DO - 10.1109/TNSRE.2023.3291239
M3 - Journal article
AN - SCOPUS:85184267441
SN - 0953-816X
VL - 59
SP - 2059
EP - 2074
JO - European Journal of Neuroscience
JF - European Journal of Neuroscience
IS - 8
ER -