Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

3 Citationer (Scopus)

Resumé

Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect. Current speech enhancement systems based on deep learning do not usually take into account this change in the speaking style, because they are trained with neutral (non-Lombard) speech utterances recorded under quiet conditions to which noise is artificially added. In this paper, we investigate the effects that the Lombard reflex has on the performance of audio-visual speech enhancement systems based on deep learning. The results show that a gap in the performance of as much as approximately 5 dB between the systems trained on neutral speech and the ones trained on Lombard speech exists. This indicates the benefit of taking into account the mismatch between neutral and Lombard speech in the design of audio-visual speech enhancement systems.

OriginalsprogEngelsk
TitelICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Antal sider5
ForlagIEEE
Publikationsdato17 apr. 2019
Sider6615-6619
Artikelnummer8682713
ISBN (Trykt)978-1-4799-8132-8
ISBN (Elektronisk)978-1-4799-8131-1
DOI
StatusUdgivet - 17 apr. 2019
Begivenhed2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) - Brighton, Storbritannien
Varighed: 12 maj 201917 maj 2019
https://2019.ieeeicassp.org/

Konference

Konference2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
LandStorbritannien
ByBrighton
Periode12/05/201917/05/2019
Internetadresse
NavnI E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings
ISSN1520-6149

Fingerprint

Speech enhancement
Deep learning

Citer dette

Michelsanti, D., Tan, Z-H., Sigurdsson, S., & Jensen, J. (2019). Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems. I ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (s. 6615-6619). [8682713] IEEE. I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings https://doi.org/10.1109/ICASSP.2019.8682713
Michelsanti, Daniel ; Tan, Zheng-Hua ; Sigurdsson, Sigurdur ; Jensen, Jesper. / Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019. s. 6615-6619 (I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings).
@inproceedings{17da3cfcd9684373acc52ccbe20201fc,
title = "Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems",
abstract = "Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect. Current speech enhancement systems based on deep learning do not usually take into account this change in the speaking style, because they are trained with neutral (non-Lombard) speech utterances recorded under quiet conditions to which noise is artificially added. In this paper, we investigate the effects that the Lombard reflex has on the performance of audio-visual speech enhancement systems based on deep learning. The results show that a gap in the performance of as much as approximately 5 dB between the systems trained on neutral speech and the ones trained on Lombard speech exists. This indicates the benefit of taking into account the mismatch between neutral and Lombard speech in the design of audio-visual speech enhancement systems.",
keywords = "Audio-visual speech enhancement, Lombard effect, deep learning",
author = "Daniel Michelsanti and Zheng-Hua Tan and Sigurdur Sigurdsson and Jesper Jensen",
year = "2019",
month = "4",
day = "17",
doi = "10.1109/ICASSP.2019.8682713",
language = "English",
isbn = "978-1-4799-8132-8",
series = "I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings",
publisher = "IEEE",
pages = "6615--6619",
booktitle = "ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",
address = "United States",

}

Michelsanti, D, Tan, Z-H, Sigurdsson, S & Jensen, J 2019, Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems. i ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)., 8682713, IEEE, I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings, s. 6615-6619, 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, Storbritannien, 12/05/2019. https://doi.org/10.1109/ICASSP.2019.8682713

Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems. / Michelsanti, Daniel; Tan, Zheng-Hua; Sigurdsson, Sigurdur; Jensen, Jesper.

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019. s. 6615-6619 8682713 (I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings).

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

TY - GEN

T1 - Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems

AU - Michelsanti, Daniel

AU - Tan, Zheng-Hua

AU - Sigurdsson, Sigurdur

AU - Jensen, Jesper

PY - 2019/4/17

Y1 - 2019/4/17

N2 - Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect. Current speech enhancement systems based on deep learning do not usually take into account this change in the speaking style, because they are trained with neutral (non-Lombard) speech utterances recorded under quiet conditions to which noise is artificially added. In this paper, we investigate the effects that the Lombard reflex has on the performance of audio-visual speech enhancement systems based on deep learning. The results show that a gap in the performance of as much as approximately 5 dB between the systems trained on neutral speech and the ones trained on Lombard speech exists. This indicates the benefit of taking into account the mismatch between neutral and Lombard speech in the design of audio-visual speech enhancement systems.

AB - Humans tend to change their way of speaking when they are immersed in a noisy environment, a reflex known as Lombard effect. Current speech enhancement systems based on deep learning do not usually take into account this change in the speaking style, because they are trained with neutral (non-Lombard) speech utterances recorded under quiet conditions to which noise is artificially added. In this paper, we investigate the effects that the Lombard reflex has on the performance of audio-visual speech enhancement systems based on deep learning. The results show that a gap in the performance of as much as approximately 5 dB between the systems trained on neutral speech and the ones trained on Lombard speech exists. This indicates the benefit of taking into account the mismatch between neutral and Lombard speech in the design of audio-visual speech enhancement systems.

KW - Audio-visual speech enhancement

KW - Lombard effect

KW - deep learning

UR - http://www.scopus.com/inward/record.url?scp=85068971184&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2019.8682713

DO - 10.1109/ICASSP.2019.8682713

M3 - Article in proceeding

SN - 978-1-4799-8132-8

T3 - I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings

SP - 6615

EP - 6619

BT - ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - IEEE

ER -

Michelsanti D, Tan Z-H, Sigurdsson S, Jensen J. Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems. I ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2019. s. 6615-6619. 8682713. (I E E E International Conference on Acoustics, Speech and Signal Processing. Proceedings). https://doi.org/10.1109/ICASSP.2019.8682713