TY - GEN
T1 - Disentangled speech representation learning based on factorized hierarchical variational autoencoder with self-supervised objective
AU - Xie, Yuying
AU - Arildsen, Thomas
AU - Tan, Zheng-Hua
PY - 2021/10/28
Y1 - 2021/10/28
N2 - Disentangled representation learning aims to extract explanatory features or factors and retain salient information. Factorized hierarchical variational autoencoder (FHVAE) presents a way to disentangle a speech signal into sequential-level and segmental-level features, which represent speaker identity and speech content information, respectively. As a self-supervised objective, autoregressive predictive coding (APC), on the other hand, has been used in extracting meaningful and transferable speech features for multiple downstream tasks. Inspired by the success of these two representation learning methods, this paper proposes to integrate the APC objective into the FHVAE framework aiming at benefiting from the additional self-supervision target. The main proposed method requires neither more training data nor more computational cost at test time, but obtains improved meaningful representations while maintaining disentanglement. The experiments were conducted on the TIMIT dataset. Results demonstrate that FHVAE equipped with the additional self-supervised objective is able to learn features providing superior performance for tasks including speech recognition and speaker recognition. Furthermore, voice conversion, as one application of disentangled representation learning, has been applied and evaluated. The results show performance similar to baseline of the new framework on voice conversion.
AB - Disentangled representation learning aims to extract explanatory features or factors and retain salient information. Factorized hierarchical variational autoencoder (FHVAE) presents a way to disentangle a speech signal into sequential-level and segmental-level features, which represent speaker identity and speech content information, respectively. As a self-supervised objective, autoregressive predictive coding (APC), on the other hand, has been used in extracting meaningful and transferable speech features for multiple downstream tasks. Inspired by the success of these two representation learning methods, this paper proposes to integrate the APC objective into the FHVAE framework aiming at benefiting from the additional self-supervision target. The main proposed method requires neither more training data nor more computational cost at test time, but obtains improved meaningful representations while maintaining disentanglement. The experiments were conducted on the TIMIT dataset. Results demonstrate that FHVAE equipped with the additional self-supervised objective is able to learn features providing superior performance for tasks including speech recognition and speaker recognition. Furthermore, voice conversion, as one application of disentangled representation learning, has been applied and evaluated. The results show performance similar to baseline of the new framework on voice conversion.
KW - Disentangled representation learning
KW - autoregressive predictive coding
KW - variational autoencoder
UR - http://www.scopus.com/inward/record.url?scp=85122805866&partnerID=8YFLogxK
U2 - 10.1109/MLSP52302.2021.9596320
DO - 10.1109/MLSP52302.2021.9596320
M3 - Article in proceeding
SN - 978-1-6654-1184-4
T3 - IEEE Workshop on Machine Learning for Signal Processing
SP - 1
EP - 6
BT - 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing, MLSP 2021
PB - IEEE
T2 - 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP)
Y2 - 25 October 2021 through 28 October 2021
ER -