Two-stage Recognition and Beyond for Compound Facial Emotion Recognition

Dorota Kamińska; Kadir Aktas; Davit Rizhinashvili; Danila Kuklyanov; Abdallah Hussein Sham; Sergio Escalera; Kamal Nasrollahi; Thomas B. Moeslund; Gholamreza Anbarjafari

doi:10.3390/electronics10222847

Two-stage Recognition and Beyond for Compound Facial Emotion Recognition

Dorota Kamińska^*, Kadir Aktas, Davit Rizhinashvili, Danila Kuklyanov, Abdallah Hussein Sham, Sergio Escalera, Kamal Nasrollahi, Thomas B. Moeslund, Gholamreza Anbarjafari

^*Corresponding author for this work

Research output: Contribution to journal › Journal article › Research › peer-review

20 Citations (Scopus)

44 Downloads (Pure)

Abstract

Facial emotion recognition is an inherently complex problem due to individual facial feature diversity and racial and cultural differences. Moreover, facial expressions are typically reflecting the mixture of emotional status of people, which can be expressed using compound emotions. Compound facial emotion recognition makes the problem even more difficult because the discrimination between dominant and complementary emotions is usually weak. We have created a database that includes $31250$ facial images with different emotions of $115$ subjects whose gender distribution is almost uniform to address compound emotion recognition. In addition, we have organized a competition based on the proposed dataset, held at FG workshop 2020. This paper analyzes the winner approach - a two-stage recognition method ($1^{st}$ stage - coarse recognition, $2^{nd}$ stage - fine recognition), which enhanced the classification of symmetrical emotion labels.

Original language	English
Article number	2847
Journal	Electronics
Volume	10
Issue number	22
ISSN	2079-9292
DOIs	https://doi.org/10.3390/electronics10222847
Publication status	Published - 2021

Bibliographical note

Funding Information:
Acknowledgments: We want to express our special thanks to Zhiyuan Zhangs, Jianping Shen, Miao Yi, Juan Xu, and Rong Zhang from Pingan Life Insurance of China for participating in the competition held at the FG workshop 2020. This work has been partially supported by the Spanish project PID2019-105093GB-I00 (MINECO/FEDER, UE), CERCA Programme/Generalitat de Catalunya), ICREA under the ICREA Academia programme, and the Estonian Centre of Excellence in IT (EXCITE) funded by the European Regional Development Fund.

Publisher Copyright:
© 2021 by the authors. Licensee MDPI, Basel, Switzerland.

Keywords

Compound emotion recognition
Deep learning
Dominant and complementary emotion recognition
Facial expression recognition

Access to Document

10.3390/electronics10222847

Open Access ArticleFinal published version, 7.98 MBLicence: CC BY 4.0

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

@article{5ea4a9ae56f14438bf1b1937b9acdf31,

title = "Two-stage Recognition and Beyond for Compound Facial Emotion Recognition",

abstract = "Facial emotion recognition is an inherently complex problem due to individual facial feature diversity and racial and cultural differences. Moreover, facial expressions are typically reflecting the mixture of emotional status of people, which can be expressed using compound emotions. Compound facial emotion recognition makes the problem even more difficult because the discrimination between dominant and complementary emotions is usually weak. We have created a database that includes $31250$ facial images with different emotions of $115$ subjects whose gender distribution is almost uniform to address compound emotion recognition. In addition, we have organized a competition based on the proposed dataset, held at FG workshop 2020. This paper analyzes the winner approach - a two-stage recognition method ($1^{st}$ stage - coarse recognition, $2^{nd}$ stage - fine recognition), which enhanced the classification of symmetrical emotion labels.",

keywords = "Compound emotion recognition, Deep learning, Dominant and complementary emotion recognition, Facial expression recognition",

author = "Dorota Kami{\'n}ska and Kadir Aktas and Davit Rizhinashvili and Danila Kuklyanov and Sham, {Abdallah Hussein} and Sergio Escalera and Kamal Nasrollahi and Moeslund, {Thomas B.} and Gholamreza Anbarjafari",

note = "Funding Information: Acknowledgments: We want to express our special thanks to Zhiyuan Zhangs, Jianping Shen, Miao Yi, Juan Xu, and Rong Zhang from Pingan Life Insurance of China for participating in the competition held at the FG workshop 2020. This work has been partially supported by the Spanish project PID2019-105093GB-I00 (MINECO/FEDER, UE), CERCA Programme/Generalitat de Catalunya), ICREA under the ICREA Academia programme, and the Estonian Centre of Excellence in IT (EXCITE) funded by the European Regional Development Fund. Publisher Copyright: {\textcopyright} 2021 by the authors. Licensee MDPI, Basel, Switzerland.",

year = "2021",

doi = "10.3390/electronics10222847",

language = "English",

volume = "10",

journal = "Electronics",

issn = "2079-9292",

publisher = "MDPI AG",

number = "22",

}

TY - JOUR

T1 - Two-stage Recognition and Beyond for Compound Facial Emotion Recognition

AU - Kamińska, Dorota

AU - Aktas, Kadir

AU - Rizhinashvili, Davit

AU - Kuklyanov, Danila

AU - Sham, Abdallah Hussein

AU - Escalera, Sergio

AU - Nasrollahi, Kamal

AU - Moeslund, Thomas B.

AU - Anbarjafari, Gholamreza

N1 - Funding Information: Acknowledgments: We want to express our special thanks to Zhiyuan Zhangs, Jianping Shen, Miao Yi, Juan Xu, and Rong Zhang from Pingan Life Insurance of China for participating in the competition held at the FG workshop 2020. This work has been partially supported by the Spanish project PID2019-105093GB-I00 (MINECO/FEDER, UE), CERCA Programme/Generalitat de Catalunya), ICREA under the ICREA Academia programme, and the Estonian Centre of Excellence in IT (EXCITE) funded by the European Regional Development Fund. Publisher Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

PY - 2021

Y1 - 2021

N2 - Facial emotion recognition is an inherently complex problem due to individual facial feature diversity and racial and cultural differences. Moreover, facial expressions are typically reflecting the mixture of emotional status of people, which can be expressed using compound emotions. Compound facial emotion recognition makes the problem even more difficult because the discrimination between dominant and complementary emotions is usually weak. We have created a database that includes $31250$ facial images with different emotions of $115$ subjects whose gender distribution is almost uniform to address compound emotion recognition. In addition, we have organized a competition based on the proposed dataset, held at FG workshop 2020. This paper analyzes the winner approach - a two-stage recognition method ($1^{st}$ stage - coarse recognition, $2^{nd}$ stage - fine recognition), which enhanced the classification of symmetrical emotion labels.

AB - Facial emotion recognition is an inherently complex problem due to individual facial feature diversity and racial and cultural differences. Moreover, facial expressions are typically reflecting the mixture of emotional status of people, which can be expressed using compound emotions. Compound facial emotion recognition makes the problem even more difficult because the discrimination between dominant and complementary emotions is usually weak. We have created a database that includes $31250$ facial images with different emotions of $115$ subjects whose gender distribution is almost uniform to address compound emotion recognition. In addition, we have organized a competition based on the proposed dataset, held at FG workshop 2020. This paper analyzes the winner approach - a two-stage recognition method ($1^{st}$ stage - coarse recognition, $2^{nd}$ stage - fine recognition), which enhanced the classification of symmetrical emotion labels.

KW - Compound emotion recognition

KW - Deep learning

KW - Dominant and complementary emotion recognition

KW - Facial expression recognition

UR - http://www.scopus.com/inward/record.url?scp=85119292851&partnerID=8YFLogxK

U2 - 10.3390/electronics10222847

DO - 10.3390/electronics10222847

M3 - Journal article

AN - SCOPUS:85119292851

SN - 2079-9292

VL - 10

JO - Electronics

JF - Electronics

IS - 22

M1 - 2847

ER -

Two-stage Recognition and Beyond for Compound Facial Emotion Recognition

Abstract

Bibliographical note

Keywords

Access to Document

AUB Link

Other files and links

Fingerprint

Cite this