Projects per year
Abstract
Despite recent significant advancements in the field of human emotion recognition, applying upper body movements along with facial expressions present severe challenges in the field of human-robot interaction. This article presents a model that learns emotions through upper body movements and corresponds with facial expressions. Once this correspondence is mapped, tasks such as emotion and gesture recognition can easily be identified using facial features and movement vectors. Our method uses a deep convolution neural network trained on benchmark datasets exhibiting various emotions and corresponding body movements. Features obtained through facial movements and body motion are fused to get emotion recognition performance. We have implemented various fusion methodologies to integrate multimodal features for non-verbal emotion identification. Our system achieves 76.8% accuracy of emotion recognition through upper body movements only, surpassing 73.1% on the FABO dataset. In addition, employing multimodal compact bilinear pooling with temporal information surpassed the state-of-the-art method with an accuracy of 94.41% on the FABO dataset. This system can lead to better human-machine interaction by enabling robots to recognize emotions and body actions and react according to their emotions, thus enriching the user experience.
Original language | English |
---|---|
Title of host publication | 16th International Conference on Computer Vision Theory and Application (VISAPP 2021) |
Editors | Giovanni Maria Farinella, Petia Radeva, Jose Braz, Kadi Bouatouch |
Number of pages | 11 |
Publisher | SCITEPRESS Digital Library |
Publication date | 2021 |
Pages | 669-679 |
Article number | 229 |
ISBN (Electronic) | 978-989-758-488-6 |
DOIs | |
Publication status | Published - 2021 |
Event | 16th International Conference on Computer Vision Theory and Application - Online Streaming Duration: 8 Feb 2021 → 10 Feb 2021 http://www.visapp.visigrapp.org/ImportantDates.aspx |
Conference
Conference | 16th International Conference on Computer Vision Theory and Application |
---|---|
Location | Online Streaming |
Period | 08/02/2021 → 10/02/2021 |
Internet address |
Keywords
- Body movements
- Convolutional neural networks
- Deep learning
- Emotion recognition
- Facial expressions
Fingerprint
Dive into the research topics of 'Deep Emotion Recognition through Upper Body Movements and Facial Expression.'. Together they form a unique fingerprint.Projects
- 1 Active
-
NET4Age-Friendly: International Interdisciplinary Network on Smart Healthy Age-friendly Environments
01/10/2020 → 30/09/2024
Project: Research
Research output
- 9 Citations
- 1 PhD thesis
-
Facial Emotion Recognition for Citizens with Traumatic Brain Injury for Therapeutic Robot Interaction
Ilyas, C. M., 2021, Aalborg Universitetsforlag. 196 p.Research output: PhD thesis
Open AccessFile204 Downloads (Pure)