Abstract

— Social signal extraction from the facial analysis is
a popular research area in human-robot interaction. However,
recognition of emotional signals from Traumatic Brain Injured
(TBI) patients with the help of robots and non-intrusive sensors
is yet to be explored. Existing robots have limited abilities to
automatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even more
challenging and complex due to unique, unusual and diverse
ways of expressing their emotions. To tackle the disparity
in a TBI patient’s Facial Expressions (FEs), a specialized
deep-trained model for automatic detection of TBI patients’
emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’s
built-in model for FE is investigated on TBI patients as well
as on healthy people. Variance in their emotional expressions
is determined by comparative studies. It is observed that the
customized trained system is highly essential for the deployment
of Pepper robot as a Socially Assistive Robot (SAR).
Original languageEnglish
Title of host publication28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)
Place of PublicationThe 28th IEEE International Conference on Robot and Human Interactive Communication
Publication date12 Oct 2019
Publication statusPublished - 12 Oct 2019
EventThe 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN): Ro-man2019 - New Delhi, India
Duration: 14 Oct 201918 Nov 2019
https://ro-man2019.org/

Conference

ConferenceThe 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
CountryIndia
CityNew Delhi
Period14/10/201918/11/2019
Internet address

Fingerprint

Brain
Teaching
Robots
Human robot interaction
Patient rehabilitation
Deep neural networks

Keywords

  • Robots in Education, Therapy and Rehabilitation, Non-verbal Cues and Expressiveness, Applications of Social Robots

Cite this

Ilyas, C. M. A., Schmuck, V., Haque, M. A., Nasrollahi, K., Rehm, M., & Moeslund, T. B. (2019). Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. In 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN) The 28th IEEE International Conference on Robot and Human Interactive Communication.
Ilyas, Chaudhary Muhammad Aqdus ; Schmuck, Viktor ; Haque, Mohammad Ahsanul ; Nasrollahi, Kamal ; Rehm, Matthias ; Moeslund, Thomas B. / Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). The 28th IEEE International Conference on Robot and Human Interactive Communication, 2019.
@inproceedings{f25ce61410e74118a6a49859556b3496,
title = "Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks",
abstract = "— Social signal extraction from the facial analysis isa popular research area in human-robot interaction. However,recognition of emotional signals from Traumatic Brain Injured(TBI) patients with the help of robots and non-intrusive sensorsis yet to be explored. Existing robots have limited abilities toautomatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even morechallenging and complex due to unique, unusual and diverseways of expressing their emotions. To tackle the disparityin a TBI patient’s Facial Expressions (FEs), a specializeddeep-trained model for automatic detection of TBI patients’emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’sbuilt-in model for FE is investigated on TBI patients as wellas on healthy people. Variance in their emotional expressionsis determined by comparative studies. It is observed that thecustomized trained system is highly essential for the deploymentof Pepper robot as a Socially Assistive Robot (SAR).",
keywords = "Robots in Education, Therapy and Rehabilitation, Non-verbal Cues and Expressiveness, Applications of Social Robots",
author = "Ilyas, {Chaudhary Muhammad Aqdus} and Viktor Schmuck and Haque, {Mohammad Ahsanul} and Kamal Nasrollahi and Matthias Rehm and Moeslund, {Thomas B.}",
year = "2019",
month = "10",
day = "12",
language = "English",
booktitle = "28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)",

}

Ilyas, CMA, Schmuck, V, Haque, MA, Nasrollahi, K, Rehm, M & Moeslund, TB 2019, Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. in 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). The 28th IEEE International Conference on Robot and Human Interactive Communication, The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, India, 14/10/2019.

Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. / Ilyas, Chaudhary Muhammad Aqdus; Schmuck, Viktor; Haque, Mohammad Ahsanul; Nasrollahi, Kamal; Rehm, Matthias; Moeslund, Thomas B.

28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). The 28th IEEE International Conference on Robot and Human Interactive Communication, 2019.

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks

AU - Ilyas, Chaudhary Muhammad Aqdus

AU - Schmuck, Viktor

AU - Haque, Mohammad Ahsanul

AU - Nasrollahi, Kamal

AU - Rehm, Matthias

AU - Moeslund, Thomas B.

PY - 2019/10/12

Y1 - 2019/10/12

N2 - — Social signal extraction from the facial analysis isa popular research area in human-robot interaction. However,recognition of emotional signals from Traumatic Brain Injured(TBI) patients with the help of robots and non-intrusive sensorsis yet to be explored. Existing robots have limited abilities toautomatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even morechallenging and complex due to unique, unusual and diverseways of expressing their emotions. To tackle the disparityin a TBI patient’s Facial Expressions (FEs), a specializeddeep-trained model for automatic detection of TBI patients’emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’sbuilt-in model for FE is investigated on TBI patients as wellas on healthy people. Variance in their emotional expressionsis determined by comparative studies. It is observed that thecustomized trained system is highly essential for the deploymentof Pepper robot as a Socially Assistive Robot (SAR).

AB - — Social signal extraction from the facial analysis isa popular research area in human-robot interaction. However,recognition of emotional signals from Traumatic Brain Injured(TBI) patients with the help of robots and non-intrusive sensorsis yet to be explored. Existing robots have limited abilities toautomatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even morechallenging and complex due to unique, unusual and diverseways of expressing their emotions. To tackle the disparityin a TBI patient’s Facial Expressions (FEs), a specializeddeep-trained model for automatic detection of TBI patients’emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’sbuilt-in model for FE is investigated on TBI patients as wellas on healthy people. Variance in their emotional expressionsis determined by comparative studies. It is observed that thecustomized trained system is highly essential for the deploymentof Pepper robot as a Socially Assistive Robot (SAR).

KW - Robots in Education, Therapy and Rehabilitation, Non-verbal Cues and Expressiveness, Applications of Social Robots

M3 - Article in proceeding

BT - 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)

CY - The 28th IEEE International Conference on Robot and Human Interactive Communication

ER -

Ilyas CMA, Schmuck V, Haque MA, Nasrollahi K, Rehm M, Moeslund TB. Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. In 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). The 28th IEEE International Conference on Robot and Human Interactive Communication. 2019