Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks

Chaudhary Muhammad Aqdus Ilyas; Viktor Schmuck; Mohammad Ahsanul Haque; Kamal Nasrollahi; Matthias Rehm; Thomas B. Moeslund

doi:10.1109/RO-MAN46459.2019.8956445

Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks

Chaudhary Muhammad Aqdus Ilyas, Viktor Schmuck, Mohammad Ahsanul Haque, Kamal Nasrollahi, Matthias Rehm, Thomas B. Moeslund

Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Konferenceartikel i proceeding › Forskning › peer review

12 Citationer (Scopus)

370 Downloads (Pure)

Abstract

Social signal extraction from the facial analysis is
a popular research area in human-robot interaction. However,
recognition of emotional signals from Traumatic Brain Injured
(TBI) patients with the help of robots and non-intrusive sensors
is yet to be explored. Existing robots have limited abilities to
automatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even more
challenging and complex due to unique, unusual and diverse
ways of expressing their emotions. To tackle the disparity
in a TBI patient’s Facial Expressions (FEs), a specialized
deep-trained model for automatic detection of TBI patients’
emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’s
built-in model for FE is investigated on TBI patients as well
as on healthy people. Variance in their emotional expressions
is determined by comparative studies. It is observed that the
customized trained system is highly essential for the deployment
of Pepper robot as a Socially Assistive Robot (SAR).

Originalsprog	Engelsk
Titel	28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)
Forlag	IEEE (Institute of Electrical and Electronics Engineers)
Publikationsdato	12 okt. 2019
Artikelnummer	8956445
ISBN (Elektronisk)	978-1-7281-2622-7
DOI	https://doi.org/10.1109/RO-MAN46459.2019.8956445
Status	Udgivet - 12 okt. 2019
Begivenhed	The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN): Ro-man2019 - New Delhi, Indien Varighed: 14 okt. 2019 → 18 nov. 2019 https://ro-man2019.org/

Konference

Konference	The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
Land/Område	Indien
By	New Delhi
Periode	14/10/2019 → 18/11/2019
Internetadresse	https://ro-man2019.org/

Navn	IEEE RO-MAN proceedings
ISSN	1944-9445

Adgang til dokumentet

10.1109/RO-MAN46459.2019.8956445

Teaching_Pepper_Robot_to_Recognize_Emotions_of_Traumatic_Brain_Injured__Patients_Using_Deep_Neural_Networks_ArchivedIndsendt manuskript, 5,2 MB

AUB Link

Søg efter materialet i Aalborg Universitetsbiblioteks søgemaskine

1 Afsluttet

Handling stress reactions in institutional care by adapting interaction behavior of a social robot
Ilyas, C. M. (PI (principal investigator)), Nasrollahi, K. (PI (principal investigator)), Rehm, M. (PI (principal investigator)) & Moeslund, T. B. (PI (principal investigator))
03/04/2017 → 14/04/2020
Projekter: Projekt › Forskning

12 Citationer
1 Ph.d.-afhandling

Facial Emotion Recognition for Citizens with Traumatic Brain Injury for Therapeutic Robot Interaction
Ilyas, C. M., 2021, Aalborg Universitetsforlag. 196 s.
Publikation: Ph.d.-afhandling

Åben adgang
Fil

496 Downloads (Pure)

Citationsformater

Ilyas, C. M. A., Schmuck, V., Haque, M. A., Nasrollahi, K., Rehm, M., & Moeslund, T. B. (2019). Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. I 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN) Artikel 8956445 IEEE (Institute of Electrical and Electronics Engineers). https://doi.org/10.1109/RO-MAN46459.2019.8956445

Ilyas, Chaudhary Muhammad Aqdus ; Schmuck, Viktor ; Haque, Mohammad Ahsanul et al. / Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). IEEE (Institute of Electrical and Electronics Engineers), 2019. (IEEE RO-MAN proceedings).

@inproceedings{f25ce61410e74118a6a49859556b3496,

title = "Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks",

abstract = "Social signal extraction from the facial analysis isa popular research area in human-robot interaction. However,recognition of emotional signals from Traumatic Brain Injured(TBI) patients with the help of robots and non-intrusive sensorsis yet to be explored. Existing robots have limited abilities toautomatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even morechallenging and complex due to unique, unusual and diverseways of expressing their emotions. To tackle the disparityin a TBI patient{\textquoteright}s Facial Expressions (FEs), a specializeddeep-trained model for automatic detection of TBI patients{\textquoteright}emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot{\textquoteright}sbuilt-in model for FE is investigated on TBI patients as wellas on healthy people. Variance in their emotional expressionsis determined by comparative studies. It is observed that thecustomized trained system is highly essential for the deploymentof Pepper robot as a Socially Assistive Robot (SAR).",

keywords = "Robots in Education, Therapy and Rehabilitation, Non-verbal Cues and Expressiveness, Applications of Social Robots",

author = "Ilyas, {Chaudhary Muhammad Aqdus} and Viktor Schmuck and Haque, {Mohammad Ahsanul} and Kamal Nasrollahi and Matthias Rehm and Moeslund, {Thomas B.}",

year = "2019",

month = oct,

day = "12",

doi = "10.1109/RO-MAN46459.2019.8956445",

language = "English",

series = "IEEE RO-MAN proceedings",

publisher = "IEEE (Institute of Electrical and Electronics Engineers)",

booktitle = "28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)",

address = "United States",

note = "The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) : Ro-man2019 ; Conference date: 14-10-2019 Through 18-11-2019",

url = "https://ro-man2019.org/",

}

Ilyas, CMA, Schmuck, V, Haque, MA, Nasrollahi, K , Rehm, M & Moeslund, TB 2019, Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. i 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)., 8956445, IEEE (Institute of Electrical and Electronics Engineers), IEEE RO-MAN proceedings, The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, Indien, 14/10/2019. https://doi.org/10.1109/RO-MAN46459.2019.8956445

Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. / Ilyas, Chaudhary Muhammad Aqdus; Schmuck, Viktor; Haque, Mohammad Ahsanul et al.
28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). IEEE (Institute of Electrical and Electronics Engineers), 2019. 8956445 (IEEE RO-MAN proceedings).

Publikation: Bidrag til bog/antologi/rapport/konference proceeding › Konferenceartikel i proceeding › Forskning › peer review

TY - GEN

T1 - Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks

AU - Ilyas, Chaudhary Muhammad Aqdus

AU - Schmuck, Viktor

AU - Haque, Mohammad Ahsanul

AU - Nasrollahi, Kamal

AU - Rehm, Matthias

AU - Moeslund, Thomas B.

PY - 2019/10/12

Y1 - 2019/10/12

N2 - Social signal extraction from the facial analysis isa popular research area in human-robot interaction. However,recognition of emotional signals from Traumatic Brain Injured(TBI) patients with the help of robots and non-intrusive sensorsis yet to be explored. Existing robots have limited abilities toautomatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even morechallenging and complex due to unique, unusual and diverseways of expressing their emotions. To tackle the disparityin a TBI patient’s Facial Expressions (FEs), a specializeddeep-trained model for automatic detection of TBI patients’emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’sbuilt-in model for FE is investigated on TBI patients as wellas on healthy people. Variance in their emotional expressionsis determined by comparative studies. It is observed that thecustomized trained system is highly essential for the deploymentof Pepper robot as a Socially Assistive Robot (SAR).

AB - Social signal extraction from the facial analysis isa popular research area in human-robot interaction. However,recognition of emotional signals from Traumatic Brain Injured(TBI) patients with the help of robots and non-intrusive sensorsis yet to be explored. Existing robots have limited abilities toautomatically identify human emotions and respond accordingly. Their interaction with TBI patients could be even morechallenging and complex due to unique, unusual and diverseways of expressing their emotions. To tackle the disparityin a TBI patient’s Facial Expressions (FEs), a specializeddeep-trained model for automatic detection of TBI patients’emotions and FE (TBI-FER model) is designed, for robotassisted rehabilitation activities. In addition, the Pepper robot’sbuilt-in model for FE is investigated on TBI patients as wellas on healthy people. Variance in their emotional expressionsis determined by comparative studies. It is observed that thecustomized trained system is highly essential for the deploymentof Pepper robot as a Socially Assistive Robot (SAR).

KW - Robots in Education, Therapy and Rehabilitation, Non-verbal Cues and Expressiveness, Applications of Social Robots

U2 - 10.1109/RO-MAN46459.2019.8956445

DO - 10.1109/RO-MAN46459.2019.8956445

M3 - Article in proceeding

T3 - IEEE RO-MAN proceedings

BT - 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN)

PB - IEEE (Institute of Electrical and Electronics Engineers)

T2 - The 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

Y2 - 14 October 2019 through 18 November 2019

ER -

Ilyas CMA, Schmuck V, Haque MA, Nasrollahi K , Rehm M , Moeslund TB. Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks. I 28th IEEE International Conference on Robot and Human Interactive Communication (ROMAN). IEEE (Institute of Electrical and Electronics Engineers). 2019. 8956445. (IEEE RO-MAN proceedings). doi: 10.1109/RO-MAN46459.2019.8956445

Teaching Pepper Robot to Recognize Emotions of Traumatic Brain Injured Patients Using Deep Neural Networks

Abstract

Konference

Adgang til dokumentet

AUB Link

Fingeraftryk

Projekter

Handling stress reactions in institutional care by adapting interaction behavior of a social robot

Publikation

Facial Emotion Recognition for Citizens with Traumatic Brain Injury for Therapeutic Robot Interaction

Citationsformater