Handling stress reactions in institutional care by adapting interaction behavior of a social robot

It is always challenging for people with physical and cognitive impairments to communicate and socialize. This project aims to determine and investigate the emotional signals, particularly negative emotions like stress, fatigue or anger cues from the brain-injured patients to explore practical strategies to improve the quality of life of the residents at the Danish neurocenter. It focuses on the extraction of social signals through facial expressions and uses emotional cues for productive human-robot interaction. This project has three phases:
Development of the multi-modal database of real patients in specific scenarios
Use of deep transfer learning approaches to make a unique and customized facial expressions recognition (FER) system for these residents.
Deployment of this FER system through social robot (in our case pepper robot) to enhance Human-Robot interaction and social interaction.
The Pepper robot performs the role of assistive technology for both residents and staff members. In the case of residents, it provides stimulus to engage more socially through its speech and visual input by analyzing mood in particular during therapy sessions. It acts as a monitoring and feedback tool for the staff members allowing to monitor emotional reactions over therapy sessions and allows for the adaptation of rehabilitation strategies based on this additional input.