Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images

Marco Bellantonio, Mohammad Ahsanul Haque, Pau Rodriguez, Kamal Nasrollahi, Taisi Telve, Sergio Escalera Guerrero, Jordi Gonzàlez, Thomas B. Moeslund, Pejman Rasti, Gholamreza Anbarjafari

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

11 Citationer (Scopus)
1007 Downloads (Pure)

Resumé

Automatic pain detection is a long expected solution to a prevalent medical problem of pain management. This is more relevant when the subject of pain is young children or patients with limited ability to communicate about their pain experience. Computer vision-based analysis of facial pain expression provides a way of efficient pain detection. When deep machine learning methods came into the scene, automatic pain detection exhibited even better performance. In this paper, we figured out three important factors to exploit in automatic pain detection: spatial information available regarding to pain in each of the facial video frames, temporal axis information regarding to pain expression pattern in a subject video sequence, and variation of face resolution. We employed a combination of convolutional neural network and recurrent neural network to setup a deep hybrid pain detection framework that is able to exploit both spatial and temporal pain information from facial video. In order to analyze the effect of different facial resolutions, we introduce a super-resolution algorithm to generate facial video frames with different resolution setups. We investigated the performance on the publicly available UNBC-McMaster Shoulder Pain database. As a contribution, the paper provides novel and important information regarding to the performance of a hybrid deep learning framework for pain detection in facial images of different resolution.
OriginalsprogEngelsk
TitelVideo Analytics : Face and Facial Expression Recognition and Audience Measurement
ForlagSpringer
Publikationsdato28 mar. 2017
ISBN (Trykt)978-3-319-56686-3
ISBN (Elektronisk)978-3-319-56687-0
DOI
StatusUdgivet - 28 mar. 2017
Begivenhed23rd international conference on pattern recognition (ICPR 2016) - Cancun, Mexico
Varighed: 4 dec. 20168 dec. 2016

Konference

Konference23rd international conference on pattern recognition (ICPR 2016)
LandMexico
ByCancun
Periode04/12/201608/12/2016
NavnLecture Notes in Computer Science
Vol/bind10165
ISSN0302-9743

Fingerprint

Recurrent neural networks
Medical problems
Computer vision
Learning systems
Neural networks
Deep learning

Citer dette

Bellantonio, M., Haque, M. A., Rodriguez, P., Nasrollahi, K., Telve, T., Guerrero, S. E., ... Anbarjafari, G. (2017). Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images. I Video Analytics: Face and Facial Expression Recognition and Audience Measurement Springer. Lecture Notes in Computer Science, Bind. 10165 https://doi.org/10.1007/978-3-319-56687-0_13
Bellantonio, Marco ; Haque, Mohammad Ahsanul ; Rodriguez, Pau ; Nasrollahi, Kamal ; Telve, Taisi ; Guerrero, Sergio Escalera ; Gonzàlez, Jordi ; Moeslund, Thomas B. ; Rasti, Pejman ; Anbarjafari, Gholamreza. / Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images. Video Analytics: Face and Facial Expression Recognition and Audience Measurement. Springer, 2017. (Lecture Notes in Computer Science, Bind 10165).
@inproceedings{ce14cfc2cf0c42f7966b44d7acba2b65,
title = "Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images",
abstract = "Automatic pain detection is a long expected solution to a prevalent medical problem of pain management. This is more relevant when the subject of pain is young children or patients with limited ability to communicate about their pain experience. Computer vision-based analysis of facial pain expression provides a way of efficient pain detection. When deep machine learning methods came into the scene, automatic pain detection exhibited even better performance. In this paper, we figured out three important factors to exploit in automatic pain detection: spatial information available regarding to pain in each of the facial video frames, temporal axis information regarding to pain expression pattern in a subject video sequence, and variation of face resolution. We employed a combination of convolutional neural network and recurrent neural network to setup a deep hybrid pain detection framework that is able to exploit both spatial and temporal pain information from facial video. In order to analyze the effect of different facial resolutions, we introduce a super-resolution algorithm to generate facial video frames with different resolution setups. We investigated the performance on the publicly available UNBC-McMaster Shoulder Pain database. As a contribution, the paper provides novel and important information regarding to the performance of a hybrid deep learning framework for pain detection in facial images of different resolution.",
keywords = "Super-Resolution, Convolutional Neural Network, Recurrent Neural Network, Pain detection",
author = "Marco Bellantonio and Haque, {Mohammad Ahsanul} and Pau Rodriguez and Kamal Nasrollahi and Taisi Telve and Guerrero, {Sergio Escalera} and Jordi Gonz{\`a}lez and Moeslund, {Thomas B.} and Pejman Rasti and Gholamreza Anbarjafari",
year = "2017",
month = "3",
day = "28",
doi = "10.1007/978-3-319-56687-0_13",
language = "English",
isbn = "978-3-319-56686-3",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
booktitle = "Video Analytics",
address = "Germany",

}

Bellantonio, M, Haque, MA, Rodriguez, P, Nasrollahi, K, Telve, T, Guerrero, SE, Gonzàlez, J, Moeslund, TB, Rasti, P & Anbarjafari, G 2017, Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images. i Video Analytics: Face and Facial Expression Recognition and Audience Measurement. Springer, Lecture Notes in Computer Science, bind 10165, Cancun, Mexico, 04/12/2016. https://doi.org/10.1007/978-3-319-56687-0_13

Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images. / Bellantonio, Marco; Haque, Mohammad Ahsanul; Rodriguez, Pau; Nasrollahi, Kamal; Telve, Taisi; Guerrero, Sergio Escalera; Gonzàlez, Jordi; Moeslund, Thomas B.; Rasti, Pejman; Anbarjafari, Gholamreza.

Video Analytics: Face and Facial Expression Recognition and Audience Measurement. Springer, 2017. (Lecture Notes in Computer Science, Bind 10165).

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

TY - GEN

T1 - Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images

AU - Bellantonio, Marco

AU - Haque, Mohammad Ahsanul

AU - Rodriguez, Pau

AU - Nasrollahi, Kamal

AU - Telve, Taisi

AU - Guerrero, Sergio Escalera

AU - Gonzàlez, Jordi

AU - Moeslund, Thomas B.

AU - Rasti, Pejman

AU - Anbarjafari, Gholamreza

PY - 2017/3/28

Y1 - 2017/3/28

N2 - Automatic pain detection is a long expected solution to a prevalent medical problem of pain management. This is more relevant when the subject of pain is young children or patients with limited ability to communicate about their pain experience. Computer vision-based analysis of facial pain expression provides a way of efficient pain detection. When deep machine learning methods came into the scene, automatic pain detection exhibited even better performance. In this paper, we figured out three important factors to exploit in automatic pain detection: spatial information available regarding to pain in each of the facial video frames, temporal axis information regarding to pain expression pattern in a subject video sequence, and variation of face resolution. We employed a combination of convolutional neural network and recurrent neural network to setup a deep hybrid pain detection framework that is able to exploit both spatial and temporal pain information from facial video. In order to analyze the effect of different facial resolutions, we introduce a super-resolution algorithm to generate facial video frames with different resolution setups. We investigated the performance on the publicly available UNBC-McMaster Shoulder Pain database. As a contribution, the paper provides novel and important information regarding to the performance of a hybrid deep learning framework for pain detection in facial images of different resolution.

AB - Automatic pain detection is a long expected solution to a prevalent medical problem of pain management. This is more relevant when the subject of pain is young children or patients with limited ability to communicate about their pain experience. Computer vision-based analysis of facial pain expression provides a way of efficient pain detection. When deep machine learning methods came into the scene, automatic pain detection exhibited even better performance. In this paper, we figured out three important factors to exploit in automatic pain detection: spatial information available regarding to pain in each of the facial video frames, temporal axis information regarding to pain expression pattern in a subject video sequence, and variation of face resolution. We employed a combination of convolutional neural network and recurrent neural network to setup a deep hybrid pain detection framework that is able to exploit both spatial and temporal pain information from facial video. In order to analyze the effect of different facial resolutions, we introduce a super-resolution algorithm to generate facial video frames with different resolution setups. We investigated the performance on the publicly available UNBC-McMaster Shoulder Pain database. As a contribution, the paper provides novel and important information regarding to the performance of a hybrid deep learning framework for pain detection in facial images of different resolution.

KW - Super-Resolution

KW - Convolutional Neural Network

KW - Recurrent Neural Network

KW - Pain detection

U2 - 10.1007/978-3-319-56687-0_13

DO - 10.1007/978-3-319-56687-0_13

M3 - Article in proceeding

SN - 978-3-319-56686-3

T3 - Lecture Notes in Computer Science

BT - Video Analytics

PB - Springer

ER -

Bellantonio M, Haque MA, Rodriguez P, Nasrollahi K, Telve T, Guerrero SE et al. Spatio-Temporal Pain Recognition in CNN-based Super-Resolved Facial Images. I Video Analytics: Face and Facial Expression Recognition and Audience Measurement. Springer. 2017. (Lecture Notes in Computer Science, Bind 10165). https://doi.org/10.1007/978-3-319-56687-0_13