Abstract

Automatic analysis of body movement to identify physical activity of patients who are at bed rest is crucial for treatment or rehabilitation purposes. Existing methods of physical activity analysis mostly focused on the detection of primitive motion/non-motion states in unimodal video data captured by either RGB or depth or thermal sensor. In this paper, we propose a multimodal visionbased approach to classify body motion of a person lying on a bed. We mimicked a real scenario of ’patient on bed’ by recording multimodal video data from healthy volunteers in a hospital room in a neurorehabilitation center. We first defined a taxonomy of possible physical activities based on observations of patients with acquired brain injuries. We then investigated different motion analysis and machine learning approaches to classify physical activities automatically. A multimodal database including RGB, depth and thermal videos was collected and annotated with eight predefined physical activities. Experimental results show that we can achieve moderately high accuracy (77.68%) to classify physical activities by tracking the body motion using an optical flow-based approach. To the best of our knowledge this is the first multimodal RGBDT video analysis for such application.
Original languageEnglish
Title of host publicationAdvances in Visual Computing : 13th International Symposium, ISVC 2018, Las Vegas, NV, USA, November 19 – 21, 2018, Proceedings
Number of pages12
PublisherSpringer
Publication date2018
Pages552-564
ISBN (Print)978-3-030-03800-7
ISBN (Electronic)978-3-030-03801-4
DOIs
Publication statusPublished - 2018
EventInternational Symposium on Visual Computing, ISVC 2018 - Las Vegas, United States
Duration: 19 Nov 201821 Nov 2018
Conference number: 13

Conference

ConferenceInternational Symposium on Visual Computing, ISVC 2018
Number13
CountryUnited States
CityLas Vegas
Period19/11/201821/11/2018
SeriesLecture Notes in Computer Science
Volume11241
ISSN0302-9743

Fingerprint

Optical flows
Taxonomies
Patient rehabilitation
Learning systems
Brain
Sensors
Hot Temperature
Motion analysis

Keywords

  • Physical activity
  • Multimodal
  • RGBDT
  • Video
  • Rest activity
  • Patient
  • hospital
  • Bed
  • Bed Rest

Cite this

Haque, M. A., Kjeldsen, S. S., Arguissain, F. G., Brunner, I., Nasrollahi, K., Andersen, O. K., ... Jørgensen, A. (2018). Patient’s Body Motion Study using Multimodal RGBDT Videos. In Advances in Visual Computing: 13th International Symposium, ISVC 2018, Las Vegas, NV, USA, November 19 – 21, 2018, Proceedings (pp. 552-564). Springer. Lecture Notes in Computer Science, Vol.. 11241 https://doi.org/10.1007/978-3-030-03801-4_48
Haque, Mohammad Ahsanul ; Kjeldsen, Simon S. ; Arguissain, Federico G. ; Brunner, Iris ; Nasrollahi, Kamal ; Andersen, Ole Kæseler ; Nielsen, Jørgen F. ; Moeslund, Thomas B. ; Jørgensen, Anders. / Patient’s Body Motion Study using Multimodal RGBDT Videos. Advances in Visual Computing: 13th International Symposium, ISVC 2018, Las Vegas, NV, USA, November 19 – 21, 2018, Proceedings. Springer, 2018. pp. 552-564 (Lecture Notes in Computer Science, Vol. 11241).
@inproceedings{946a770ab8ba425aa54ff6534b78fb15,
title = "Patient’s Body Motion Study using Multimodal RGBDT Videos",
abstract = "Automatic analysis of body movement to identify physical activity of patients who are at bed rest is crucial for treatment or rehabilitation purposes. Existing methods of physical activity analysis mostly focused on the detection of primitive motion/non-motion states in unimodal video data captured by either RGB or depth or thermal sensor. In this paper, we propose a multimodal visionbased approach to classify body motion of a person lying on a bed. We mimicked a real scenario of ’patient on bed’ by recording multimodal video data from healthy volunteers in a hospital room in a neurorehabilitation center. We first defined a taxonomy of possible physical activities based on observations of patients with acquired brain injuries. We then investigated different motion analysis and machine learning approaches to classify physical activities automatically. A multimodal database including RGB, depth and thermal videos was collected and annotated with eight predefined physical activities. Experimental results show that we can achieve moderately high accuracy (77.68{\%}) to classify physical activities by tracking the body motion using an optical flow-based approach. To the best of our knowledge this is the first multimodal RGBDT video analysis for such application.",
keywords = "Physical activity, Multimodal, RGBDT, Video, Rest activity, Patient, hospital, Bed, Bed Rest",
author = "Haque, {Mohammad Ahsanul} and Kjeldsen, {Simon S.} and Arguissain, {Federico G.} and Iris Brunner and Kamal Nasrollahi and Andersen, {Ole K{\ae}seler} and Nielsen, {J{\o}rgen F.} and Moeslund, {Thomas B.} and Anders J{\o}rgensen",
year = "2018",
doi = "10.1007/978-3-030-03801-4_48",
language = "English",
isbn = "978-3-030-03800-7",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "552--564",
booktitle = "Advances in Visual Computing",
address = "Germany",

}

Haque, MA, Kjeldsen, SS, Arguissain, FG, Brunner, I, Nasrollahi, K, Andersen, OK, Nielsen, JF, Moeslund, TB & Jørgensen, A 2018, Patient’s Body Motion Study using Multimodal RGBDT Videos. in Advances in Visual Computing: 13th International Symposium, ISVC 2018, Las Vegas, NV, USA, November 19 – 21, 2018, Proceedings. Springer, Lecture Notes in Computer Science, vol. 11241, pp. 552-564, International Symposium on Visual Computing, ISVC 2018, Las Vegas, United States, 19/11/2018. https://doi.org/10.1007/978-3-030-03801-4_48

Patient’s Body Motion Study using Multimodal RGBDT Videos. / Haque, Mohammad Ahsanul; Kjeldsen, Simon S.; Arguissain, Federico G.; Brunner, Iris; Nasrollahi, Kamal; Andersen, Ole Kæseler; Nielsen, Jørgen F. ; Moeslund, Thomas B.; Jørgensen, Anders.

Advances in Visual Computing: 13th International Symposium, ISVC 2018, Las Vegas, NV, USA, November 19 – 21, 2018, Proceedings. Springer, 2018. p. 552-564 (Lecture Notes in Computer Science, Vol. 11241).

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - Patient’s Body Motion Study using Multimodal RGBDT Videos

AU - Haque, Mohammad Ahsanul

AU - Kjeldsen, Simon S.

AU - Arguissain, Federico G.

AU - Brunner, Iris

AU - Nasrollahi, Kamal

AU - Andersen, Ole Kæseler

AU - Nielsen, Jørgen F.

AU - Moeslund, Thomas B.

AU - Jørgensen, Anders

PY - 2018

Y1 - 2018

N2 - Automatic analysis of body movement to identify physical activity of patients who are at bed rest is crucial for treatment or rehabilitation purposes. Existing methods of physical activity analysis mostly focused on the detection of primitive motion/non-motion states in unimodal video data captured by either RGB or depth or thermal sensor. In this paper, we propose a multimodal visionbased approach to classify body motion of a person lying on a bed. We mimicked a real scenario of ’patient on bed’ by recording multimodal video data from healthy volunteers in a hospital room in a neurorehabilitation center. We first defined a taxonomy of possible physical activities based on observations of patients with acquired brain injuries. We then investigated different motion analysis and machine learning approaches to classify physical activities automatically. A multimodal database including RGB, depth and thermal videos was collected and annotated with eight predefined physical activities. Experimental results show that we can achieve moderately high accuracy (77.68%) to classify physical activities by tracking the body motion using an optical flow-based approach. To the best of our knowledge this is the first multimodal RGBDT video analysis for such application.

AB - Automatic analysis of body movement to identify physical activity of patients who are at bed rest is crucial for treatment or rehabilitation purposes. Existing methods of physical activity analysis mostly focused on the detection of primitive motion/non-motion states in unimodal video data captured by either RGB or depth or thermal sensor. In this paper, we propose a multimodal visionbased approach to classify body motion of a person lying on a bed. We mimicked a real scenario of ’patient on bed’ by recording multimodal video data from healthy volunteers in a hospital room in a neurorehabilitation center. We first defined a taxonomy of possible physical activities based on observations of patients with acquired brain injuries. We then investigated different motion analysis and machine learning approaches to classify physical activities automatically. A multimodal database including RGB, depth and thermal videos was collected and annotated with eight predefined physical activities. Experimental results show that we can achieve moderately high accuracy (77.68%) to classify physical activities by tracking the body motion using an optical flow-based approach. To the best of our knowledge this is the first multimodal RGBDT video analysis for such application.

KW - Physical activity

KW - Multimodal

KW - RGBDT

KW - Video

KW - Rest activity

KW - Patient

KW - hospital

KW - Bed

KW - Bed Rest

U2 - 10.1007/978-3-030-03801-4_48

DO - 10.1007/978-3-030-03801-4_48

M3 - Article in proceeding

SN - 978-3-030-03800-7

T3 - Lecture Notes in Computer Science

SP - 552

EP - 564

BT - Advances in Visual Computing

PB - Springer

ER -

Haque MA, Kjeldsen SS, Arguissain FG, Brunner I, Nasrollahi K, Andersen OK et al. Patient’s Body Motion Study using Multimodal RGBDT Videos. In Advances in Visual Computing: 13th International Symposium, ISVC 2018, Las Vegas, NV, USA, November 19 – 21, 2018, Proceedings. Springer. 2018. p. 552-564. (Lecture Notes in Computer Science, Vol. 11241). https://doi.org/10.1007/978-3-030-03801-4_48