Challenges in multimodal gesture recognition

Sergio Escalera; Vassilis Athitsos; Isabelle Guyon

Challenges in multimodal gesture recognition

Sergio Escalera, Vassilis Athitsos, Isabelle Guyon

Research output: Contribution to journal › Journal article › Research › peer-review

Abstract

This paper surveys the state of the art on multimodal gesture recognition and introduces the JMLR special topic on gesture recognition 2011-2015. We began right at the start of the Kinect™ revolution when inexpensive infrared cameras providing image depth recordings became available. We published papers using this technology and other more conventional methods, including regular video cameras, to record data, thus providing a good overview of uses of machine learning and computer vision using multimodal data in this area of application. Notably, we organized a series of challenges and made available several datasets we recorded for that purpose, including tens of thousands of videos, which are available to conduct further research. We also overview recent state of the art works on gesture recognition based on a proposed taxonomy for gesture recognition, discussing challenges and future lines of research.

Original language	English
Journal	Journal of Machine Learning Research
Volume	17
ISSN	1532-4435
Publication status	Published - 1 Apr 2016
Externally published	Yes

Bibliographical note

Funding Information:
This work has been partially supported by ChaLearn Challenges in Machine Learning http://chalearn.org, the Human Pose Recovery and Behavior Analysis Group7 , the Pascal2 network of excellence, NSF grants 1128296, 1059235, 1055062, 1338118, 1035913, 0923494, and Spanish project TIN2013-43478-P. Our sponsors include Microsoft and Texas Instrument who donated prizes and provided technical support. The challenges were hosted by Kaggle.com and Coralab.org who are gratefully acknowledged. We thank our co-organizers of ChaLearn gesture and action recognition challenges: Miguel Reyes, Jordi Gonzalez, Xavier Baro, Jamie Shotton, Victor Ponce, Miguel Angel Bautista, and Hugo Jair Escalante.

Publisher Copyright:
©2016 Sergio Escalera, Vassilis Athitsos, and Isabelle Guyon.

Keywords

Computer vision
Gesture recognition
Infrared cameras
Kinect™
Multimodal data analysis
Pattern recognition
Time series analysis
Wearable sensors

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

@article{154bee2968f94d2e8154ca86b04d6f19,

title = "Challenges in multimodal gesture recognition",

abstract = "This paper surveys the state of the art on multimodal gesture recognition and introduces the JMLR special topic on gesture recognition 2011-2015. We began right at the start of the Kinect{\texttrademark} revolution when inexpensive infrared cameras providing image depth recordings became available. We published papers using this technology and other more conventional methods, including regular video cameras, to record data, thus providing a good overview of uses of machine learning and computer vision using multimodal data in this area of application. Notably, we organized a series of challenges and made available several datasets we recorded for that purpose, including tens of thousands of videos, which are available to conduct further research. We also overview recent state of the art works on gesture recognition based on a proposed taxonomy for gesture recognition, discussing challenges and future lines of research.",

keywords = "Computer vision, Gesture recognition, Infrared cameras, Kinect{\texttrademark}, Multimodal data analysis, Pattern recognition, Time series analysis, Wearable sensors",

author = "Sergio Escalera and Vassilis Athitsos and Isabelle Guyon",

note = "Funding Information: This work has been partially supported by ChaLearn Challenges in Machine Learning http://chalearn.org, the Human Pose Recovery and Behavior Analysis Group7 , the Pascal2 network of excellence, NSF grants 1128296, 1059235, 1055062, 1338118, 1035913, 0923494, and Spanish project TIN2013-43478-P. Our sponsors include Microsoft and Texas Instrument who donated prizes and provided technical support. The challenges were hosted by Kaggle.com and Coralab.org who are gratefully acknowledged. We thank our co-organizers of ChaLearn gesture and action recognition challenges: Miguel Reyes, Jordi Gonzalez, Xavier Baro, Jamie Shotton, Victor Ponce, Miguel Angel Bautista, and Hugo Jair Escalante. Publisher Copyright: {\textcopyright}2016 Sergio Escalera, Vassilis Athitsos, and Isabelle Guyon.",

year = "2016",

month = apr,

day = "1",

language = "English",

volume = "17",

journal = "Journal of Machine Learning Research",

issn = "1532-4435",

publisher = "MIT Press",

}

TY - JOUR

T1 - Challenges in multimodal gesture recognition

AU - Escalera, Sergio

AU - Athitsos, Vassilis

AU - Guyon, Isabelle

N1 - Funding Information: This work has been partially supported by ChaLearn Challenges in Machine Learning http://chalearn.org, the Human Pose Recovery and Behavior Analysis Group7 , the Pascal2 network of excellence, NSF grants 1128296, 1059235, 1055062, 1338118, 1035913, 0923494, and Spanish project TIN2013-43478-P. Our sponsors include Microsoft and Texas Instrument who donated prizes and provided technical support. The challenges were hosted by Kaggle.com and Coralab.org who are gratefully acknowledged. We thank our co-organizers of ChaLearn gesture and action recognition challenges: Miguel Reyes, Jordi Gonzalez, Xavier Baro, Jamie Shotton, Victor Ponce, Miguel Angel Bautista, and Hugo Jair Escalante. Publisher Copyright: ©2016 Sergio Escalera, Vassilis Athitsos, and Isabelle Guyon.

PY - 2016/4/1

Y1 - 2016/4/1

N2 - This paper surveys the state of the art on multimodal gesture recognition and introduces the JMLR special topic on gesture recognition 2011-2015. We began right at the start of the Kinect™ revolution when inexpensive infrared cameras providing image depth recordings became available. We published papers using this technology and other more conventional methods, including regular video cameras, to record data, thus providing a good overview of uses of machine learning and computer vision using multimodal data in this area of application. Notably, we organized a series of challenges and made available several datasets we recorded for that purpose, including tens of thousands of videos, which are available to conduct further research. We also overview recent state of the art works on gesture recognition based on a proposed taxonomy for gesture recognition, discussing challenges and future lines of research.

AB - This paper surveys the state of the art on multimodal gesture recognition and introduces the JMLR special topic on gesture recognition 2011-2015. We began right at the start of the Kinect™ revolution when inexpensive infrared cameras providing image depth recordings became available. We published papers using this technology and other more conventional methods, including regular video cameras, to record data, thus providing a good overview of uses of machine learning and computer vision using multimodal data in this area of application. Notably, we organized a series of challenges and made available several datasets we recorded for that purpose, including tens of thousands of videos, which are available to conduct further research. We also overview recent state of the art works on gesture recognition based on a proposed taxonomy for gesture recognition, discussing challenges and future lines of research.

KW - Computer vision

KW - Gesture recognition

KW - Infrared cameras

KW - Kinect™

KW - Multimodal data analysis

KW - Pattern recognition

KW - Time series analysis

KW - Wearable sensors

UR - http://www.scopus.com/inward/record.url?scp=84979936643&partnerID=8YFLogxK

M3 - Journal article

AN - SCOPUS:84979936643

SN - 1532-4435

VL - 17

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

ER -

Challenges in multimodal gesture recognition

Abstract

Bibliographical note

Keywords

AUB Link

Other files and links

Fingerprint

Cite this