ChaLearn LAP large scale signer independent isolated sign language recognition challenge: Design, results and future research

Ozge Mercanoglu Sincan, Julio C.S. Jacques, Sergio Escalera, Hacer Yalim Keles

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

20 Citations (Scopus)

Abstract

The performances of Sign Language Recognition (SLR) systems have improved considerably in recent years. However, several open challenges still need to be solved to allow SLR to be useful in practice. The research in the field is in its infancy in regards to the robustness of the models to a large diversity of signs and signers, and to fairness of the models to performers from different demographics. This work summarises the ChaLearn LAP Large Scale Signer Independent Isolated SLR Challenge, organised at CVPR 2021 with the goal of overcoming some of the aforementioned challenges. We analyse and discuss the challenge design, top winning solutions and suggestions for future research. The challenge attracted 132 participants in the RGB track and 59 in the RGB+Depth track, receiving more than 1.5K submissions in total. Participants were evaluated using a new large-scale multi-modal Turkish Sign Language (AUTSL) dataset, consisting of 226 sign labels and 36, 302 isolated sign video samples performed by 43 different signers. Winning teams achieved more than 96% recognition rate, and their approaches benefited from pose/hand/face estimation, transfer learning, external data, fusion/ensemble of modalities and different strategies to model spatio-temporal information. However, methods still fail to distinguish among very similar signs, in particular those sharing similar hand trajectories.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
Number of pages10
PublisherIEEE Computer Society Press
Publication dateSept 2021
Pages3467-3476
ISBN (Electronic)9781665448994
DOIs
Publication statusPublished - Sept 2021
Externally publishedYes
Event2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021 - Virtual, Online, United States
Duration: 19 Jun 202125 Jun 2021

Conference

Conference2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
Country/TerritoryUnited States
CityVirtual, Online
Period19/06/202125/06/2021
SeriesIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN2160-7508

Bibliographical note

Funding Information:
This work has been partially supported by the Scientific and Technological Research Council of Turkey (TUBITAK) project 217E022, the Spanish project PID2019-105093GB-I00 (MINECO/FEDER, UE) and CERCA Programme/Generalitat de Catalunya, and ICREA under the ICREA Academia programme.

Publisher Copyright:
© 2021 IEEE.

Fingerprint

Dive into the research topics of 'ChaLearn LAP large scale signer independent isolated sign language recognition challenge: Design, results and future research'. Together they form a unique fingerprint.

Cite this