Multi-agent attentional activity recognition

Kaixuan Chen, Lina Yao, Dalin Zhang, Bin Guo, Zhiwen Yu

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

19 Citations (Scopus)


Multi-modality is an important feature of sensor based activity recognition. In this work, we consider two inherent characteristics of human activities, the spatially-temporally varying salience of features and the relations between activities and corresponding body part motions. Based on these, we propose a multi-agent spatial-temporal attention model. The spatial-temporal attention mechanism helps intelligently select informative modalities and their active periods. And the multiple agents in the proposed model represent activities with collective motions across body parts by independently selecting modalities associated with single motions. With a joint recognition goal, the agents share gained information and coordinate their selection policies to learn the optimal recognition model. The experimental results on four real-world datasets demonstrate that the proposed model outperforms the state-of-the-art methods.

Original languageEnglish
Title of host publicationProceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019
EditorsSarit Kraus
Number of pages7
PublisherInternational Joint Conferences on Artificial Intelligence
Publication date2019
ISBN (Electronic)9780999241141
Publication statusPublished - 2019
Externally publishedYes
Event28th International Joint Conference on Artificial Intelligence, IJCAI 2019 - Macao, China
Duration: 10 Aug 201916 Aug 2019


Conference28th International Joint Conference on Artificial Intelligence, IJCAI 2019
SponsorBaidu, et al., Huawei Technologies Co., Ltd., International Joint Conferences on Artifical Intelligence (IJCAI), Sony Group Corporation, Xiao-i
SeriesIJCAI International Joint Conference on Artificial Intelligence

Bibliographical note

Publisher Copyright:
© 2019 International Joint Conferences on Artificial Intelligence. All rights reserved.


Dive into the research topics of 'Multi-agent attentional activity recognition'. Together they form a unique fingerprint.

Cite this