Multi-modality is an important feature of sensor based activity recognition. In this work, we consider two inherent characteristics of human activities, the spatially-temporally varying salience of features and the relations between activities and corresponding body part motions. Based on these, we propose a multi-agent spatial-temporal attention model. The spatial-temporal attention mechanism helps intelligently select informative modalities and their active periods. And the multiple agents in the proposed model represent activities with collective motions across body parts by independently selecting modalities associated with single motions. With a joint recognition goal, the agents share gained information and coordinate their selection policies to learn the optimal recognition model. The experimental results on four real-world datasets demonstrate that the proposed model outperforms the state-of-the-art methods.
|Title of host publication||Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019|
|Number of pages||7|
|Publisher||International Joint Conferences on Artificial Intelligence|
|Publication status||Published - 2019|
|Event||28th International Joint Conference on Artificial Intelligence, IJCAI 2019 - Macao, China|
Duration: 10 Aug 2019 → 16 Aug 2019
|Conference||28th International Joint Conference on Artificial Intelligence, IJCAI 2019|
|Period||10/08/2019 → 16/08/2019|
|Sponsor||Baidu, et al., Huawei Technologies Co., Ltd., International Joint Conferences on Artifical Intelligence (IJCAI), Sony Group Corporation, Xiao-i|
|Series||IJCAI International Joint Conference on Artificial Intelligence|
Bibliographical notePublisher Copyright:
© 2019 International Joint Conferences on Artificial Intelligence. All rights reserved.