Abstract
Recently, the memory mechanism has been widely implemented in target tracking. However, these trackers hardly balance the stability of long-term memory with the plasticity of short-term memory through an elegant and efficient mechanism. A residual memory inference network (RMIT) is proposed to exploit the history of target states and last visual features. Specifically, RMIT consists of a base layer and a residual memory layer by synergizing short-and long-term memories. The base layer can be regarded as Discriminative Correlation Filter (DCF) reformulation that maintains the short-term memory to accommodate rapid appearance changes. The residual memory layer can extend residual learning from the spatial domain to the Spatio-temporal domain via ConvLSTM to obtain long-term memory of the target appearance. To avoid model degradation due to sample imbalance, we introduce a weighted gradient harmonized loss to improve the discrimination of the tracker. Then, response scores can be served as a basis of the adaptive learning strategy to ensure the reliability of memory updates. The proposed method performs favorably and has been extensively validated on six benchmark datasets, including OTB-50/100, TC-128, UAV-123, and VOT-2016/2018 against several advanced methods.
Original language | English |
---|---|
Journal | Information Sciences |
Volume | 597 |
Pages (from-to) | 105-124 |
Number of pages | 20 |
ISSN | 0020-0255 |
DOIs | |
Publication status | Published - Jun 2022 |
Bibliographical note
Funding Information:This work was supported by the National Natural Science Foundation of China under Grant (61873246, 62072416, 62102373, 61806181, 62006213), Program for Science & Technology Innovation Talents in Universities of Henan Province, China (21HASTIT028), Natural Science Foundation of Henan Province, China (202300410495) and Zhongyuan Science and Technology Innovation Leadership Program, China (214200510026).
Funding Information:
Huanlong Zhang received the Ph.D. degree from the School of Aeronautics and Astronautics, Shanghai Jiao Tong University, China, in 2015. He is currently an Associate Professor with the College of Electric and Information Engineering, Zhengzhou University of Light Industry, Henan, Zhengzhou, China. His research has been funded by the National Natural Science Foundation of China (NSFC), the Key Science and Technology. Henan Province et al. He has published more than 40 technical articles in refereed journals and conference proceedings. His research interests include pattern recognition, machine learning, image processing, computer vision, and intelligent human-machine systems.
Publisher Copyright:
© 2022 Elsevier Inc.
Keywords
- Long-short term memory
- Residual network
- Visual tracking