Back-dropout Transfer Learning for Action Recognition

Huamin Ren, Nattiya Kanhabua, Andreas Møgelmose, Weifeng Liu, Sergio Escalera, Xavier Baro, Thomas B. Moeslund

Research output: Contribution to journalReview articleResearchpeer-review

Abstract

Transfer learning aims at adapting a model learned from source dataset to target dataset. It is a beneficial approach especially when annotating on the target dataset is expensive or infeasible. Transfer learning has demonstrated its powerful learning capabilities in various vision tasks. Despite transfer learning being a promising approach, it is still an open question how to adapt the model learned from the source dataset to the target dataset. One big challenge is to prevent the impact of category bias on classification performance. Dataset bias exists when two images from the same category, but from different datasets, are not classified as the same. To address this problem, we propose a transfer learning algorithm which takes advantage of misclassified images and applies back-dropout strategy to punish errors. We call this Negative Back-dropout Transfer Learning (NB-TL). Experimental results demonstrate the effectiveness of our algorithm. Moreover, our NB-TL algorithm has obtained 88.9% accuracy on UCF101 Action Recognition dataset, which achieves state-of-the-art performance.
Original languageEnglish
JournalIET Computer Vision
Volume12
Issue number4
Pages (from-to)1-15
Number of pages15
ISSN1751-9632
DOIs
Publication statusPublished - 2018

Fingerprint

Learning algorithms

Cite this

Ren, Huamin ; Kanhabua, Nattiya ; Møgelmose, Andreas ; Liu, Weifeng ; Escalera, Sergio ; Baro, Xavier ; Moeslund, Thomas B. / Back-dropout Transfer Learning for Action Recognition. In: IET Computer Vision. 2018 ; Vol. 12, No. 4. pp. 1-15.
@article{7de5407fe91a479d8d5e729e1737b09a,
title = "Back-dropout Transfer Learning for Action Recognition",
abstract = "Transfer learning aims at adapting a model learned from source dataset to target dataset. It is a beneficial approach especially when annotating on the target dataset is expensive or infeasible. Transfer learning has demonstrated its powerful learning capabilities in various vision tasks. Despite transfer learning being a promising approach, it is still an open question how to adapt the model learned from the source dataset to the target dataset. One big challenge is to prevent the impact of category bias on classification performance. Dataset bias exists when two images from the same category, but from different datasets, are not classified as the same. To address this problem, we propose a transfer learning algorithm which takes advantage of misclassified images and applies back-dropout strategy to punish errors. We call this Negative Back-dropout Transfer Learning (NB-TL). Experimental results demonstrate the effectiveness of our algorithm. Moreover, our NB-TL algorithm has obtained 88.9{\%} accuracy on UCF101 Action Recognition dataset, which achieves state-of-the-art performance.",
author = "Huamin Ren and Nattiya Kanhabua and Andreas M{\o}gelmose and Weifeng Liu and Sergio Escalera and Xavier Baro and Moeslund, {Thomas B.}",
year = "2018",
doi = "10.1049/iet-cvi.2016.0309",
language = "English",
volume = "12",
pages = "1--15",
journal = "IET Computer Vision",
issn = "1751-9632",
publisher = "Institution of Engineering and Technology",
number = "4",

}

Back-dropout Transfer Learning for Action Recognition. / Ren, Huamin; Kanhabua, Nattiya; Møgelmose, Andreas; Liu, Weifeng; Escalera, Sergio; Baro, Xavier; Moeslund, Thomas B.

In: IET Computer Vision, Vol. 12, No. 4, 2018, p. 1-15.

Research output: Contribution to journalReview articleResearchpeer-review

TY - JOUR

T1 - Back-dropout Transfer Learning for Action Recognition

AU - Ren, Huamin

AU - Kanhabua, Nattiya

AU - Møgelmose, Andreas

AU - Liu, Weifeng

AU - Escalera, Sergio

AU - Baro, Xavier

AU - Moeslund, Thomas B.

PY - 2018

Y1 - 2018

N2 - Transfer learning aims at adapting a model learned from source dataset to target dataset. It is a beneficial approach especially when annotating on the target dataset is expensive or infeasible. Transfer learning has demonstrated its powerful learning capabilities in various vision tasks. Despite transfer learning being a promising approach, it is still an open question how to adapt the model learned from the source dataset to the target dataset. One big challenge is to prevent the impact of category bias on classification performance. Dataset bias exists when two images from the same category, but from different datasets, are not classified as the same. To address this problem, we propose a transfer learning algorithm which takes advantage of misclassified images and applies back-dropout strategy to punish errors. We call this Negative Back-dropout Transfer Learning (NB-TL). Experimental results demonstrate the effectiveness of our algorithm. Moreover, our NB-TL algorithm has obtained 88.9% accuracy on UCF101 Action Recognition dataset, which achieves state-of-the-art performance.

AB - Transfer learning aims at adapting a model learned from source dataset to target dataset. It is a beneficial approach especially when annotating on the target dataset is expensive or infeasible. Transfer learning has demonstrated its powerful learning capabilities in various vision tasks. Despite transfer learning being a promising approach, it is still an open question how to adapt the model learned from the source dataset to the target dataset. One big challenge is to prevent the impact of category bias on classification performance. Dataset bias exists when two images from the same category, but from different datasets, are not classified as the same. To address this problem, we propose a transfer learning algorithm which takes advantage of misclassified images and applies back-dropout strategy to punish errors. We call this Negative Back-dropout Transfer Learning (NB-TL). Experimental results demonstrate the effectiveness of our algorithm. Moreover, our NB-TL algorithm has obtained 88.9% accuracy on UCF101 Action Recognition dataset, which achieves state-of-the-art performance.

U2 - 10.1049/iet-cvi.2016.0309

DO - 10.1049/iet-cvi.2016.0309

M3 - Review article

VL - 12

SP - 1

EP - 15

JO - IET Computer Vision

JF - IET Computer Vision

SN - 1751-9632

IS - 4

ER -