Projekter pr. år
Abstract
Since the introduction of the Vision Transformer (ViT), researchers have sought to make ViTs more efficient by removing redundant information in the processed tokens. While different methods have been explored to achieve this goal, we still lack understanding of the resulting reduction patterns and how those patterns differ across token reduction methods and datasets. To close this gap, we set out to understand the reduction patterns of 10 different token reduction methods using four image classification datasets. By systematically comparing these methods on the different classification tasks, we find that the Top-K pruning method is a surprisingly strong baseline. Through in-depth analysis of the different methods, we determine that: the reduction patterns are generally not consistent when varying the capacity of the backbone model, the reduction patterns of pruning-based methods significantly differ from fixed radial patterns, and the reduction patterns of pruning-based methods are correlated across classification datasets. Finally we report that the similarity of reduction patterns is a moderate-to-strong proxy for model performance. Project page at https://vap.aau.dk/tokens.
Originalsprog | Engelsk |
---|---|
Titel | Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 |
Antal sider | 11 |
Forlag | IEEE (Institute of Electrical and Electronics Engineers) |
Publikationsdato | 2023 |
Sider | 773-783 |
Artikelnummer | 10350983 |
ISBN (Trykt) | 979-8-3503-0745-0 |
ISBN (Elektronisk) | 9798350307443 |
DOI | |
Status | Udgivet - 2023 |
Begivenhed | 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 - Paris, Frankrig Varighed: 2 okt. 2023 → 6 okt. 2023 |
Konference
Konference | 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 |
---|---|
Land/Område | Frankrig |
By | Paris |
Periode | 02/10/2023 → 06/10/2023 |
Navn | IEEE International Conference on Computer Vision Workshops (ICCVW) |
---|---|
ISSN | 2473-9944 |
Bibliografisk note
Publisher Copyright:© 2023 IEEE.
Fingeraftryk
Dyk ned i forskningsemnerne om 'Which Tokens to Use? Investigating Token Reduction in Vision Transformers'. Sammen danner de et unikt fingeraftryk.Projekter
- 1 Igangværende
-
Pioneer Centre for AI
Tan, Z.-H. (CoPI), Moeslund, T. B. (CoPI) & Larsen, T. (Projektdeltager)
01/07/2021 → …
Projekter: Projekt › Forskning