8 Citationer (Scopus)

Abstract

Since the introduction of the Vision Transformer (ViT), researchers have sought to make ViTs more efficient by removing redundant information in the processed tokens. While different methods have been explored to achieve this goal, we still lack understanding of the resulting reduction patterns and how those patterns differ across token reduction methods and datasets. To close this gap, we set out to understand the reduction patterns of 10 different token reduction methods using four image classification datasets. By systematically comparing these methods on the different classification tasks, we find that the Top-K pruning method is a surprisingly strong baseline. Through in-depth analysis of the different methods, we determine that: the reduction patterns are generally not consistent when varying the capacity of the backbone model, the reduction patterns of pruning-based methods significantly differ from fixed radial patterns, and the reduction patterns of pruning-based methods are correlated across classification datasets. Finally we report that the similarity of reduction patterns is a moderate-to-strong proxy for model performance. Project page at https://vap.aau.dk/tokens.

OriginalsprogEngelsk
TitelProceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023
Antal sider11
ForlagIEEE (Institute of Electrical and Electronics Engineers)
Publikationsdato2023
Sider773-783
Artikelnummer10350983
ISBN (Trykt)979-8-3503-0745-0
ISBN (Elektronisk)9798350307443
DOI
StatusUdgivet - 2023
Begivenhed2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 - Paris, Frankrig
Varighed: 2 okt. 20236 okt. 2023

Konference

Konference2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023
Land/OmrådeFrankrig
ByParis
Periode02/10/202306/10/2023
NavnIEEE International Conference on Computer Vision Workshops (ICCVW)
ISSN2473-9944

Bibliografisk note

Publisher Copyright:
© 2023 IEEE.

Fingeraftryk

Dyk ned i forskningsemnerne om 'Which Tokens to Use? Investigating Token Reduction in Vision Transformers'. Sammen danner de et unikt fingeraftryk.

Citationsformater