1 Citation (Scopus)
6 Downloads (Pure)

Abstract

The ability to detect unfamiliar or unexpected images is essential for safe deployment of computer vision systems. In the context of classification the task of detecting images outside of a model's training domain is known as out-of-distribution (OOD) detection. While there has been a growing research interest in developing post-hoc OOD detection methods there has been comparably little discussion around how these methods perform when the underlying classifier is not trained on a clean carefully curated dataset. In this work we take a closer look at 20 state-of-the-art OOD detection methods in the (more realistic) scenario where the labels used to train the underlying classifier are unreliable (e.g. crowd-sourced or web-scraped labels). Extensive experiments across different datasets noise types & levels architectures and checkpointing strategies provide insights into the effect of class label noise on OOD detection and show that poor separation between incorrectly classified ID samples vs. OOD samples is an overlooked yet important limitation of existing methods. Code: https://github.com/glhr/ood-labelnoise
Original languageEnglish
Title of host publication2024 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Number of pages11
PublisherIEEE (Institute of Electrical and Electronics Engineers)
Publication date16 Sept 2024
Pages22626-22636
ISBN (Print)979-8-3503-5301-3
ISBN (Electronic)979-8-3503-5300-6
DOIs
Publication statusPublished - 16 Sept 2024
Event2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) - Seattle Convention Center, Seattle, United States
Duration: 17 Jun 202421 Jun 2024
https://cvpr.thecvf.com/Conferences/2024

Conference

Conference2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
LocationSeattle Convention Center
Country/TerritoryUnited States
CitySeattle
Period17/06/202421/06/2024
Internet address

Keywords

  • Image Classification
  • aleatoric uncertainty
  • benchmark
  • computer vision
  • data uncertainty
  • label noise
  • noisy labels
  • open-set recognition
  • out-of-distribution detection
  • post-hoc
  • Out-of-distribution detection
  • Noisy labels
  • Image classification

Fingerprint

Dive into the research topics of 'A Noisy Elephant in the Room: Is Your out-of-Distribution Detector Robust to Label Noise?'. Together they form a unique fingerprint.

Cite this