Classification Accuracy Is Not Enough: On the Evaluation of Music Genre Recognition Systems

Research output: Research - peer-reviewJournal article

Abstract

A recent review of the research literature evaluating music genre recognition (MGR) systems over the past two decades shows that most works (81\%) measure the capacity of a system to recognize genre by its classification accuracy. We show here, by implementing and testing three categorically different state-of-the-art MGR systems, that classification accuracy does not necessarily reflect the capacity of a system to recognize genre in musical signals. We argue that a more comprehensive analysis of behavior at the level of the music is needed to address the problem of MGR, and that measuring classification accuracy obscures the aim of MGR: to select labels indistinguishable from those a person would choose.
Close

Details

A recent review of the research literature evaluating music genre recognition (MGR) systems over the past two decades shows that most works (81\%) measure the capacity of a system to recognize genre by its classification accuracy. We show here, by implementing and testing three categorically different state-of-the-art MGR systems, that classification accuracy does not necessarily reflect the capacity of a system to recognize genre in musical signals. We argue that a more comprehensive analysis of behavior at the level of the music is needed to address the problem of MGR, and that measuring classification accuracy obscures the aim of MGR: to select labels indistinguishable from those a person would choose.
Original languageEnglish
JournalJournal of Intelligent Information Systems
Volume41
Issue number3
Pages (from-to)371-406
Number of pages36
ISSN0925-9902
DOI
StatePublished - 2013
Publication categoryResearch
Peer-reviewedYes

Activities

Projects

Download statistics

No data available
ID: 70797939