Most research in automatic music genre recognition
has used a dataset assembled by Tzanetakis et al. \cite{Tzanetakis2002},
though its integrity has never been validated.
Here we catalog numerous serious problems with this dataset,
which include replications, mislabelings, versions, and corruptions.
These of course affect the validity of results derived from it;
but they also present new challenges,
especially now that we are using datasets so large
that manual validation is impossible.