Bioscan-5M: A Multimodal Dataset for Insect Biodiversity

BIOSCAN-5M is a curated subset of above dataset containing multi-modal information for 5,150,850 insect specimens, and it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, and geographical information. The dataset includes specimens collected from 1,650 sites across 47 countries. Images were resized to 1024 x 768 pixels.
We propose three benchmark experiments to demonstrate the impact of the multi-modal data types on the classification and clustering accuracy.

First, we pretrain a masked language model on the DNA barcode sequences of the BIOSCAN-5M dataset and demonstrate the impact of using this large reference library on species- and genus-level classification performance.

Second, we propose a zero-shot transfer learning task applied to images and DNA barcodes to cluster feature embeddings obtained from self-supervised learning, to investigate whether meaningful clusters can be derived from these representation embeddings.

Third, we benchmark multi-modality by performing contrastive learning on DNA barcodes, image data, and taxonomic information. This yields a general shared embedding space enabling taxonomic classification using multiple types of information and modalities.