Re-Identification of Giant Sunfish using Keypoint Matching

Malte Pedersen; Joakim Bruslund Haurum; Thomas B. Moeslund; Marianne Nyegaard

doi:10.7557/18.6234

Re-Identification of Giant Sunfish using Keypoint Matching

Malte Pedersen^*, Joakim Bruslund Haurum, Thomas B. Moeslund, Marianne Nyegaard

^*Corresponding author for this work

Research output: Contribution to journal › Conference article in Journal › Research › peer-review

74 Downloads (Pure)

Abstract

We present the first work where re-identification ofthe Giant Sunfish (Mola alexandrini) is automated using computer vision and deep learning. We propose a pipeline that scores an mAP of 60.34% on a full rank of the novel TinyMola dataset which includes 31 IDs and 91 images. The method requires no domain-adaptation or training which makes it especially suited for low-budget or volunteer-based projects, like Match My Mola, as part of a human-in-the-loop model.

The pipeline includes segmentation, keypoint detection and description, keypoint matching, and ranking. The choice of feature descriptor has the largest impact on the performance and we show that the deep learning based SuperPoint descriptor greatly outperforms handcrafted descriptors like SIFT and RootSIFT independent of the segmentation level and matching method. Combining SuperPoint and the graph neural network based SuperGlue matching method produces the best results.

Original language	English
Journal	Proceedings of the Northern Lights Deep Learning Workshop
Volume	3
Number of pages	9
ISSN	2703-6928
DOIs	https://doi.org/10.7557/18.6234
Publication status	Published - 28 Mar 2022
Event	Northern Lights Deep Learning Workshop - Tromsø, Norway Duration: 10 Jan 2022 → 12 Jan 2022 https://www.nldl.org/

Workshop

Workshop	Northern Lights Deep Learning Workshop
Country/Territory	Norway
City	Tromsø
Period	10/01/2022 → 12/01/2022
Internet address	https://www.nldl.org/

Bibliographical note

This work has been funded by the Independent ResearchFund Denmark under case number 9131-00128B.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.7557/18.6234Licence: CC BY 4.0

Open Access articleFinal published version, 3.23 MBLicence: CC BY 4.0

AUB Link

Search for the material in Aalborg University Library's search engine

Cite this

@inproceedings{0ddf9b8a21c34e33a1e4530a557b3d6b,

title = "Re-Identification of Giant Sunfish using Keypoint Matching",

abstract = "We present the first work where re-identification ofthe Giant Sunfish (Mola alexandrini) is automated using computer vision and deep learning. We propose a pipeline that scores an mAP of 60.34% on a full rank of the novel TinyMola dataset which includes 31 IDs and 91 images. The method requires no domain-adaptation or training which makes it especially suited for low-budget or volunteer-based projects, like Match My Mola, as part of a human-in-the-loop model.The pipeline includes segmentation, keypoint detection and description, keypoint matching, and ranking. The choice of feature descriptor has the largest impact on the performance and we show that the deep learning based SuperPoint descriptor greatly outperforms handcrafted descriptors like SIFT and RootSIFT independent of the segmentation level and matching method. Combining SuperPoint and the graph neural network based SuperGlue matching method produces the best results.",

author = "Malte Pedersen and Haurum, {Joakim Bruslund} and Moeslund, {Thomas B.} and Marianne Nyegaard",

note = "This work has been funded by the Independent ResearchFund Denmark under case number 9131-00128B.; Northern Lights Deep Learning Workshop ; Conference date: 10-01-2022 Through 12-01-2022",

year = "2022",

month = mar,

day = "28",

doi = "10.7557/18.6234",

language = "English",

volume = "3",

journal = "Proceedings of the Northern Lights Deep Learning Workshop",

issn = "2703-6928",

url = "https://www.nldl.org/",

}

TY - GEN

T1 - Re-Identification of Giant Sunfish using Keypoint Matching

AU - Pedersen, Malte

AU - Haurum, Joakim Bruslund

AU - Moeslund, Thomas B.

AU - Nyegaard, Marianne

N1 - This work has been funded by the Independent ResearchFund Denmark under case number 9131-00128B.

PY - 2022/3/28

Y1 - 2022/3/28

N2 - We present the first work where re-identification ofthe Giant Sunfish (Mola alexandrini) is automated using computer vision and deep learning. We propose a pipeline that scores an mAP of 60.34% on a full rank of the novel TinyMola dataset which includes 31 IDs and 91 images. The method requires no domain-adaptation or training which makes it especially suited for low-budget or volunteer-based projects, like Match My Mola, as part of a human-in-the-loop model.The pipeline includes segmentation, keypoint detection and description, keypoint matching, and ranking. The choice of feature descriptor has the largest impact on the performance and we show that the deep learning based SuperPoint descriptor greatly outperforms handcrafted descriptors like SIFT and RootSIFT independent of the segmentation level and matching method. Combining SuperPoint and the graph neural network based SuperGlue matching method produces the best results.

AB - We present the first work where re-identification ofthe Giant Sunfish (Mola alexandrini) is automated using computer vision and deep learning. We propose a pipeline that scores an mAP of 60.34% on a full rank of the novel TinyMola dataset which includes 31 IDs and 91 images. The method requires no domain-adaptation or training which makes it especially suited for low-budget or volunteer-based projects, like Match My Mola, as part of a human-in-the-loop model.The pipeline includes segmentation, keypoint detection and description, keypoint matching, and ranking. The choice of feature descriptor has the largest impact on the performance and we show that the deep learning based SuperPoint descriptor greatly outperforms handcrafted descriptors like SIFT and RootSIFT independent of the segmentation level and matching method. Combining SuperPoint and the graph neural network based SuperGlue matching method produces the best results.

U2 - 10.7557/18.6234

DO - 10.7557/18.6234

M3 - Conference article in Journal

SN - 2703-6928

VL - 3

JO - Proceedings of the Northern Lights Deep Learning Workshop

JF - Proceedings of the Northern Lights Deep Learning Workshop

T2 - Northern Lights Deep Learning Workshop

Y2 - 10 January 2022 through 12 January 2022

ER -

Re-Identification of Giant Sunfish using Keypoint Matching

Abstract

Workshop

Bibliographical note

UN SDGs

Access to Document

AUB Link

Fingerprint

Cite this