WildlifeDatasets: an Open-source Toolkit for Animal Re-identification: Conclusion and References

Table of Links

• Abstract and Introduction
• Related work
• The WildlifeDatasets toolkit
• MegaDescriptor – Methodology
• Ablation studies
• Performance evaluation
• Conclusion and References

7. Conclusion

We have introduced the WildlifeDatasets toolkit, an open-source, user-friendly library that provides (i) convenient access and manipulation of all publicly available wildlife datasets for individual re-identification, (ii) access to a variety of state-of-the-art models for animal reidentification, and (iii) simple API that allows inference and matching over new datasets. Besides, we have provided the most comprehensive experimental comparison of these datasets and essential methods in wildlife re-identification using local descriptors and deep learning approaches. Using insights from our ablation studies led to the creation of a MegaDescriptor, the first-ever foundation model for animal re-identification, which delivers state-of-the-art performance on a wide range of species. We anticipate that this toolkit will be widely used by both computer vision scientists and ecologists interested in wildlife re-identification and will significantly facilitate progress in this field.

A. WildlifeDatasets: Supplementary Materials

A.1. Ablation study on model size

To showcase and quantify the performance of different MegaDescriptor flavors, we compare five variants, e.g., Base, Small, Tiny, and Large-224 and Large-384, originating from corresponding variations of the Swin architecture. All the models were trained and evaluated using the same setting. Naturally, the model performance in terms of accuracy increased with an increasing model size, i.e., the MegaDescriptor-L-384 outperformed smaller flavors by a considerable margin in most cases. Overall, higher model complexity achieved higher performance with few exceptions, where it underperformed by a small margin, e.g., by 2.53%, 0.48%, and 0.08% on FriesianCattle2017, LeopardID2022, and MacaqueFaces respectively. This is more or less statistically insignificant, given the poor quality of the data and the data acquisition.

We visualized the accuracy of all provided MegaDescriptor flavors in Figure 9 and Table 5.

A.2. Online Documentation – Dataset samples and tutorials

We provide extensive documentation to give users a better orientation within the WildlifeDatasets toolkit and available features. It covers a wide range of use cases of the toolkit, including a guide to installation and dataset downloading, tutorials, and how to contribute. Notably, the documentation includes a detailed description of the datasets, including image samples.

References

[1] Right whale recognition, 2015. 1, 3

[2] Humpback whale identification, 2019. 1, 3

[3] Beluga ID 2022, 2022. 1, 3

[4] Turtle recall: Conservation challenge, 2022. 1, 3

[5] Luka´s Adam, Vojt ˇ ech ˇ Cerm ˇ ak, Kostas Papafitsoros, and ´ Lukas Picek. SeaTurtleID2022: A long-span dataset for reliable sea turtle re-identification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024. 3, 5

[6] Carlos JR Anderson, Niels Da Vitoria Lobo, James D Roth, and Jane M Waterman. Computer-aided photo-identification system with an application to polar bears based on whisker spot patterns. Journal of Mammalogy, 91(6):1350–1359, 2010. 2

[7] William Andrew, Jing Gao, Siobhan Mullan, Neill Campbell, Andrew W Dowsey, and Tilo Burghardt. Visual identification of individual holstein-friesian cattle via deep metric learning. Computers and Electronics in Agriculture, 185:106133, 2021. 3

[8] William Andrew, Colin Greatwood, and Tilo Burghardt. Visual localisation and individual identification of holstein friesian cattle via deep learning. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 2850–2859, 2017. 3

[9] William Andrew, Sion Hannuna, Neill Campbell, and Tilo Burghardt. Automatic individual holstein friesian cattle identification via selective local coat pattern matching in RGB-D imagery. In 2016 IEEE International Conference on Image Processing (ICIP), pages 484–488. IEEE, 2016. 2, 3

[10] Anka Bedetti, Cathy Greyling, Barry Paul, Jennifer Blondeau, Amy Clark, Hannah Malin, Jackie Horne, Ronny Makukule, Jessica Wilmot, Tammy Eggeling, et al. System for elephant ear-pattern knowledge (seek) to identify individual african elephants. Pachyderm, 61:63–77, 2020. 2

[11] Douglas T Bolger, Thomas A Morrison, Bennet Vance, Derek Lee, and Hany Farid. A computer-assisted system for photographic mark–recapture analysis. Methods in Ecology and Evolution, 3(5):813–822, 2012. 2

[12] Joakim Bruslund Haurum, Anastasija Karpova, Malte Pedersen, Stefan Hein Bengtson, and Thomas B Moeslund. Reidentification of zebrafish using metric learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, pages 1–11, 2020. 2, 3

[13] Ted Cheeseman, Ken Southerland, Jinmo Park, Marilia Olio, Kiirsten Flynn, John Calambokidis, Lindsey Jones, Claire Garrigue, Astrid Frisch Jordan, Addison Howard, et al. Advanced image recognition: a fully automated, high-accuracy photo-identification matching system for humpback whales. Mammalian Biology, 102(3):915–929, 2022. 1, 3

[14] Weihua Chen, Xiaotang Chen, Jianguo Zhang, and Kaiqi Huang. Beyond triplet loss: a deep quadruplet network for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 403–412, 2017. 5

[15] Jonathan P Crall, Charles V Stewart, Tanya Y Berger-Wolf, Daniel I Rubenstein, and Siva R Sundaresan. Hotspotter patterned species instance recognition. In 2013 IEEE Workshop on Applications of Computer Vision (WACV), pages 230–237. IEEE, 2013. 2

[16] Debayan Deb, Susan Wiper, Sixue Gong, Yichun Shi, Cori Tymoszek, Alison Fletcher, and Anil K Jain. Face recognition: Primates in the wild. In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–10. IEEE, 2018. 2, 5

[17] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4690–4699, 2019. 5

[18] Nkosikhona Dlamini and Terence L van Zyl. Automated identification of individuals in wildlife population using siamese neural networks. In 2020 7th International Conference on Soft Computing & Machine Intelligence (ISCMI), pages 224–228. IEEE, 2020. 1, 3

[19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 4

[20] Axel Drechsler, Tobias Helling, and Sebastian Steinfartz. Genetic fingerprinting proves cross-correlated automatic photo-identification of individuals as highly efficient in large capture–mark–recapture studies. Ecology and Evolution, 5(1):141–151, 2015. 2

[21] Stephen G Dunbar, Edward C Anger, Jason R Parham, Colin Kingen, Marsha K Wright, Christian T Hayes, Shahnaj Safi, Jason Holmberg, Lidia Salinas, and Dustin S Baumbach. Hotspotter: Using a computer-driven photo-id application to identify sea turtles. Journal of Experimental Marine Biology and Ecology, 535:151490, 2021. 1, 2, 5

[22] Andre C Ferreira, Liliana R Silva, Francesco Renna, Hanja B ´ Brandl, Julien P Renoult, Damien R Farine, Rita Covas, and Claire Doutrelant. Deep learning-based methods for individual recognition in small birds. Methods in Ecology and Evolution, 11(9):1072–1085, 2020. 3

[23] Alexander Freytag, Erik Rodner, Marcel Simon, Alexander Loos, Hjalmar S Kuhl, and Joachim Denzler. Chimpanzee ¨ faces in the wild: Log-Euclidean CNNs for predicting identities and attributes of primates. In German Conference on Pattern Recognition, pages 51–63. Springer, 2016. 1, 3

[24] Jing Gao, Tilo Burghardt, William Andrew, Andrew W Dowsey, and Neill W Campbell. Towards self-supervision for video identification of individual holstein-friesian cattle: The Cows2021 dataset. arXiv preprint arXiv:2105.01938, 2021. 3

[25] Andrew Gilman, Krista Hupman, Karen A Stockin, and Matthew DM Pawley. Computer-assisted recognition of dolphin individuals using dorsal fin pigmentations. In 2016 International Conference on Image and Vision Computing New Zealand (IVCNZ), pages 1–6. IEEE, 2016. 2

[26] Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017. 5

[27] Jason Holmberg, Bradley Norman, and Zaven Arzoumanian. Estimating population size, structure, and residency time for whale sharks Rhincodon typus through collaborative photoidentification. Endangered Species Research, 7(1):39–53, 2009. 3

[28] Jeff Johnson, Matthijs Douze, and Herve J ´ egou. Billion- ´ scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2019. 4

[29] Marcella J Kelly. Computer-aided photograph matching in studies using individual identification: an example from Serengeti cheetahs. Journal of Mammalogy, 82(2):440–449, 2001. 2

[30] Mayank Lahiri, Chayant Tantipathananandh, Rosemary Warungu, Daniel I Rubenstein, and Tanya Y Berger-Wolf. Biometric animal databases from field photographs: identification of individual zebra in the wild. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval, pages 1–8, 2011. 3

[31] Shuyuan Li, Jianguo Li, Hanlin Tang, Rui Qian, and Weiyao Lin. ATRW: A benchmark for amur tiger re-identification in the wild. arXiv preprint arXiv:1906.05586, 2019. 1, 2, 3, 5

[32] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021. 4

[33] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022. 4

[34] Vincent Miele, Gaspard Dussert, Bruno Spataro, Simon Chamaille-Jammes, Dominique Allain ´ e, and Christophe Bo- ´ nenfant. Revisiting animal photo-identification using deep metric learning and network analysis. Methods in Ecology and Evolution, 12(5):863–873, 2021. 2, 3, 5

[35] Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. A metric learning reality check. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16, pages 681–699. Springer, 2020. 5

[36] Ekaterina Nepovinnykh, Tuomas Eerola, Vincent Biard, Piia Mutka, Marja Niemi, Heikki Kalvi ¨ ainen, and Mervi Kunnas- ¨ ranta. SealID: Saimaa ringed seal re-identification dataset. arXiv preprint arXiv:2206.02260, 2022. 3

[37] Maxime Oquab, Timothee Darcet, Th ´ eo Moutakanni, Huy ´ Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2

[38] Kostas Papafitsoros, Luka´s Adam, Vojt ˇ ech ˇ Cerm ˇ ak, and ´ Luka´s Picek. SeaTurtleID: A novel long-span dataset ˇ highlighting the importance of timestamps in wildlife reidentification. arXiv preprint arXiv:2211.10307, 2022. 3, 5

[39] Kostas Papafitsoros, Aliki Panagopoulou, and Gail Schofield. Social media reveals consistently disproportionate tourism pressure on a threatened marine vertebrate. Animal Conservation, 24(4):568–579, 2021. 1

[40] Jason Remington Parham, Jonathan Crall, Charles Stewart, Tanya Berger-Wolf, and Daniel Rubenstein. Animal population censusing at scale with citizen science and photographic identification. In 2017 AAAI Spring Symposium Series, 2017. 3

[41] Malte Pedersen, Joakim Bruslund Haurum, Thomas B Moeslund, and Marianne Nyegaard. Re-identification of giant sunfish using keypoint matching. In Proceedings of the Northern Lights Deep Learning Workshop, volume 3, 2022. 5

[42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748–8763. PMLR, 2021. 2

[43] Vito Reno, Giovanni Dimauro, G Labate, Ettore Stella, ` Carmelo Fanizza, Giulia Cipriano, Roberto Carlucci, and Rosalia Maglietta. A SIFT-based software system for the photo-identification of the Risso’s dolphin. Ecological informatics, 50:95–101, 2019. 2

[44] Jonathan Schneider, Nihal Murali, Graham W Taylor, and Joel D Levine. Can Drosophila melanogaster tell who’s who? PloS one, 13(10):e0205043, 2018. 3

[45] Gail Schofield, Kostas Papafitsoros, Chloe Chapman, Akanksha Shah, Lucy Westover, Liam CD Dickson, and Kostas A Katselidis. More aggressive sea turtles win fights over foraging resources independent of body size and years of presence. Animal Behaviour, 190:209–219, 2022. 1

[46] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 815–823, 2015. 5

[47] Cameron Trotter, Georgia Atkinson, Matt Sharpe, Kirsten Richardson, A Stephen McGough, Nick Wright, Ben Burville, and Per Berggren. NDD20: A large-scale few-shot dolphin dataset for coarse and fine-grained categorisation. arXiv preprint arXiv:2005.13359, 2020. 1, 3

[48] Botswana Predator Conservation Trust. Panthera pardus csv custom export, 2022. 1, 3

[49] Masataka Ueno, Ryosuke Kabata, Hidetaka Hayashi, Kazunori Terada, and Kazunori Yamada. Automatic individual recognition of Japanese macaques (Macaca fuscata) from sequential images. Ethology, 128(5):461–470, 2022. 2

[50] Maxime Vidal, Nathan Wolf, Beth Rosenberg, Bradley P Harris, and Alexander Mathis. Perspectives on individual animal identification from biology and computer vision. Integrative and Comparative Biology, 61(3):900–916, 2021. 1

[51] Le Wang, Rizhi Ding, Yuanhao Zhai, Qilin Zhang, Wei Tang, Nanning Zheng, and Gang Hua. Giant panda identification. IEEE Transactions on Image Processing, 30:2837– 2849, 2021. 3

[52] Hendrik Weideman, Chuck Stewart, Jason Parham, Jason Holmberg, Kiirsten Flynn, John Calambokidis, D Barry Paul, Anka Bedetti, Michelle Henley, Frank Pope, and Jerenimo Lepirei. Extracting identifying contours for African elephants and humpback whales using a learned appearance model. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1276–1285, 2020. 2

[53] Ross Wightman. Pytorch image models. https : / / github . com / rwightman / pytorch - image - models, 2019. 4

[54] Claire L Witham. Automated face recognition of rhesus macaques. Journal of Neuroscience Methods, 300:157–165, 2018. 1, 3

[55] Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, and ´ Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1492–1500, Honolulu, 2017. 4

[56] Cheng Yan, Guansong Pang, Xiao Bai, Changhong Liu, Xin Ning, Lin Gu, and Jun Zhou. Beyond triplet loss: person re-identification with fine-grained difference-aware pairwise loss. IEEE Transactions on Multimedia, 24:1665–1677, 2021. 5

[57] Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, and Michael J Black. Three-D safari: Learning to estimate zebra pose, shape, and texture from images ”In the wild”. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5359–5368, 2019. 1, 3