A software that detects sharks and rays in underwater videos and computes MaxN 21 times faster, with machine learning.

Here we provide a guide to learn about SharkTrack. The guide is divided in the following 3 chapters. We suggest to follow the order. It should take less than 15m.

Click on the image above to watch a demo

Shark and Ray (Elasmobranch) researchers monitor their populations using Baited Remote Underwater Video Systems (BRUVS). This is a time-consuming process, as each video needs to be manually annotated.

SharkTrack is a Machine Learning model that uses computer vision to detect and track Elasmobranchii in BRUVS videos and compute the MaxN metrics, used by ecologists.

Therefore SharkTrack is an AI-enhanced workflow to convert raw BRUVS videos to MaxN, which has been tested 21x faster than traditional methods.

• 🏃‍♀️ Computes MaxN semi-automatically, 21x faster
• 🏝 Can detect any elasmobranch species (as a single 'elasmobranch' class) in any location with 89% accuracy
• 👨‍💻 Can run on a standard laptop - no experience or advanced tech requirement

SharkTrack analyses BRUVS in two steps, as showed below: The SharkTrack pipeline to compute MaxN, divided into Step 1 (top) and Step 2 (bottom)

• (a) Ingests all underwater videos in a hard drive or folder
• (b) Automatically detects elasmobranchs
• (c) Save sightings in a CSV
• (d) Save a screenshot for each detected elasmobranch with video Each detection image shows the video and time it was captured

• (e) Classify the species of detected elasmobranchs by renaming the relative screenshot filename.
• (f) SharkTrack updates all sightings of detected elasmobranchs with the new species classification
• (g) And outputs the species-specific MaxN

• 👀 Peek Mode After a day of sampling, run it on your laptop overnight and automatically detect where and when sharks and rays appeared in the videos.
• 🔎 Analyst Mode Accurately analyse the footage to derive relative abundance with the MaxN metrics.

Both modes can be run on a standard laptop and do not require WIFI.

Overall, SharkTrack is the most helpful to process videos where elasmobranch appears rarely, as it extracts them from the mainly empty footage, removeing the need to watch it manually.

SharkTrack is a publicly-available model and you can install and run it by following this guide.

If you don't have experience with Python, we know it can be daunting but don't fear! By following the guide step-by-step you will have SharkTrack up and running on your videos in less than 10 minutes.

Reach out if have any quesions here

• Anguilla by University of Exeter and Anguilla National Trust
• Revillagigedo Archipelago, Mexico by Pelagios Kakunja
• La Paz, Mexico by Pelagios Kakunja
• Northern Gulf of California, Mexico by Pelagios Kakunja
• Maldives, by the University of Leeds
• Cabo Verde, by Dr. Francesco Garzon and Adam Whiting of the University of Exeter
• Red Sea, by Dr. Francesco Garzon, University of Exeter
• Cape Cod, by the Virginia Tech University
• Hawaii, by the Virginia Tech University
• Azores, by the Okeanos Institute of Marine Sciences

This software and related work was supported by the efforts of Filippo Varini, Joel H. Gayford, Jeremy Jenrette, Matthew J. Witt, Francesco Garzon, Francesco Ferretti, Sophie Wilday, Mark E. Bond, Michael R. Heithaus, Danielle Robinson, Devon Carter, Najee Gumbs, Vincent Webster, Ben Glocker, Fabio De Sousa Ribeiro, Rajat Rasal, Orlando Timmerman, Natalie Ng, Rui Wen Lim, Michael Sellgren, Lara Tse, Steven Chen, Maria Pia Donrelas, Manfredi Minervini, Xuen Bei (Bay) Chin, Adam Whiting, Aurora Crocini, Gabriele Bai, Stephanie Guerinfor.

This repository is licensed with the MIT license. If you use SharkTrack, please cite:

Varini, F. et al (2024). SharkTrack. GitHub. Available at https://github.com/filippovarini/sharktrack

The same citation, in BibTex format:

@article{varini2024sharktrack,
  title={SharkTrack},
  author={Filippo Varini et al},
  year={2024}
}

We update the citation to reference the paper once it has been pubblished.

Please submit any issue on GitHub. We aim to respond to it within a week.

This project welcomes contributions, as pull requests, issues, or suggestions by email.

This is the first step of a broader effort to develop generalisable marine species classifiers. We are looking for contributors for this project. If you want to get involved in AI-driven Ocean Conservation please email us.

Run the model on your BRUVS by following this guide