BIIGLE Magic SAM Module

This is a BIIGLE module that offers the Magic SAM image annotation instrument.

Installation

1. Run composer require biigle/magic-sam.
2. Run php artisan vendor:publish --tag=public to refresh the public assets of the modules. Do this for every update of this module.
3. Configure a storage disk for the image embedding files. Set the MAGIC_SAM_EMBEDDING_STORAGE_DISK variable in the .env file to the name of the respective storage disk. The content of the storage disk should be publicly accessible. Example for local disks:

   'magic-sam' => [
       'driver' => 'local',
       'root' => storage_path('app/public/magic-sam'),
       'url' => env('APP_URL').'/storage/magic-sam',
       'visibility' => 'public',
   ],

   This requires the link storage -> ../storage/app/public in the public directory.

Configuration

Image embeddings are generated by a Python worker process that requires a dedicated Docker Compose configuration. The Docker container can be based on ghcr.io/biigle/pyworker but also must satisfy the Python requirements of this repository. The SAM model type and checkpoint URL are configured via environment variables, as shown below. Here is a full example configuration:

  magic-sam-pyworker:
    image: magic-sam-pyworker
    user: ${USER_ID}:${GROUP_ID}
    working_dir: /var/www
    volumes:
      - ./:/var/www
    tmpfs:
      - /tmp
    init: true
    environment:
      # The SAM model type.
      # See: https://github.com/facebookresearch/segment-anything#model-checkpoints
      - SAM_MODEL_TYPE=vit_h
      # URL from which to download the model checkpoint (must match the model type above).
      # Important: The model checkpoint must match with the ONNX file (see config/
      # magic_sam.php)!
      - SAM_MODEL_URL=https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
    command: "python vendor/biigle/magic-sam/src/resources/scripts/GenerateEmbeddingWorker.py"

While image embeddings can be generated on the CPU it is much faster on a GPU.

Image embedding files are automatically deleted after 30 days. You can configure this with the MAGIC_SAM_PRUNE_AGE_DAYS environment variable.

References

Reference publications that you should cite if you use Magic SAM for one of your studies.

• BIIGLE 2.0 Langenkämper, D., Zurowietz, M., Schoening, T., & Nattkemper, T. W. (2017). Biigle 2.0-browsing and annotating large marine image collections. Frontiers in Marine Science, 4, 83. doi: 10.3389/fmars.2017.00083

• Segment Anything Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y. and Dollár, P., (2023). Segment anything. arXiv preprint arXiv:2304.02643. doi: 10.48550/arXiv.2304.02643

Developing

Take a look at the development guide of the core repository to get started with the development setup.

Want to develop a new module? Head over to the biigle/module template repository.

Contributions and bug reports

Contributions to BIIGLE are always welcome. Check out the contribution guide to get started.