This project is an effort to establish a unified containerization strategy for the scverse ecosystem. Beyond individual tool containerization, efforts have been made to assemble multi-tool images around commonly used workflows and integrations.
-
Container images are hosted on GitHub Container Registry (ghcr.io/scleestacks). The catalogue of available images can be found in the SCleeStacks packages section.
-
To access the registry, you'll need to authenticate with GitHub. You can do this using your GitHub credentials or a personal access token.
Our registry contains two types of images:
- Individual Tools: Separate images for each tool from the scverse ecosystem. Package releases capture for containerization begin with versions current as of June 2024.
- Stacks: Integrated images containing multiple tools commonly used together.
- airr-multi (scirpy, muon, scanpy)
- scvi-cpu-base (scvi-tools, scanpy)
- scvi-cpu-multi (scvi-tools, muon, scanpy)
- scvi-gpu-base (cuda, scvi-tools, scanpy) very large
- scvi-gpu-multi (cuda, scvi-tools, muon, scanpy) very large
- spatial-base (scanpy, spatialdata, squidpy)
To pull an image from our registry, use the following command:
docker pull ghcr.io/scleestacks[image-name]:[tag]Replace [image-name], and [tag] with the appropriate values.
We welcome contributions to our container registry project. Much of the basis for this work depends on community consensus for stadardization, so please feel free to weigh in whether you are a maintainer or user of scverse tools.
-
Fork the Repository: Start by forking the main repository to your GitHub account.
-
Clone Your Fork: Clone your forked repository to your local machine.
-
Create a Branch: Create a new branch for your contribution.
-
Make Changes: Implement your changes, whether it's adding a new image, updating an existing one, or modifying documentation.
-
Commit Your Changes: Commit your changes with a clear and descriptive commit message.
-
Push to Your Fork: Push your changes to your forked repository.
-
Create a Pull Request: Go to the original repository on GitHub and create a new pull request from your feature branch.
- Follow best practices for writing Dockerfiles
- Minimize layers
- Establish deterministic builds in all possible cases
- Start with lean base images. Current base image defaults:
- CPU: python:3.10-slim
- GPU: nvidia/cuda:12.1.0-devel-ubuntu22.04
- Include necessary metadata (maintainer, version information, etc.).
- Place your Dockerfile in the appropriate directory within the repository.
We use GitHub Actions to automatically build and publish images. To set up a workflow for your image:
- Create a new
.ymlfile in the.github/workflows/directory of the repository. - Use the
workflow-template.ymllocated in the templates directory as a starting point. Replace the env and name vars in square brackets with the appropriate values. - Customize the workflow as needed for your specific image or stack.
- Stack workflow file names should be prefixed with "zz-" to help with organization.
To create a new stack (an integrated image of multiple tools):
- Create a new Dockerfile that includes all necessary tools.
- Use multi-stage builds, isolation of individual tool dependencies, and pip-based reconciliation.
- Ensure your stack image is well-documented, explaining which tools are included and how to use them together.
- Include appropriate tests for your images in the repository.
- Ensure your workflow includes steps to run these tests before pushing the image.
- Update the main README.md file if you're adding new features or changing existing ones.
- Provide clear usage instructions for any new images or stacks you create.
If you encounter issues while contributing, please check our Issues page for known problems and solutions. If your issue isn't addressed, feel free to open a new issue.
- Always tag your images with specific versions to ensure reproducibility.
- Include detailed metadata for stacks in your Dockerfiles and in-place using the
generate-stack-metadata.shscript. - Regularly update base images to include security patches.
- Write clear and concise commit messages and pull request descriptions.
- Engage in respectful and constructive communication in issues and pull requests.