Beaker Gantry

Gantry is a CLI that streamlines running experiments in Beaker.

---

⚡️Easy to use

• No Docker required! 🚫 🐳
• No writing Beaker YAML experiment specs.
• Easy setup.
• Simple CLI.

🏎 Fast

• Fire off Beaker experiments from your laptop instantly!
• No local image build or upload.

🪶 Lightweight

• Pure Python (built on top of beaker's Python client).
• Minimal dependencies.

Gantry is for both new and seasoned Beaker users who need to run batch jobs (as opposed to interactive sessions) from a rapidly changing repository, especially Python-based jobs.

Without Gantry, this workflow usually looks like this:

1. Add a Dockerfile to your repository.
2. Build the Docker image locally.
3. Push the Docker image to Beaker.
4. Write a YAML Beaker experiment spec that points to the image you just uploaded.
5. Submit the experiment spec.
6. Make changes and repeat from step 2.

This requires experience with Docker, experience writing Beaker experiment specs, and a fast and reliable internet connection.

With Gantry, on the other hand, that same workflow simplifies down to this:

1. (Optional) Write a pyproject.toml/setup.py file, a PIP requirements.txt file, a or conda environment.yml file to specify your Python environment.
2. Commit and push your changes.
3. Submit and track a Beaker experiment with the gantry run command.
4. Make changes and repeat from step 2.

• 💾 Installing
• 🚀 Quick start
• ❓ FAQ

• Savings results / metrics from an experiment

• CHANGELOG
• CONTRIBUTING

Gantry is available on PyPI. Just run

pip install beaker-gantry

Gantry can be installed and made available on the PATH using uv:

uv tool install beaker-gantry

With this command, beaker-gantry is automatically installed to an isolated virtual environment.

To install Gantry from source, first clone the repository:

git clone https://github.com/allenai/beaker-gantry.git
cd beaker-gantry

Then run

pip install -e .

1. Create and clone your repository.

   If you haven't already done so, create a GitHub repository for your project and clone it locally. Every gantry command you run must be invoked from the root directory of your repository.

2. Configure Gantry.

   If you've already configured the Beaker command-line client, Gantry will find and use the existing configuration file (usually located at $HOME/.beaker/config.yml). Otherwise just set the environment variable BEAKER_TOKEN to your Beaker user token.

   Some gantry settings can also be specified in a pyproject.toml file under the section [tool.gantry]. For now those settings are:

   1. workspace - The default Beaker workspace to use.
   2. gh_token_secret - The name of the Beaker secret with your GitHub API token.
   3. budget - The default Beaker budget to use.
   4. log_level - The (local) Python log level. Defaults to "warning".
   5. quiet - A boolean. If true the gantry logo won't be displayed on the command line.

   For example:

   # pyproject.toml
   [tool.gantry]
   workspace = "ai2/my-default-workspace"
   gh_token_secret = "GITHUB_TOKEN"
   budget = "ai2/my-teams-budget"
   log_level = "warning"
   quiet = false

   The first time you call gantry run ... you'll also be prompted to provide a GitHub personal access token with the repo scope if your repository is private. This allows Gantry to clone your private repository when it runs in Beaker. You don't have to do this just yet (Gantry will prompt you for it), but if you need to update this token later you can use the gantry config set-gh-token command.

3. (Optional) Specify your Python environment.

   Typically you'll have to create one of several different files to specify your Python environment. There are three widely used options:

   1. A pyproject.toml or setup.py file.
   2. A PIP requirements.txt file.
   3. A conda environment.yml file.

   Gantry will automatically find and use these files to reconstruct your Python environment at runtime. Alternatively you can provide a custom Python install command with the --install option to gantry run, or skip the Python setup completely with --no-python.

Let's spin up a Beaker experiment that just prints "Hello, World!" from Python.

First make sure you've committed and pushed all changes so far in your repository. Then (from the root of your repository) run:

gantry run --show-logs -- python -c 'print("Hello, World!")'

❗Note: Everything after the -- is the command + arguments you want to run on Beaker. It's necessary to include the -- if any of your arguments look like options themselves (like -c in this example) so gantry can differentiate them from its own options.

In this case we didn't request any GPUs nor a specific cluster, so this could run on any Beaker cluster. We can use the --gpu-type and --gpus options to get GPUs. For example:

gantry run --show-logs --gpu-type=h100 --gpus=1 -- python -c 'print("Hello, World!")'

Or we can use the --cluster option to request clusters by their name or aliases. For example:

gantry run --show-logs --cluster=ai2/jupiter --gpus=1 -- python -c 'print("Hello, World!")'

Try gantry run --help to see all of the available options.

gantry run --show-logs --docker-image='python:3.10' --system-python -- python --version

gantry run --show-logs --dataset='petew/squad-train:/input-data' -- ls /input-data

gantry run --show-logs --weka='oe-training-default:/mount/weka' -- ls -l /mount/weka

gantry run \
  --show-logs \
  --gpus=8 \
  --gpu-type='h100' \
  --replicas=2 \
  --torchrun \
  --install 'uv pip install . torch numpy --torch-backend=cu129' \
  -- python -m gantry.all_reduce_bench

gantry run \
  --show-logs \
  --gpus=8 \
  --gpu-type='h100' \
  --replicas=2 \
  --leader-selection \
  --host-networking \
  --propagate-failure \
  --propagate-preemption \
  --synchronized-start-timeout='5m' \
  --install 'uv pip install . torch numpy --torch-backend=cu129' \
  --exec-method=bash \
  -- torchrun \
    '--nnodes="$BEAKER_REPLICA_COUNT:$BEAKER_REPLICA_COUNT"' \
    '--nproc-per-node="$BEAKER_ASSIGNED_GPU_COUNT"' \
    '--rdzv-id=12347' \
    '--rdzv-backend=static' \
    '--rdzv-endpoint="$BEAKER_LEADER_REPLICA_HOSTNAME:29400"' \
    '--node-rank="$BEAKER_REPLICA_RANK"' \
    '--rdzv-conf="read_timeout=420"' \
    -m gantry.all_reduce_bench

 gantry run \
   --show-logs \
   --gpus=8 \
   --gpu-type='h100' \
   --replicas=2 \
   --leader-selection \
   --host-networking \
   --propagate-failure \
   --propagate-preemption \
   --synchronized-start-timeout='5m' \
   --install 'uv pip install . torch numpy --torch-backend=cu129' \
-  --exec-method='bash' \
-  -- torchrun \
-    '--nnodes="$BEAKER_REPLICA_COUNT:$BEAKER_REPLICA_COUNT"' \
-    '--nproc-per-node="$BEAKER_ASSIGNED_GPU_COUNT"' \
-    '--rdzv-id=12347' \
-    '--rdzv-backend=static' \
-    '--rdzv-endpoint="$BEAKER_LEADER_REPLICA_HOSTNAME:29400"' \
-    '--node-rank="$BEAKER_REPLICA_RANK"' \
-    '--rdzv-conf="read_timeout=420"' \
-    -m gantry.all_reduce_bench
+  -- ./launch-torchrun.sh

gantry run --show-logs --install='pip install -r custom_requirements.txt' -- echo "Hello, World!"

gantry run --show-logs --python-manager=conda -- which python

name: Beaker

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main

jobs:
  gpu_tests:
    name: GPU Tests
    runs-on: ubuntu-latest
    timeout-minutes: 15
    env:
      BEAKER_TOKEN: ${{ secrets.BEAKER_TOKEN }}
      GANTRY_GITHUB_TESTING: 'true'  # force better logging for CI
      BEAKER_WORKSPACE: 'ai2/your-workspace'  # TODO: change this to your Beaker workspace
    steps:
      - uses: actions/checkout@v5
        with:
          ref: ${{ github.event.pull_request.head.sha }}  # check out PR head commit instead of merge commit

      - uses: astral-sh/setup-uv@v6
        with:
          python-version: '3.12'

      - name: install gantry
        run:
          uv tool install 'beaker-gantry>=3.1,<4.0'

      - name: Determine current commit SHA (pull request)
        if: github.event_name == 'pull_request'
        run: |
          echo "COMMIT_SHA=${{ github.event.pull_request.head.sha }}" >> $GITHUB_ENV
          echo "BRANCH_NAME=${{ github.head_ref }}" >> $GITHUB_ENV

      - name: Determine current commit SHA (push)
        if: github.event_name != 'pull_request'
        run: |
          echo "COMMIT_SHA=$GITHUB_SHA" >> $GITHUB_ENV
          echo "BRANCH_NAME=${{ github.ref_name }}" >> $GITHUB_ENV

      - name: launch job
        run: |
          gantry run \
            --show-logs \
            --yes \
            --workspace ${{ env.BEAKER_WORKSPACE }} \
            --description 'GitHub Actions GPU tests' \
            --ref ${{ env.COMMIT_SHA }} \
            --branch ${{ env.BRANCH_NAME }} \
            --priority normal \
            --preemptible \
            --gpus 1 \
            --gpu-type h100 \
            --gpu-type a100 \
            -- pytest -v tests/cuda_tests/  # TODO: change to your own command