Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Cache not working self-hosted kubernetes runners #112

@MarcHagen

Description

@MarcHagen

Hi! So I'm a bit clueless how to set this up.
I'm not quite sure if you are familiar with the GitHub ARC concept.

Essentially, you have a runner pod/container with an extra runner pod for Docker (aka dind)

Because this repo is running a docker container, it will be running on the extra dind pod.
This pod has three volumes:

  • /home/runner/_work
  • /run/docker
  • /home/runner/externals

Meaning the mount to /tmp/composer-cache in in the dind container.
So the actions/cache script is running in the main runner pod, had thus has no clue about this /tmp/composer-cache

flowchart TD
    Startup

    Init[Init container]
    Runner(Runner)
    DinD(DinD)

    VolumeWork[(/home/runner/_work)]
    VolumeDinDSockWork[(/run/docker)]

    Startup -->|Create POD| Init
    Startup -->|Create POD| Runner

    Init -->|copy /home/runner/externals | DinD

    DinD-- volume mount ---VolumeWork
    DinD-- volume mount ---VolumeDinDSockWork

    Runner-- volume mount ---VolumeWork
    Runner-- volume mount ---VolumeDinDSockWork
Loading

So what I've done now it changed the bash script to point composer to /app/composer-cache
Where /app is the _work directory, aka GITHUB_WORKSPACE

mkdir -p "${GITHUB_WORKSPACE}/composer-cache"
export COMPOSER_CACHE_DIR="/app/composer-cache"
...
docker run --rm \
    --user "$(id -u):$(id -g)" \
    --volume "${github_action_path}/composer.phar":/usr/local/bin/composer \
    --volume ~/.gitconfig:/root/.gitconfig \
    --volume ~/.ssh:/root/.ssh \
    --volume "${GITHUB_WORKSPACE}":/app \
    --workdir ${container_workdir} \
    --env-file ./DOCKER_ENV \
    --network host \
    ${memory_limit} \
    ${docker_tag} /bin/sh -c "${command_string}"

Removed the /tmp/composer-cache mount.
And set the action/cache to use ${{ github.workspace }}/composer-cache

This works, but is a bit modified. So I'm making this issue to open a discussion on how we can improve this.

Another side effect is that the ~/.gitconfig and ~/.ssh will never be mounted, because they dont exist on the DinD pod.

How can we make this action work better for self-hosted ARC runners?


Resource links:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions