[BUG] Build secret files not mounted in /run/secrets when using docker compose build with profiles #13020
Description
Description
If I set up a Dockerfile for build secrets, docker build --secret ... works just fine per the official docs.
If I set up a docker compose project per the official docs (this one too) secrets do still seem to work just fine.
However, if I add a profile to my service that has build secrets, and try to specifically build the service in isolation, I run into issues!
Further, if i do a direct up --build on the command behind the profile, that still works though...
Note: this is highly related to other issues such as #9822 & #13019 (sorry for the confusion on that latter one). But it describes a different issue with profiles and build vs. up behavior on the latest docker engine.
Steps To Reproduce
- Create a simple local test file:
echo "SECRET TEST" > test-secret.txt - Create a Dockerfile with this content
#Dockerfile
FROM debian:stable-slim
RUN --mount=type=secret,id=test-secret ls -la /run/secrets/; cp /run/secrets/test-secret /tmp
CMD ["cat", "/tmp/test-secret"]- Create a docker-compose file with this content:
#docker-compose.yaml
secrets:
test-secret:
file: test-secret.txt
services:
secret-build-test:
profiles: ["test"]
build:
context: .
dockerfile: Dockerfile
secrets:
- test-secret- Verify that a docker compose build with profiles activated works as expected
docker builder prune -fa
docker compose --progress=plain --profile test build secret-build-testshould output something like the following:
...
#6 [stage-0 2/2] RUN --mount=type=secret,id=test-secret ls -la /run/secrets/; cp /run/secrets/test-secret /tmp
#6 0.156 total 12
#6 0.156 drwxr-xr-x 2 root root 4096 Jul 4 15:42 .
#6 0.156 drwxr-xr-x 1 root root 4096 Jul 4 15:42 ..
#6 0.156 -r-------- 1 root root 12 Jul 4 15:42 test-secret
#6 DONE 0.2s
...
- Demonstrate that a direct docker compose build fails!
docker builder prune -fa
docker compose build secret-build-test --progress=plain --no-cacheThis outputs the following for me
...
#6 [stage-0 2/2] RUN --mount=type=secret,id=test-secret ls -la /run/secrets/; cp /run/secrets/test-secret /tmp
#6 0.149 ls: cannot access '/run/secrets/': No such file or directory
#6 0.150 cp: cannot stat '/run/secrets/test-secret': No such file or directory
#6 ERROR: process "/bin/sh -c ls -la /run/secrets/; cp /run/secrets/test-secret /tmp" did not complete successfully: exit code: 1
------
> [stage-0 2/2] RUN --mount=type=secret,id=test-secret ls -la /run/secrets/; cp /run/secrets/test-secret /tmp:
0.149 ls: cannot access '/run/secrets/': No such file or directory
0.150 cp: cannot stat '/run/secrets/test-secret': No such file or directory
------
...
- Show that a plain
upcommand still works though...
docker builder prune -fa
docker compose up --build secret-build-testThis outputs the following for me:
...
=> [stage-0 2/2] RUN --mount=type=secret,id=test-secret ls -la /run/secrets/; cp /run/secrets/test-secret /tmp 0.2s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:fe3ec2d88d0be58cdfc31774c851e50c985d05b7f86d36b2bf24da0d9e511ea7 0.0s
=> => naming to docker.io/library/compose-secrets-issue-secret-build-test 0.0s
=> resolving provenance for metadata file 0.0s
[+] Running 2/2
✔ secret-build-test Built 0.0s
✔ Container compose-secrets-issue-secret-build-test-1 Recreated 0.1s
Attaching to secret-build-test-1
secret-build-test-1 | SECRET TEST
secret-build-test-1 exited with code 0
Compose Version
Docker Compose version v2.37.1-desktop.1
Docker Environment
Client:
Version: 28.2.2
Context: default
Debug Mode: false
Plugins:
ai: Docker AI Agent - Ask Gordon (Docker Inc.)
Version: v1.4.0
Path: /usr/local/lib/docker/cli-plugins/docker-ai
buildx: Docker Buildx (Docker Inc.)
Version: v0.24.0-desktop.2
Path: /usr/local/lib/docker/cli-plugins/docker-buildx
cloud: Docker Cloud (Docker Inc.)
Version: v0.3.9
Path: /usr/local/lib/docker/cli-plugins/docker-cloud
compose: Docker Compose (Docker Inc.)
Version: v2.37.1-desktop.1
Path: /usr/local/lib/docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.41
Path: /usr/local/lib/docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Docker Inc.)
Version: v0.1.9
Path: /usr/local/lib/docker/cli-plugins/docker-desktop
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.29
Path: /usr/local/lib/docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.4.0
Path: /usr/local/lib/docker/cli-plugins/docker-init
mcp: Docker MCP Plugin (Docker Inc.)
Version: dev
Path: /usr/local/lib/docker/cli-plugins/docker-mcp
model: Docker Model Runner (Docker Inc.)
Version: v0.1.24
Path: /usr/local/lib/docker/cli-plugins/docker-model
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /usr/local/lib/docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.18.1
Path: /home/kevin/.docker/cli-plugins/docker-scout
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-dev" is not valid: failed to fetch metadata: fork/exec /usr/local/lib/docker/cli-plugins/docker-dev: no such file or directory
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-feedback" is not valid: failed to fetch metadata: fork/exec /usr/local/lib/docker/cli-plugins/docker-feedback: no such file or directory
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /usr/local/lib/docker/cli-plugins/docker-scan: no such file or directory
Server:
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 6
Server Version: 28.2.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Discovered Devices:
cdi: docker.com/gpu=webgpu
Swarm: inactive
Runtimes: nvidia runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.6.87.2-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 31.18GiB
Name: docker-desktop
ID: adac6913-bc44-4345-b8d0-ba432c46a5ce
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///var/run/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
::1/128
127.0.0.0/8
Live Restore Enabled: false
WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW is set
Anything else?
As a general note: in the past, I've typically been able to expect that direct service callouts to build, up etc. will work in isolation regardless of profiles. But it seems there's been some tumult in profile behavior recently. Is there a desire / trend from compose developers to make profiled services only work when profiles are provided? Or can I still reliably count on isolated callouts to work regardless?