feat(bazel): wire up prebuilt protoc toolchain #24115

alexeagle · 2025-10-22T01:58:00Z

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo.

Adds:

private API: module_extension + tag protoc.prebuilt_toolchain to provide and register toolchain for bzlmod users
WORKSPACE users can live with the inconvenience of having to create toolchains themselves
example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature

This needs work with @thesayyn to:

constrain users to choosing a protoc version agreeing with design doc
https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit
also register lang toolchains when the flag is enabled

Work towards #19558

examples/example_without_cc_toolchain/MODULE.bazel

bazel/private/prebuilt_protoc_extension.bzl

bazel/private/prebuilt_protoc_toolchain.bzl

bazel/private/prebuilt_protoc_extension.bzl

bazel/toolchains/BUILD

bazel/private/toolchains/prebuilt/BUILD.bazel

examples/example_without_cc_toolchain/.bazelrc

bazel/private/prebuilt_tool_integrity.bzl

mkruskal-google · 2025-11-15T04:41:41Z

This looks reasonable to me. But it's still missing the validation layer AFAICT. Is this PR waiting for that? Also the tests are all still broken

alexeagle · 2025-11-17T00:53:52Z

Sure, I can pick up something like #23539 - the only comment I got there was "cannot reproduce" but I'd like to know if the guarantee is strong enough.

Separately I'd be very happy to land this one first. That could unblock the flag flip in time for Bazel 9 due in a month.

zhangskz · 2025-11-17T20:48:37Z

Sure, I can pick up something like #23539 - the only comment I got there was "cannot reproduce" but I'd like to know if the guarantee is strong enough.

Separately I'd be very happy to land this one first. That could unblock the flag flip in time for Bazel 9 due in a month.

I'd probably like to see version check to prevent use of arbitrarily old versions of prebuilt protoc or newer versions of protoc from being used like w https://protobuf.dev/support/cross-version-runtime-guarantee/.

MODULE.bazel

zhangskz · 2025-11-17T20:23:54Z

bazel/private/prebuilt_tool_integrity.bzl

-}
+# An arbitrary version of protobuf that includes pre-built binaries.
+# See /examples/example_without_cc_toolchain which uses this for testing.
+_TEST_VERSION = "v33.0"


We probably still want this updated to ensure it doesn't end up too ancient. Perhaps this can get updated via GHA too triggered by a new release to update this when there are new minor releases (e.g. v33.0) in at least main and the corresponding release branch.

Sure, that's possible, I can propose some GHA automation after this lands.

zhangskz · 2025-11-17T20:28:45Z

bazel/toolchains/BUILD

+bool_flag(
+    name = "prefer_prebuilt_protoc",
+    # TODO(alexeagle): this should be True after the feature is vetted with some adoption
+    build_setting_default = False,


This will need to be set to false for most of our CI tests at least.

Sure, I wouldn't expect the default for end-users to be motivated similarly to how protobuf team will test. In fact, this default could vary between this repo and the Bazel module we publish, using the BCR patches feature.

mkruskal-google · 2025-11-18T00:03:07Z

Sure, I can pick up something like #23539 - the only comment I got there was "cannot reproduce" but I'd like to know if the guarantee is strong enough.

Separately I'd be very happy to land this one first. That could unblock the flag flip in time for Bazel 9 due in a month.

+1 to Sandy, while we might be ok with landing this PR first, we definitely wouldn't want to flip the flag without the validation we laid out in the design.

I'm not sure what guarantee is in that prototype, but it does seem like an appropriate place to add the validation

alexeagle · 2025-11-20T11:44:50Z

okay @zhangskz I've added the version verification to this PR, PTAL

.github/workflows/test_bazel.yml

bazel/private/toolchains/prebuilt/protoc_authenticity.bzl

Users can manually register this in their MODULE.bazel so that registration overrides the default, which is still building protoc from source.

tmp

@thesayyn

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo. Adds: - private API: module_extension + tag `protoc.prebuilt_toolchain` to provide and register toolchain for bzlmod users - WORKSPACE users can live with the inconvenience of having to create toolchains themselves - example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature This needs work with @thesayyn to: - constrain users to choosing a protoc version agreeing with design doc https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit - also register lang toolchains when the flag is enabled Work towards #19558 Closes #24115 COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d FUTURE_COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d PiperOrigin-RevId: 843732324

@thesayyn

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo. Adds: - private API: module_extension + tag `protoc.prebuilt_toolchain` to provide and register toolchain for bzlmod users - WORKSPACE users can live with the inconvenience of having to create toolchains themselves - example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature This needs work with @thesayyn to: - constrain users to choosing a protoc version agreeing with design doc https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit - also register lang toolchains when the flag is enabled Work towards #19558 Closes #24115 COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d FUTURE_COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d PiperOrigin-RevId: 843732324

@thesayyn

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo. Adds: - private API: module_extension + tag `protoc.prebuilt_toolchain` to provide and register toolchain for bzlmod users - WORKSPACE users can live with the inconvenience of having to create toolchains themselves - example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature This needs work with @thesayyn to: - constrain users to choosing a protoc version agreeing with design doc https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit - also register lang toolchains when the flag is enabled Work towards #19558 Closes #24115 COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d FUTURE_COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d PiperOrigin-RevId: 843732324

examples/example_without_cc_toolchain/BUILD.bazel

@thesayyn

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo. Adds: - private API: module_extension + tag `protoc.prebuilt_toolchain` to provide and register toolchain for bzlmod users - WORKSPACE users can live with the inconvenience of having to create toolchains themselves - example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature This needs work with @thesayyn to: - constrain users to choosing a protoc version agreeing with design doc https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit - also register lang toolchains when the flag is enabled Work towards #19558 Closes #24115 COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d FUTURE_COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d PiperOrigin-RevId: 844815128

@thesayyn

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo. Adds: - private API: module_extension + tag `protoc.prebuilt_toolchain` to provide and register toolchain for bzlmod users - WORKSPACE users can live with the inconvenience of having to create toolchains themselves - example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature This needs work with @thesayyn to: - constrain users to choosing a protoc version agreeing with design doc https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit - also register lang toolchains when the flag is enabled Work towards #19558 Closes #24115 COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d FUTURE_COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d PiperOrigin-RevId: 844815128

@thesayyn

Upstreamed from my https://github.com/aspect-build/toolchains_protoc which provided this feature outside the protobuf repo. Adds: - private API: module_extension + tag `protoc.prebuilt_toolchain` to provide and register toolchain for bzlmod users - WORKSPACE users can live with the inconvenience of having to create toolchains themselves - example of proto_library rule with intentionally non-functional cc toolchain, exercising the feature This needs work with @thesayyn to: - constrain users to choosing a protoc version agreeing with design doc https://docs.google.com/document/d/16N-eU-0zHbWmxEuaUAIFTJwQLekIXZ8zIC3K4XDr20E/edit - also register lang toolchains when the flag is enabled Work towards #19558 Closes #24115 COPYBARA_INTEGRATE_REVIEW=#24115 from protocolbuffers:alexeagle/use_prebuilt_toolchain b18667d PiperOrigin-RevId: 844859103

alexeagle requested a review from a team as a code owner October 22, 2025 01:58

alexeagle requested review from Logofile and removed request for a team October 22, 2025 01:58

fmeum reviewed Oct 22, 2025

View reviewed changes

examples/example_without_cc_toolchain/MODULE.bazel Outdated Show resolved Hide resolved

Logofile requested review from honglooker and removed request for Logofile October 22, 2025 12:25

fmeum reviewed Oct 22, 2025

View reviewed changes

honglooker removed their request for review October 22, 2025 15:49

alexeagle requested a review from thesayyn October 22, 2025 20:48

alexeagle changed the title ~~WIP: wire up prebuilt protoc toolchain~~ feat(bazel): wire up prebuilt protoc toolchain Nov 5, 2025

alexeagle force-pushed the alexeagle/use_prebuilt_toolchain branch from 46f35dc to 7e9cd2c Compare November 5, 2025 02:02

mkruskal-google requested review from mkruskal-google and zhangskz November 8, 2025 00:22

mkruskal-google reviewed Nov 8, 2025

View reviewed changes

examples/example_without_cc_toolchain/.bazelrc Show resolved Hide resolved

bazel/private/prebuilt_tool_integrity.bzl Outdated Show resolved Hide resolved

alexeagle force-pushed the alexeagle/use_prebuilt_toolchain branch from 7e9cd2c to d76fbd6 Compare November 14, 2025 18:36

alexeagle force-pushed the alexeagle/use_prebuilt_toolchain branch from d76fbd6 to f45a2a4 Compare November 17, 2025 01:24

zhangskz reviewed Nov 17, 2025

View reviewed changes

alexeagle mentioned this pull request Nov 20, 2025

chore: show a validation action #23539

Closed

alexeagle force-pushed the alexeagle/use_prebuilt_toolchain branch 2 times, most recently from 1fd7854 to 6dce826 Compare November 20, 2025 11:44

mkruskal-google reviewed Nov 27, 2025

View reviewed changes

.github/workflows/test_bazel.yml Outdated Show resolved Hide resolved

.github/workflows/test_bazel.yml Outdated Show resolved Hide resolved

.github/workflows/test_bazel.yml Show resolved Hide resolved

zhangskz reviewed Dec 1, 2025

View reviewed changes

bazel/private/toolchains/prebuilt/protoc_authenticity.bzl Outdated Show resolved Hide resolved

bazel/private/toolchains/prebuilt/protoc_authenticity.bzl Show resolved Hide resolved

alexeagle force-pushed the alexeagle/use_prebuilt_toolchain branch 2 times, most recently from ebad988 to ab033ab Compare December 9, 2025 20:57

feat: add toolchain providing pre-built protoc

4dd57da

Users can manually register this in their MODULE.bazel so that registration overrides the default, which is still building protoc from source.

alexeagle added 13 commits December 9, 2025 13:57

more output to diagnose

4199082

turn off validations for toolchain test

622e085

add another bazel build test under new example folder

749972e

more scary warning message

1596edb

fixup! add another bazel build test under new example folder

2f1bdd0

fixup! add another bazel build test under new example folder

75923a2

fixup! add another bazel build test under new example folder

255a464

fixup! add another bazel build test under new example folder

3f0f307

fixup! add another bazel build test under new example folder

103938e

add windows test sha

4c0237e

chore: relax validation of -dev protoc versions

8825c25

grep -e since it starts with hypen

bae2434

add another opt-out flag

3d734ea

alexeagle force-pushed the alexeagle/use_prebuilt_toolchain branch from ab033ab to 3d734ea Compare December 9, 2025 21:58

mkruskal-google requested a review from haberman December 11, 2025 21:39

mkruskal-google reviewed Dec 11, 2025

View reviewed changes

tmp Outdated Show resolved Hide resolved

fix: delete extra file

b18667d

mkruskal-google approved these changes Dec 12, 2025

View reviewed changes

copybara-service bot mentioned this pull request Dec 12, 2025

feat(bazel): wire up prebuilt protoc toolchain (#24115) #24819

Closed

meteorcloudy reviewed Dec 15, 2025

View reviewed changes

examples/example_without_cc_toolchain/BUILD.bazel Show resolved Hide resolved

copybara-service bot mentioned this pull request Dec 15, 2025

feat(bazel): wire up prebuilt protoc toolchain (#24115) #24888

Merged

copybara-service bot closed this in cc23698 Dec 15, 2025

thesayyn mentioned this pull request Jan 5, 2026

feat: introduce version constraints for proto toolchain #25163

Open

feat(bazel): wire up prebuilt protoc toolchain #24115

feat(bazel): wire up prebuilt protoc toolchain #24115

Uh oh!

Conversation

alexeagle commented Oct 22, 2025 • edited by haberman Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mkruskal-google commented Nov 15, 2025

Uh oh!

alexeagle commented Nov 17, 2025

Uh oh!

zhangskz commented Nov 17, 2025

Uh oh!

Uh oh!

zhangskz Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

alexeagle Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

zhangskz Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

alexeagle Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

mkruskal-google commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexeagle commented Nov 20, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

alexeagle commented Oct 22, 2025 •

edited by haberman

Loading

mkruskal-google commented Nov 18, 2025 •

edited

Loading