Implement incompatible_load_externally #23016
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Issue: bazelbuild#22928 The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel. Cycles are prevented with a list of repositories where autoloads should not be used. The new environments are injected into BzlCompile, BzlLoad and Package function. StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.
comius
added a commit
to comius/bazel
that referenced
this pull request
Oct 9, 2024
Issue: bazelbuild#22928 Incompatible flag issue: bazelbuild#23043 The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel. Cycles are prevented with a list of repositories where autoloads should not be used. The new environments are injected into BzlCompile, BzlLoad and Package function. StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols. Closes bazelbuild#23016. Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003 PiperOrigin-RevId: 666710259 Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius
added a commit
to comius/bazel
that referenced
this pull request
Oct 9, 2024
Issue: bazelbuild#22928 Incompatible flag issue: bazelbuild#23043 The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel. Cycles are prevented with a list of repositories where autoloads should not be used. The new environments are injected into BzlCompile, BzlLoad and Package function. StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols. Closes bazelbuild#23016. Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003 PiperOrigin-RevId: 666710259 Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius
added a commit
to comius/bazel
that referenced
this pull request
Oct 9, 2024
Issue: bazelbuild#22928 Incompatible flag issue: bazelbuild#23043 The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel. Cycles are prevented with a list of repositories where autoloads should not be used. The new environments are injected into BzlCompile, BzlLoad and Package function. StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols. Closes bazelbuild#23016. Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003 PiperOrigin-RevId: 666710259 Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
comius
added a commit
to comius/bazel
that referenced
this pull request
Oct 9, 2024
Issue: bazelbuild#22928 Incompatible flag issue: bazelbuild#23043 The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel. Cycles are prevented with a list of repositories where autoloads should not be used. The new environments are injected into BzlCompile, BzlLoad and Package function. StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols. Closes bazelbuild#23016. Downstream test: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4003 PiperOrigin-RevId: 666710259 Change-Id: Idaa9b6b13c9a474f700e69e22b6162ed59b7fab0
mbland
added a commit
to EngFlow-Academy/bzlmod-bootcamp
that referenced
this pull request
Nov 3, 2025
Introduces mechanisms to ensure that `rules_magic` remains compatible with a range of dependency versions. Based on the advice from: - https://blog.engflow.com/2025/07/08/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-1/ - https://blog.engflow.com/2025/09/03/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-3/ All builds pass _except_ the Bazel 8 legacy `WORKSPACE` build, which we'll fix in the next commit. See the end of this commit message for details. Changes include: - Updates `rules_magic/MODULE.bazel` to define the lowest supported dependency versions in each `bazel_dep`, and the highest versions in `single_version_override`. This ensures that builds in `rules_magic` use the latest dependency versions, but only requires users specify the minimum or later versions of other dependencies. In other words, upgrading to a new version of `rules_magic` shouldn't require upgrades to other dependencies, or staying on older versions. Note that the shared Bazel worker proto definition from `@bazel_worker_api` is a completely internal detail that shouldn't really affect users. For that reason, it doesn't have a corresponding `single_version_override`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/MODULE.bazel#L30-L67 - Updates `rules_magic/MODULE.bazel` and `bootcamp/MODULE.bazel` to use the latest available `rules_cc` and `rules_python` modules as dev dependencies. These versions only apply to a module when built as the main module and do not convey to other modules. - Adds `latest_dependencies/MODULE.bazel` to define a module with the most recent versions of `rules_magic` module dependencies. It only includes `protobuf` and `rules_java`, which is enough for demonstration purposes. This module does not have `.bazelrc`, `.bazelversion`, `BUILD`, `MODULE.bazel.lock`, or `tools/bazel` files because it's only providing dependency versions for tests. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/deps/latest/MODULE.bazel - Updates `bootcamp/MODULE.bazel` to depend upon the `latest_dependencies` module, ensuring that tests will run against the latest dependency versions. (Except for the minimum dependency versions compatibility test, coming in the next commit.) Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/examples/crossbuild/MODULE.bazel#L11-L15 - Includes the earliest dependency versions in `rules_magic/magic/deps.bzl` that are compatible with legacy `WORKSPACE` builds. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/deps.bzl - Adds `rules_magic/magic/latest_deps.bzl` to instantiate the latest dependency versions that are compatible with legacy `WORKSPACE` builds. Includes `bazel_skylib` and `bazel_features`, which aren't included by `latest_dependencies/MODULE.bazel`, for the reasons documented in the comments. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/latest_deps.bzl - Adds `rules_magic/magic/private/dev_deps.bzl` to instantiate dependencies only necessary for `rules_magic` when developing in legacy `WORKSPACE` mode. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/extensions/dev_deps.bzl - Extracts `rules_magic/magic/private/bazel_worker_api.bzl` to share `@bazel_worker_api` instantiation between `deps.bzl` and `latest_deps.bzl`. This shared Bazel worker proto definition is a completely internal detail that can stay the same in both `deps.bzl` and `latest_deps.bzl`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/macros/workspace_compat.bzl - Updates `bootcamp/WORKSPACE` and `rules_magic/WORKSPACE` to use `latest_deps.bzl`. Adds a commented-out block to `bootcamp/WORKSPACE` to switch to `deps.bzl` if desired. Updates `rules_magic/WORKSPACE` to include `dev_deps.bzl` and invoke `rules_magic_dev_deps` before `rules_magic_deps`. Analogous to (except for the dev deps first part): https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/WORKSPACE#L4 --- The Bazel 8 legacy `WORKSPACE` build fails with: ```txt ERROR: Cycle caused by autoloads, failed to load .bzl file '@@cc_compatibility_proxy//:symbols.bzl'. Add 'cc_compatibility_proxy' to --repositories_without_autoloads or disable autoloads by setting '--incompatible_autoload_externally=' More information on bazelbuild/bazel#23043. ERROR: cycles detected during target parsing ``` This error comes from: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/skyframe/BzlmodRepoCycleReporter.java#L184-L199 This is because this change updates `rules_cc` from 0.2.11 to 0.2.13, and 0.2.12 introduced `@cc_compatibility_proxy//:symbols.bzl`. Loading Bazel native symbols from this file causes Bazel 8 to autoload them from `@rules_cc`. This leads to a cycle while autoloading `@rules_cc`, something like: - A legacy `WORKSPACE` file invokes `compatibility_proxy_repo()`, which generates `@cc_compatibility_proxy//:symbols.bzl`. On Bazels before 9.0.0-pre.20250911, this file loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. Namely: `cc_common`, `CcInfo`, `DebugPackageInfo`, `CcToolchainConfigInfo`. - bazelbuild/rules_cc: Enable C++ symbols using compatiblity proxy bazelbuild/rules_cc@b81a4f4 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/extensions.bzl#L135-L150 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/private/rules_impl/native.bzl - Some `.bzl` file loads a symbol from a `@rules_cc` file, which in turn loads from `@cc_compatibility_proxy//:symbols.bzl`. - `@cc_compatibility_proxy//:symbols.bzl` loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. All of these imported native symbols are listed in `AutoloadSymbols.java` from: - bazelbuild/bazel#23016: Implement incompatible_autoload_externally bazelbuild/bazel@f9ed0b0 bazelbuild/bazel#22928 bazelbuild/bazel#23043 - These autoloaded symbols redirect back to `@rules_cc`, which then tries to load `@cc_compatibility_proxy`, and goes boom. It appears that a future Bazel 8 release should add `cc_compatibility_proxy` to `PREDECLARED_REPOS_DISALLOWING_AUTOLOADS`: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/packages/AutoloadSymbols.java#L679-L698
mbland
added a commit
to EngFlow-Academy/bzlmod-bootcamp
that referenced
this pull request
Nov 4, 2025
Introduces mechanisms to ensure that `rules_magic` remains compatible with a range of dependency versions. Based on the advice from: - https://blog.engflow.com/2025/07/08/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-1/ - https://blog.engflow.com/2025/09/03/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-3/ All builds pass _except_ the Bazel 8 legacy `WORKSPACE` build, which we'll fix in the next commit. See the end of this commit message for details. Changes include: - Updates `rules_magic/MODULE.bazel` to define the lowest supported dependency versions in each `bazel_dep`, and the highest versions in `single_version_override`. This ensures that builds in `rules_magic` use the latest dependency versions, but only requires users specify the minimum or later versions of other dependencies. In other words, upgrading to a new version of `rules_magic` shouldn't require upgrades to other dependencies, or staying on older versions. Note that the shared Bazel worker proto definition from `@bazel_worker_api` is a completely internal detail that shouldn't really affect users. For that reason, it doesn't have a corresponding `single_version_override`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/MODULE.bazel#L30-L67 - Updates `rules_magic/MODULE.bazel` and `bootcamp/MODULE.bazel` to use the latest available `rules_cc` and `rules_python` modules as dev dependencies. These versions only apply to a module when built as the main module and do not convey to other modules. - Adds `latest_dependencies/MODULE.bazel` to define a module with the most recent versions of `rules_magic` module dependencies. It only includes `protobuf` and `rules_java`, which is enough for demonstration purposes. This module does not have `.bazelrc`, `.bazelversion`, `BUILD`, `MODULE.bazel.lock`, or `tools/bazel` files because it's only providing dependency versions for tests. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/deps/latest/MODULE.bazel - Updates `bootcamp/MODULE.bazel` to depend upon the `latest_dependencies` module, ensuring that tests will run against the latest dependency versions. (Except for the minimum dependency versions compatibility test, coming in the next commit.) Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/examples/crossbuild/MODULE.bazel#L11-L15 - Includes the earliest dependency versions in `rules_magic/magic/deps.bzl` that are compatible with legacy `WORKSPACE` builds. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/deps.bzl - Adds `rules_magic/magic/latest_deps.bzl` to instantiate the latest dependency versions that are compatible with legacy `WORKSPACE` builds. Includes `bazel_skylib` and `bazel_features`, which aren't included by `latest_dependencies/MODULE.bazel`, for the reasons documented in the comments. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/latest_deps.bzl - Adds `rules_magic/magic/private/dev_deps.bzl` to instantiate dependencies only necessary for `rules_magic` when developing in legacy `WORKSPACE` mode. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/extensions/dev_deps.bzl - Extracts `rules_magic/magic/private/bazel_worker_api.bzl` to share `@bazel_worker_api` instantiation between `deps.bzl` and `latest_deps.bzl`. This shared Bazel worker proto definition is a completely internal detail that can stay the same in both `deps.bzl` and `latest_deps.bzl`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/macros/workspace_compat.bzl - Updates `bootcamp/WORKSPACE` and `rules_magic/WORKSPACE` to use `latest_deps.bzl`. Adds a commented-out block to `bootcamp/WORKSPACE` to switch to `deps.bzl` if desired. Updates `rules_magic/WORKSPACE` to include `dev_deps.bzl` and invoke `rules_magic_dev_deps` before `rules_magic_deps`. Analogous to (except for the dev deps first part): https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/WORKSPACE#L4 --- The Bazel 8 legacy `WORKSPACE` build fails with: ```txt ERROR: Cycle caused by autoloads, failed to load .bzl file '@@cc_compatibility_proxy//:symbols.bzl'. Add 'cc_compatibility_proxy' to --repositories_without_autoloads or disable autoloads by setting '--incompatible_autoload_externally=' More information on bazelbuild/bazel#23043. ERROR: cycles detected during target parsing ``` This error comes from: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/skyframe/BzlmodRepoCycleReporter.java#L184-L199 This is because this change updates `rules_cc` from 0.2.11 to 0.2.13, and 0.2.12 introduced `@cc_compatibility_proxy//:symbols.bzl`. Loading Bazel native symbols from this file causes Bazel 8 to autoload them from `@rules_cc`. This leads to a cycle while autoloading `@rules_cc`, something like: - A legacy `WORKSPACE` file invokes `compatibility_proxy_repo()`, which generates `@cc_compatibility_proxy//:symbols.bzl`. On Bazels before 9.0.0-pre.20250911, this file loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. Namely: `cc_common`, `CcInfo`, `DebugPackageInfo`, `CcToolchainConfigInfo`. - bazelbuild/rules_cc: Enable C++ symbols using compatiblity proxy bazelbuild/rules_cc@b81a4f4 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/extensions.bzl#L135-L150 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/private/rules_impl/native.bzl - Some `.bzl` file loads a symbol from a `@rules_cc` file, which in turn loads from `@cc_compatibility_proxy//:symbols.bzl`. - `@cc_compatibility_proxy//:symbols.bzl` loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. All of these imported native symbols are listed in `AutoloadSymbols.java` from: - bazelbuild/bazel#23016: Implement incompatible_autoload_externally bazelbuild/bazel@f9ed0b0 bazelbuild/bazel#22928 bazelbuild/bazel#23043 - These autoloaded symbols redirect back to `@rules_cc`, which then tries to load `@cc_compatibility_proxy`, and goes boom. It appears that a future Bazel 8 release should add `cc_compatibility_proxy` to `PREDECLARED_REPOS_DISALLOWING_AUTOLOADS`: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/packages/AutoloadSymbols.java#L679-L698
mbland
added a commit
to EngFlow-Academy/bzlmod-bootcamp
that referenced
this pull request
Nov 4, 2025
Introduces mechanisms to ensure that `rules_magic` remains compatible with a range of dependency versions. Based on the advice from: - https://blog.engflow.com/2025/07/08/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-1/ - https://blog.engflow.com/2025/09/03/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-3/ All builds pass _except_ the Bazel 8 legacy `WORKSPACE` build, which we'll fix in the next commit. See the end of this commit message for details. Changes include: - Updates `rules_magic/MODULE.bazel` to define the lowest supported dependency versions in each `bazel_dep`, and the highest versions in `single_version_override`. This ensures that builds in `rules_magic` use the latest dependency versions, but only requires users specify the minimum or later versions of other dependencies. In other words, upgrading to a new version of `rules_magic` shouldn't require upgrades to other dependencies, or staying on older versions. Note that the shared Bazel worker proto definition from `@bazel_worker_api` is a completely internal detail that shouldn't really affect users. For that reason, it doesn't have a corresponding `single_version_override`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/MODULE.bazel#L30-L67 - Updates `rules_magic/MODULE.bazel` and `bootcamp/MODULE.bazel` to use the latest available `rules_cc` and `rules_python` modules as dev dependencies. These versions only apply to a module when built as the main module and do not convey to other modules. - Adds `latest_dependencies/MODULE.bazel` to define a module with the most recent versions of `rules_magic` module dependencies. It only includes `protobuf` and `rules_java`, which is enough for demonstration purposes. This module does not have `.bazelrc`, `.bazelversion`, `BUILD`, `MODULE.bazel.lock`, or `tools/bazel` files because it's only providing dependency versions for tests. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/deps/latest/MODULE.bazel - Updates `bootcamp/MODULE.bazel` to depend upon the `latest_dependencies` module, ensuring that tests will run against the latest dependency versions. (Except for the minimum dependency versions compatibility test, coming in the next commit.) Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/examples/crossbuild/MODULE.bazel#L11-L15 - Includes the earliest dependency versions in `rules_magic/magic/deps.bzl` that are compatible with legacy `WORKSPACE` builds. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/deps.bzl - Adds `rules_magic/magic/latest_deps.bzl` to instantiate the latest dependency versions that are compatible with legacy `WORKSPACE` builds. Includes `bazel_skylib` and `bazel_features`, which aren't included by `latest_dependencies/MODULE.bazel`, for the reasons documented in the comments. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/latest_deps.bzl - Adds `rules_magic/magic/private/dev_deps.bzl` to instantiate dependencies only necessary for `rules_magic` when developing in legacy `WORKSPACE` mode. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/extensions/dev_deps.bzl - Extracts `rules_magic/magic/private/bazel_worker_api.bzl` to share `@bazel_worker_api` instantiation between `deps.bzl` and `latest_deps.bzl`. This shared Bazel worker proto definition is a completely internal detail that can stay the same in both `deps.bzl` and `latest_deps.bzl`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/macros/workspace_compat.bzl - Updates `bootcamp/WORKSPACE` and `rules_magic/WORKSPACE` to use `latest_deps.bzl`. Adds a commented-out block to `bootcamp/WORKSPACE` to switch to `deps.bzl` if desired. Updates `rules_magic/WORKSPACE` to include `dev_deps.bzl` and invoke `rules_magic_dev_deps` before `rules_magic_deps`. Analogous to (except for the dev deps first part): https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/WORKSPACE#L4 --- The Bazel 8 legacy `WORKSPACE` build fails with: ```txt ERROR: Cycle caused by autoloads, failed to load .bzl file '@@cc_compatibility_proxy//:symbols.bzl'. Add 'cc_compatibility_proxy' to --repositories_without_autoloads or disable autoloads by setting '--incompatible_autoload_externally=' More information on bazelbuild/bazel#23043. ERROR: cycles detected during target parsing ``` This error comes from: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/skyframe/BzlmodRepoCycleReporter.java#L184-L199 This is because this change updates `rules_cc` from 0.2.11 to 0.2.13, and 0.2.12 introduced `@cc_compatibility_proxy//:symbols.bzl`. Loading Bazel native symbols from this file causes Bazel 8 to autoload them from `@rules_cc`. This leads to a cycle while autoloading `@rules_cc`, something like: - A legacy `WORKSPACE` file invokes `compatibility_proxy_repo()`, which generates `@cc_compatibility_proxy//:symbols.bzl`. On Bazels before 9.0.0-pre.20250911, this file loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. Namely: `cc_common`, `CcInfo`, `DebugPackageInfo`, `CcToolchainConfigInfo`. - bazelbuild/rules_cc: Enable C++ symbols using compatiblity proxy bazelbuild/rules_cc@b81a4f4 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/extensions.bzl#L135-L150 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/private/rules_impl/native.bzl - Some `.bzl` file loads a symbol from a `@rules_cc` file, which in turn loads from `@cc_compatibility_proxy//:symbols.bzl`. - `@cc_compatibility_proxy//:symbols.bzl` loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. All of these imported native symbols are listed in `AutoloadSymbols.java` from: - bazelbuild/bazel#23016: Implement incompatible_autoload_externally bazelbuild/bazel@f9ed0b0 bazelbuild/bazel#22928 bazelbuild/bazel#23043 - These autoloaded symbols redirect back to `@rules_cc`, which then tries to load `@cc_compatibility_proxy`, and goes boom. It appears that a future Bazel 8 release should add `cc_compatibility_proxy` to `PREDECLARED_REPOS_DISALLOWING_AUTOLOADS`: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/packages/AutoloadSymbols.java#L679-L698
mbland
added a commit
to EngFlow-Academy/bzlmod-bootcamp
that referenced
this pull request
Nov 4, 2025
Introduces mechanisms to ensure that `rules_magic` remains compatible with a range of dependency versions. Based on the advice from: - https://blog.engflow.com/2025/07/08/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-1/ - https://blog.engflow.com/2025/09/03/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-3/ All builds pass _except_ the Bazel 8 legacy `WORKSPACE` build, which we'll fix in the next commit. See the end of this commit message for details. Changes include: - Updates `rules_magic/MODULE.bazel` to define the lowest supported dependency versions in each `bazel_dep`, and the highest versions in `single_version_override`. This ensures that builds in `rules_magic` use the latest dependency versions, but only requires users specify the minimum or later versions of other dependencies. In other words, upgrading to a new version of `rules_magic` shouldn't require upgrades to other dependencies, or staying on older versions. Note that the shared Bazel worker proto definition from `@bazel_worker_api` is a completely internal detail that shouldn't really affect users. For that reason, it doesn't have a corresponding `single_version_override`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/MODULE.bazel#L30-L67 - Updates `rules_magic/MODULE.bazel` and `bootcamp/MODULE.bazel` to use the latest available `rules_cc` and `rules_python` modules as dev dependencies. These versions only apply to a module when built as the main module and do not convey to other modules. - Adds `latest_dependencies/MODULE.bazel` to define a module with the most recent versions of `rules_magic` module dependencies. It only includes `protobuf` and `rules_java`, which is enough for demonstration purposes. This module does not have `.bazelrc`, `.bazelversion`, `BUILD`, `MODULE.bazel.lock`, or `tools/bazel` files because it's only providing dependency versions for tests. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/deps/latest/MODULE.bazel - Updates `bootcamp/MODULE.bazel` to depend upon the `latest_dependencies` module, ensuring that tests will run against the latest dependency versions. (Except for the minimum dependency versions compatibility test, coming in the next commit.) Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/examples/crossbuild/MODULE.bazel#L11-L15 - Includes the earliest dependency versions in `rules_magic/magic/deps.bzl` that are compatible with legacy `WORKSPACE` builds. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/deps.bzl - Adds `rules_magic/magic/latest_deps.bzl` to instantiate the latest dependency versions that are compatible with legacy `WORKSPACE` builds. Includes `bazel_skylib` and `bazel_features`, which aren't included by `latest_dependencies/MODULE.bazel`, for the reasons documented in the comments. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/latest_deps.bzl - Adds `rules_magic/magic/private/dev_deps.bzl` to instantiate dependencies only necessary for `rules_magic` when developing in legacy `WORKSPACE` mode. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/extensions/dev_deps.bzl - Extracts `rules_magic/magic/private/bazel_worker_api.bzl` to share `@bazel_worker_api` instantiation between `deps.bzl` and `latest_deps.bzl`. This shared Bazel worker proto definition is a completely internal detail that can stay the same in both `deps.bzl` and `latest_deps.bzl`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/macros/workspace_compat.bzl - Updates `bootcamp/WORKSPACE` and `rules_magic/WORKSPACE` to use `latest_deps.bzl`. Adds a commented-out block to `bootcamp/WORKSPACE` to switch to `deps.bzl` if desired. Updates `rules_magic/WORKSPACE` to include `dev_deps.bzl` and invoke `rules_magic_dev_deps` before `rules_magic_deps`. Analogous to (except for the dev deps first part): https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/WORKSPACE#L4 --- The Bazel 8 legacy `WORKSPACE` build fails with: ```txt ERROR: Cycle caused by autoloads, failed to load .bzl file '@@cc_compatibility_proxy//:symbols.bzl'. Add 'cc_compatibility_proxy' to --repositories_without_autoloads or disable autoloads by setting '--incompatible_autoload_externally=' More information on bazelbuild/bazel#23043. ERROR: cycles detected during target parsing ``` This error comes from: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/skyframe/BzlmodRepoCycleReporter.java#L184-L199 This is because this change updates `rules_cc` from 0.2.11 to 0.2.13, and 0.2.12 introduced `@cc_compatibility_proxy//:symbols.bzl`. Loading Bazel native symbols from this file causes Bazel 8 to autoload them from `@rules_cc`. This leads to a cycle while autoloading `@rules_cc`, something like: - A legacy `WORKSPACE` file invokes `compatibility_proxy_repo()`, which generates `@cc_compatibility_proxy//:symbols.bzl`. On Bazels before 9.0.0-pre.20250911, this file loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. Namely: `cc_common`, `CcInfo`, `DebugPackageInfo`, `CcToolchainConfigInfo`. - bazelbuild/rules_cc: Enable C++ symbols using compatiblity proxy bazelbuild/rules_cc@b81a4f4 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/extensions.bzl#L135-L150 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/private/rules_impl/native.bzl - Some `.bzl` file loads a symbol from a `@rules_cc` file, which in turn loads from `@cc_compatibility_proxy//:symbols.bzl`. - `@cc_compatibility_proxy//:symbols.bzl` loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. All of these imported native symbols are listed in `AutoloadSymbols.java` from: - bazelbuild/bazel#23016: Implement incompatible_autoload_externally bazelbuild/bazel@f9ed0b0 bazelbuild/bazel#22928 bazelbuild/bazel#23043 - These autoloaded symbols redirect back to `@rules_cc`, which then tries to load `@cc_compatibility_proxy`, and goes boom. It appears that a future Bazel 8 release should add `cc_compatibility_proxy` to `PREDECLARED_REPOS_DISALLOWING_AUTOLOADS`: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/packages/AutoloadSymbols.java#L679-L698
mbland
added a commit
to EngFlow-Academy/bzlmod-bootcamp
that referenced
this pull request
Nov 4, 2025
Introduces mechanisms to ensure that `rules_magic` remains compatible with a range of dependency versions. Based on the advice from: - https://blog.engflow.com/2025/07/08/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-1/ - https://blog.engflow.com/2025/09/03/migrating-to-bazel-modules-aka-bzlmod---maintaining-compatibility-part-3/ All builds pass _except_ the Bazel 8 legacy `WORKSPACE` build, which we'll fix in the next commit. See the end of this commit message for details. Changes include: - Updates `rules_magic/MODULE.bazel` to define the lowest supported dependency versions in each `bazel_dep`, and the highest versions in `single_version_override`. This ensures that builds in `rules_magic` use the latest dependency versions, but only requires users specify the minimum or later versions of other dependencies. In other words, upgrading to a new version of `rules_magic` shouldn't require upgrades to other dependencies, or staying on older versions. Note that the shared Bazel worker proto definition from `@bazel_worker_api` is a completely internal detail that shouldn't really affect users. For that reason, it doesn't have a corresponding `single_version_override`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/MODULE.bazel#L30-L67 - Updates `rules_magic/MODULE.bazel` and `bootcamp/MODULE.bazel` to use the latest available `rules_cc` and `rules_python` modules as dev dependencies. These versions only apply to a module when built as the main module and do not convey to other modules. - Adds `latest_dependencies/MODULE.bazel` to define a module with the most recent versions of `rules_magic` module dependencies. It only includes `protobuf` and `rules_java`, which is enough for demonstration purposes. This module does not have `.bazelrc`, `.bazelversion`, `BUILD`, `MODULE.bazel.lock`, or `tools/bazel` files because it's only providing dependency versions for tests. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/deps/latest/MODULE.bazel - Updates `bootcamp/MODULE.bazel` to depend upon the `latest_dependencies` module, ensuring that tests will run against the latest dependency versions. (Except for the minimum dependency versions compatibility test, coming in the next commit.) Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/examples/crossbuild/MODULE.bazel#L11-L15 - Includes the earliest dependency versions in `rules_magic/magic/deps.bzl` that are compatible with legacy `WORKSPACE` builds. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/deps.bzl - Adds `rules_magic/magic/latest_deps.bzl` to instantiate the latest dependency versions that are compatible with legacy `WORKSPACE` builds. Includes `bazel_skylib` and `bazel_features`, which aren't included by `latest_dependencies/MODULE.bazel`, for the reasons documented in the comments. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/latest_deps.bzl - Adds `rules_magic/magic/private/dev_deps.bzl` to instantiate dependencies only necessary for `rules_magic` when developing in legacy `WORKSPACE` mode. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/extensions/dev_deps.bzl - Extracts `rules_magic/magic/private/bazel_worker_api.bzl` to share `@bazel_worker_api` instantiation between `deps.bzl` and `latest_deps.bzl`. This shared Bazel worker proto definition is a completely internal detail that can stay the same in both `deps.bzl` and `latest_deps.bzl`. Analogous to: https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/scala/private/macros/workspace_compat.bzl - Updates `bootcamp/WORKSPACE` and `rules_magic/WORKSPACE` to use `latest_deps.bzl`. Adds a commented-out block to `bootcamp/WORKSPACE` to switch to `deps.bzl` if desired. Updates `rules_magic/WORKSPACE` to include `dev_deps.bzl` and invoke `rules_magic_dev_deps` before `rules_magic_deps`. Analogous to (except for the dev deps first part): https://github.com/bazel-contrib/rules_scala/blob/v7.1.2/WORKSPACE#L4 --- The Bazel 8 legacy `WORKSPACE` build fails with: ```txt ERROR: Cycle caused by autoloads, failed to load .bzl file '@@cc_compatibility_proxy//:symbols.bzl'. Add 'cc_compatibility_proxy' to --repositories_without_autoloads or disable autoloads by setting '--incompatible_autoload_externally=' More information on bazelbuild/bazel#23043. ERROR: cycles detected during target parsing ``` This error comes from: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/skyframe/BzlmodRepoCycleReporter.java#L184-L199 This is because this change updates `rules_cc` from 0.2.11 to 0.2.13, and 0.2.12 introduced `@cc_compatibility_proxy//:symbols.bzl`. Loading Bazel native symbols from this file causes Bazel 8 to autoload them from `@rules_cc`. This leads to a cycle while autoloading `@rules_cc`, something like: - A legacy `WORKSPACE` file invokes `compatibility_proxy_repo()`, which generates `@cc_compatibility_proxy//:symbols.bzl`. On Bazels before 9.0.0-pre.20250911, this file loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. Namely: `cc_common`, `CcInfo`, `DebugPackageInfo`, `CcToolchainConfigInfo`. - bazelbuild/rules_cc: Enable C++ symbols using compatiblity proxy bazelbuild/rules_cc@b81a4f4 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/extensions.bzl#L135-L150 - https://github.com/bazelbuild/rules_cc/blob/0.2.13/cc/private/rules_impl/native.bzl - Some `.bzl` file loads a symbol from a `@rules_cc` file, which in turn loads from `@cc_compatibility_proxy//:symbols.bzl`. - `@cc_compatibility_proxy//:symbols.bzl` loads symbols from `@rules_cc//cc/private/rules_impl:native.bzl`. All of these imported native symbols are listed in `AutoloadSymbols.java` from: - bazelbuild/bazel#23016: Implement incompatible_autoload_externally bazelbuild/bazel@f9ed0b0 bazelbuild/bazel#22928 bazelbuild/bazel#23043 - These autoloaded symbols redirect back to `@rules_cc`, which then tries to load `@cc_compatibility_proxy`, and goes boom. It appears that a future Bazel 8 release should add `cc_compatibility_proxy` to `PREDECLARED_REPOS_DISALLOWING_AUTOLOADS`: - https://github.com/bazelbuild/bazel/blob/8.4.2/src/main/java/com/google/devtools/build/lib/packages/AutoloadSymbols.java#L679-L698
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue: #22928
The flag supports Bazel users in migrating the rules from being embedded in Bazel to external repositories. Listing a symbol or a rule in the flag automatically adds a load to the respective repository. Listing it with a '+' keeps the symbol available in the rules_repository. Listing a symbol with "-" forcefully removes it from Bazel.
Cycles are prevented with a list of repositories where autoloads should not be used.
The new environments are injected into BzlCompile, BzlLoad and Package function.
StarlarkBuiltinsFunction with autoloads true key is used to load the external symbols.