-
Notifications
You must be signed in to change notification settings - Fork 580
Upgrade Bazel from 6.5.0 to 8.5.0 #9801
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
536e6cc to
4b272ca
Compare
amomchilov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed everything here, LGTM except for the version bumps. Could we do this in a separate PR?
| # This statement defines the @com_google_protobuf repo. | ||
| http_archive( | ||
| name = "com_google_protobuf", | ||
| url = "https://github.com/protocolbuffers/protobuf/archive/v3.27.0.zip", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This got bumped up to 29.3:
Line 13 in 46000ba
| bazel_dep(name = "protobuf", version = "29.3", repo_name = "com_google_protobuf") |
| http_archive( | ||
| name = "bazel_skylib", | ||
| sha256 = "cd55a062e763b9349921f0f5db8c3933288dc8ba4f76dd9416aac68acee3cb94", | ||
| url = "https://github.com/bazelbuild/bazel-skylib/releases/download/1.5.0/bazel-skylib-1.5.0.tar.gz", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This got bumped up to 1.9.0:
Line 8 in 46000ba
| bazel_dep(name = "bazel_skylib", version = "1.9.0") |
third_party/externals.bzl
Outdated
| build_file = "@com_stripe_ruby_typer//third_party:crcpp.BUILD", | ||
| strip_prefix = "CRCpp-51fbc35ef892e98abe91a51f7320749c929d72bd", | ||
| name = "rules_foreign_cc", | ||
| url = "https://github.com/bazelbuild/rules_foreign_cc/archive/d74623f0ad47f4e375de81baa454eb106715a416.zip", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note to other reviewers, rules_foreign_cc was already here, just got moved within the file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This stuff mostly just got moved to WORKSPACE.bzlmod:
git diff master:WORKSPACE WORKSPACE.bzlmodgives:
diff --git a/WORKSPACE b/WORKSPACE.bzlmod
index ba7ec32f7..fd8c9505a 100644
--- a/WORKSPACE
+++ b/WORKSPACE.bzlmod
@@ -5,6 +5,12 @@ load("//third_party:externals.bzl", "register_sorbet_dependencies")
register_sorbet_dependencies()
+load("@bazel_features//:deps.bzl", "bazel_features_deps")
+bazel_features_deps()
+
+load("@rules_cc//cc:extensions.bzl", "compatibility_proxy_repo")
+compatibility_proxy_repo()
+
load("@rules_foreign_cc//foreign_cc:repositories.bzl", "rules_foreign_cc_dependencies")
# We need to explicitly pull in make here for rules_foreign_cc
@@ -42,7 +48,6 @@ llvm_toolchain(
"https://github.com/sorbet/llvm-project/releases/download/llvmorg-{llvm_version}/{basename}",
],
llvm_version = "15.0.7",
- # The sysroots are needed for cross-compiling
sysroot = {
"": "",
"darwin-x86_64": "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk",
@@ -88,14 +93,3 @@ bison_register_toolchains(
extra_copts = ["-Wno-implicit-const-int-float-conversion"],
)
-load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
-
-protobuf_deps()
-
-load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
-
-bazel_skylib_workspace()
-
-load("@aspect_bazel_lib//lib:repositories.bzl", "aspect_bazel_lib_dependencies")
-
-aspect_bazel_lib_dependencies()
| module_name = "rapidjson", | ||
| urls = ["https://github.com/Tencent/rapidjson/archive/f376690822cbc2d17044e626be5df21f7d91ca8f.zip"], | ||
| strip_prefix = "rapidjson-f376690822cbc2d17044e626be5df21f7d91ca8f", | ||
| integrity = "sha256-lCUnZYPf+QIM7mMyRysM8keuMly18m2+FXGD90faORA=", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Verified these hashes are the same as tomorrow, by rewriting them from this Base64 style back to hex, with this little script:
#!/usr/bin/env ruby
require 'base64'
# Read the MODULE.bazel file
file_path = 'MODULE.bazel'
content = File.read(file_path)
# Replace all integrity parameters from base64 to hex format
# Pattern matches: integrity = "sha256-<base64>",
modified_content = content.gsub(/integrity\s*=\s*"sha256-([A-Za-z0-9+\/=]+)"/) do
base64_hash = $1
# Decode base64 to binary, then convert to hex
binary_hash = Base64.decode64(base64_hash)
hex_hash = binary_hash.unpack1('H*')
# Return the new format
%Q{sha256 = "#{hex_hash}"}
end
# Write the modified content back to the file
File.write(file_path, modified_content)
puts "Successfully converted #{content.scan(/integrity\s*=/).length} integrity parameters to sha256 format"Answers to reviewer questions from PR sorbet#9801 covering: - lmdb directory structure - rapidjson linkstatic simplification - RBS Parser upstream contribution - MODULE.bazel location requirements - protobuf compiler flags - progressbar.BUILD removal - BCR dependencies origin
- Update .bazelversion to 8.5.0 - Add --enable_workspace flag (disabled by default in Bazel 8+) - Add --enable_bzlmod=false (project uses WORKSPACE for dependencies) - Update installer checksums for all platforms
This is a work-in-progress upgrade to Bazel 8.5.0. The build does not yet work due to toolchain compatibility issues. Changes made: - .bazelrc: Enable WORKSPACE mode, disable Bzlmod, allow empty globs - Add bazel_features 1.38.0 (required for Bazel 8) - Add rules_java 8.9.0 (required for Bazel 8) - Add rules_python 1.0.0 (required for Bazel 8) - Update rules_cc from commit to 0.2.14 - Update protobuf from 3.27.0 to 29.3 - Update bazel_skylib from 1.5.0 to 1.9.0 - Add bazel_features_deps() and compatibility_proxy_repo() calls Remaining work: - The sorbet/bazel-toolchain fork needs to be updated to work with rules_cc 0.2.14. The new rules_cc uses a different archiver action format that is incompatible with the current fork. - To fix: rebase sorbet/bazel-toolchain onto a newer upstream version of bazel-contrib/toolchains_llvm (e.g., v1.5.0) that has Bazel 8 support, then update the SHA in third_party/externals.bzl. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Additional fixes needed for the Bazel 8 upgrade: - Add --host_features=-libtool for exec configuration (required because sorbet fork uses llvm-ar with traditional ar-style arguments, not the libtool-darwin format that rules_cc 0.2.14 defaults to on macOS) - Add -Wno-incomplete-umbrella to suppress LLVM modulemap warnings - Add per_file_copt to suppress missing-field-initializers warnings in protobuf 29.3 (incompatible with our -Werror flag) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Migrate core dependencies to Bazel Central Registry (BCR) via MODULE.bazel while keeping project-specific dependencies in WORKSPACE. This is a stepping stone toward full Bzlmod migration. Dependencies moved to BCR (MODULE.bazel): - [email protected] - [email protected] - [email protected] - [email protected] (with repo_name com_google_protobuf) - [email protected] (with repo_name com_google_absl) - zlib (pulled in as protobuf transitive dependency) Changes: - MODULE.bazel: New file with BCR dependencies - .bazelrc: Enable hybrid mode (--enable_workspace + --enable_bzlmod), add warning suppressions for BCR repos (protobuf+, zlib+) - WORKSPACE: Remove protobuf_deps() call (now handled by BCR) - externals.bzl: Remove dependencies that now come from BCR 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Continue migrating dependencies from WORKSPACE to Bazel Central Registry via MODULE.bazel. This reduces the WORKSPACE footprint and prepares for eventual full Bzlmod migration. Additional dependencies moved to BCR: - [email protected] - [email protected] - [email protected] - [email protected] (as io_bazel_rules_go) - [email protected] Dependencies remaining in WORKSPACE (not on BCR or have custom BUILD files): - toolchains_llvm (sorbet fork) - rules_foreign_cc (using specific commit) - emsdk (version mismatch - we use 3.1.59, BCR has 4.x) - Various C++ libraries with custom BUILD files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Continue migrating dependencies from WORKSPACE to Bazel Central Registry via MODULE.bazel. rules_cc must remain in WORKSPACE because the sorbet toolchains_llvm fork depends on it being loaded before compatibility_proxy_repo(). Additional dependencies moved to BCR: - [email protected] - [email protected] - [email protected] (as io_bazel_rules_go) - [email protected] - [email protected] - [email protected] - [email protected] Dependencies remaining in WORKSPACE: - rules_cc (required by toolchains_llvm fork before compatibility_proxy_repo) - toolchains_llvm (sorbet fork) - rules_foreign_cc, rules_ragel (not on BCR or using specific commits) - emsdk (version mismatch - we use 3.1.59, BCR has 4.x) - Various C++ libraries with custom BUILD files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
The hybrid WORKSPACE + Bzlmod approach had too many compatibility issues with module extensions (compatibility_proxy, proto_bazel_features, etc.). Each BCR dependency requires its own set of module extensions to be configured, leading to cascading complexity. Reverting to pure WORKSPACE mode which works reliably with Bazel 8: - Remove MODULE.bazel (not needed with --noenable_bzlmod) - Restore all dependencies to WORKSPACE/externals.bzl - Clean up BCR-related warning suppressions from .bazelrc The WORKSPACE mode will be deprecated in Bazel 9, but gives time to: 1. Wait for better hybrid mode support in Bazel 2. Fully migrate all deps to BCR when ready 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Migrate core dependencies to Bazel Central Registry (BCR) via MODULE.bazel while keeping project-specific dependencies in WORKSPACE. This is a stepping stone toward full Bzlmod migration. Dependencies moved to BCR (MODULE.bazel): - [email protected] - [email protected] - [email protected] (with repo_name com_google_protobuf) - [email protected] (with repo_name com_google_absl) - [email protected] - [email protected] - [email protected] (with repo_name io_bazel_rules_go) - [email protected] - [email protected] Dependencies remaining in WORKSPACE: - [email protected] (needed before BCR deps load) - [email protected] (required by toolchains_llvm fork) - toolchains_llvm (sorbet fork) - rules_foreign_cc, rules_ragel, rules_m4, rules_bison - All C++ libraries with custom BUILD files Key configuration: - .bazelrc: --enable_workspace --enable_bzlmod for hybrid mode - Warning suppressions for BCR protobuf+ and zlib+ repos - compatibility_proxy_repo() called from WORKSPACE rules_cc 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Move doctest from WORKSPACE to MODULE.bazel (BCR). Dependencies that could NOT be migrated to BCR due to rules_cc conflicts: - rules_m4, rules_bison, rules_foreign_cc all transitively depend on rules_cc from BCR, which conflicts with our WORKSPACE rules_cc (needed for sorbet toolchains_llvm fork) Also removed duplicate bazel_skylib and aspect_bazel_lib from externals.bzl (they are already in MODULE.bazel). Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Bazel 8 defaults to targeting macOS 10.11.0, but the LLVM toolchain libraries were built for macOS 12.6.0. This mismatch causes ld64.lld warnings. Setting --macos_minimum_os=13.0 silences these warnings. Also improved the comment for --noincompatible_disallow_empty_glob to document why it's needed (glob patterns like *.cc in ast/treemap).
- Add lz4 1.10.0.bcr.1 to MODULE.bazel - Remove lz4 from externals.bzl and delete custom BUILD file - Add warning suppression for lz4+ implicit-fallthrough
- Add yaml-cpp 0.8.0 and xxhash 0.8.3.bcr.1 to MODULE.bazel - Remove from externals.bzl and delete custom BUILD files - Update main/options/BUILD to use @yaml-cpp
- Add mimalloc 2.2.4 to MODULE.bazel - Remove mimalloc from externals.bzl and delete custom BUILD file - BCR version uses cmake internally (same as our custom BUILD)
- Add spdlog 1.16.0.bcr.2 to MODULE.bazel - Remove spdlog from externals.bzl and delete custom BUILD file - Using header-only mode (BCR default) instead of SPDLOG_COMPILED_LIB
- Add libprotobuf-mutator 1.3 to MODULE.bazel - Remove libprotobuf-mutator from externals.bzl and delete custom BUILD file
These files are not referenced in externals.bzl
- Add rapidjson via bazel_dep + archive_override with custom BUILD file - Create third_party/rapidjson/ directory with module.patch and BUILD - Simplify linkstatic to True (removes @com_stripe_ruby_typer reference) - Remove rapidjson from externals.bzl
- Add lmdb via bazel_dep + archive_override with custom BUILD file - Create third_party/lmdb_override/ for module.patch and lmdb.BUILD - Create third_party/lmdb/BUILD to export strdup.patch - Move rapidjson.BUILD into third_party/rapidjson/ - Remove lmdb from externals.bzl
Continue migrating dependencies from WORKSPACE to Bazel Central Registry via MODULE.bazel with archive_override for custom BUILD files. Dependencies migrated: - cxxopts (header-only options parsing library) - dtl (diff template library) - pdqsort (pattern-defeating quicksort) - rang (terminal colors library) - concurrentqueue (lock-free concurrent queue) All use archive_override to supply MODULE.bazel patch and custom BUILD files since they are not on BCR or need specific configurations. Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Continue migrating dependencies from WORKSPACE to Bazel Central Registry via MODULE.bazel with archive_override for custom BUILD files. Dependencies migrated: - statsd (C client for StatsD metrics) - mpack (MessagePack library, repo_name com_github_ludocode_mpack) - crcpp (CRC++ library, repo_name com_github_d_bahr_crcpp) - cpp_subprocess (C++ subprocess library) All use archive_override to supply MODULE.bazel patch and custom BUILD files since they are not on BCR or need specific configurations. Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Continue migrating dependencies from WORKSPACE to Bazel Central Registry via MODULE.bazel with archive_override for custom BUILD files. Dependencies migrated: - blake2 (BLAKE2 reference implementation, repo_name com_github_blake2_blake2) - libb2 (BLAKE2 optimized SSE version, repo_name com_github_blake2_libb2) These are hashing algorithm libraries used for file hashing. Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Continue migrating dependencies from WORKSPACE to Bazel Central Registry via MODULE.bazel with archive_override for custom BUILD files. Dependencies migrated: - prism (Ruby parser library) - jemalloc (memory allocator with complex genrule build) - rbs_parser (RBS type signature parser) Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
The lockfile pins exact versions and hashes of all Bzlmod dependencies to ensure reproducible builds across different machines and CI. Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Update BUILD files to use shorter dependency names: - @com_github_ludocode_mpack -> @mpack - @com_github_d_bahr_crcpp -> @crcpp - @com_github_blake2_libb2 -> @libb2 - @com_github_blake2_blake2 -> @BLAKE2 Also update the target names in the custom BUILD files to match. Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Rename module to 'sorbet' with repo_name for backwards compat - Mark doctest as dev_dependency (won't propagate to consumers) - Move WORKSPACE content to WORKSPACE.bzlmod (takes precedence with Bzlmod) - Keep minimal WORKSPACE file to mark workspace root Based on https://bazel.build/external/migration recommendations. Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Continue BCR migration for rule dependencies: - Add [email protected] to MODULE.bazel (was in externals.bzl) - Add [email protected] to MODULE.bazel (was in externals.bzl) - Keep rules_foreign_cc in externals.bzl (has WORKSPACE setup functions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
The linkstatic select pattern is unnecessary - the -fPIC flags in --config=shared-libs are sufficient for shared library builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Revert bazel_skylib from 1.9.0 to 1.5.0 (matches original) - Revert protobuf from 29.3 to 27.0 (matches original v3.27.0) - Remove rules_java, rules_python, rules_license (not in original WORKSPACE) Version bumps should be done in a separate PR as requested by reviewer.
Answers to reviewer questions from PR sorbet#9801 covering: - lmdb directory structure - rapidjson linkstatic simplification - RBS Parser upstream contribution - MODULE.bazel location requirements - protobuf compiler flags - progressbar.BUILD removal - BCR dependencies origin
652aa8d to
d524d26
Compare
- Add single_version_override for rules_python to 1.2.0 (fixes PyInfo/PyRuntimeInfo removal) - Update rules_foreign_cc to 0.15.0 for Bazel 8 compatibility
This upgrades Bazel to version 8.5.0 using a hybrid WORKSPACE + Bzlmod
approach. Core dependencies come from the Bazel Central Registry (BCR)
via MODULE.bazel, while toolchain-related deps remain in WORKSPACE.
Key changes:
Dependencies migrated to BCR:
Dependencies remaining in WORKSPACE:
Motivation
I'm going to want to pull in bazelbuild/bazel#28088 once it lands and thought this pre-work is useful
Test plan
See included automated tests.