Support Bzlmod and add rules_scala to bazel-central-registry #1482

sfc-gh-pbennes · 2023-03-16T23:26:47Z

Greetings!

It looks like rules_scala isn't part of the Bzlmod effort or added to bazel-central-registry yet. I've opened an issue both here and in https://github.com/bazelbuild/bazel-central-registry to request it: bazelbuild/bazel-central-registry#522

chrislovecnm · 2023-07-11T15:30:04Z

Just a note to anyone that starts the work native.register_toolchains is not supported when using bzlmod.

sluongng · 2023-07-11T15:35:37Z

@chrislovecnm it's still available here no? https://bazel.build/rules/lib/globals/module#register_toolchains

chrislovecnm · 2023-07-11T15:42:51Z

@sluongng When I last tried it, it did not work when using it inside an extension. In rules_python we are skipping the native call:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/python/repositories.bzl#L569

And build the toolchains here:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/python/extensions/private/pythons_hub.bzl#L56

We build the toolchains and use the following:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/MODULE.bazel#L49

sluongng · 2023-07-11T15:47:13Z

Yeah you would only want to prepare the toolchain (as a new repository) inside the extension I think. Registering it must be called from the MODULE.bazel file.

So native.register_toolchains call from the extension's starlark does not work, but having the extension create @pythons_hub//:all and then MODULE.bazel calling register_toolchains("@pythons_hub//:all") worked. 👍

mateuszkuta256 · 2024-01-24T08:56:24Z

I prepared a minimal bzlmod support here and gonna split it into a few PRs
TODOs after merging it:

write documentation on how to use rules_scala with bzlmod (some properties like SCALA_VERSION will probably be configurable via repo-env)
prepare release notes
contribute repo to the Bazel Central Registry
add tests that the project builds using both WORKSPACE and MODULE.bazel
migrate test/external workspaces to blzmod too

pcj · 2024-06-30T17:14:28Z

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

mateuszkuta256 · 2024-07-01T08:54:25Z

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

hi, unfortunately I'm working on other things now and won't continue with this PR in the foreseeable future
I remember this migration was on hold because interfered with 'cross compilation support'
It looks like great progress was done it this area, so maybe someone can already take over the PR

mbland · 2024-10-02T17:41:34Z

I'd like to take on the task of Bzlmodifying this repo through a series of pull requests.

I've already created a (mostly) working branch in my own fork. Though I saw a couple draft pull requests here, I ended up taking a different approach and got it mostly working. In fact, I did exactly what @sluongng suggested in #1482 (comment). (I didn't notice this comment before just now—I might've read it, but not understood it at the time—but I did study rules_python and rules_go, and ended up doing exactly that.)

There are still issues I need help to address (recorded in some of the commit messages), I didn't strictly maintain WORKSPACE compatibility (fixable), and I only tested against Scala 2.13. But I can start teasing chunks out of it in a methodical fashion, to ensure that I maintain WORKSPACE and cross-version compatibility.

For an example of what it looks like from a client perspective, here's what the MODULE.bazel stanza from my local EngFlow/example repo looks like (as opposed to the current stanza, explanatory comment notwithstanding):

bazel_dep(name = "rules_scala", repo_name = "io_bazel_rules_scala")
local_path_override(
    module_name = "rules_scala",
    path = "../../bazelbuild/rules_scala"
)   

scala_dev_deps = use_extension(
    "@io_bazel_rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
    dev_dependency = True,
)   
scala_dev_deps.toolchains(
    scalatest = True,
)

So if folks are game for me to do this, I'll start carving off pieces as separate pull requests, and we can resolve any outstanding problems in the process.

cc: @BillyAutrey @jayconrod @benjaminp @TheGrizzlyDev

This begins the Bzlmod compatibility migration by updating Bazel to version 7.3.2 and adding initial `MODULE.bazel` and `WORKSPACE.bzlmod` files. Part of: bazelbuild#1482 Though Bzlmod remains disabled, updating to Bazel 7.3.2 requred updating or adding the following packages to maintain `WORKSPACE` compatibility. In `rules_scala_setup()`: - bazel_skylib: 1.4.1 => 1.7.1 - rules_cc: 0.0.6 => 0.0.10 - rules_java: 5.4.1 => 7.9.0 - rules_proto: 5.3.0-21.7 => 6.0.2 Dev dependencies in `WORKSPACE`: - com_google_protobuf: 28.2 - rules_pkg: 1.0.1 - rules_jvm_external: 6.4 - com_google_absl: abseil-cpp-20240722.0 - zlib: 1.3.1 Of all of the new, explicit dev dependencies, only `com_google_protobuf` will be necessary to include in `MODULE.bazel`. The Bzlmod mechanism will discover these other transitive dev dependencies automatically. Also removed the `rules_java_extra` repo from `WORKSPACE`, which appeared unused. --- Though the current `rules_java` version is 7.12.1, and largely works with this repo, it requires a few temporary workarounds. Rather than commit the workarounds, upgrading only to 7.9.0 now seems less crufty. What follows is a very detailed explanation of what happens with 7.12.1 with Bazel 7.3.2, just to have it on the record. --- The workaround is to change a few toolchain and macro file targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. This isn't a terribly bad or invasive workaround, but `@bazel_tools//tools/jdk:` is clearly the canonical path. Best to keep it that way, lest we build up technical debt. Without the workaround, these targets would fail: - //test/src/main/resources/java_sources:CompiledWithJava11 - //test/src/main/resources/java_sources:CompiledWithJava8 - //test/toolchains:java21_toolchain - //test:JunitRuntimePlatform - //test:JunitRuntimePlatform_test_runner - //test:scala_binary_jdk_11 with this error: ```txt ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. ``` This appears to be a consequence of both upgrading the Bazel version from 6.3.0 to 7.3.2 and updating `rules_java` to 7.12.1. The `rules_java_builtin` repo is part of the `WORKSPACE` prefix that adds implicit dependencies: - https://bazel.build/external/migration#builtin-default-deps This repo was added to 7.0.0-pre.20231011.2 in the following change, mapped to `@rules_java` within the scope of the `@bazel_tools` repo: - bazelbuild/bazel: Add rules_java_builtin to the users of Java modules bazelbuild/bazel@ff1abb2 This change tried to ensure `rules_java` remained compatible with earlier Bazel versions. However, it changed all instances of `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` to `//toolchains:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java: Make rules_java backwards compatible with Bazel 6.3.0 bazelbuild/rules_java@30ecf3f Bazel has bumped `rules_java` in its `workspace_deps.bzl` from 7.9.0 to 7.11.0, but it's only available as of 8.0.0-pre.20240911.1. - bazelbuild/bazel: Update rules_java 7.11.1 / java_tools 13.8 bazelbuild/bazel#23571 bazelbuild/bazel@f92124a --- What I believe is happening is, under Bazel 7.3.2 and `rules_java` 7.12.1: - Bazel creates `rules_java` 7.9.0 as `@rules_java_builtin` in the `WORKSPACE` prefix. - `@bazel_tools` has `@rules_java` mapped to `@rules_java_builtin` when initialized during the `WORKSPACE` prefix, during which `@bazel_tools//tools/jdk` registers `alias()` targets to `@rules_java` toolchain targets. These aliased toolchains specify `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` in their `toolchains` attribute. - `WORKSPACE` loads `@rules_java` 7.12.1 and registers all its toolchains with type `@rules_java//toolchains:bootstrap_runtime_toolchain_type`. - Some `@rules_java` rules explicitly specifying toolchains from `@bazel_tools//tools/jdk` can't find them, because the `@bazel_tools//tools/jdk` toolchain aliases expect toolchains of type `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`. This has broken other projects in the same way: - bazelbuild/bazel: [Bazel CI] Downstream project broken by rules_java upgrade #23619 bazelbuild/bazel#23619 These problems don't appear under Bzlmod, and `@rules_java_builtin` was never required. This is because `WORKSPACE` executes its statements sequentially, while Bzlmod builds the module dependency graph _before_ instantiating repositories (within module extensions). It seems a fix is on the way that removes `@rules_java_builtin` from the `WORKSPACE` prefix, and adds `@rules_java` to the suffix. At this moment, though, it's not even in a prerelase: - bazelbuild/bazel: Remove rules_java_builtin in WORKSPACE prefix bazelbuild/bazel@7506690 --- Note that the error message matches that from the following resolved issue, but that issue was for non-Bzlmod child repos when `WORKSPACE` was disabled. - bazelbuild/bazel: Undefined @@rules_java_builtin repository with --noenable_workspace option bazelbuild/bazel#22754

Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation documented at: - bazelbuild#1619 (comment) Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources` values between Bazel 6 and Bazel 7. Without this change, `_import_paths()` emits incorrect values under Bazel 7, causing targets containing generated `.proto` inputs to fail, e.g. `//test/proto3:test_generated_proto`. See also: - Fix paths for sibling repository setup and generated .proto files bazelbuild/bazel@6c6c196 - The docstring for `ProtoInfo.proto_source_root` in the Bazel sources: https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172 - Remove incompatible_generated_protos_in_virtual_imports bazelbuild/bazel@3efaa32 - Comment from: Generated Protos are no longer considered as virtual_imports in Bazel 7 bazelbuild/bazel#21075 (comment) --- I cherrypicked this commit into bazelbuild#1618. While it fixed the `//test/proto3` build failure, it does _not_ fix the hanging scalapb_workers from the ProtoScalaPBRule aspect. I'll have to investiate further whether than hang is related to Bazel, rules_proto, com_google_protobuf, or some mixture thereof. Still, progress!

Part of bazelbuild#1482. Splits the last component off of canonical repo names to produce the expected repo name. Without Bzlmod, it returns the original name. With Bzlmod enabled, it avoids generating output like: scala_import( name = "_main~scala_deps~io_bazel_rules_scala_scala_compiler", jars = ["scala-compiler-2.12.18.jar"], ) resulting in errors like: ``` ERROR: .../_main~_repo_rules~io_bazel_rules_scala/scala/BUILD: no such target '@@_main~scala_deps~io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler': target 'io_bazel_rules_scala_scala_compiler' not declared in package '' defined by .../_main~scala_deps~io_bazel_rules_scala_scala_compiler/BUILD and referenced by '@@_main~_repo_rules~io_bazel_rules_scala//scala:default_toolchain_scala_compile_classpath_provider' ``` Also fixes the following error when attaching resources from custom repos to targets under Bzlmod: ```txt $ bazel test //test/src/main/scala/scalarules/test/resources:all 1) Scala library depending on resources from external resource-only jar::allow to load resources(scalarules.test.resources.ScalaLibResourcesFromExternalDepTest) java.lang.NullPointerException at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.get(ScalaLibResourcesFromExternalDepTest.scala:17) at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$3(ScalaLibResourcesFromExternalDepTest.scala:11) at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$2(ScalaLibResourcesFromExternalDepTest.scala:11) ``` Can be replaced with a future bazel-skylib implementation, if accepted into that repo. --- We can't rely on the specific canonical repository name format: > Repos generated by extensions have canonical names in the form of > `module_repo_canonical_name+extension_name+repo_name`. For extensions > hosted in the root module, the `module_repo_canonical_name` part is > replaced with the string `_main`. Note that the canonical name format is > not an API you should depend on — it's subject to change at any time. > > - https://bazel.build/external/extension#repository_names_and_visibility The change to no longer encode module versions in canonical repo names in Bazel 7.1.0 is a recent example of Bazel maintainers altering the format: - bazelbuild/bazel#21316 And the maintainers recently replaced the `~` delimiter with `+` in the upcoming Bazel 8 release due to build performance issues on Windows: - bazelbuild/bazel#22865 This function assumes the only valid `repo_name` characters are letters, numbers, '_', '-', and '.'. It finds the last character not in this set, and returns the contents of `name` following this character. This is valid so long as this condition holds: - https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/java/com/google/devtools/build/lib/cmdline/RepositoryName.java#L159-L162

Related to #1482, #1618, and #1619. Results from the investigation documented at: - #1619 (comment) Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources` values between Bazel 6 and Bazel 7. Without this change, `_import_paths()` emits incorrect values under Bazel 7, causing targets containing generated `.proto` inputs to fail, e.g. `//test/proto3:test_generated_proto`. See also: - Fix paths for sibling repository setup and generated .proto files bazelbuild/bazel@6c6c196 - The docstring for `ProtoInfo.proto_source_root` in the Bazel sources: https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172 - Remove incompatible_generated_protos_in_virtual_imports bazelbuild/bazel@3efaa32 - Comment from: Generated Protos are no longer considered as virtual_imports in Bazel 7 bazelbuild/bazel#21075 (comment) --- I cherrypicked this commit into #1618. While it fixed the `//test/proto3` build failure, it does _not_ fix the hanging scalapb_workers from the ProtoScalaPBRule aspect. I'll have to investiate further whether than hang is related to Bazel, rules_proto, com_google_protobuf, or some mixture thereof. Still, progress!

Part of bazelbuild#1482. Splits the last component off of canonical repo names to produce the expected repo name. Without Bzlmod, it returns the original name. With Bzlmod enabled, it avoids generating output like: scala_import( name = "_main~scala_deps~io_bazel_rules_scala_scala_compiler", jars = ["scala-compiler-2.12.18.jar"], ) resulting in errors like: ``` ERROR: .../_main~_repo_rules~io_bazel_rules_scala/scala/BUILD: no such target '@@_main~scala_deps~io_bazel_rules_scala_scala_compiler//:io_bazel_rules_scala_scala_compiler': target 'io_bazel_rules_scala_scala_compiler' not declared in package '' defined by .../_main~scala_deps~io_bazel_rules_scala_scala_compiler/BUILD and referenced by '@@_main~_repo_rules~io_bazel_rules_scala//scala:default_toolchain_scala_compile_classpath_provider' ``` Also fixes the following error when attaching resources from custom repos to targets under Bzlmod: ```txt $ bazel test //test/src/main/scala/scalarules/test/resources:all 1) Scala library depending on resources from external resource-only jar::allow to load resources(scalarules.test.resources.ScalaLibResourcesFromExternalDepTest) java.lang.NullPointerException at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.get(ScalaLibResourcesFromExternalDepTest.scala:17) at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$3(ScalaLibResourcesFromExternalDepTest.scala:11) at scalarules.test.resources.ScalaLibResourcesFromExternalDepTest.$anonfun$new$2(ScalaLibResourcesFromExternalDepTest.scala:11) ``` Can be replaced with a future bazel-skylib implementation, if accepted into that repo. --- We can't rely on the specific canonical repository name format: > Repos generated by extensions have canonical names in the form of > `module_repo_canonical_name+extension_name+repo_name`. For extensions > hosted in the root module, the `module_repo_canonical_name` part is > replaced with the string `_main`. Note that the canonical name format is > not an API you should depend on — it's subject to change at any time. > > - https://bazel.build/external/extension#repository_names_and_visibility The change to no longer encode module versions in canonical repo names in Bazel 7.1.0 is a recent example of Bazel maintainers altering the format: - bazelbuild/bazel#21316 And the maintainers recently replaced the `~` delimiter with `+` in the upcoming Bazel 8 release due to build performance issues on Windows: - bazelbuild/bazel#22865 This function assumes the only valid `repo_name` characters are letters, numbers, '_', '-', and '.'. It finds the last character not in this set, and returns the contents of `name` following this character. This is valid so long as this condition holds: - https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/java/com/google/devtools/build/lib/cmdline/RepositoryName.java#L159-L162

mbland · 2024-10-11T16:49:17Z

A quick update for visibility: I'm very close to having the Bzlmodified rules_scala passing 100% of the tests, down to the last couple of failures:

I need to replace the bind() calls from twitter_scrooge with alias() or some such to get test_version.sh to pass.
I need to fix the Scala 3.1.2 and Scala 3.3.3 failures for the Scalafmt targets from test/shell/test_cross_build.sh (corresponding to the test3 and library3 test cases in test_scalafmt()).

I also need to get test_lint.sh working again, and update the dt_patches repos for Bzlmod. But the following all pass:

test_rules_scala.sh
test_reproducibility.sh
test_examples.sh
test_coverage.sh

Hopefully I can get these fixed up today, and I'll start peeling off the next pull request or two. And thanks to @simuons and @liucijus for reviewing and merging #1619 and #1620 already.

All but finishes bazelbuild#1482, barring the release and publication to the Bazel Central Registry. New `MODULE.bazel` files mirroring existing `WORKSPACE` configurations comprise the bulk of the commit. The empty `WORKSPACE.bzlmod` files ensure that Bzlmod won't evaluate the existing `WORKSPACE` files. These new files comprise the most significant part of the change: - `scala/extensions/config.bzl` - `scala/extensions/deps.bzl` - `scala/private/extensions/dev_deps.bzl` - `scala/private/macros/bzlmod.bzl` `config.bzl`, `deps.bzl`, and `dev_deps.bzl` are thoroughly tested by existing tests. The following files thoroughly test the helpers from `scala/private/macros/bzlmod.bzl` specifically: - scala/private/macros/test/... - test/shell/test_bzlmod_macros.sh The pattern employed throughout the new module extensions is explained in the `scala/private/macros/bzlmod.bzl` docstring. Adding `test/shell/test_bzlmod_macros.sh` also precipitated adding a new `assert_matches` helper to `test/shell/test_helper.sh`. `test/shell/test_helper.sh` also introduces a mechanism for automatically finding and skipping tests, documented in the comment at the bottom of the file. "Publish to BCR" configuration in the `.bcr` directory comes from: - https://github.com/bazel-contrib/publish-to-bcr/tree/main/templates The bazel-contrib/publish-to-bcr README contains further guidance on configuring the app. Once that's done, we can use the app to publish a new version to https://registry.bazel.build/ and then close bazelbuild#1482. This change enables Bazel 7 and 8 users to migrate their `rules_scala` dependency from `WORKSPACE` to `MODULE.bazel` whenever they're ready.

Registers a precompiled protocol compiler toolchain when `--incompatible_enable_proto_toolchain_resolution` is `True`. Part of bazelbuild#1482 and bazelbuild#1652. Stops `protoc` recompilation, and fixes the build breakage in bazelbuild#1710 due to `protobuf` include paths exceeding the Visual Studio path length limit. The updates to `scala_proto/scala_proto_toolchain.bzl` were inspired by: - protocolbuffers/protobuf: bazel: Remove hardcoded dependency on //:protoc from language runtimes #19679 protocolbuffers/protobuf#19679 The `proto_lang_toolchain` call was inspired by the `README` from: - https://github.com/aspect-build/toolchains_protoc/ Adds `scripts/update_protoc_integrity.py` to automatically update `scala/private/protoc/protoc_integrity.bzl`. This should make builds of `rules_scala` much faster all around. Given the fact that this feature depends on recent `protobuf` versions, and the Windows `protobuf` build breaks without it, we have a catch-22. It likely can't be separated from the rest of bazelbuild#1710, though I would prefer that. It also seems likely that we'd eventually need to do this to continue supporting Windows, per: - protocolbuffers/protobuf#12947 - https://protobuf.dev/news/v30/#poison-msvc--bazel - protocolbuffers/protobuf#20085 More background on proto toolchainization: - Proto Toolchainisation Design Doc https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit - bazelbuild/bazel: Protobuf repo recompilation sensitivity bazelbuild/bazel#7095 - bazelbuild/rules_proto: Implement proto toolchainisation bazelbuild/rules_proto#179 - rules_proto 6.0.0 release notes mentioning Protobuf Toolchainisation https://github.com/bazelbuild/rules_proto/releases/tag/6.0.0

All but finishes bazelbuild#1482, barring the release and publication to the Bazel Central Registry. New `MODULE.bazel` files mirroring existing `WORKSPACE` configurations comprise the bulk of the commit. The empty `WORKSPACE.bzlmod` files ensure that Bzlmod won't evaluate the existing `WORKSPACE` files. These new files comprise the most significant part of the change: - `scala/extensions/config.bzl` - `scala/extensions/deps.bzl` - `scala/private/extensions/dev_deps.bzl` - `scala/private/macros/bzlmod.bzl` `config.bzl`, `deps.bzl`, and `dev_deps.bzl` are thoroughly tested by existing tests. The following files thoroughly test the helpers from `scala/private/macros/bzlmod.bzl` specifically: - scala/private/macros/test/... - test/shell/test_bzlmod_macros.sh The pattern employed throughout the new module extensions is explained in the `scala/private/macros/bzlmod.bzl` docstring. Adding `test/shell/test_bzlmod_macros.sh` also precipitated adding a new `assert_matches` helper to `test/shell/test_helper.sh`. `test/shell/test_helper.sh` also introduces a mechanism for automatically finding and skipping tests, documented in the comment at the bottom of the file. "Publish to BCR" configuration in the `.bcr` directory comes from: - https://github.com/bazel-contrib/publish-to-bcr/tree/main/templates The bazel-contrib/publish-to-bcr README contains further guidance on configuring the app. Once that's done, we can use the app to publish a new version to https://registry.bazel.build/ and then close bazelbuild#1482. This change enables Bazel 7 and 8 users to migrate their `rules_scala` dependency from `WORKSPACE` to `MODULE.bazel` whenever they're ready.

mbland · 2025-03-03T05:23:39Z

It turns out @crt-31 and I were able to diagnose the Windows problem in #1710 (the classic 260 character path limit). I've been able to get it working with protocol compiler toolchainization (i.e., downloading precompiled protoc packages) with a little help from protocolbuffers/protobuf#19679.

Please check out #1710 for the latest details.

All but finishes bazelbuild#1482, barring the release and publication to the Bazel Central Registry. New `MODULE.bazel` files mirroring existing `WORKSPACE` configurations comprise the bulk of the commit. The empty `WORKSPACE.bzlmod` files ensure that Bzlmod won't evaluate the existing `WORKSPACE` files. These new files comprise the most significant part of the change: - `scala/extensions/config.bzl` - `scala/extensions/deps.bzl` - `scala/private/extensions/dev_deps.bzl` - `scala/private/macros/bzlmod.bzl` `config.bzl`, `deps.bzl`, and `dev_deps.bzl` are thoroughly tested by existing tests. The following files thoroughly test the helpers from `scala/private/macros/bzlmod.bzl` specifically: - scala/private/macros/test/... - test/shell/test_bzlmod_macros.sh The pattern employed throughout the new module extensions is explained in the `scala/private/macros/bzlmod.bzl` docstring. Adding `test/shell/test_bzlmod_macros.sh` also precipitated adding a new `assert_matches` helper to `test/shell/test_helper.sh`. `test/shell/test_helper.sh` also introduces a mechanism for automatically finding and skipping tests, documented in the comment at the bottom of the file. "Publish to BCR" configuration in the `.bcr` directory comes from: - https://github.com/bazel-contrib/publish-to-bcr/tree/main/templates The bazel-contrib/publish-to-bcr README contains further guidance on configuring the app. Once that's done, we can use the app to publish a new version to https://registry.bazel.build/ and then close bazelbuild#1482. This change enables Bazel 7 and 8 users to migrate their `rules_scala` dependency from `WORKSPACE` to `MODULE.bazel` whenever they're ready. Stardoc inspired doc updates

All but finishes bazelbuild#1482, barring the release and publication to the Bazel Central Registry. Closes bazelbuild#1625. New `MODULE.bazel` files mirroring existing `WORKSPACE` configurations and copies of the `protobuf` toolchainization patch comprise the bulk of the commit. The empty `WORKSPACE.bzlmod` files ensure that Bzlmod won't evaluate the existing `WORKSPACE` files. Reenables the `last_green` Bazel build in CI. Other than that, these new files comprise the most significant part of the change: - `.bazelignore` - `.bcr/*` - `scala/extensions/config.bzl` - `scala/extensions/deps.bzl` - `scala/private/extensions/dev_deps.bzl` - `scala/private/macros/bzlmod.bzl` `config.bzl`, `deps.bzl`, and `dev_deps.bzl` are thoroughly tested by existing tests. These new test files thoroughly test the helpers from `scala/private/macros/bzlmod.bzl` specifically: - scala/private/macros/test/... - test/shell/test_bzlmod_macros.sh The pattern employed throughout the new module extensions is explained in the `scala/private/macros/bzlmod.bzl` docstring. Adding `test/shell/test_bzlmod_macros.sh` also precipitated adding a new `assert_matches` helper to `test/shell/test_helper.sh`. `test/shell/test_helper.sh` also introduces a mechanism for automatically finding and skipping tests, documented in the comment at the bottom of the file. "Publish to BCR" configuration in the `.bcr` directory comes from: - https://github.com/bazel-contrib/publish-to-bcr/tree/main/templates The bazel-contrib/publish-to-bcr README contains further guidance on configuring the app. --- This change enables Bazel 7 and 8 users to migrate their `rules_scala` dependency from `WORKSPACE` to `MODULE.bazel` whenever they're ready. Once the Publish to BCR GitHub app is ready, we can use it to publish a new version to https://registry.bazel.build/. At that point, we can close bazelbuild#1482.

mbland · 2025-03-04T20:29:04Z

Time for another (not so) weekly update!

Overview

We're potentially two pull requests and one release (or two) away from closing this issue:

Bazel 8 + rules_java 8 compatibility updates #1710
One more pull request from my bzlmod-enable working branch
Set up the Publish to BCR GitHub app
Publish 7.0.0 (or maybe 7.1.0) to https://registry.bazel.build/
Profit! (i.e., close this issue)

I'm OK breaking more pull requests out of #1710, as it's gotten quite large. But all the tests pass, including new ones I've added!

As discussed here last week, I created #1710 to bump all the versions, since @simuons is OK with dropping Bazel 6 support. This would also close #1652. This likely means one major release instead of two, with Bazel 7 and 8 compatibility out of the box.

Bazel 6.5.0 WORKSPACE builds will actually still work, with either the C++ compiler flags from #1647, with precompiled protoc toolchains (discussed below), or both. From the updated README.md:

Bazel/Dependency	`rules_scala` 7.x
Bazel versions using Bzlmod	7.5.0, 8.x, `rolling`, `last_green`
Bazel versions using `WORKSPACE`	6.5.0, 7.5.0, 8.x
`protobuf`	v29.3
`abseil-cpp`	20250127.0
`rules_java`	8.10.0
`ScalaPB`	1.0.0-alpha.1

Both #1710 and bzlmod-enable contain many README updates, explaining all the changes, how to use the new APIs, and how to handle breaking changes. bzlmod-enable also contains the files required by the Publish to BCR GitHub app, and will close #1625.

Once the bzlmod-enable changes land, @simuons and/or @liucijus will need to configure the Publish to BCR app and cut a release. Once rules_scala appears in the BCR after that, we can close this issue (and bazelbuild/bazel-central-registry#522).

Merged

Closed

Use rctx.name if rctx.original_name is empty #1702

Open

Bazel 8 + rules_java 8 compatibility updates #1710

Releases

There's some discussion about releasing 7.0.0 without Bzlmod, then 7.1.0 with. From a technical perspective, there's no need; nothing in the Bzlmod implementation will affect WORKSPACE users at all. Upgrading to 7.x will require migrating to a new API regardless, and the new WORKSPACE macros and module extensions both converge upon scala_toolchains().

The only advantage I see to releasing 7.0.0 without Bzlmod is that we can release it sooner without waiting for the final Bzlmod pull request. Either way, I'm able to support whichever option @simuons and @liucijus prefer.

Windows and precompiled `protoc` toolchains

Recall that @crt-31 and I found protocolbuffers/protobuf#12947, which confirmed the 260 character path limit was responsible for breaking the Windows build in #1710. I was able to fix the Windows build by implementing a precompiled protoc toolchain to avoid rebuilding @com_google_protobuf//:protoc, currenly also in #1710. This required making a patch out of protocolbuffers/protobuf#19679 as well as applying the same changes to scala_proto_library.

This is pretty much the only option for Windows MSVC builds to stay current with protobuf releases at this point, per:

For all other platforms, the precompiled protoc toolchain is completely optional, but it does make builds much faster. It's easily configurable, and the implementation has zero impact on builds that don't explicitly configure it. See the README updates in #1710 for details.

Trying Bzlmod before the BCR release via `git_override()`

As always, anyone can try my bzlmod and bzlmod-bazel-8 branches. (bzlmod-enable is not guaranteed to remain stable; please don't depend on it!)

At this point, the only differences between the stable branches and bzlmod-enable are:

Both use Scala 2.13.16 by default instead of 2.12.20 (still the current default on master).
bzlmod-bazel-8 sets all .bazelversion files to 8.1.1 by default instead of 7.5.0 (which actually has zero impact on consumers).

For examples, see:

Stardoc experimentation

I did tinker with Stardoc on my bzlmod-enable branch yesterday afternoon, but have decided not to invest in it yet. It does some very cool things, but it requires feeding all the dependencies of a .bzl input file into the stardoc() rule. This makes it a non-starter for files that depend on external repository files, which is certainly the case for many .bzl files.

Maybe there's a way to hack around this limitation, and that would also avoid having to add bzl_library targets and exports_files directives everywhere. (I'm thinking of something that would strip out all load() statements, and possibly replace macro/function bodies with pass or something.) But I don't anticipate making it a priority.

Thanks to...

In addition to @simuons and @liucijus (as always!), several folks have stepped up to help with some code reviews and drive some useful conversations. I just wanted to say thanks while it's fresh in my mind to:

@dmivankov for catching the 'overridden_artifacts' design error in the Bzlmod implementation (fixed in my Bzlmod branches).
@WojciechMazur for jumping in to try to help @dmivankov with his remaining issue.
@michalbogacz for pointing out the need to document the repo scope changes in README.md (in the Bazel Slack workspace).
@bartoszkosiorek for:
- opening Please add support to Scala 2.13.16 and make new release #1699, which inspired big README changes and old API removals in Update docs for version 7.0.0, remove old APIs #1703.
- reviewing Update docs for version 7.0.0, remove old APIs #1703.
@grepwood for feedback on Update docs for version 7.0.0, remove old APIs #1703 leading to significant improvements.
@gergelyfabian for reviewing Update docs for version 7.0.0, remove old APIs #1703 and trying out the documentation changes using the latest changes on master, leading to several important updates.
@crt-31 for:
- filing setup_scala_testing_toolchain() failing when passed xxx_classpath #1706 (fixed in Fix setup_scala_testing_toolchain() classpaths #1707).
- getting me to explain several aspects the work on this issue, including
- getting clarity on dropping out-of-the-box support for Bazel 6 (thanks also to @simuons for that one, of course).
- finding and posting src build on windows not working protocolbuffers/protobuf#12947 to help diagnose the Windows breakage in Bazel 8 + rules_java 8 compatibility updates #1710.
@fmeum for:
- resolving New repository_ctx.original_name produces "canonical reproducible" warnings bazel#25192 as part of Bazel 8.1.1, allowing me to close Use rctx.name if rctx.original_name is empty #1702.
- opening bazel: Remove hardcoded dependency on //:protoc from language runtimes protocolbuffers/protobuf#19679, without which fixing Bazel 8 + rules_java 8 compatibility updates #1710 on Windows by using a precompiled protoc wouldn't've been possible.

All but finishes bazelbuild#1482, barring the release and publication to the Bazel Central Registry. Closes bazelbuild#1625. New `MODULE.bazel` files mirroring existing `WORKSPACE` configurations and copies of the `protobuf` toolchainization patch comprise the bulk of the commit. The empty `WORKSPACE.bzlmod` files ensure that Bzlmod won't evaluate the existing `WORKSPACE` files. Reenables the `last_green` Bazel build in CI. Other than that, these new files comprise the most significant part of the change: - `.bazelignore` - `.bcr/*` - `scala/extensions/config.bzl` - `scala/extensions/deps.bzl` - `scala/private/extensions/dev_deps.bzl` - `scala/private/macros/bzlmod.bzl` `config.bzl`, `deps.bzl`, and `dev_deps.bzl` are thoroughly tested by existing tests. These new test files thoroughly test the helpers from `scala/private/macros/bzlmod.bzl` specifically: - scala/private/macros/test/... - test/shell/test_bzlmod_macros.sh The pattern employed throughout the new module extensions is explained in the `scala/private/macros/bzlmod.bzl` docstring. Adding `test/shell/test_bzlmod_macros.sh` also precipitated adding a new `assert_matches` helper to `test/shell/test_helper.sh`. `test/shell/test_helper.sh` also introduces a mechanism for automatically finding and skipping tests, documented in the comment at the bottom of the file. "Publish to BCR" configuration in the `.bcr` directory comes from: - https://github.com/bazel-contrib/publish-to-bcr/tree/main/templates The bazel-contrib/publish-to-bcr README contains further guidance on configuring the app. --- This change enables Bazel 7 and 8 users to migrate their `rules_scala` dependency from `WORKSPACE` to `MODULE.bazel` whenever they're ready. Once the Publish to BCR GitHub app is ready, we can use it to publish a new version to https://registry.bazel.build/. At that point, we can close bazelbuild#1482.

Dependency version updates that still work with Bazel 6.5.0 and 7.5.0. Broken out from bazelbuild#1710, and part of bazelbuild#1482 and bazelbuild#1652. Updates `.bazelversion` files to 7.5.0 and the CI builds in `.bazelci/presubmit.yml` to use Bazel 7.5.0. Bumps the following dependencies, which should not cause build breakages on Windows + MSVC: - Go: 1.24.0 => 1.24.1 - Scalafmt: 3.9.1 => 3.9.2 - `abseil-cpp`: 20220623.1 => 20250127.0 - `grpc`: 1.70.0 => 1.71.1 - `protobuf-java`: 4.29.3 => 4.30.0 - `sbt-compiler-interface`: 1.10.7 => 1.10.8 - `sbt-compiler-util`: 1.10.7 => 1.10.10 - `google-common-protos`: 2.52.0 => 2.53.0 Defers the following updates, which are already present in bazelbuild#1710: - `protobuf`: v21.7 => v28.3 (or v30.0) - `rules_cc`: 0.0.9 => 0.1.1 - `rules_proto`: 6.0.2 => 7.1.0 - `rules_python`: 0.38.0 => 1.2.0 - `scalapb`: 0.11.17 => 1.0.0-alpha1 These deferred updates all need to happen together, as updating only a subset of them will break the build. This change is smaller and more focused than bazelbuild#1710, and should ultimately make that pull request smaller and/or easier to review.

Updates the `protobuf` version to one that's still compatible with Bazel 6.5.0 and 7.5.0 without updating `scalapb`. Broken out from bazelbuild#1710, and part of bazelbuild#1482 and bazelbuild#1652. Like bazelbuild#1711, updates `.bazelversion` files to 7.5.0 and the CI builds in `.bazelci/presubmit.yml` to use Bazel 7.5.0. Unlike bazelbuild#1711, contains only these updates: - `abseil-cpp`: 20220623.1 => 20250127.0 - `protobuf`: v21.7 => v25.6 This change aims make bazelbuild#1710 smaller and more focused, and should ultimately make that pull request smaller and/or easier to review. Specifically, this is an attempt to see whether MSVC will build successfully with `protobuf` v25.6. If it doesn't, I will update this change to include the protocol compiler toolchainization changes from bazelbuild#1710.

@dmivankov

Adds the `examples/overridden_artifacts` repository and the corresponding `overridden_artifacts_example` test case in `test/shell/test_examples.sh`. Broken out from bazelbuild#1710, and part of bazelbuild#1482 and bazelbuild#1652. @dmivankov noticed the design bug in the upcoming Bzlmod API for `overridden_artifacts` that this change addresses. See: - bazelbuild#1482 (comment) - bazelbuild#1482 (comment) Makes `_validate_scalac_srcjar()` and `dt_patched_compiler_setup()` in `scala/private/macros/scala_repositories.bzl` more tolerant of dictionaries containing keys mapped to `None`. The new `overridden_artifacts_example` test covers this. Sets `.bazelversion` in the new repo to 7.5.0 to match changes in both bazelbuild#1710 and bazelbuild#1711. This change is smaller and more focused than bazelbuild#1710, and should ultimately make that pull request smaller and/or easier to review.

Contains many editorial improvements to and some extra information in the README, along with a few small improvements to the code. Broken out from bazelbuild#1710, and part of bazelbuild#1482 and bazelbuild#1652. Specifically: - Adds more info on translating `@rules_scala` to `@io_bazel_rules_scala` for dependencies via repo mapping. - Merges information about the previously planned `rules_scala` 8.0.0 release into the information for 7.0.0, since it seems we may make only one major release. - Improves information about `protobuf` support for versions before v28, Scala 2.11, and the upcoming Bzlmod `compatibility_level` setting. - In `scala_config.bzl`, changes the private `_default_scala_version()` to the public `DEFAULT_SCALA_VERSION`. - Adds `allow_empty = True` to a `glob` expression in `//test/semanticdb:lib_with_tempsrc`. - Removes Scala 2.11 test cases from `test_thirdparty_version.sh` and `test_version.sh`. This change is smaller and more focused than bazelbuild#1710, and should ultimately make that pull request smaller and/or easier to review. The motivations for the individual changes are: - The public `DEFAULT_SCALA_VERSION` constant makes this value accessible to the upcoming module extension. - `glob` requires an explicit `allow_empty = True` parameter in Bazel 8, in which `--incompatible_disallow_empty_glob` defaults to `True`. - ScalaPB 0.9.8, the last version compatible with Scala 2.11, does not support `protobuf` v25.6 or later. For this reason, we must remove the Scala 2.11 test cases, as documented in the `README.md` updates. See also bazelbuild#1712. We should consider dropping Scala 2.11 support at this point, since there's no ScalaPB release for it that supports later versions of `protobuf`. That, and we could remove some of the special case code added in the following changes, amongst other 2.11 support details: - bazelbuild#1631 - bazelbuild#1648 - bazelbuild#1687 - bazelbuild#1688

All but finishes bazelbuild#1482, barring the release and publication to the Bazel Central Registry. Closes bazelbuild#1625. New `MODULE.bazel` files mirroring existing `WORKSPACE` configurations and copies of the `protobuf` toolchainization patch comprise the bulk of the commit. The empty `WORKSPACE.bzlmod` files ensure that Bzlmod won't evaluate the existing `WORKSPACE` files. Reenables the `last_green` Bazel build in CI. Other than that, these new files comprise the most significant part of the change: - `.bazelignore` - `.bcr/*` - `scala/extensions/config.bzl` - `scala/extensions/deps.bzl` - `scala/private/extensions/dev_deps.bzl` - `scala/private/macros/bzlmod.bzl` `config.bzl`, `deps.bzl`, and `dev_deps.bzl` are thoroughly tested by existing tests. These new test files thoroughly test the helpers from `scala/private/macros/bzlmod.bzl` specifically: - scala/private/macros/test/... - test/shell/test_bzlmod_macros.sh The pattern employed throughout the new module extensions is explained in the `scala/private/macros/bzlmod.bzl` docstring. Adding `test/shell/test_bzlmod_macros.sh` also precipitated adding a new `assert_matches` helper to `test/shell/test_helper.sh`. `test/shell/test_helper.sh` also introduces a mechanism for automatically finding and skipping tests, documented in the comment at the bottom of the file. "Publish to BCR" configuration in the `.bcr` directory comes from: - https://github.com/bazel-contrib/publish-to-bcr/tree/main/templates The bazel-contrib/publish-to-bcr README contains further guidance on configuring the app. --- This change enables Bazel 7 and 8 users to migrate their `rules_scala` dependency from `WORKSPACE` to `MODULE.bazel` whenever they're ready. Once the Publish to BCR GitHub app is ready, we can use it to publish a new version to https://registry.bazel.build/. At that point, we can close bazelbuild#1482.

* 7.0.0 README updates and minor code changes Contains many editorial improvements to and some extra information in the README, along with a few small improvements to the code. Broken out from #1710, and part of #1482 and #1652. Specifically: - Adds more info on translating `@rules_scala` to `@io_bazel_rules_scala` for dependencies via repo mapping. - Merges information about the previously planned `rules_scala` 8.0.0 release into the information for 7.0.0, since it seems we may make only one major release. - Improves information about `protobuf` support for versions before v28, Scala 2.11, and the upcoming Bzlmod `compatibility_level` setting. - In `scala_config.bzl`, changes the private `_default_scala_version()` to the public `DEFAULT_SCALA_VERSION`. - Adds `allow_empty = True` to a `glob` expression in `//test/semanticdb:lib_with_tempsrc`. - Removes Scala 2.11 test cases from `test_thirdparty_version.sh` and `test_version.sh`. This change is smaller and more focused than #1710, and should ultimately make that pull request smaller and/or easier to review. The motivations for the individual changes are: - The public `DEFAULT_SCALA_VERSION` constant makes this value accessible to the upcoming module extension. - `glob` requires an explicit `allow_empty = True` parameter in Bazel 8, in which `--incompatible_disallow_empty_glob` defaults to `True`. - ScalaPB 0.9.8, the last version compatible with Scala 2.11, does not support `protobuf` v25.6 or later. For this reason, we must remove the Scala 2.11 test cases, as documented in the `README.md` updates. See also #1712. We should consider dropping Scala 2.11 support at this point, since there's no ScalaPB release for it that supports later versions of `protobuf`. That, and we could remove some of the special case code added in the following changes, amongst other 2.11 support details: - #1631 - #1648 - #1687 - #1688 * Tweak `rules_python` and Bazel 6.5.0 `README` info Provided a better explanation for using `rules_python` 0.38.0 for now. Improved some of the language in the Bazel 6.5.0 compatibility section.

@dmivankov

Adds the `examples/overridden_artifacts` repository and the corresponding `overridden_artifacts_example` test case in `test/shell/test_examples.sh`. Broken out from #1710, and part of #1482 and #1652. @dmivankov noticed the design bug in the upcoming Bzlmod API for `overridden_artifacts` that this change addresses. See: - #1482 (comment) - #1482 (comment) Makes `_validate_scalac_srcjar()` and `dt_patched_compiler_setup()` in `scala/private/macros/scala_repositories.bzl` more tolerant of dictionaries containing keys mapped to `None`. The new `overridden_artifacts_example` test covers this. Sets `.bazelversion` in the new repo to 7.5.0 to match changes in both #1710 and #1711. This change is smaller and more focused than #1710, and should ultimately make that pull request smaller and/or easier to review.

* Bazel 7 and MSVC compatible version updates Dependency version updates that still work with Bazel 6.5.0 and 7.5.0. Broken out from #1710, and part of #1482 and #1652. Updates `.bazelversion` files to 7.5.0 and the CI builds in `.bazelci/presubmit.yml` to use Bazel 7.5.0. Bumps the following dependencies, which should not cause build breakages on Windows + MSVC: - Go: 1.24.0 => 1.24.1 - Scalafmt: 3.9.1 => 3.9.2 - `abseil-cpp`: 20220623.1 => 20250127.0 - `grpc`: 1.70.0 => 1.71.1 - `protobuf-java`: 4.29.3 => 4.30.0 - `sbt-compiler-interface`: 1.10.7 => 1.10.8 - `sbt-compiler-util`: 1.10.7 => 1.10.10 - `google-common-protos`: 2.52.0 => 2.53.0 Defers the following updates, which are already present in #1710: - `protobuf`: v21.7 => v28.3 (or v30.0) - `rules_cc`: 0.0.9 => 0.1.1 - `rules_proto`: 6.0.2 => 7.1.0 - `rules_python`: 0.38.0 => 1.2.0 - `scalapb`: 0.11.17 => 1.0.0-alpha1 These deferred updates all need to happen together, as updating only a subset of them will break the build. This change is smaller and more focused than #1710, and should ultimately make that pull request smaller and/or easier to review. * Add `abseil-cpp` repo mapping to `protobuf` This will make sure `protobuf` uses the version of `abseil-cpp` that we import. * Set `common --enable_workspace --noenable_bzlmod` These flags affect `bazel query`, and Bazel 8 defaults to `--noenable_workspace --enable_bzlmod`. Using `common` ensures `bazel query` sees the same settings as `bazel build`. This prevents the `WORKSPACE` run of `test_semanticdb_handles_removed_sourcefiles` from failing under Bazel 8. Bazel 6.5.0 doesn't define `--[no]enable_workspace`, so it makes sense to include it in the change that sets all `.bazelversion` files to 7.5.0.

@simuons

Bumps dependencies to versions that are compatible with both Bazel 7.5.0 and 8.0.0, and adds protocol compiler toolchainization in `//protoc` for `protobuf` v29 and later. Closes bazelbuild#1652. Part of bazelbuild#1482. - ScalaPB jars: 0.11.17 => 1.0.0-alpha.1 - Scalafmt: 3.9.2 => 3.9.3 - `rules_python`: 0.38.0 => 1.2.0 - `rules_cc`: 0.0.9 => 0.1.1 - `rules_java`: 7.12.4 => 8.10.0 - `protobuf`: 21.7 => 30.0 - `rules_proto`: 6.0.2 => 7.1.0 Bazel 6 is officially unsupported as of this change and the upcoming `rules_scala` 7.0.0 release. Updates `.bazelci/presubmit.yml` to bump the `7.x` build to `last_rc`. Registers a precompiled protocol compiler toolchain when `--incompatible_enable_proto_toolchain_resolution` is `True`. Otherwise, `register_toolchains(//protoc:all)` toolchains is a no-op, as it will be empty. `scripts/update_protoc_integrity.py` automatically updates `scala/private/protoc/protoc_integrity.bzl`. The `protobuf` patch is the `git diff` output from protocolbuffers/protobuf#19679, which also inspired the updates to `scala_proto/scala_proto_toolchain.bzl`. The `proto_lang_toolchain` call in the `BUILD` file generated by `protoc/private/protoc_toolchain.bzl` was inspired by the `README` from: - https://github.com/aspect-build/toolchains_protoc/ Loads `java_proto_library` from `com_google_protobuf`, replacing the officially deprecated version from `rules_java`. Bumps Scalafmt to 3.9.3 out of convenience. Updates to `README.md`, and updates to `WORKSPACE` and `third_party/repositories` files precipitated by the dependency updates, comprise the remainder of this change. --- We're no longer planning to support Bazel 6 in the next major release per @simuons's decision in: - bazelbuild#1482 (comment) The plan is now to land the Bazel 7 and 8 compatibility updates first, then land the Bzlmod change. This enables us to make only one new major version release, instead of two (whereby the first release would've continued supporting Bazel 6). It turns out the two major version plan wouldn't've been possible. Bazel 8 and `rules_java` 8 require `protobuf` >= v29, but this bump caused Windows builds to break when compiling `protoc` in bazelbuild#1710. `src/google/protobuf/compiler/java/java_features.pb.h`, the path specified in the error message, doesn't exist until `protobuf` v25.0. @crt-31 and I found that this was related to the Windows/MSVC 260 character file path length limit. What's more, the `protobuf` team plans to drop MSVC support specifically because of this path length limit. The protocol compiler toolchain prevents `protoc` recompilation, which fixes the Windows breakage while making all builds faster. Since Windows builds break since at least `protobuf` v25, but `protoc` toolchainization requires v29, the version bump and the `protoc` toolchain must land together. --- I tried several things to get protocol compiler toolchainization to work with `protobuf` v28.2, described below. However, each path only led to the same suffering described in the new "Why this requires `protobuf` v29 or later" section of the README. I discovered along the way that `protobuf` v30 isn't compatible with Bazel 6.5.0 at all. I added an explanation to the "Limited Bazel 6.5.0 compatibility" section of `README.md`. --- I experimented with using `protobuf` v28.2, `rules_proto` 6.0.2, and `rules_java` 7.12.4 and 8.10.0. I updated the `protobuf` patch for v28.2 with the following statements: ```py load("//bazel/common:proto_common.bzl", "proto_common") load("@rules_proto//proto:proto_common.bzl", "toolchains") _PROTO_TOOLCHAIN = "@rules_proto//proto:toolchain_type" _PROTO_TOOLCHAIN_ATTR = "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION" _PROTOC_TOOLCHAINS = toolchains.use_toolchain(_PROTO_TOOLCHAIN) def _protoc_files_to_run(ctx): if getattr(proto_common, _PROTO_TOOLCHAIN_ATTR, False): ``` I updated `protoc/private/protoc_toolchain.bzl` to use `proto_common` from `rules_proto`. I also created a `rules_proto` 6.0.2 patch for `proto_toolchain()` to fix a "no such package: //proto" breakage: ```diff 6.0.2 patch for `proto_toolchain()`: ```diff diff --git i/proto/private/rules/proto_toolchain.bzl w/proto/private/rules/proto_toolchain.bzl index a091b80..def2699 100644 --- i/proto/private/rules/proto_toolchain.bzl +++ w/proto/private/rules/proto_toolchain.bzl @@ -33,7 +33,7 @@ def proto_toolchain(*, name, proto_compiler, exec_compatible_with = []): native.toolchain( name = name + "_toolchain", - toolchain_type = "//proto:toolchain_type", + toolchain_type = Label("//proto:toolchain_type"), exec_compatible_with = exec_compatible_with, target_compatible_with = [], toolchain = name, ``` I tried adding combinations of the following `--incompatible_autoload_externally` flag values to .bazelrc`: ```txt common --incompatible_autoload_externally=+@protobuf,+@rules_java ``` Nothing worked. --- After the `protobuf` v29 bump, and before the ScalaPB 1.0.0-alpha.1 bump, `scala_proto` targets would fail with the following error: ```txt ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14: ProtoScalaPBRule external/com_google_protobuf/src/google/protobuf/any_proto_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/.../bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) --jvm_extra_protobuf_generator_out: java.lang.NoSuchMethodError: 'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)' at scalapb.compiler.DescriptorImplicits$ExtendedFieldDescriptor.fieldOptions(DescriptorImplicits.scala:329) [ ...snip... ] java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:30) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14 Building source jar external/com_google_protobuf/src/google/protobuf/any_proto_scalapb-src.jar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target @@com_google_protobuf//src/google/protobuf:any_proto) bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) ```

sfc-gh-pbennes mentioned this issue Mar 16, 2023

wanted: bazelbuild/rules_scala bazelbuild/bazel-central-registry#522

Open

1 task

liucijus added the help wanted label Mar 17, 2023

mateuszkuta256 mentioned this issue Jan 24, 2024

Draft: Add bzlmod minimal support #1541

Closed

mbland mentioned this issue Apr 15, 2024

Convert WORKSPACE to MODULE.bazel, update README EngFlow/example#312

Merged

agluszak mentioned this issue Apr 22, 2024

Migrate from sbt to bazel build-server-protocol/build-server-protocol#615

Merged

agluszak mentioned this issue Aug 7, 2024

Draft: Add bzlmod minimal support (continuation) #1595

Closed

mbland mentioned this issue Oct 2, 2024

Transfer rules_scala to bazel-contrib #1616

Open

4 tasks

This was referenced Oct 3, 2024

Bump Bazel and dependencies to prepare for Bzlmod #1618

Closed

Bazel 7 compatibility updates #1619

Merged

mbland mentioned this issue Oct 5, 2024

Fix Bazel 7 related protobuf build failures #1620

Merged

liucijus closed this as completed in #1619 Oct 8, 2024

simuons reopened this Oct 8, 2024

mbland mentioned this issue Mar 6, 2025

Bazel 7 and MSVC compatible version updates #1711

Merged

mbland mentioned this issue Mar 6, 2025

Upgrade to protobuf v25.6 #1712

Closed

mbland mentioned this issue Mar 7, 2025

Add examples/overridden_artifacts and test #1713

Merged

mbland mentioned this issue Mar 7, 2025

7.0.0 README updates and minor code changes #1714

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Bzlmod and add rules_scala to bazel-central-registry #1482

Support Bzlmod and add rules_scala to bazel-central-registry #1482

sfc-gh-pbennes commented Mar 16, 2023 •

edited

Loading

chrislovecnm commented Jul 11, 2023

sluongng commented Jul 11, 2023

chrislovecnm commented Jul 11, 2023 •

edited

Loading

sluongng commented Jul 11, 2023

mateuszkuta256 commented Jan 24, 2024

pcj commented Jun 30, 2024 •

edited

Loading

mateuszkuta256 commented Jul 1, 2024

mbland commented Oct 2, 2024

mbland commented Oct 11, 2024

mbland commented Mar 3, 2025

mbland commented Mar 4, 2025 •

edited

Loading

Support Bzlmod and add rules_scala to bazel-central-registry #1482

Support Bzlmod and add rules_scala to bazel-central-registry #1482

Comments

sfc-gh-pbennes commented Mar 16, 2023 • edited Loading

chrislovecnm commented Jul 11, 2023

sluongng commented Jul 11, 2023

chrislovecnm commented Jul 11, 2023 • edited Loading

sluongng commented Jul 11, 2023

mateuszkuta256 commented Jan 24, 2024

pcj commented Jun 30, 2024 • edited Loading

mateuszkuta256 commented Jul 1, 2024

mbland commented Oct 2, 2024

mbland commented Oct 11, 2024

mbland commented Mar 3, 2025

mbland commented Mar 4, 2025 • edited Loading

Overview

Merged

Closed

Open

Releases

Windows and precompiled protoc toolchains

Trying Bzlmod before the BCR release via git_override()

Stardoc experimentation

Thanks to...

sfc-gh-pbennes commented Mar 16, 2023 •

edited

Loading

chrislovecnm commented Jul 11, 2023 •

edited

Loading

pcj commented Jun 30, 2024 •

edited

Loading

mbland commented Mar 4, 2025 •

edited

Loading

Windows and precompiled `protoc` toolchains

Trying Bzlmod before the BCR release via `git_override()`