The Windows RBE test has been using Bazel 7.3.1 for running tests.
However, the RBE configuration itself was built with an older Bazel
version (6.3.2). While this hasn't caused any issues so far, it's best
to use the same Bazel version (7.3.1) for both building the RBE
configuration and running tests to ensure consistency and avoid
potential problems in the future.
Related to https://github.com/grpc/grpc/pull/37987
- Update to windows RBE docker image, followed by subsequent regenerate
of win RBE toolchain (basically verifying that the README.md for windows
RBE toolchain works well)
- Also move windows docker image from GCR to GAR (which is where our
other testing images already live).
- (cleanup) Move windows RBE docker image to a better location in our
repo.
This PR should be sufficient to demonstrate how we would go about
upgrading the windows RBE build to VS2022 once needed.
- Upgrade windows RBE builds to bazel 6.3.2 (supersedes
https://github.com/grpc/grpc/pull/33880). To be able to do that, the RBE
toolchain needed to be regenerated and
- Also added "--dynamic_mode=off" option for windows build since it
seemed necessary for bazel 6.x builds to pass.
- Wrote instructions for generating windows RBE toolchain using the
`rbe_configs_gen` tool (the original windows RBE toolchain was out of
data and also it was generated by a custom script from
go/rbe-windows-user-guide - using a standard tool is better)
- Wrote instructions for rebuilding the windows RBE docker image.
This addresses the problem where windows RBE is stuck on bazel 5.x
(unlike the rest of the repository) and also documents the steps for
making changes to the RBE docker image (e.g. upgrading the visual studio
version used by RBE).
Since we were planning on adding testing for VS2022 (for which we don't
have any test ATM), this will definitely come handy. With the
documentation the process should now be relatively straightforward.
- Remove no-longer-needed workaround for b/275571385 (and switch back to
using docker image from GAR)
- regenerate RBE linux toolchain for bazel 6.1.2 while at it.
- Add a new docker image "rbe_ubuntu2004" that is built in a way that's
analogous to how our other testing docker images are built (this gives
us control over what exactly is contained in the docker image and
ability to fine-tune our RBE configuration)
- Switch RBE on linux to the new image (which gives us ubuntu20.04-based
builds)
For some reason, RBE seems to have trouble pulling the docker image from
Google Artifact Registry (GAR), which is where our public testing images
normally live, so for now, I used a workaround and I upload a copy of
the rbe_ubuntu2004 docker image to GCR as well, and that makes RBE works
just fine (see comment in the `renerate_linux_rbe_configs.sh` script).
More followup items (config cleanup, getting local sanitizer builds
working etc.) are in go/grpc-rbe-tech-debt-2023
First step in the modernization of our RBE stack (see
go/rbe-tech-debt-notes).
- Get rid of the deprecated rbe_autoconfig and start using
[rbe_configs_gen](https://github.com/bazelbuild/bazel-toolchains#rbe_configs_gen---cli-tool-to-generate-configs)
+ check in the generated toolchain configs.
- Switch from marketplace.gcr.io/google/rbe-ubuntu16-04 to
marketplace.gcr.io/google/rbe-ubuntu18-04 (this image is still not owned
by us, but at least it's newer and demonstrates how a switch to a newer
docker image is done).
- provide script for generating the linux RBE toolchain configs.
- cleanup RBE configuration in the bazelrc files used for remote build
* update win RBE docker image and toolchain
and switch win RBE builds to using it.
* delete legacy RBE windows toolchain config
* switching to VS2019 makes #28544 obsolete
* Remove the toolchains //third_party/toolchains:local and //third_party/toolchains:local_large.
* Remove the platforms :rbe_ubuntu1604, :rbe_ubuntu1604_large, :local and :local_large.
* No longer inherit from @rbe_default//config:platform but instead use it directly. It is now the only non-windows platform.
* When creating @rbe_default//config:platform directly with rbe_autoconfig, set dockerAddCapabilities and dockerPrivileged directly in the exec_properties field. No need to set dockerNetwork to "off" and dockerSiblingContainers to false since these are the defaults.
* Also set gceMachineType = "n1-highmem-2" on the default platform. This value can be overridden by specific targets that want to use LARGE_MACHINE.
* Use create_exec_properties_dict where appropriate.
* Use custom_exec_properties to define LARGE_MACHINE.
I wasn't able to test thoroughly that this PR does not break any existing targets. I was not able to run anything on windows/mac and I also don't have access to gRPC's RBE setup.
This should fix Bazel Basic Tests for Python (Local) and other
local CI runs that are now failing.
I also have a plan to fix Bazel RBE Opt C/C++.
Bazel RBE UBSAN C/C++ might need to wait for gRPC to upgrade to
using 0.23.0
Esun Kim
Marcel
Jan Tattermusch
Ara Nguyen
Jan Tattermusch
Lidi Zheng
Eytan Kidron
Nicolas Noble
Nicolas "Pixel" Noble
Bill Feng
Nicolas Lopez