Use `dockcross/manylinux2014-aarch64` for aarch64 artifact docker images to improve portability. Also existing docker images are updated to use the latest instead of pinned image.
Closes#38084
PiperOrigin-RevId: 694241161
This reverts commit 574b19ec31.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#38074
PiperOrigin-RevId: 693803071
This might not be needed? (From https://github.com/grpc/grpc/issues/37976)
- Boringssl commit requiring -msse2: 56d3ad9d23 (diff-f628d148f94bbab9e22a1ad426ccd94311f1fb87bbe5cc533cb85aee18b07a20R248)
- gRPC change to add -msse2 https://github.com/grpc/grpc/pull/36089
---
Answer to the quesiton above is yes; The -msse2 option remains necessary for gRPC on i686 due to BoringSSL's requirements. However, the existing CMake condition for this option was too broad, potentially including ARM architectures where SSE2 isn't supported, leading to compilation errors. I've refined the condition to specifically target 32-bit x86 architectures.
Furthermore, to ensure accurate architecture detection within our dockerized tests, I've configured x86 tests to utilize the linux32 command. This ensures that uname -a correctly reports i686, allowing gRPC's CMake to identify the architecture and apply the -msse2 option as needed.
It's important to note that RBE overrides the default entrypoint, so RBE-based tests must explicitly invoke linux32 even if the Docker image already has it set.
Fixes https://github.com/grpc/grpc/issues/37976Closes#38024
PiperOrigin-RevId: 693026079
This aims to resolve segfaults encountered when rebuilding ARM64 PHP Docker images. This segfaults came from gcc during ARM64 builds using QEMU. To workaround this problem, the PHP Docker images have been restructured to use apt-get for installation, thereby avoiding the use of gcc. This also needed to upgrade Debian and PHP to their latest versions, aligning with our minimum supported PHP 8.
Closes#37895
PiperOrigin-RevId: 686162640
With the CL-first approach, the docker test configs for Binder need to be deleted before the Binder code and tests themselves can be deleted in the next step. Sanity checks fail otherwise.
Closes#37862
PiperOrigin-RevId: 683691175
This PR adds templating for Python versions and updates the maximum supported Python version to 3.13. The following major changes related to templating are added:
- Minimum supported Python version and list of supported versions in `setup.py` are fetched using new template generated files called `python_version.py`
- Dockerfiles for the different Python Linux builds are now template generated.
- The "Supported Python Versions" section from READMEs of ancillary and main packages have been removed
Note: All the `python_version.py` files and Linux build `Dockerfiles` except `tools/dockerfile/grpc_artifact_python_linux_armv7/Dockerfile` in the PR are generated from the respective templates.
Further non-templated additions to add support for Python 3.13:
- install scripts and artifacts for windows, macos and linux are added manually. Later, these can be templated as well.
- updated cython bounds to 3.x
- updated twine version to solve [cgi module import error](https://github.com/pypa/twine/issues/1046)
- the twine update introduces a dependency on cryptography>=2.0. But the cryptography package doesn't support 32-bit Linux images and hence `twine check` has been disabled for x86 manylinux and x86 musllinux artifacts.
Closes#37643
PiperOrigin-RevId: 678954495
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37572
PiperOrigin-RevId: 670631284
The oldest gcc version that gRPC supports as of today is gcc 7 but gcc 7 has an issue with template supports that gRPC already picked up. Recently we managed to fix it in gRPC library code but we still have some in our test code. Given that it's not easy to fix since it requires many trial error approach to find a way to satisfy gcc 7 and eventually gcc 7 will be dropped from our supported compilers, let's have this mitigation where just main grpc++ target is being tested for gcc 7 so that users can use grpc with it without having to fix this hairy issue.
Fixes https://github.com/grpc/grpc/issues/36751Closes#37257
PiperOrigin-RevId: 654076384
GCC 7 was disabled due to its gcc internal error while building protobuf C++. But we still need to support gcc7 and there appear to be issues causing gcc 7 to not build grpc so let's reintroduce gcc 7 test.
Closes#36565
PiperOrigin-RevId: 631947203
Typo introduced in: https://github.com/grpc/grpc/pull/34450
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
### NOTE
* We shouldn't merge this PR until GCP cloud functions drops support for Python 3.7 ([Currently scheduled for GCF](https://cloud.google.com/functions/docs/runtime-support#python))
As part of supporting Python 3.12, we're now officially drop support for Python 3.7.
This PR:
* Changed supported Python version from 3.7 to 3.8 in README.
* Replaced distribution test image from `debian:buster` to `debian:bullseye` since the default Python version in buster is 3.7.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34450
PiperOrigin-RevId: 622899511
It turned out that the previous change missed two things which this PR
has
- Fix function `_dockerized_genrule` not to have `timeout` and `flaky`
which Bazel doesn't have. (Bazel 7 may either drop these arguments or
become more strict about passing unrecognized ones)
- Disabled python bazel distribution tests with Bazel 7. This needs to
be addressed by https://github.com/grpc/grpc/issues/35391.
- Added Bazel 7 to the support bazel versions.
- Changed the default Bazel version to 7.
- Fixed Android Binder build issue.
Closes#35362
PiperOrigin-RevId: 592946781
Enable OpenSSL 1.0.2 tests and add a container for 1.1.1 so that it is tested during portability testing as well.
Closes#35236
PiperOrigin-RevId: 590345568
The `bazel_build_with_strict_warnings_linux` job was taking nearly an
hour to complete. This change brings it down to ~20m by splitting up the
build targets into 7 separate RBE actions.
---------
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Foundation for being able to bazelify the build artifact -> build
package -> distribtest workflow tests.
Main ideas:
- "build artifact" and "build packages" will be represented by a custom
genrule (that runs the build on RBE under a docker container).
- since genrule doesn't support displaying logs for each target as a
separate "target log" (in the same way that bazel tests do), and we
generally want readable per-target logs for the bazelified test, a pair
of targets will be created for each "build artifact task":
- a genrule that actually performs the build, creates an archive with
artifacts and stores the exitcode and build log as rule outputs
- a corresponding "build_test" sh_test that simply looks at the result
of the genrule and presents the build log and build result a "target
log" for this test.
Bazelify tests from "linux/grpc_bazel_build" kokoro job by creating 3
bazelified tests - "build with strict warning", "build with no_xds=True"
and "build with no_xds=True negative test".
- also make the original "linux/grpc_bazel_build" kokoro job a no-op
(since bazelified tests now provide the same coverage).
- make C-core basictests use `--build_only` when running as bazelified
tests. This is because the volume of C core tests is expected to grow
very significantly after https://github.com/grpc/grpc/pull/34419 and
currently the non-bazelified counterpart of the tests (the presubmit
grpc_basictests_c_cpp_build_only job) is also "build only".
- make the linux presubmit job `grpc_basictests_c_cpp_build_only` a
noop, since the bazelified tests already give the same coverage on
presubmit.
Basically run each of the subtests (buildtest, distribtest_cpp,
distribtest_python) as a separate bazel target.
- currently the bazel distribtest are the slowest targets in
grpc_bazel_rbe_nonbazel
- the shards are basically independent tests anyway
- when split into multiple targets, they each get a separate target log
so it's easier debug issues since there isn't multiple bazel invocations
in each log.
- Upgrade bazel
- Reduce the number of places where bazel version needs to be upgraded
in future.
- also make sure the list of bazel versions to test by bazelified tests
is loaded from supported_versions.txt (it was hardcoded before).
- ~~Try upgrading windows RBE build to bazel 6.3.2 as well.~~
The core idea:
- the source of truth for supported bazel versions is in
`bazel/supported_versions.txt`
- the first version listed in `bazel/supported_versions.txt` is
considered to be the "primary" bazel version and is going to be used in
most places thoroughout the repo.
- use templates to include the primary bazel version in testing
dockerfiles and in a newly introduced `.bazelversion` files (which gets
loaded by our existing `tools/bazel` wrapper).
~~Supersedes https://github.com/grpc/grpc/pull/33880~~
Needs https://github.com/grpc/grpc/pull/34035 to be merged first. With
newer ccache in available in the gcc12 test image, the build is now
faster so we can reenable the test.
Reintroduce https://github.com/grpc/grpc/pull/33959.
I added a fix for the python arm64 build (which is the reason why the
change has been reverted earlier).
- some cleanup
- add bazel distribtests (to run the existing
test_single_bazel_version.sh locally under a docker container created by
RBE).
- add c++ distribtests
- rename `grpc_run_tests_py_test` macro to `grpc_run_tests_harness_test`
- as requested previously to avoid confusion with `*_py_test` rules
Add "bazelified" non-bazel tests. See tools/bazelify_tests/README.md for
the core idea.
- add a bunch of test targets that run under docker and execute tests
that correspond to `run_tests.py -l LANG ...`
- many more tests can be added in the future
- to enable running some of the C/C++ portability tests easily, added
support for `--cmake_extra_configure_args` in run_tests.py (the change
is fairly small).
Example passing build that shows how test results are structured:
https://source.cloud.google.com/results/invocations/21295351-a3e3-4be1-b6e9-aaf52195a044/targets
Esun Kim
AJ Heller
Sreenithi Sridharan
Yijie Ma
Nathan Baulch
Xuan Wang
Gregory Cooke
Jan Tattermusch