The stubs generated by grpcio_tools should always be used with [the same or higher version of grpcio](https://github.com/grpc/grpc/blob/master/tools/distrib/python/grpcio_tools/setup.py#L313), this change will add a run time check for this requirement inside the generated stubs and therefor enforce this requirement.
Please note for now we're just printing a warning for incorrect usage, we'll **change it to an error** soon.
Example warning message:
```
/usr/local/google/home/xuanwn/workspace/misc/grpc/examples/python/helloworld/helloworld_pb2_grpc.py:21: RuntimeWarning: The grpc package installed is at version 1.60.1, but the generated code in helloworld_pb2_grpc.py depends on grpcio>=1.63.0.dev0. Please upgrade your grpc module to grpcio>=1.63.0.dev0 or downgrade your generated code using grpcio-tools<=1.60.1. This warning will become an error in 1.64.0, scheduled for release on May 14,2024.
```
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35906
PiperOrigin-RevId: 615659471
Earlier, the tests just had bazel support. With CMake support added in #36063, we can also add CI CMake support for the tests. A major benefit of this is that we also get coverage for the various platforms that we test from our portability test suite.
Closes#36087
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36087 from yashykt:OTelCISupport b28fbe02e5
PiperOrigin-RevId: 615543685
Refactored android-ndk docker image to have
- Replaced a full Android SDK with a Android command line tool (generally good for docker usage) and bumped its version to the latest among ones that support JDK 11
- Upgraded Debian from 10 to 11
- Upgraded Java SDK 8 to 11
- Upgraded Google CLI to the latest
Closes#36091
PiperOrigin-RevId: 615088973
Change was created by the release automation script. See go/grpc-release.
Additional Changes:
* Boring SSL started to [Require SSE2 when targetting 32-bit x86](56d3ad9d23), thus added `-msse2` to fix some build failures.
Closes#36089
PiperOrigin-RevId: 614822548
gRPC swift package depends on [boringSSL-SwiftPM](https://github.com/firebase/boringSSL-SwiftPM), which hasn't been updated since more than two years ago (https://github.com/firebase/boringSSL-SwiftPM/pull/8).
Updating it is not trivial as the `Package.swift` file is maintained manually, a recent [upgrade to swift 5.5](https://github.com/firebase/boringSSL-SwiftPM/pull/16) took good amount of time only to repeatedly exclude one (sometimes two) non-code files and retry build to get new errors, until the build succeeds.
This PR uses the gRPC build system to generate the swift package file from the same boringssl version in third_party.
Will use the output to update the boringSSL-SwiftPM repo after this PR is merged.
Tested with
```
cp BoringSSL-Package.swift Package.swift
swift build --target openssl_grpc
swift test
```
Closes#36062
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36062 from HannahShiSFB:add-boringssl-swift-package ac3e525bf2
PiperOrigin-RevId: 614088802
Changes -
* Add CMake build support to `grpcpp_otel_plugin`. Currently, we are only supporting the `find_package CONFIG` method for depending on `opentelemetry-cpp`.
* Since, `grpcpp_otel_plugin` is an extension of gRPC, it will not be built by default. To enable building of this target, a new CMake option `gRPC_BUILD_GRPCPP_OTEL_PLUGIN` is being added.
* Also add `CMakeLists.txt` to the otel example.
The `otel_plugin_test` can also be built through cmake but, for the CI to work, there are some additional changes. Those will be made in an upcoming PR.
Closes#36063
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36063 from yashykt:OTelCMakeSupport 3bc783823b
PiperOrigin-RevId: 613734473
- PHP 7.4 and 8.0 have been EOL https://www.php.net/supported-versions.php
- See go/drop-php-8.0 for more details
Most of the changes came from `generate_projects.sh` after updating the `tools/buildgen/plugins/expand_version.py` file. The change to `tools/internal_ci/helper_scripts/prepare_build_macos_rc` is manual.
Closes#35964
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35964 from stanley-cheung:php-update-to-8_1 0c24c4eac2
PiperOrigin-RevId: 609495798
This PR is copied from #34276, since I did not have permissions to add commits to it. That PR has been verified to work (see the top-level description). This PR just makes the gRPC tests pass (e.g. adding includes, clang formatting).
Closes#34874
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34874 from matthewstevenson88:pull_34276 d5fb73e5b2
PiperOrigin-RevId: 609107146
- Dropped Ruby 2.7 test (Ruby 2.7 hit EOL 1 year ago)
- Upgraded the base image of Ruby docker images from Debian:10 to Debian:11 (Debian:10 hit EOL)
Closes#35911
PiperOrigin-RevId: 607088383
Change was created by the release automation script. See go/grpc-release.
On top of changes made by the script, `setup.py` had to be updated to have `/std:c11` option for MSVC to use c11 features for boringssl.
Closes#35768
PiperOrigin-RevId: 603403329
BoringSSL-GRPC uses sources from https://github.com/google/boringssl.git, which doesn't contain PrivacyInfo.xcprivacy file.
This PR embeds the manifest data into the podspec and extract during pod install, similar to file boringssl_prefix_symbols.h
--
Will need to regenerate the podspec after #35542 merged.
Closes#35616
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35616 from HannahShiSFB:privacy-manifests-boring-ssl d068be244b
PiperOrigin-RevId: 601274520
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Follow-up to add privacy manifests to #35042
- [x] Update podspec template files
- [x] Align on manifest update:
```
git grep mach_absolute_time
src/core/lib/gpr/posix/time.cc:static uint64_t g_time_start = mach_absolute_time();
src/core/lib/gpr/posix/time.cc: ((double)(mach_absolute_time() - g_time_start)) * g_time_scale;
```
cc: @paulb777Closes#35542
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35542 from ncooke3:nc/privacy-manifests b0e0e57d2e
PiperOrigin-RevId: 600595032
Fixes#30385 for C++/cmake.
Add a `gRPC_DOWNLOAD_ARCHIVES` switch that is on by default but can be disabled for environments building without Internet connectivity.
Passes build tests:
```
$ python tools/run_tests/run_tests.py -l c++ --build_only
PASSED: tools/run_tests/helper_scripts/build_cxx.sh [time=7116.3sec, retries=0:0SUCCESS: All tests passed
=== run_tests.py DEBUG INFO ===
command: "tools/run_tests/run_tests.py -l c++ --build_only"
===============================
$
```
And tested in a Docker container with `eth0` down to ensure we can now build when `-DgRPC_DOWNLOAD_ARCHIVES="OFF"` is passed to cmake.
Closes#34587
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34587 from matthewjmiller1:no-download-option 9c07bdce87
PiperOrigin-RevId: 599954208
`GPR_BACKWARDS_COMPATIBILITY_MODE` was devised to support old Linux kernel with old glibc but gRPC is now expected to support glibc 2.17 and later (2.17 is derived from the oldest but supported Linux distro, CentOS 7). Effectively, gRPC doesn't need `GPR_BACKWARDS_COMPATIBILITY_MODE` anymore. Hence, let's remove it.
gRPC Python and Ruby that distribute binary artifacts already began to use `manylinux` environment dealing with this issue in a better way so they don't need this flag anymore.
Closes#35602
PiperOrigin-RevId: 599895270
We've been trying to upgrade Cmake to 3.13 or later as OSS policy bumped it. But we couldn't as Android has a weird linker error with Cmake 3.18 (you can see the error from https://github.com/grpc/grpc/pull/34331) This PR instead upgrades it to use 3.22 for Android test.
Closes#35572
PiperOrigin-RevId: 599317285
Updates ruby-compiler-dock to 1.4.0 which brings Ruby 3.3 final support per https://github.com/rake-compiler/rake-compiler-dock/releases/tag/1.4.0 and starts cross-compiling for ruby 3.3.
I can't find obviously where the test infrastructure configuration is to run the tests under Ruby 3.3, so might need pointers or an accompanying PR for the test infra. (I note #31991 from @apolcyn so perhaps currently they are not run against newer versions)
Fixes#35396
- Backport to `1.60` is desirable since currently the Ruby gems cannot be installed with Ruby 3.3 and have to be built from source.
Closes#35399
PiperOrigin-RevId: 599200628
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35484
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35484 from dawidcha:win_shared_gs_api f8f244da99
PiperOrigin-RevId: 598863993
This reverts commit 96b9e8d3e3.
[Implement OpenTelemetry PR](https://github.com/grpc/grpc/pull/35292) was [reverted](96b9e8d3e3) because some tests started failing after import the changes to g3.
After investigation, we found root cause, it can be fixed both on our side and on gapic API side, we opened an issue to [gapic API team](https://github.com/googleapis/python-api-core/issues/579), this PR will includes the fixes on our side.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35439
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35439 from XuanWang-Amos:reapply_otel 0133564438
PiperOrigin-RevId: 595746222
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35292
PiperOrigin-RevId: 595188404
- Fixed the bazel distrib tests with Bazel 7 by disabling bzlmod option.
- Added a new note for bzlmod to the doc.
Closes#35390
PiperOrigin-RevId: 593816700
Fix: https://github.com/grpc/grpc/issues/35085
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35325
PiperOrigin-RevId: 592635611
Enable OpenSSL 1.0.2 tests and add a container for 1.1.1 so that it is tested during portability testing as well.
Closes#35236
PiperOrigin-RevId: 590345568
`_parallel_compile_patch.py` exist in multiple places, we should use template system to generate the content.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34847
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34847 from XuanWang-Amos:use_template_for__parallel_compile_patch 15b98ea826
PiperOrigin-RevId: 590262888
### Changes in this PR
* Refactor and remove some Core/C++ dependencies to simplify Python Observability package build process.
* Refactored code to read config at Python layer.
* Enable observability build from source.
* Add observability to run_test.
* Currently it's only enabled in Linux.
* Add error handler in run_test loaders.
* Current framework will always visit modules in test directory then decide which tests to skip.
* Since we're not building Observability for MacOS and Windows this step will fail with error `No module named 'grpc_observability'`.
* After the change we'll just skip those modules.
* We still have `_sanity_test` to make sure all tests are loaded correctly for each platform.
* Remov OC dependency as we're migrating to OTel.
* Also removed trace from testing.
* Note that trace propagation function was also removed because of this.
### Testing
* Passed existing tests.
* Tested locally, able to build observability from source using `GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install .`.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34207
PiperOrigin-RevId: 590258014
Not sure if we need to do the same for all the podspec files.
Will create a separate PR for swift package.
Closes#35042
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35042 from HannahShiSFB:privacy-manifests c9b19d3c20
PiperOrigin-RevId: 588937122
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
Not all POSIX platforms have rt as a separate library. QNX has rt as
part of libc (same as pthreads).
Add condition to check if the library can be found and link with it only
in positive scenario.
A solution for: https://github.com/grpc/grpc/issues/34254
Reverts grpc/grpc#34667
The change was reverted because it failed to import to g3, after some
changes, now it's safe to reapply those changes.
Tested by importing this PR internally, it passed presubmit:
cl/573836270
Partial rollback of https://github.com/grpc/grpc/pull/34473 for RBE as
RBE needs Ubuntu 20.04 to get clang artifacts which Google publishes.
There aren't artifacts for Ubuntu 22.04 yet so there is no point of
upgrading RBE image to 22.04.
Introduce `grpc-native-debug` gems containing debug symbol packages that
complement the shared libraries shipped in pre-compiled binary gems.
src/ruby/nativedebug/README.md has details on usage.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
We're not running any test at all from `run_test.py` because of the way
we filter test cases:
1d136fd05f/src/python/grpcio_tests/tests/_runner.py (L137)
* `testcase_filter` is read from a json file (like [this
one](https://github.com/grpc/grpc/blob/master/src/python/grpcio_tests/tests/tests.json))
and test name is similar to `unit._metadata_test.MetadataTest`.
* `case.id()` is loaded by `iterate_suite_cases` and will always have a
prefix of `tests`, an example of case id will be:
`tests.unit._metadata_test.MetadataTest`.
Because of the prefix, none of the test case will be matched thus we're
not running any of the tests.
This PR fixes the prefix issue and all the regressions comes from not
running tests using `run_test.py`.
#### Other Changes
* Added couple of `__init__.py` file since it's required to load tests.
* Added `py_status_code` to Aio rpc state.
* `code()` is expecting to return a python gRPC code but current
`status_code` is a Cython code.
* Added `libsqlite3-dev` to our dockers because it's required for
`coverage==7.2.0`.
* Renamed csds and admin test because test case file have to end with
`_test`:
1d136fd05f/src/python/grpcio_tests/tests/_loader.py (L26)
* Removed gevent test from `run_test.py` because Bazel gevent tests
should be good enough for us.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Yash Tibrewal
Xuan Wang
Esun Kim
Hannah Shi
Stanley Cheung
Matthew Stevenson
Eugene Ostroukhov
apolcyn
Nick Cooke
Matt Miller
Chad Wilson
David Chamberlin
gRPC Team Bot
Gregory Cooke
Craig Tiller
Adam Cozzette
Roman-Byshliaha-Bose
Yijie Ma