Follow up from #37917 to add `noexcept` to multiple Cython functions using `nogil` with a `void` return type.
Below are the performance hints that were encountered as part of the Cython translation which are solved by this PR:
The below performance hint still exists, and is not resolved.
This is because `noexcept` expects that the function doesn't raise an exception, or a raised exception is just displayed as a warning and not propagated. But `_poll` raises an `AssertionError` at `QUEUE_TIMEOUT`, and hence cannot use `noexcept` with `_poll`.
As [PR 37917](https://github.com/grpc/grpc/pull/37917) and this PR now solves the Cython Asyncio test timeouts caused by Cython upgrade, this PR also reverts the Bazel Cython downgrade PR #37884
### Testing for timeout
Tested using `bazel test -c dbg --runs_per_test=3000 --test_timeout=10 "//examples/python/auth:_auth_example_test"`
```
//examples/python/auth:_auth_example_test PASSED in 4.6s
Stats over 3000 runs: max = 4.6s, min = 2.6s, avg = 3.2s, dev = 0.2s
```
Closes#37922
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37922 from sreenithi:fix_bazel_cython_asyncio_timeout 2162cd28b1
PiperOrigin-RevId: 686524195
This PR adds templating for Python versions and updates the maximum supported Python version to 3.13. The following major changes related to templating are added:
- Minimum supported Python version and list of supported versions in `setup.py` are fetched using new template generated files called `python_version.py`
- Dockerfiles for the different Python Linux builds are now template generated.
- The "Supported Python Versions" section from READMEs of ancillary and main packages have been removed
Note: All the `python_version.py` files and Linux build `Dockerfiles` except `tools/dockerfile/grpc_artifact_python_linux_armv7/Dockerfile` in the PR are generated from the respective templates.
Further non-templated additions to add support for Python 3.13:
- install scripts and artifacts for windows, macos and linux are added manually. Later, these can be templated as well.
- updated cython bounds to 3.x
- updated twine version to solve [cgi module import error](https://github.com/pypa/twine/issues/1046)
- the twine update introduces a dependency on cryptography>=2.0. But the cryptography package doesn't support 32-bit Linux images and hence `twine check` has been disabled for x86 manylinux and x86 musllinux artifacts.
Closes#37643
PiperOrigin-RevId: 678954495
Change was created by the release automation script. See go/grpc-release.
Closes#37279
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37279 from drfloob:bump_dev_version_202407222027 4e6607411e
PiperOrigin-RevId: 654925894
Change was created by the release automation script. See go/grpc-release.
Closes#36892
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36892 from XuanWang-Amos:bump_dev_version_202406112301 9b2898d716
PiperOrigin-RevId: 643027546
Change was created by the release automation script. See go/grpc-release.
Closes#36544
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36544 from yashykt:bump_dev_version_202405061956 69ee5c869e
PiperOrigin-RevId: 631187829
Since we already finished v26.1 upgrade in https://github.com/grpc/grpc/pull/35796
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36230
PiperOrigin-RevId: 623960260
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36333
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36333 from yijiem:bump_dev_version_202404101721 0a7cc5b4b7
PiperOrigin-RevId: 623878150
### NOTE
* We shouldn't merge this PR until GCP cloud functions drops support for Python 3.7 ([Currently scheduled for GCF](https://cloud.google.com/functions/docs/runtime-support#python))
As part of supporting Python 3.12, we're now officially drop support for Python 3.7.
This PR:
* Changed supported Python version from 3.7 to 3.8 in README.
* Replaced distribution test image from `debian:buster` to `debian:bullseye` since the default Python version in buster is 3.7.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34450
PiperOrigin-RevId: 622899511
This is a prerequisite for the upcoming Protobuf v26 upgrade and has the following changes;
- Reduced the optimization level from `O3` to `O1` on Linux to workaround a segfault issue on `manylinux2014/x86`. This is believed to be caused by an aggressive but buggy optimization that `gcc` 10 made so workaround is to reduce the optimization level. `gcsio_tools` isn't that performance sensitive so we should be able to afford this change. (internal b/329134877)
- Added a Windows library dependency `Shell32.lib` to support the use of `CommandLineToArgvW`. (internal b/328455319)
Closes#36124
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36124 from veblush:grpcio-x e7b555bfcf
PiperOrigin-RevId: 616245600
The stubs generated by grpcio_tools should always be used with [the same or higher version of grpcio](https://github.com/grpc/grpc/blob/master/tools/distrib/python/grpcio_tools/setup.py#L313), this change will add a run time check for this requirement inside the generated stubs and therefor enforce this requirement.
Please note for now we're just printing a warning for incorrect usage, we'll **change it to an error** soon.
Example warning message:
```
/usr/local/google/home/xuanwn/workspace/misc/grpc/examples/python/helloworld/helloworld_pb2_grpc.py:21: RuntimeWarning: The grpc package installed is at version 1.60.1, but the generated code in helloworld_pb2_grpc.py depends on grpcio>=1.63.0.dev0. Please upgrade your grpc module to grpcio>=1.63.0.dev0 or downgrade your generated code using grpcio-tools<=1.60.1. This warning will become an error in 1.64.0, scheduled for release on May 14,2024.
```
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35906
PiperOrigin-RevId: 615659471
Protobuf is beginning to rely more on upb, pulling in more C files. Hence, grpcio_tool needs the following changes to absorb this.
- Changing the setup.py to support both C and C++ by removing explicit language=c++. Rather it can choose the right compiler by its extension.
- Adding build_extensions injection to deal with C/C++ option conflict as grpcio does.
- Adding protobuf and upb directories to the build script so that it can find newly added source files in protobuf.
Tested by https://github.com/grpc/grpc/pull/35796Closes#36074
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36074 from veblush:grpcio-tool-ready 84abce9083
PiperOrigin-RevId: 613975137
Discuss thread about this change: [link](https://mail.google.com/mail/u/0/#sent/QgrcJHsBpWNGRlrMktwbppGGfFTVCFLcQgL?compose=new)
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35849
PiperOrigin-RevId: 607144827
Change was created by the release automation script. See go/grpc-release.
Closes#35899
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35899 from eugeneo:bump_dev_version_202402131133 73950425c9
PiperOrigin-RevId: 606777850
We changed `pkg_resources.resource_filename` to `importlib.resources.files`, but the return of `resources.files()` API is a traversable object implementing a subset of the [pathlib.Path](https://docs.python.org/3/library/pathlib.html#pathlib.Path) interface instead of string, thus we're seeing errors like `AttributeError: 'PosixPath' object has no attribute 'rstrip'`.
This PR converts the result of `files()` to str to prevent those kinds of errors.
Test run:
* [x] [grpc/core/master/linux/grpc_interop_tocloud](http://sponge/ee5d493b-e23e-4358-8084-3dba1df1d42f)
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35810
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35810 from XuanWang-Amos:fix_import_pkg_resources_2 70fff7a7c3
PiperOrigin-RevId: 604398041
Change was created by the release automation script. See go/grpc-release.
On top of changes made by the script, `setup.py` had to be updated to have `/std:c11` option for MSVC to use c11 features for boringssl.
Closes#35768
PiperOrigin-RevId: 603403329
This PR is required in order to upgrade to the `master` branch version of protobuf. To do this, we upgrade to the latest version of `rules_python` by:
- Adding in explicit dependencies on all PyPi Bazel test dependencies
- Resolving the circular dependency this creates for `xds-protos` by giving it a Bazel build
- Generating the Python code for `xds-protos` as part of `generate-projects.sh` since doing so directly in Bazel would be highly fragily, dependent on many other projects' Bazel builds
Closes#34945
PiperOrigin-RevId: 599875020
Change was created by the release automation script. See go/grpc-release.
Closes#35580
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35580 from stanley-cheung:bump_dev_version_202401171835 b363888ca5
PiperOrigin-RevId: 599267642
`_parallel_compile_patch.py` exist in multiple places, we should use template system to generate the content.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34847
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34847 from XuanWang-Amos:use_template_for__parallel_compile_patch 15b98ea826
PiperOrigin-RevId: 590262888
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
Some user reported an issue that symbols from CoreFoundation are missing
when install grpc/grpc-tools inside Conda:
* https://github.com/grpc/grpc/issues/33714
* https://github.com/grpc/grpc/issues/34135
This PR adds the `-framework CoreFoundation` flag so no workarounds are
required from user.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Reverts grpc/grpc#34667
The change was reverted because it failed to import to g3, after some
changes, now it's safe to reapply those changes.
Tested by importing this PR internally, it passed presubmit:
cl/573836270
We're not running any test at all from `run_test.py` because of the way
we filter test cases:
1d136fd05f/src/python/grpcio_tests/tests/_runner.py (L137)
* `testcase_filter` is read from a json file (like [this
one](https://github.com/grpc/grpc/blob/master/src/python/grpcio_tests/tests/tests.json))
and test name is similar to `unit._metadata_test.MetadataTest`.
* `case.id()` is loaded by `iterate_suite_cases` and will always have a
prefix of `tests`, an example of case id will be:
`tests.unit._metadata_test.MetadataTest`.
Because of the prefix, none of the test case will be matched thus we're
not running any of the tests.
This PR fixes the prefix issue and all the regressions comes from not
running tests using `run_test.py`.
#### Other Changes
* Added couple of `__init__.py` file since it's required to load tests.
* Added `py_status_code` to Aio rpc state.
* `code()` is expecting to return a python gRPC code but current
`status_code` is a Cython code.
* Added `libsqlite3-dev` to our dockers because it's required for
`coverage==7.2.0`.
* Renamed csds and admin test because test case file have to end with
`_test`:
1d136fd05f/src/python/grpcio_tests/tests/_loader.py (L26)
* Removed gevent test from `run_test.py` because Bazel gevent tests
should be good enough for us.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Esun Kim
Marcel
Richard Belleville
Sreenithi Sridharan
Craig Tiller
apolcyn
Nathan Baulch
AJ Heller
Mark D. Roth
Eugene Ostroukhov
Xuan Wang
Yash Tibrewal
Yijie Ma
Fahad Ahammed
Stanley Cheung
Esun Kim
Adam Cozzette