We really should not have our own copy of these protos in the first
place, but there's some tech debt here that hasn't been cleaned up yet.
Until we do that, we should at least ensure that this tech debt doesn't
spread by having more code depend on them.
PiperOrigin-RevId: 683631165
This was put in as a temporary kludge to carry us until `work_serializer_dispatch` was ready, however since #37637 this is a no-op for all known use cases (and I think it's time for `work_serializer_dispatch` to stick).
Closes#37851
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37851 from ctiller:tc 8ae59415bd
PiperOrigin-RevId: 683255110
This PR introduces a new CONTRIBUTING.md file to the gRPC Python repository. The goal of this document is to provide clear and comprehensive guidelines for new contributors who are interested in contributing to the project.
- **Code of Conduct**: Establishes expectations for respectful and inclusive interactions within the community.
- **Guidelines**: Offers advice and best practices for contributing effectively, including how to find issues, get started with the codebase, and follow the project's standards.
- **Ways to Contribute**: Highlights the diverse ways in which people can contribute beyond just code, including documentation, community support, and testing.
- **Steps to Contributing**: Provides a step-by-step guide on how to set up a development environment, make changes, run tests, and submit a pull request.
- **Code Style**: Outlines the coding conventions and tools used in the project to ensure consistency.
Closes#36749
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36749 from sourabhsinghs:documentation/python-contributing-guide bcd1123742
PiperOrigin-RevId: 681889662
All known flakes/bugs have been fixed.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37839
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37839 from yousukseung:work_serializer_dispatch a82a068ebf
PiperOrigin-RevId: 681609545
* grpc_google_default_channel_credentials
* TlsServerCredentials
* grpc_alts(_server)_credentials
* UrlExternalAccountCredentials
* FileExternalAccountCredentials
* AwsExternalAccountCredentials
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37795
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37795 from AlienCookie:patch-1 34ffa5eca5
PiperOrigin-RevId: 681554756
This seems to be a vestige from around 8 years ago. The code has changed a lot since then, and this makes no sense anymore.
(Making this change separately to be safe.)
Closes#37824
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37824 from yashykt:RemoveUnnecessaryExecCtxStuff 97a7625033
PiperOrigin-RevId: 681184980
[PH2][Refactor] Creating call_tracer_wrapper.h and cc and moving code as-is into it. The content of the functions or classes has not changed.
In the next few iterations, more code will come into this file.
Closes#37786
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37786 from tanvi-jagtap:ph2_internal_h_split_01 31af14a4e7
PiperOrigin-RevId: 680861154
Fix https://github.com/grpc/grpc/issues/37701
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37822
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37822 from yijiem:debug-check-metadata 67df0da89c
PiperOrigin-RevId: 680734161
Since we don't actually use the `desc` parameter when building without latent_see we had an unused parameter error.
More though, I think the `GRPC_LATENT_SEE_METADATA_RAW` macro was flawed, as the inner `metadata` argument would only be created once, and so all uses would get tagged by whatever passed through `Flusher` first. Correct this by passing in the `latent_see::Metadata*` directly to the function.
Closes#37815
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37815 from ctiller:woah-bubba 606a6350fe
PiperOrigin-RevId: 680136826
This is the last piece of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Note that although this is the first use-case for this "blackboard" mechanism, we will also use it in the future for the xDS rate-limiting filter on the gRPC server side.
Closes#37646
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37646 from markdroth:gcp_auth_filter_state 72d0d96c79
PiperOrigin-RevId: 679707134
Allows use of the party <-> party wakeup batching stuff, which reduces threadhops drastically for this transport.
Closes#37078
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37078 from ctiller:chaotic-party-3 75c32e6a64
PiperOrigin-RevId: 679685211
Moving out related code into a separate file.
`internal.h -> stream_lists.h`
Closes#37775
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37775 from tanvi-jagtap:ph2_stream_lists_h 13b6030ceb
PiperOrigin-RevId: 679205824
This PR adds templating for Python versions and updates the maximum supported Python version to 3.13. The following major changes related to templating are added:
- Minimum supported Python version and list of supported versions in `setup.py` are fetched using new template generated files called `python_version.py`
- Dockerfiles for the different Python Linux builds are now template generated.
- The "Supported Python Versions" section from READMEs of ancillary and main packages have been removed
Note: All the `python_version.py` files and Linux build `Dockerfiles` except `tools/dockerfile/grpc_artifact_python_linux_armv7/Dockerfile` in the PR are generated from the respective templates.
Further non-templated additions to add support for Python 3.13:
- install scripts and artifacts for windows, macos and linux are added manually. Later, these can be templated as well.
- updated cython bounds to 3.x
- updated twine version to solve [cgi module import error](https://github.com/pypa/twine/issues/1046)
- the twine update introduces a dependency on cryptography>=2.0. But the cryptography package doesn't support 32-bit Linux images and hence `twine check` has been disabled for x86 manylinux and x86 musllinux artifacts.
Closes#37643
PiperOrigin-RevId: 678954495
[PH2][Refactor][is_client] Split two functions for better readability
1. send_initial_metadata_locked - Moved out the tracing part
2. grpc_chttp2_config_default_keepalive_args - Split the client and server path to reduce cyclomatic complexity.
Closes#37641
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37641 from tanvi-jagtap:remove_is_client_02 bfbc9361db
PiperOrigin-RevId: 678096501
Basic building block for retries, hedging: buffer outgoing messages & metadata, allow for replay whilst buffered (with a single reader able to read once buffering ends)
Closes#37448
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37448 from ctiller:once-again-into-the-breach-my-friends 79cb121054
PiperOrigin-RevId: 677959212
Log error message instead of crashing for this API misuse.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37764
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37764 from yijiem:report-different-gauge-wont-crash 1b6e912bfc
PiperOrigin-RevId: 677944595
Fix https://github.com/grpc/grpc/issues/37727.
A better idea might be to set up Fuzzing for these APIs to find those sort of things. Maybe that can be a next step if we want to further hardening things.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37737
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37737 from yijiem:memory-leak-alts-2 6be8a49e63
PiperOrigin-RevId: 677880955
Add a ValidateCredentials API to the TLS certificate provider interface. A user can call this API to check that the credentials currently held by the certificate provider instance are valid. The definition of "valid" depends on provider that is being used. For the static data and file watcher providers, "valid" means that the credentials consist of valid PEM.
~Currently there is no check to ensure that credentials consist of valid PEM blocks before a TLS handshake commences. This PR creates a static factory for FileWatcherCertificateProvider (and marks the constructor as deprecated) which performs this validation check. The analogous work for StaticDataCertificateProvider will be done in a follow-up PR.~
Closes#37565
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37565 from matthewstevenson88:filewatcher f223228023
PiperOrigin-RevId: 677847751
Some of our tests failed because RPC was canceled by server, error message:
```
I0000 00:00:1726598495.788119 64528 chttp2_transport.cc:1182] ipv4:127.0.0.1:41825: Got goaway [2] err=UNAVAILABLE:GOAWAY received; Error code: 2; Debug Text: Cancelling all calls {grpc_status:14, http2_error:2, created_time:"2024-09-17T18:41:35.78810597+00:00"}
```
It's possible that the request waited too long to be processed and thus canceled by server (default maximum time gRPC server allows a request to be received by the server but not handled is `30s`, controlled by `grpc.server_max_unrequested_time_in_server`).
This PR will print the test start/finish time so we know if we should increase `grpc.server_max_unrequested_time_in_server` time.
Example before change:
```
Testing gRPC Python...
SUCCESS tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_default
Running tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_disabled
SUCCESS tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_disabled
Running tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_enabled
```
Example after change:
```
[2024-09-20 18:32:15.228947]Testing gRPC Python...
[2024-09-20 18:32:15.237855]START tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_default
[2024-09-20 18:32:15.265477]SUCCESS tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_default[Duration: 0:00:00.027622s]
Running tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_disabled
[2024-09-20 18:32:15.278142]START tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_disabled
[2024-09-20 18:32:15.316065]FAILURE tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_disabled[Duration: 0:00:00.037911]
Running tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_enabled
[2024-09-20 18:32:15.328680]START tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_enabled
[2024-09-20 18:32:16.938745]SUCCESS tests.unit._metadata_flags_test.MetadataFlagsTest.test_call_wait_for_ready_enabled[Duration: 0:00:01.610065s]
```
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37761
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37761 from XuanWang-Amos:add_time_to_unit_test d2ecd8ce22
PiperOrigin-RevId: 677805375
Ensure OPENSSL global clean up happens after gRPC shutdown completes. OPENSSL registers an exit handler to clean up global objects, which may happen before gRPC removes all references to OPENSSL.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37768
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37768 from yousukseung:openssl-atexit-wait d3d1c964a8
PiperOrigin-RevId: 677284514
This is a retry of #37762 with a fix. We no longer hold the C mutex and GIL at the same to avoid a deadlock.
MetadataPluginCallCredentials passes a function pointer to C-core to be called during destruction. This function grabs GIL which may race between GIL destruction during process shutdown. Since GIL destruction happens after Python's exit handlers, we cana mark that Python is shutting down from an exit handler and don't grab GIL in this function afterwards using a C mutex.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37784
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37784 from yousukseung:python-atexit-retry b17addf805
PiperOrigin-RevId: 677280657
The following files have been moved:
- src/core/lib/avl/*
- src/core/lib/backoff/*
- src/core/lib/debug/event_log*
- src/core/lib/iomgr/gethostname*
- src/core/lib/iomgr/grpc_if_nametoindex*
- src/core/lib/matchers/*
- src/core/lib/uri/* (renamed from uri_parser.* to uri.*)
- src/core/lib/gprpp/* (existing src/core/util/time.cc was renamed to gpr_time.cc to avoid conflict)
Closes#36792
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36792 from markdroth:reorg_util d4e8996f48
PiperOrigin-RevId: 676947640
AJ Heller
Mark D. Roth
Craig Tiller
Alisha Nanda
Sourabh Singh
Yousuk Seung
Kukki
Yash Tibrewal
Tanvi Jagtap
Yijie Ma
Esun Kim
Tanvi Jagtap
Vignesh Babu
Sreenithi Sridharan
erm-g
Eugene Ostroukhov
Matthew Stevenson
Xuan Wang