This eliminates the need for the `grpc_cc_proto_library` bazel BUILD rule introduced in #37863.
To make this work, I had to upgrade several bazel dependencies and apply a patch to rules_go to work around https://github.com/bazelbuild/bazel/issues/11636.
Closes#37902
PiperOrigin-RevId: 685868647
Instead of getting value of `csm_mesh_id` from the bootstrap file, get it from the env var `CSM_MESH_ID`
Closes#37801
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37801 from yashykt:CsmMeshIdChange d0f149e023
PiperOrigin-RevId: 685864223
### What's happening
Some of our asyncio tests began timing out following a Cython upgrade to 3.0. This issue occurs consistently across both our Bazel and setup.py test environments.
### Why the time out
After some investigation, we found that our code here:4ffcdd4ab7/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi (L115-L116)
Was translated to this in Cython 0.29:
```
__pyx_f_7_cython_6cygrpc__unified_socket_write(__pyx_v_self->_write_fd);
```
And it changed to this in Cython 3.0:
```
__pyx_f_7_cython_6cygrpc__unified_socket_write(__pyx_v_self->_write_fd); if (unlikely(__Pyx_ErrOccurredWithGIL())) __PYX_ERR(7, 136, __pyx_L1_error)
```
Which indicates that this `nogil` function `_unified_socket_write` now requires GIL.
### What's new in Cython 3
* Cython 3 `cdef` functions with `void` return type will default to use `except *` as exception specification.
* If function have `void` return type and defined as `nogil`, Cython will always re-acquire the GIL after the function call to check if an exception has been raised.
* In some cases, this will cause a deadlock, especially if the function was called inside another `nogil` function.
### What's the fix
* This PR changes those functions to use `noexcept` as exception specification since we don't expect them to throw any exception, and this is also the suggested workarounds in Cython documentation: https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#error-return-values
### Test
* Tested locally by running `bazel test`, time our rate decreased from 5% to 0.3% in 3000 runs and 10s test time out.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37917
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37917 from XuanWang-Amos:fix_cython_aio 063d27aee9
PiperOrigin-RevId: 685851320
There's a timeout flake when running windows tests for the Static CRL Provider, let's see if increasing the timeout helps
Closes#37915
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37915 from gtcooke94:increase_test_deadline 0386aab786
PiperOrigin-RevId: 685752852
These look too large for the configured timeouts internally... will revisit later once the system is starting to be more ready.
Closes#37905
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37905 from ctiller:lol-nope 6d44e515b3
PiperOrigin-RevId: 685223450
We'll probably disable some next week :)
But I want to watch a good selection and refine criteria for acceptance.
Closes#37903
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37903 from ctiller:ALL-the-things 5f829db870
PiperOrigin-RevId: 685010911
This will allow latent_see to be used with different build configurations in the future (and eventually perhaps defaulted on for some CI runs to aid postmortem debugging of test failures)
Closes#37897
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37897 from ctiller:latent-peek 52cbf36ce5
PiperOrigin-RevId: 684981143
So far missing for HTTP/2 style flow control has been a primitive to query whether there's a receiver for flow control data at the other end of the message pipes.
Here I'm updating the state machine accessors to accommodate that functionality.
No new states were needed.
Whilst here, document the current member functions on `CallState`.
Closes#37867
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37867 from ctiller:like-the-river c9814c737d
PiperOrigin-RevId: 684972125
Previously if a request was cancelled whilst being matched the matcher would leak a ref to the call forever.
Note that the only change here is `s/SpawnGuarded/SpawnGuardedUntilCallCompletes` and clang-format noise.
Closes#37886
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37886 from ctiller:flake-fightas-18 edc045102b
PiperOrigin-RevId: 684955517
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37883
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37883 from yousukseung:work_serializer_dispatch_extend 04c4602b73
PiperOrigin-RevId: 684887827
This is a trial baloon to see if we can actually make this work. If it does, I'll change the remaining xDS tests to use the real xDS protos and completely remove our local copies.
Closes#37863
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37863 from markdroth:xds_tests_use_real_protos 3ad2fe12be
PiperOrigin-RevId: 684877750
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37853
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37853 from yijiem:dns-migration-chttp2-server b830720b20
PiperOrigin-RevId: 684631881
Prior to this change events could conspire such that newly read streams got added after the AbortWithError() code ran, and so those calls would be orphaned in the transport forever - continuing to hold a ref.
Closes#37887
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37887 from ctiller:flake-fightas-20 2648d7f37f
PiperOrigin-RevId: 684609806
This reverts commit e8aa408bba.
It looks like this increased flakiness... I'm going to roll forward the separate pieces after this.
Closes#37882
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37882 from ctiller:flake-fightas-17 6e666cd985
PiperOrigin-RevId: 684577945
We're seeing some Bazel tests timing out due to this upgrade, revert this for now until we resolved the issue.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37884
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37884 from XuanWang-Amos:revert_cython_upgrade 0071889b5c
PiperOrigin-RevId: 684538459
I'm continuing to look into some flakes here, but in the meantime these shouldn't halt submissions. Marking them flaky.
Closes#37880
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37880 from ctiller:mark-flaky 27427c7978
PiperOrigin-RevId: 684526341
I'm unable to reproduce some of the flakiness here. Enabling tracers to get more information.
Closes#37875
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37875 from yashykt:MoreLogsInXdsE2ETest 4a7eb67202
PiperOrigin-RevId: 684201791
1. Simplify refcounting in the server by not keeping a reference to a call around when it's not necessary
2. Handle the case that we get a NewStream on the server transport after the transport is closed
3. If a call is cancelled whilst matching in the server, ensure that we clean that up
Closes#37865
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37865 from ctiller:flake-fightas-16 b3dfc1e9b1
PiperOrigin-RevId: 684105108
With the CL-first approach, the docker test configs for Binder need to be deleted before the Binder code and tests themselves can be deleted in the next step. Sanity checks fail otherwise.
Closes#37862
PiperOrigin-RevId: 683691175
We really should not have our own copy of these protos in the first
place, but there's some tech debt here that hasn't been cleaned up yet.
Until we do that, we should at least ensure that this tech debt doesn't
spread by having more code depend on them.
PiperOrigin-RevId: 683631165
This was put in as a temporary kludge to carry us until `work_serializer_dispatch` was ready, however since #37637 this is a no-op for all known use cases (and I think it's time for `work_serializer_dispatch` to stick).
Closes#37851
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37851 from ctiller:tc 8ae59415bd
PiperOrigin-RevId: 683255110
This allows to restrict CQ based generic stub/service use.
See https://github.com/grpc/proposal/pull/426
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37838
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37838 from yousukseung:generic-stub 3e7c558372
PiperOrigin-RevId: 681932616
In some rare occasions on Win machines (0,3-0,4%), the tests are stuck when we execute the loop of 10 DoRpc calls. We receive Deadline Exceeded for such cases. The PR bumps the deadline from 10 to 60s (no flakes for --runs_per_test=10000).
Closes#37844
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37844 from erm-g:seqFix 8644db8194
PiperOrigin-RevId: 681891281
This PR introduces a new CONTRIBUTING.md file to the gRPC Python repository. The goal of this document is to provide clear and comprehensive guidelines for new contributors who are interested in contributing to the project.
- **Code of Conduct**: Establishes expectations for respectful and inclusive interactions within the community.
- **Guidelines**: Offers advice and best practices for contributing effectively, including how to find issues, get started with the codebase, and follow the project's standards.
- **Ways to Contribute**: Highlights the diverse ways in which people can contribute beyond just code, including documentation, community support, and testing.
- **Steps to Contributing**: Provides a step-by-step guide on how to set up a development environment, make changes, run tests, and submit a pull request.
- **Code Style**: Outlines the coding conventions and tools used in the project to ensure consistency.
Closes#36749
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36749 from sourabhsinghs:documentation/python-contributing-guide bcd1123742
PiperOrigin-RevId: 681889662
All known flakes/bugs have been fixed.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37839
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37839 from yousukseung:work_serializer_dispatch a82a068ebf
PiperOrigin-RevId: 681609545
* grpc_google_default_channel_credentials
* TlsServerCredentials
* grpc_alts(_server)_credentials
* UrlExternalAccountCredentials
* FileExternalAccountCredentials
* AwsExternalAccountCredentials
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37795
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37795 from AlienCookie:patch-1 34ffa5eca5
PiperOrigin-RevId: 681554756
Mark D. Roth
Yash Tibrewal
Xuan Wang
Craig Tiller
AJ Heller
Gregory Cooke
Yijie Ma
Yousuk Seung
Richard Belleville
Nana Pang
Nathan Baulch
Kannan J
Alisha Nanda
Alexander Polcyn
erm-g
Sourabh Singh
Kukki