This will disable the jobs in both bazel and cmake builds, which is necessary for our CI to remain happy. These end2end tests were enabled on Windows at some point without any notice (they had been intentionally disabled for a while), but the RBE jobs have been silently failing for 7 months, so it's unclear when that happened.
These failures still need to be examined so we can re-enable these tests.
Closes#37983
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37983 from drfloob:winguh 64a62fd5b9
PiperOrigin-RevId: 689118231
Possible fix for sporadic flakes like `detected corruption in /Volumes/BuildData/tmpfs/altsrc/github/grpc/workspace_ruby_macos_dbg_native/src/ruby/lib/grpc/grpc_c.bundle` we've been seeing in ruby.
`rake` (invoked by run_ruby.sh) "shouldn't" be modifying the binary if it's already been built (which is the case in these tests) but isn't really guaranteed not to. Running rspec directly ensures that we don't accidentally write to any pre-built artifacts while tests are possibly using them.
A side benefit here is to get better test reporting granularity.
Closes#37975
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37975 from apolcyn:ruby_flake_attempt c03931dfa4
PiperOrigin-RevId: 689064491
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37973
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37973 from yijiem:core-end2end-windows-hack a2da4ae4eb
PiperOrigin-RevId: 688727248
The default versions to run tests on master using `run_tests.py` are usually set to min and max supported Python versions. Looks like I missed updating the max version to Python 3.13 when adding support recently
Closes#37945
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37945 from sreenithi:add_missing_py313_test_config 708aa70f2b
PiperOrigin-RevId: 688702425
Three problems:
1. We have an owning waker, but on the `Expire` path we never wake it, leading to calls being stranded until the pending timer runs out - instead we now call Finish and have it always wake things up (slightly more expensive in shutdown case, but not on the fast path)
2. Avoid a race condition whereby two threads could wake the same waker
3. Don't add new requests to the pending queue after we've removed all requests
Closes#37972
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37972 from ctiller:flake-fightas-21 2bbd1cf667
PiperOrigin-RevId: 688310530
I hit this crash working on fork support but there's a chance it happens if the file descriptor becomes bad for some other reason.
Closes#37952
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37952 from eugeneo:listener-no-address-crash 1e2e82f8e6
PiperOrigin-RevId: 688238823
Skipping the `tests_aio.unit.channel_ready_test.TestChannelReady.channel_ready_blocked` unit test due to a flake of the test not timing out and raising TimeoutError as expected
Can be reverted once #37949 is investigated and fixed
Closes#37948
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37948 from sreenithi:check_aio_channel_ready_test f48edea5f2
PiperOrigin-RevId: 688061272
This log can be hit under normal-ish circumstances, e.g. if the handshaker service fails to respond or is unreachable. For that reason, it should not be an error log.
Closes#37962
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37962 from matthewstevenson88:downgrade-log 36d4f006a0
PiperOrigin-RevId: 687986132
Porting from #37829.
This ensures that we wait to create the stream to the handshaker service until handshake frames arrive from the client. Without this change, a TCP connection to the ALTS server triggers the stream to the handshaker service to be created, even if no handshake frames have arrived from the client. This waste resources and can potentially trigger the ALTS server to freeze up, because there is a cap on the number of concurrent ALTS handshakes that a server can perform.
Closes#37961
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37961 from matthewstevenson88:alts-fix f8f07e59bb
PiperOrigin-RevId: 687977457
This fixes connecting to a local ATLS server with grpc_cli.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37950
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37950 from rainwoodman:patch-1 1172618fb2
PiperOrigin-RevId: 687975188
Use the POSIX code like FreeBSD does.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37700
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37700 from 0-wiz-0:master 0846ff5ef0
PiperOrigin-RevId: 687539488
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37901
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37901 from yijiem:dns-migration-chttp2-server-2 70d29b3b6c
PiperOrigin-RevId: 687466646
Don't complete writes of messages until they make it to the transports outbound loop. Since payloads could be large this introduces just enough pushback that, once #37868 goes in also we should be able to sense when a transport is busy writing and stop sending at higher layers.
Closes#37894
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37894 from ctiller:send-acked 0cb3d7f8ad
PiperOrigin-RevId: 686689473
This is missing in v3 vs v2
- in v2 we had Pipe setup so that multiple Pipe stages could be chained and only complete when the last stage had passed flow control, whereas in v3 the top stage will start accepting requests as soon as the first stage in the pipeline takes the message.
Closes#37868
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37868 from ctiller:drizzling 69209da8a7
PiperOrigin-RevId: 686652402
Passed interop test:
- [x] [grpc/core/master/linux/psm-csm-python](https://source.cloud.google.com/results/invocations/b9ba256b-31a9-4002-bd59-b21817aa9978)
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37837
PiperOrigin-RevId: 686643728
Currently the destructive reclaimer single threaded cancels existing requests, but we admit new rpcs on every channel (to be eventually cancelled, probably).
We've got evidence that this (shockingly) doesn't scale and senders can easily overwhelm and oom a server.
Instead under this experiment now we'll always reject new work under very high load, and allow the reclaimer to mop up any remaining work to get back to within bounds.
Closes#37927
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37927 from ctiller:fast_reject 835726473a
PiperOrigin-RevId: 686553599
You would think `always_inline` implies inline, but GCC complains loudly (`-Wattribute`) if the `always_inline` attribute is not also accompanied by the normal `inline` keyword.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37834
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37834 from benjaminp:always-inline 3ecfaa6d50
PiperOrigin-RevId: 686553476
`std::string_view` is not allowed to use per Google C++ style guide. This test will prevent us from accidentally using it after C++17 upgrade.
Closes#37930
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37930 from veblush:string-view 88f56c12ba
PiperOrigin-RevId: 686526151
Follow up from #37917 to add `noexcept` to multiple Cython functions using `nogil` with a `void` return type.
Below are the performance hints that were encountered as part of the Cython translation which are solved by this PR:
The below performance hint still exists, and is not resolved.
This is because `noexcept` expects that the function doesn't raise an exception, or a raised exception is just displayed as a warning and not propagated. But `_poll` raises an `AssertionError` at `QUEUE_TIMEOUT`, and hence cannot use `noexcept` with `_poll`.
As [PR 37917](https://github.com/grpc/grpc/pull/37917) and this PR now solves the Cython Asyncio test timeouts caused by Cython upgrade, this PR also reverts the Bazel Cython downgrade PR #37884
### Testing for timeout
Tested using `bazel test -c dbg --runs_per_test=3000 --test_timeout=10 "//examples/python/auth:_auth_example_test"`
```
//examples/python/auth:_auth_example_test PASSED in 4.6s
Stats over 3000 runs: max = 4.6s, min = 2.6s, avg = 3.2s, dev = 0.2s
```
Closes#37922
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37922 from sreenithi:fix_bazel_cython_asyncio_timeout 2162cd28b1
PiperOrigin-RevId: 686524195
To prepare for the upcoming upgrade to C++17, the following changes were made:
Increased minimum supported operating system versions:
- iOS: 11 (previously 10)
- macOS: 10.14 (previously 10.12)
- tvOS: 13.0 (previously 12.0)
In addition to this, version requirements across different projects were updated to use these for consistency.
Closes#37931
PiperOrigin-RevId: 686519641
Currently this is still causing log spam for ruby.
Though this is in core, I believe this code path is specific to ruby.
Closes#37633
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37633 from apolcyn:reduce_log ca58824fdd
PiperOrigin-RevId: 686280644
Corrects bazel run command in README.
It does not match the bazel command in the BUILD file.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37664
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37664 from nolasconapoleao:fix_interceptor_readme 42d828f2ec
PiperOrigin-RevId: 686223940
This migrates all of the xDS unit tests except for the fuzzer, which I'll get in a subsequent PR.
This also does not include the xDS e2e tests, which I will also do separately.
Closes#37896
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37896 from markdroth:xds_tests_use_real_protos2 de568b4e53
PiperOrigin-RevId: 686197812
This aims to resolve segfaults encountered when rebuilding ARM64 PHP Docker images. This segfaults came from gcc during ARM64 builds using QEMU. To workaround this problem, the PHP Docker images have been restructured to use apt-get for installation, thereby avoiding the use of gcc. This also needed to upgrade Debian and PHP to their latest versions, aligning with our minimum supported PHP 8.
Closes#37895
PiperOrigin-RevId: 686162640
The existing code fails to enable vsock on Android because it tries to enable vsock on Android when:
a) a certain Linux version is detected
b) and a certain libc is present.
For a), it requires that `<linux/version.h>` is included.
For b), Android does not use glibc. Instead `AF_VSOCK` is included since NDK version 14.
This commit fixes both issues by including the needed Linux header and detecting the `__NDK_MAJOR__` version.
For context, AOSP, has it always enabled:
06fb97455f%5E%21/src/core/lib/iomgr/port.h
But this patch compiles with older NDK version, in case someone is still using them.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37733
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37733 from ricardoquesadawork:android 9b27187337
PiperOrigin-RevId: 685868655
This eliminates the need for the `grpc_cc_proto_library` bazel BUILD rule introduced in #37863.
To make this work, I had to upgrade several bazel dependencies and apply a patch to rules_go to work around https://github.com/bazelbuild/bazel/issues/11636.
Closes#37902
PiperOrigin-RevId: 685868647
Instead of getting value of `csm_mesh_id` from the bootstrap file, get it from the env var `CSM_MESH_ID`
Closes#37801
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37801 from yashykt:CsmMeshIdChange d0f149e023
PiperOrigin-RevId: 685864223
### What's happening
Some of our asyncio tests began timing out following a Cython upgrade to 3.0. This issue occurs consistently across both our Bazel and setup.py test environments.
### Why the time out
After some investigation, we found that our code here:4ffcdd4ab7/src/python/grpcio/grpc/_cython/_cygrpc/aio/completion_queue.pyx.pxi (L115-L116)
Was translated to this in Cython 0.29:
```
__pyx_f_7_cython_6cygrpc__unified_socket_write(__pyx_v_self->_write_fd);
```
And it changed to this in Cython 3.0:
```
__pyx_f_7_cython_6cygrpc__unified_socket_write(__pyx_v_self->_write_fd); if (unlikely(__Pyx_ErrOccurredWithGIL())) __PYX_ERR(7, 136, __pyx_L1_error)
```
Which indicates that this `nogil` function `_unified_socket_write` now requires GIL.
### What's new in Cython 3
* Cython 3 `cdef` functions with `void` return type will default to use `except *` as exception specification.
* If function have `void` return type and defined as `nogil`, Cython will always re-acquire the GIL after the function call to check if an exception has been raised.
* In some cases, this will cause a deadlock, especially if the function was called inside another `nogil` function.
### What's the fix
* This PR changes those functions to use `noexcept` as exception specification since we don't expect them to throw any exception, and this is also the suggested workarounds in Cython documentation: https://cython.readthedocs.io/en/latest/src/userguide/language_basics.html#error-return-values
### Test
* Tested locally by running `bazel test`, time our rate decreased from 5% to 0.3% in 3000 runs and 10s test time out.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37917
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37917 from XuanWang-Amos:fix_cython_aio 063d27aee9
PiperOrigin-RevId: 685851320
Craig Tiller
AJ Heller
Esun Kim
apolcyn
Yijie Ma
Sreenithi Sridharan
Mark D. Roth
Eugene Ostroukhov
Richard Belleville
Matthew Stevenson
Sourabh Singh
Yu Feng
Thomas Klausner
Yash Tibrewal
Xuan Wang
Benjamin Peterson
Nolasco Napoleão
dependabot[bot]
Nana Pang
Ricardo Quesada