It's not obvious to me what the reason for the increase in test time is, but it seems that we were already at ~3hr45min earlier and now we are timing out at 4 hours. Increasing to 6 hours.
Closes#37989
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37989 from yashykt:IncreaseWindowsTimeout a0daa97486
PiperOrigin-RevId: 689416681
Possible fix for sporadic flakes like `detected corruption in /Volumes/BuildData/tmpfs/altsrc/github/grpc/workspace_ruby_macos_dbg_native/src/ruby/lib/grpc/grpc_c.bundle` we've been seeing in ruby.
`rake` (invoked by run_ruby.sh) "shouldn't" be modifying the binary if it's already been built (which is the case in these tests) but isn't really guaranteed not to. Running rspec directly ensures that we don't accidentally write to any pre-built artifacts while tests are possibly using them.
A side benefit here is to get better test reporting granularity.
Closes#37975
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37975 from apolcyn:ruby_flake_attempt c03931dfa4
PiperOrigin-RevId: 689064491
The default versions to run tests on master using `run_tests.py` are usually set to min and max supported Python versions. Looks like I missed updating the max version to Python 3.13 when adding support recently
Closes#37945
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37945 from sreenithi:add_missing_py313_test_config 708aa70f2b
PiperOrigin-RevId: 688702425
`std::string_view` is not allowed to use per Google C++ style guide. This test will prevent us from accidentally using it after C++17 upgrade.
Closes#37930
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37930 from veblush:string-view 88f56c12ba
PiperOrigin-RevId: 686526151
Follow up from #37917 to add `noexcept` to multiple Cython functions using `nogil` with a `void` return type.
Below are the performance hints that were encountered as part of the Cython translation which are solved by this PR:
The below performance hint still exists, and is not resolved.
This is because `noexcept` expects that the function doesn't raise an exception, or a raised exception is just displayed as a warning and not propagated. But `_poll` raises an `AssertionError` at `QUEUE_TIMEOUT`, and hence cannot use `noexcept` with `_poll`.
As [PR 37917](https://github.com/grpc/grpc/pull/37917) and this PR now solves the Cython Asyncio test timeouts caused by Cython upgrade, this PR also reverts the Bazel Cython downgrade PR #37884
### Testing for timeout
Tested using `bazel test -c dbg --runs_per_test=3000 --test_timeout=10 "//examples/python/auth:_auth_example_test"`
```
//examples/python/auth:_auth_example_test PASSED in 4.6s
Stats over 3000 runs: max = 4.6s, min = 2.6s, avg = 3.2s, dev = 0.2s
```
Closes#37922
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37922 from sreenithi:fix_bazel_cython_asyncio_timeout 2162cd28b1
PiperOrigin-RevId: 686524195
This aims to resolve segfaults encountered when rebuilding ARM64 PHP Docker images. This segfaults came from gcc during ARM64 builds using QEMU. To workaround this problem, the PHP Docker images have been restructured to use apt-get for installation, thereby avoiding the use of gcc. This also needed to upgrade Debian and PHP to their latest versions, aligning with our minimum supported PHP 8.
Closes#37895
PiperOrigin-RevId: 686162640
This eliminates the need for the `grpc_cc_proto_library` bazel BUILD rule introduced in #37863.
To make this work, I had to upgrade several bazel dependencies and apply a patch to rules_go to work around https://github.com/bazelbuild/bazel/issues/11636.
Closes#37902
PiperOrigin-RevId: 685868647
This will allow latent_see to be used with different build configurations in the future (and eventually perhaps defaulted on for some CI runs to aid postmortem debugging of test failures)
Closes#37897
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37897 from ctiller:latent-peek 52cbf36ce5
PiperOrigin-RevId: 684981143
This is a trial baloon to see if we can actually make this work. If it does, I'll change the remaining xDS tests to use the real xDS protos and completely remove our local copies.
Closes#37863
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37863 from markdroth:xds_tests_use_real_protos 3ad2fe12be
PiperOrigin-RevId: 684877750
I'm continuing to look into some flakes here, but in the meantime these shouldn't halt submissions. Marking them flaky.
Closes#37880
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37880 from ctiller:mark-flaky 27427c7978
PiperOrigin-RevId: 684526341
With the CL-first approach, the docker test configs for Binder need to be deleted before the Binder code and tests themselves can be deleted in the next step. Sanity checks fail otherwise.
Closes#37862
PiperOrigin-RevId: 683691175
[PH2][Refactor] Creating call_tracer_wrapper.h and cc and moving code as-is into it. The content of the functions or classes has not changed.
In the next few iterations, more code will come into this file.
Closes#37786
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37786 from tanvi-jagtap:ph2_internal_h_split_01 31af14a4e7
PiperOrigin-RevId: 680861154
IDK how many times I've messed this up over the years accidentally and had to spend thoughts trying to fix it... when really we could do that trivially with code.
Closes#37811
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37811 from ctiller:bamboozled 0cfbeae925
PiperOrigin-RevId: 679741188
This is the last piece of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Note that although this is the first use-case for this "blackboard" mechanism, we will also use it in the future for the xDS rate-limiting filter on the gRPC server side.
Closes#37646
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37646 from markdroth:gcp_auth_filter_state 72d0d96c79
PiperOrigin-RevId: 679707134
Building armv7 artifacts after adding Python 3.13 is failing with a timeout. Hence increasing timeout to 2 hours.
Closes#37807
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37807 from sreenithi:increase_armv7_artifact_build_timeout 60034378f5
PiperOrigin-RevId: 679608393
Moving out related code into a separate file.
`internal.h -> stream_lists.h`
Closes#37775
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37775 from tanvi-jagtap:ph2_stream_lists_h 13b6030ceb
PiperOrigin-RevId: 679205824
This PR adds templating for Python versions and updates the maximum supported Python version to 3.13. The following major changes related to templating are added:
- Minimum supported Python version and list of supported versions in `setup.py` are fetched using new template generated files called `python_version.py`
- Dockerfiles for the different Python Linux builds are now template generated.
- The "Supported Python Versions" section from READMEs of ancillary and main packages have been removed
Note: All the `python_version.py` files and Linux build `Dockerfiles` except `tools/dockerfile/grpc_artifact_python_linux_armv7/Dockerfile` in the PR are generated from the respective templates.
Further non-templated additions to add support for Python 3.13:
- install scripts and artifacts for windows, macos and linux are added manually. Later, these can be templated as well.
- updated cython bounds to 3.x
- updated twine version to solve [cgi module import error](https://github.com/pypa/twine/issues/1046)
- the twine update introduces a dependency on cryptography>=2.0. But the cryptography package doesn't support 32-bit Linux images and hence `twine check` has been disabled for x86 manylinux and x86 musllinux artifacts.
Closes#37643
PiperOrigin-RevId: 678954495
1. Added script to install all dependencies then run all unit tests for python grpcio.
Closes#37070
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37070 from sourabhsinghs:feature/developer-setup-script 954892f20a
PiperOrigin-RevId: 678718224
Basic building block for retries, hedging: buffer outgoing messages & metadata, allow for replay whilst buffered (with a single reader able to read once buffering ends)
Closes#37448
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37448 from ctiller:once-again-into-the-breach-my-friends 79cb121054
PiperOrigin-RevId: 677959212
Passing manual build: https://btx.cloud.google.com/invocations/9adb3cd8-628c-42e5-a7a3-c2ba1ab6127f
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37723
PiperOrigin-RevId: 677900305
The following files have been moved:
- src/core/lib/avl/*
- src/core/lib/backoff/*
- src/core/lib/debug/event_log*
- src/core/lib/iomgr/gethostname*
- src/core/lib/iomgr/grpc_if_nametoindex*
- src/core/lib/matchers/*
- src/core/lib/uri/* (renamed from uri_parser.* to uri.*)
- src/core/lib/gprpp/* (existing src/core/util/time.cc was renamed to gpr_time.cc to avoid conflict)
Closes#36792
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36792 from markdroth:reorg_util d4e8996f48
PiperOrigin-RevId: 676947640
`grpc/core/master/macos/grpc_basictests_python` test is flaky, all test cases passed by the test itself [failed with timeout](https://btx.cloud.google.com/invocations/33bdec48-e17b-431b-b473-c8a335e1650c/log):
```
2024-09-11 16:48:57,385 TIMEOUT: py38.asyncio.tests_aio.unit.done_callback_test.TestServerSideDoneCallback [pid=11128, time=486.5sec]
```
This PR does two things:
1. Increase timeout for test case from `480s` to `600s`.
2. Add timestamp to test starting and passing so we know which test case takes a long time to run.
Log before timestamp change:
```
Running tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_stream_stream
Running tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_stream_unary
Running tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_unary_stream
Running tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_unary_unary
Testing gRPC Python...
SUCCESS tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_error_in_callback
SUCCESS tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_error_in_handler
SUCCESS tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_stream_stream
SUCCESS tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_stream_unary
SUCCESS tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_unary_stream
SUCCESS tests_aio.unit.done_callback_test.TestServerSideDoneCallback.test_unary_unary
```
Log after timestamp change:
```
Running tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_stream_stream
Running tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_stream_unary
Running tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_unary_stream
Running tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_unary_unary
[2024-09-12 10:47:13.877185]Testing gRPC Python...
[2024-09-12 10:47:13.901135]SUCCESS tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_add_after_done
[2024-09-12 10:47:13.933705]SUCCESS tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_stream_stream
[2024-09-12 10:47:13.962419]SUCCESS tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_stream_unary
[2024-09-12 10:47:13.990368]SUCCESS tests_aio.unit.done_callback_test.TestClientSideDoneCallback.test_unary_stream
```
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37703
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37703 from XuanWang-Amos:add_test_timestamp 4e13d29021
PiperOrigin-RevId: 675703890
Eric Anderson
Mark D. Roth
Esun Kim
Vignesh Babu
Marcel
apolcyn
Richard Belleville
Yash Tibrewal
Craig Tiller
Sreenithi Sridharan
AJ Heller
Arjan Singh Bal
Tanvi Jagtap
Paulo Castello da Costa
erm-g
Sourabh Singh
Hannah Shi
Xuan Wang