`//src/python/grpcio_tests/tests/unit:_contextvars_propagation_test` is very flaky, mainly in two ways:
1. Failing with error `Error in bind for address '/tmp/grpc_fullstack_test.sock': Address already in use`.
2. Failing with timeout without any error.
#### Address already in use error
This is because we're reusing the same path for all test cases: 5011420f16/src/python/grpcio_tests/tests/unit/_contextvars_propagation_test.py (L31)
#### Timeout error
We're deleting tmp file after test is done:
5011420f16/src/python/grpcio_tests/tests/unit/_contextvars_propagation_test.py (L64-L66)
This might cause Core fail to connect to channel with error: `connect failed: addr: unix:/tmp/grpc_fullstack_test.sock error: No such file or directory`, Core will keep retrying and thus causing the test to timeout.
To make things worse, we're using multiple threads in one of the test case, leading to an even higher rate of flakiness.
This PR fix the issue by using different address for different test runs.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#38076
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38076 from XuanWang-Amos:fix_contextvar_test 93ab2b350f
PiperOrigin-RevId: 693812629
This reverts commit 574b19ec31.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#38074
PiperOrigin-RevId: 693803071
This log can be hit under normal circumstances (e.g. a client has an expired cert and authenticates to the server), so this should be an INFO-level log rather than an ERROR-level log.
Closes#38058
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38058 from matthewstevenson88:downgrade23 1cbdd5a3e7
PiperOrigin-RevId: 693375018
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#38056
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38056 from yijiem:ee-dns-non-client-channel-chaotic 74c3b20731
PiperOrigin-RevId: 693112346
This might not be needed? (From https://github.com/grpc/grpc/issues/37976)
- Boringssl commit requiring -msse2: 56d3ad9d23 (diff-f628d148f94bbab9e22a1ad426ccd94311f1fb87bbe5cc533cb85aee18b07a20R248)
- gRPC change to add -msse2 https://github.com/grpc/grpc/pull/36089
---
Answer to the quesiton above is yes; The -msse2 option remains necessary for gRPC on i686 due to BoringSSL's requirements. However, the existing CMake condition for this option was too broad, potentially including ARM architectures where SSE2 isn't supported, leading to compilation errors. I've refined the condition to specifically target 32-bit x86 architectures.
Furthermore, to ensure accurate architecture detection within our dockerized tests, I've configured x86 tests to utilize the linux32 command. This ensures that uname -a correctly reports i686, allowing gRPC's CMake to identify the architecture and apply the -msse2 option as needed.
It's important to note that RBE overrides the default entrypoint, so RBE-based tests must explicitly invoke linux32 even if the Docker image already has it set.
Fixes https://github.com/grpc/grpc/issues/37976Closes#38024
PiperOrigin-RevId: 693026079
We're about to completely change the wire format here... land one additional copy of the transport and tests as a hedge against bugs. Enable the hedge with an experiment.
Closes#38026
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38026 from ctiller:legacy-admission 5a32bb105d
PiperOrigin-RevId: 692984545
Improve metadata redaction comment to help people who are seeing the redaction statement in their logs.
Closes#38033
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38033 from tanvi-jagtap:improve_redaction_comment 18ba7e18c9
PiperOrigin-RevId: 692906001
[PH2][NewFile][ClassStructure][Important] Add client and server class
1. New classes Http2ServerTransport and Http2ClientTransport
2. Similar to the classes in [Chaotic Good Client Transport](https://github.com/grpc/grpc/blob/master/src/core/ext/transport/chaotic_good/client_transport.h) and [Chaotic Good Server Transport](https://github.com/grpc/grpc/blob/master/src/core/ext/transport/chaotic_good/server_transport.h)
3. Added new Test files. For now, the 2 new tests just call the constructor of Http2ServerTransport and Http2ClientTransport.
Tested locally using
```
CC=cc bazel test --test_output=all -c dbg --config=asan --verbose_failures //test/core/transport/chttp2:http2_client_transport_test
```
```
CC=cc bazel test --test_output=all -c dbg --config=asan --verbose_failures //test/core/transport/chttp2:http2_server_transport_test
```
Closes#37840
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37840 from tanvi-jagtap:ph2_add_client_server_class c6c3a0d5fb
PiperOrigin-RevId: 692824127
The Windows RBE test has been using Bazel 7.3.1 for running tests.
However, the RBE configuration itself was built with an older Bazel
version (6.3.2). While this hasn't caused any issues so far, it's best
to use the same Bazel version (7.3.1) for both building the RBE
configuration and running tests to ensure consistency and avoid
potential problems in the future.
Related to https://github.com/grpc/grpc/pull/37987
This test is flaky, example failure: https://btx.cloud.google.com/invocations/ea6d91f6-655a-41a4-bcbd-96c1a75118e1/targets, error message:
```
[91mtests_aio.unit.call_test.TestStreamStreamCall.test_cancel_after_done_writing[0m
[1mtraceback:[0m
Traceback (most recent call last):
File "/usr/lib/python3.9/unittest/case.py", line 59, in testPartExecutor
yield
File "/usr/lib/python3.9/unittest/case.py", line 593, in run
self._callTestMethod(testMethod)
File "/usr/lib/python3.9/unittest/case.py", line 550, in _callTestMethod
method()
File "/var/local/git/grpc/src/python/grpcio_tests/tests_aio/unit/_test_base.py", line 31, in wrapper
return loop.run_until_complete(f(*args, **kwargs))
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/var/local/git/grpc/src/python/grpcio_tests/tests_aio/unit/call_test.py", line 821, in test_cancel_after_done_writing
self.assertTrue(call.cancel())
File "/usr/lib/python3.9/unittest/case.py", line 682, in assertTrue
raise self.failureException(msg)
AssertionError: False is not true
```
The test is trying to cancel the RPC after calling `done_writing()`, but it's possible that the RPC will finish before we checks the status and thus `call.cancel()` will return false.
We're changing this test to add some delays in server side so that we can properly cancel the RPC before it ends.
Tested locally and flake rate decreased from about 5/10000 to 0/10000.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#38051
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38051 from XuanWang-Amos:fix_test_cancel_after_done_writing d72170c3de
PiperOrigin-RevId: 692266861
Protobuf 6.30.0 will change the return types of Descriptor::name() and other methods to absl::string_view. This makes the code work both before and after such a change.
PiperOrigin-RevId: 692216341
gRPC requires at least CMake 3.16 or later so the test script no longer needs to install CMake 3.16.
Closes#38048
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38048 from veblush:cmake-no 3e364d9770
PiperOrigin-RevId: 691976945
Certificate verification can fail for more than 50 different reasons. Simplify troubleshooting by including the reason in the error message.
Closes#37207
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37207 from csapuntz:master a8c0855f36
PiperOrigin-RevId: 691962134
The ring buffer was inconsistently removing items making it hard to reason about the state in general.
Additionally adjust API such that we'll guarantee concurrent one flush and fail to collect others. This ultimately makes it a lot easier to limit the impact of running this system.
PiperOrigin-RevId: 691657279
tl;dr: If data is received on the underlying endpoint, it must never be discarded.
Closes#38036
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38036 from drfloob:endpoint-cb-xor ab763cac30
PiperOrigin-RevId: 691508829
Fix for b/365993761.
Noticed that XdsClient metrics were not being reported due to authority not being properly set.
This solution is not perfect since channels created later can possibly use a different authority, so preferring to use the default authority from the first channel.
Closes#38009
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38009 from yashykt:AddAuthorityToXdsClientMetricsScope 00071efa23
PiperOrigin-RevId: 691149703
There was an edge case in which a socket or endpoint was shut down, a socket `read` call returned zero bytes, and there was unread in the read buffer from a previous read operation. The endpoint callbacks were called with an error status to indicate the end of the stream, and the callbacks did not consume that final chunk of data.
My current hunch is that something inside gRPC is violating the EventEngine Endpoint::Read contract, but I'm not certain what, yet. 88b5c9e3ab/include/grpc/event_engine/event_engine.h (L197-L199)
However, by modifying WindowsEndpoint to return an `absl::OkStatus()` if there's any data in the buffer, tests appear to pass.
Closes#38014
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38014 from drfloob:win-endpoint-data-leak b24b2d9f8a
PiperOrigin-RevId: 691063044
In EventEngineClientChannelDNSResolver, it needs to acquire `on_resolved_mu_` lock when calling dns resolver APIs as well as in on_resolve callback.
In DNSServiceResolverImpl::LookupHostname, it acquires `request_mu_` lock, so the lock order is:
EventEngineClientChannelDNSResolver::on_resolved_mu_ -> DNSServiceResolverImpl::request_mu_
Upon the resolution successful or failed, DNSServiceResolverImpl calls the on_resolved callback without holding any locks, so only one lock here:
EventEngineClientChannelDNSResolver::on_resolved_mu_
However when DNSServiceResolver was deleted, in DNSServiceResolverImpl::Shutdown, it calls the on_resolve callbacks while holding the `request_mu_` lock, as a result the lock order becomes:
DNSServiceResolverImpl::request_mu_ -> EventEngineClientChannelDNSResolver::on_resolved_mu_
which triggers the deadlock check in absl::Mutex as these two locks are acquired in different orders.
This PR release the on_resolved_mu_ first before calling on_resolve, so abls won't complain.
Added a new test which fails with GPR_ABSEIL_SYNC=1 (not enabled by default) without this PR.
Closes#38010
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38010 from HannahShiSFB:fix-lock-invert-in-dns-service-resolver-shutdown 5adc8f32b3
PiperOrigin-RevId: 690781049
`//test/core/end2end/...` tests stopped running back in March 2024, when the linking process for test binaries changed. See https://github.com/grpc/grpc/pull/36197.
The test targets ran on Windows, but zero tests were found inside those targets, so the tests succeeded instantly. This fix results in longer linking steps, and more disk space consumed, but tests are getting discovered now. To illustrate the problem, run `bazel test //test/core/end2end:ping_pong_streaming_test --test_output=all --test_arg=--gtest_list_tests`. Before this PR, zero tests were found. Now:
```
CoreEnd2endTest.
PingPongStreaming1/Inproc
PingPongStreaming1/Chttp2FakeSecurityFullstack
PingPongStreaming1/Chttp2Fullstack
PingPongStreaming1/Chttp2FullstackCompression
PingPongStreaming1/Chttp2FullstackLocalIpv4
...
```
Closes#37918
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37918 from drfloob:fix-win-core-e2e e60f832a5d
PiperOrigin-RevId: 690741492
Also remove a now-unnecessary dependency from the dump_args target that
previously had been needed just to get the right includes to be used.
PiperOrigin-RevId: 690619490
This will resolve a bunch of warnings we're seeing in Envoy, of the form:
```
INFO: From Action external/com_github_grpc_grpc/src/proto/grpc/reflection/v1alpha/reflection.grpc.pb.h:
bazel-out/k8-opt/bin/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
INFO: From Action external/envoy_api/envoy/service/ext_proc/v3/external_processor.grpc.pb.h:
bazel-out/k8-opt/bin/external/envoy_api/external/envoy_api: warning: directory does not exist.
INFO: From Action external/opencensus_proto/opencensus/proto/agent/trace/v1/trace_service.grpc.pb.h:
bazel-out/k8-opt/bin/external/opencensus_proto/external/opencensus_proto: warning: directory does not exist.
INFO: From Action external/com_google_googleapis/google/devtools/cloudtrace/v2/trace.grpc.pb.h:
bazel-out/k8-opt/bin/external/com_google_googleapis/external/com_google_googleapis: warning: directory does not exist.
INFO: From Action external/com_github_grpc_grpc/src/proto/grpc/reflection/v1/reflection.grpc.pb.h:
bazel-out/k8-opt/bin/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
```
```
bazel-out/k8-opt/bin/external/com_github_grpc_grpc/external/com_github_grpc_grpc: warning: directory does not exist.
|----copy-1-from-dir_out-----||---copy-2-from-proto_root---|
```
Closes#37990
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37990 from asedeno:warnings 801da4c5cd
PiperOrigin-RevId: 689517449
Yash Tibrewal
Xuan Wang
Esun Kim
Mark D. Roth
gRPC Team Bot
AJ Heller
Vignesh Babu
Matthew Stevenson
Kannan J
Yijie Ma
Alisha Nanda
Craig Tiller
Tanvi Jagtap
Nathan Baulch
Lev Kandel
Constantine Sapuntzakis
Hannah Shi
Eric Anderson
Marcel
apolcyn
Richard Belleville
Alejandro R. Sedeño