This eliminates the need for the `grpc_cc_proto_library` bazel BUILD rule introduced in #37863.
To make this work, I had to upgrade several bazel dependencies and apply a patch to rules_go to work around https://github.com/bazelbuild/bazel/issues/11636.
Closes#37902
PiperOrigin-RevId: 685868647
We'll probably disable some next week :)
But I want to watch a good selection and refine criteria for acceptance.
Closes#37903
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37903 from ctiller:ALL-the-things 5f829db870
PiperOrigin-RevId: 685010911
So far missing for HTTP/2 style flow control has been a primitive to query whether there's a receiver for flow control data at the other end of the message pipes.
Here I'm updating the state machine accessors to accommodate that functionality.
No new states were needed.
Whilst here, document the current member functions on `CallState`.
Closes#37867
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37867 from ctiller:like-the-river c9814c737d
PiperOrigin-RevId: 684972125
This is a trial baloon to see if we can actually make this work. If it does, I'll change the remaining xDS tests to use the real xDS protos and completely remove our local copies.
Closes#37863
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37863 from markdroth:xds_tests_use_real_protos 3ad2fe12be
PiperOrigin-RevId: 684877750
I'm continuing to look into some flakes here, but in the meantime these shouldn't halt submissions. Marking them flaky.
Closes#37880
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37880 from ctiller:mark-flaky 27427c7978
PiperOrigin-RevId: 684526341
This is the last piece of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Note that although this is the first use-case for this "blackboard" mechanism, we will also use it in the future for the xDS rate-limiting filter on the gRPC server side.
Closes#37646
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37646 from markdroth:gcp_auth_filter_state 72d0d96c79
PiperOrigin-RevId: 679707134
Allows use of the party <-> party wakeup batching stuff, which reduces threadhops drastically for this transport.
Closes#37078
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37078 from ctiller:chaotic-party-3 75c32e6a64
PiperOrigin-RevId: 679685211
Basic building block for retries, hedging: buffer outgoing messages & metadata, allow for replay whilst buffered (with a single reader able to read once buffering ends)
Closes#37448
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37448 from ctiller:once-again-into-the-breach-my-friends 79cb121054
PiperOrigin-RevId: 677959212
Fix https://github.com/grpc/grpc/issues/37727.
A better idea might be to set up Fuzzing for these APIs to find those sort of things. Maybe that can be a next step if we want to further hardening things.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37737
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37737 from yijiem:memory-leak-alts-2 6be8a49e63
PiperOrigin-RevId: 677880955
Add a ValidateCredentials API to the TLS certificate provider interface. A user can call this API to check that the credentials currently held by the certificate provider instance are valid. The definition of "valid" depends on provider that is being used. For the static data and file watcher providers, "valid" means that the credentials consist of valid PEM.
~Currently there is no check to ensure that credentials consist of valid PEM blocks before a TLS handshake commences. This PR creates a static factory for FileWatcherCertificateProvider (and marks the constructor as deprecated) which performs this validation check. The analogous work for StaticDataCertificateProvider will be done in a follow-up PR.~
Closes#37565
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37565 from matthewstevenson88:filewatcher f223228023
PiperOrigin-RevId: 677847751
Ensure OPENSSL global clean up happens after gRPC shutdown completes. OPENSSL registers an exit handler to clean up global objects, which may happen before gRPC removes all references to OPENSSL.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37768
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37768 from yousukseung:openssl-atexit-wait d3d1c964a8
PiperOrigin-RevId: 677284514
The following files have been moved:
- src/core/lib/avl/*
- src/core/lib/backoff/*
- src/core/lib/debug/event_log*
- src/core/lib/iomgr/gethostname*
- src/core/lib/iomgr/grpc_if_nametoindex*
- src/core/lib/matchers/*
- src/core/lib/uri/* (renamed from uri_parser.* to uri.*)
- src/core/lib/gprpp/* (existing src/core/util/time.cc was renamed to gpr_time.cc to avoid conflict)
Closes#36792
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36792 from markdroth:reorg_util d4e8996f48
PiperOrigin-RevId: 676947640
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37773
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37773 from yijiem:alts-concurrent-connect-timeout 99e371f3ac
PiperOrigin-RevId: 676895049
In some rare occasions on Win machines (0,2-0,5%), the tests are stuck before the handshake when we execute `grpc_call_start_batch`. We receive OP_COMPLETE with `Deadline Exceeded {grpc_status:4}` for such cases. The PR bumps it from 5 to 30s (no flakes for --runs_per_test=1000).
Closes#37767
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37767 from erm-g:h2test 9d1208ee1f
PiperOrigin-RevId: 676531975
Previously, if we pulled server trailing metadata *before* the call was added to the client transport then we'd never call `on_done_` on the spine and consequently never remove the call from the map. This change fixes that edge case.
In fixing it, I noticed a state in `CallState` that was both complicating the fix and completely irrelevant because we respecced earlier this year to say that ServerTrailingMetadata processing cannot be asynchronous, so I'm removing that state also.
Closes#37749
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37749 from ctiller:flake-fightas-11 847814a286
PiperOrigin-RevId: 676246259
The gRPC Core API currently requires callers to provide initial metadata before trailing metadata. You can see the C++ Callback API do this bookkeeping, for example. There is an eventual goal to be able to provide these in any order, and have gRPC do the right thing, but core is not there yet.
The proxy fixture in our end2end tests had a rare scenario in which trailing metadata from the server would show up at the proxy before initial metadata. This is part of the proxy's job: to split up batches into singular-operations that can complete in any order. There was, however, a rare flake wherein trailing metadata would complete before initial metadata, and the result was both client and server waiting on each other to respond.
This change adds a way for the proxy to defer sending trailing metadata back to the client, until after initial metadata has been sent to the client. In my testing, this eliminates the flake I had been able to reproduce 1 in 10k times using a single test. It happened more frequently across the full set of tests in our CI test suites.
Closes#37738
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37738 from drfloob:fix-proxy-fixture 6e0d7b7e6f
PiperOrigin-RevId: 676026493
Without this, we see GOAWAYs with "enter idle" irrespective of the reason being idleness or max connection age.
Closes#37709
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37709 from yashykt:ChannelIdleFilterMessage 236072e7e2
PiperOrigin-RevId: 675762380
This test has been flaking for a while with a WSAEACCESS error on the `bind` call.
Change the loop to only create on socket at a time (on Windows) to rule out something windows-specific is not liking the fact that we are opening multiple listen sockets on the same port.
Closes#37669
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37669 from apolcyn:change_loop ffa105ba46
PiperOrigin-RevId: 675172803
This change adds an experiment to move time caching from `ExecCtx` (which is the wrong place for this mechanism) and moves it to the party update path (the expectation being that a single poll of a call is the granularity at which we expect time caching to be a useful optimization, whilst avoiding the unbounded hold times associated with the current mechanism).
This requires fixing up a few tests that grew to depend on time caching (would appreciate close eyes on the credentials test, as it's unclear to me why this is required or what the effect is).
This should also fix b/232544809.
Closes#37637
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37637 from ctiller:closer-to-the-sun 8bbde2d0bd
PiperOrigin-RevId: 672574762
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37632
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37632 from yijiem:alts-concurrent-connectivity-test-flake 68df4b1327
PiperOrigin-RevId: 671117730
Looks like there are some odd interactions, but call-v3 doesn't (and will never) handle wakeup sets, so disable for now until iomgr is removed.
Closes#37630
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37630 from ctiller:cgg 7c37893667
PiperOrigin-RevId: 671104484
Fallback interop test is fully deployed. This variable is no longer needed.
Closes#37620
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37620 from eugeneo:no-fallback-var c21509d0a5
PiperOrigin-RevId: 670738146
Final piece of gRFC A83 (https://github.com/grpc/proposal/pull/438): the GCP authentication filter itself.
Infrastructure changes include:
- Added a general-purpose LRU cache library that can be reused elsewhere.
- Fixed the client channel code to use the channel args returned by the resolver for the dynamic filters. This was necessary so that the GCP auth filter could access the `XdsConfig` object, which is passed via a channel arg.
- Unlike the other xDS HTTP filters we support, the GCP auth filter does not support config overrides, and its configuration includes a cache size parameter that we always need at the channel level, not per-call. As a result, I had to change the xDS HTTP filter API to give it the ability to set top-level fields in the service config, not just per-method fields. (We use the service config as a way of passing configuration down into xDS HTTP filters.) Note that for now, this works only on the client side, because we don't have machinery for a top-level service config on the server side.
- The GCP auth filter is also the first case where the filter needs to know its instance name from the xDS config, so I changed the xDS HTTP filter API to plumb that through.
- Fixed a bug in the HTTP client library that prevented the override functions from declining to override a particular request.
Closes#37550
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37550 from markdroth:xds_gcp_auth_filter 19eaefb52f
PiperOrigin-RevId: 669371249
Fixes a bug in the backoff implementation whereby we were incorrectly failing to apply jitter to the initial backoff.
Also change the API to return `Duration` instead of `Timestamp`. The only caller that actually wants to count the backoff from the start of the previous attempt instead of the end of the previous attempt is the subchannel code, and it handles that on its end.
Closes#37595
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37595 from markdroth:backoff_fixes_and_api_improvement 39d083c0f4
PiperOrigin-RevId: 669112557
Add validation of the `Audience` cluster metadata type, as per gRFC A83 (https://github.com/grpc/proposal/pull/438).
I had previously changed the metadata to be represented as JSON in #37468. However, while working on the GCP Authentication filter implementation, I realized that that's not an ideal representation, because it would have required us to validate the JSON on a per-RPC basis, which would be bad for performance. So I've changed the representation of metadata to be an abstract type, and we now store the `Audience` metadata as a simple string. I've also moved metadata into its own type with its own validation code, so that in the future we can use it in places other than CDS (many xDS resource types have metadata fields).
While I was at it, I also add some helper functions for validating the `UInt32Value` and `UInt64Value` wrapper protos.
Closes#37566
PiperOrigin-RevId: 668281729
The first commit is a pure revert of the revert, and the second one has the fix.
Closes#37573
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37573 from markdroth:call_creds_roll_forward 2476329534
PiperOrigin-RevId: 667672832
The changes in #37531 are causing test failures under run_tests.py (but not bazel), and #37544 was built on top of #37531, so both need to be reverted.
Closes#37567
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37567 from markdroth:call_creds_revert d086e066f5
PiperOrigin-RevId: 666978406
This will fix timestamps on logs and show all `VLOG(2)` logs on tests by default.
Currently, timestamps on logs are shown as -
```
I0000 00:00:1724385276.681936 1894892 config.cc:262] gRPC experiments enabled: call_tracer_in_transport, event_engine_dns, event_engine_listener, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
```
After invoking `absl::InitializeLog()`, this gets fixed to -
```
I0823 03:55:53.993928 1895644 config.cc:262] gRPC experiments enabled: call_tracer_in_transport, event_engine_dns, event_engine_listener, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
```
Closes#37560
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37560 from yashykt:ImproveLoggingForTests 66433336c8
PiperOrigin-RevId: 666956421
Mark D. Roth
Craig Tiller
AJ Heller
erm-g
Tanvi Jagtap
Yijie Ma
Matthew Stevenson
Yousuk Seung
Chris Kennelly
Yash Tibrewal
Nathan Baulch
apolcyn
Eugene Ostroukhov
Esun Kim