One test (`CallerPollentsAreNotReferencedAfterCallbackIsRan`) was not waiting for a callback to be run before finishing when the EventEngine client is in use. With the iomgr implementation, `exec_ctx.Flush();` previously guaranteed that the callback would be executed before the test ended, but the EventEngine runs it in a separate thread, with the EventEngine shim providing its own isolated ExecCtx. This fix simply waits for the callback to be run before exiting, in the same way many other tests in this file wait on iomgr-based callback execution.
Closes#35776
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35776 from drfloob:httpcli-test-async 3c920da741
PiperOrigin-RevId: 603175531
This PR:
* adds FD extensions to the public headers
* Adds the query extension interface to EventEngine, Listener, and Endpoint, via a new `Extensible` interface
* Refactors the PosixEventEngine to use the Extensible interface.
Closes#35648
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35648 from drfloob:posix-ee-query-interface 7cae28e0b3
PiperOrigin-RevId: 601794970
Instead, connect a real client, which gives us the ability to exercise the listener portion of the code too.
Closes#35625
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35625 from ctiller:shush-connect b8f78993c6
PiperOrigin-RevId: 600836565
`ProtoBitGen` provides a random number generator that returns values directly from fuzzer selected values, which allows us to test-into random selection algorithms deterministically.
Since the list of values provided by the fuzzer is limited, we need a fallback implementation. Previously we'd used something that was very correlated, and some of the distribution algorithms get into a very slow convergence mode when we do that (so we repeatedly return the same value for billions of iterations and cause timeouts in fuzzers).
Instead, when we run out of fuzzer supplied values, seed an mt19937 generator with the fuzzer selected values and use that from there on. Said generator will then produce values deterministically (for a given fuzzer input), but with a better distribution to allow convergence for fiddly algorithms.
Closes#35621
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35621 from ctiller:cg-timeout 6c9ef9cac5
PiperOrigin-RevId: 600607424
Using `AF_UNSPEC` for both IPv4 and IPv6 queries does not work in all cases. Specifically, for `localhost:<>`, c-ares only returns the IPv6 record i.e. `::1`.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35530
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35530 from yijiem:enable-oss-ee-dns-posix 452b5a2d81
PiperOrigin-RevId: 599989537
A few improvements to the promise context system (more coming)
Allow subclassed contexts:
If we have multiple different kinds of a base context, allow `GetContext<Derived>()` to mean `down_cast<Derived*>(GetContext<Base>())` everywhere for brevity.
Allow custom context lookup:
For a base context type, allow customization of how that context is looked up.
These two together allow:
1. normalization of activity lookup and context lookup to the same syntax (so we can write `GetContext<Activity>()` everywhere now
2. Party & Activity to share a context, so that anywhere we need to do a party specific operation we can write `GetContext<Party>()->...` and safely know that it's the current activity *and* it's a party.
Closes#35592
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35592 from ctiller:contextual-types 37ef948a36
PiperOrigin-RevId: 599651708
We've got a few situations coming up with promises that will want a "broadcast new value to everywhere" situation.
Closes#35552
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35552 from ctiller:obs 30fd697ae3
PiperOrigin-RevId: 599609399
Only call constructors when absolutely necessary (empty trivially constructible types don't need construction!!)
Similarly for destructors, if the destructor is trivial it means C++ will do no work destructing it... let's not even do the virtual function call to get there.
(also fix a bug where we weren't calling this stuff anyway, and add a test that would have caught that)
Closes#35591
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35591 from ctiller:filter-min 2933152d61
PiperOrigin-RevId: 599521371
A call execution environment for the V3 runtime.
The `CallFilters` class will ultimately be a (private) member of `CallSpine`, and the `StackBuilder` component will be used by a channel when all of the filters it needs are known to allow the call spine to start processing a call.
This is accompanied by a reasonably extensive test suite.
I expect to fine tune semantics, implementation, and tests over the coming weeks/months as we iterate to bring up the rest of the pieces.
Closes#35533
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35533 from ctiller:filters 689c7b527b
PiperOrigin-RevId: 599220150
We were getting errors due to insane amounts of padding: enforce limits, fix b/319533934.
Closes#35537
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35537 from ctiller:fff 9f5f31ef27
PiperOrigin-RevId: 597899598
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35210
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35210 from yijiem:csm-service-label 6a6a7d1774
PiperOrigin-RevId: 597641393
I realized that this field wasn't actually necessary, since the string is already present in the map key.
Closes#35503
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35503 from markdroth:xds_config_remove_cluster_name 94d5edc133
PiperOrigin-RevId: 597375018
Eventually for call-v3 we're going to want to have registration of filters generate the appropriate glue into the channel runtime to execute a call.
Begin that process now and gradually by introducing the new syntax and allowing a piecemeal migration to it - by the time we're done converting filters to the v3 APIs we'll also have the registration piece done.
PiperOrigin-RevId: 596013927
Mirrors what we had with combiner, but allows it to occur at arbitrary points.
We'll use this in chaotic-good to:
1. combine fragments into a single frame
2. combine writes from different calls into a single syscall
Closes#35413
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35413 from ctiller:group-on 9f20f34523
PiperOrigin-RevId: 596004767
We encountered an api_fuzzer test case that adds a huge number of addresses that all immediately fail to connect, but it set max_backoff to 0, so there was a giant busy loop where pick_first was constantly trying to connect to subchannels with no delay. The FuzzingEventEngine was getting stuck in a tick loop, always accumulating more tasks that needed to be executed immediately, so it could never make forward progress on the test case.
This PR fixes the problem by adding a fixed 1us delay if the task's delay is 0 and the test case has not provided any more fixed delays.
(Unfortunately, I cannot include the test case that triggered the problem in this PR, because it winds up exceeding the RBE stdout limit.)
Fixes b/310664846.
Closes#35447
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35447 from markdroth:api_fuzzer_busy_loop_fix 90055d3d92
PiperOrigin-RevId: 595853516
There are a select few tests that are failing when building with OpenSSL102 - disable them until we can fix.
Closes#35354
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35354 from gtcooke94:fix_ossl_102 8708d6ce86
PiperOrigin-RevId: 595761932
- `memory_pressure_controller` finally - allows deletion of pid_controller throughout the codebase
- `overload_protection` - one of the http2 rapid reset mitigations
- `red_max_concurrent_streams` - another http2 rapid reset mitigation
Closes#35426
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35426 from ctiller:new-years-cleanse 4651672e7e
PiperOrigin-RevId: 595205029
Remove the old `switch` library - this used to be an implementation detail of `Seq`, `TrySeq` - but has become unused.
Add a new user facing primitive `Switch` that fills a similar role to `switch` in C++ - selecting a promise to execute based on a primitive discriminator - much like `If` allows selection based on a boolean discriminator now.
A future change will optimize this to actually lower the `Switch` into an actual `switch` statement, but for right now I want to get the functionality in.
Closes#35424
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35424 from ctiller:switchy 5308a914c6
PiperOrigin-RevId: 595140965
Whilst here, eliminate unnecessary mutexes and streamline some complexity in the read variants.
Closes#35409
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35409 from ctiller:pbe 4f9588101a
PiperOrigin-RevId: 595006455
Adds temporary `call.cc` and `connected_channel.cc` scaffolding to run `CallInterceptor`/`CallHandler` style calls.
This will get ripped out as soon as the v3 transition is completed.
Closes#35312
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35312 from ctiller:v3-accept ae0bf81f8b
PiperOrigin-RevId: 594128029
Previously, `RefCountedPtr<>` and `WeakRefCountedPtr<>` incorrectly allowed
implicit casting of any type to any other type. This hadn't caused a
problem until recently, but now that it has, we need to fix it. I have
fixed this by changing these smart pointer types to allow type
conversions only when the type used is convertible to the type of the
smart pointer. This means that if `Subclass` inherits from `Base`, then
we can set a `RefCountedPtr<BaseClass>` to a value of type
`RefCountedPtr<Subclass>`, but we cannot do the reverse.
We had been (ab)using this bug to make it more convenient to deal with
down-casting in subclasses of ref-counted types. For example, because
`Resolver` inherits from `InternallyRefCounted<Resolver>`, calling
`Ref()` on a subclass of `Resolver` will return `RefCountedPtr<Resolver>`
rather than returning the subclass's type. The ability to implicitly
convert to the subclass type made this a bit easier to deal with. Now
that that ability is gone, we need a different way of dealing with that
problem.
I considered several ways of dealing with this, but none of them are
quite as ergonomic as I would ideally like. For now, I've settled on
requiring callers to explicitly down-cast as needed, although I have
provided some utility functions to make this slightly easier:
- `RefCounted<>`, `InternallyRefCounted<>`, and `DualRefCounted<>` all
provide a templated `RefAsSubclass<>()` method that will return a new
ref as a subclass. The type used with `RefAsSubclass()` must be a
subclass of the type passed to `RefCounted<>`, `InternallyRefCounted<>`,
or `DualRefCounted<>`.
- In addition, `DualRefCounted<>` provides a templated `WeakRefAsSubclass<T>()`
method. This is the same as `RefAsSubclass()`, except that it returns
a weak ref instead of a strong ref.
- In `RefCountedPtr<>`, I have added a new `Ref()` method that takes
debug tracing parameters. This can be used instead of calling `Ref()`
on the underlying object in cases where the caller already has a
`RefCountedPtr<>` and is calling `Ref()` only to specify the debug
tracing parameters. Using this method on `RefCountedPtr<>` is more
ergonomic, because the smart pointer is already using the right
subclass, so no down-casting is needed.
- In `WeakRefCountedPtr<>`, I have added a new `WeakRef()` method that
takes debug tracing parameters. This is the same as the new `Ref()`
method on `RefCountedPtr<>`.
- In both `RefCountedPtr<>` and `WeakRefCountedPtr<>`, I have added a
templated `TakeAsSubclass<>()` method that takes the ref out of the
smart pointer and returns a new smart pointer of the down-casted type.
Just as with the `RefAsSubclass()` method above, the type used with
`TakeAsSubclass()` must be a subclass of the type passed to
`RefCountedPtr<>` or `WeakRefCountedPtr<>`.
Note that I have *not* provided an `AsSubclass<>()` variant of the
`RefIfNonZero()` methods. Those methods are used relatively rarely, so
it's not as important for them to be quite so ergonomic. Callers of
these methods that need to down-cast can use
`RefIfNonZero().TakeAsSubclass<>()`.
PiperOrigin-RevId: 592327447
This fixes#21619. This experimental ALPN protocol has already been removed from the other gRPC stacks.
Closes#34876
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34876 from matthewstevenson88:remove-grpc-exp 1cb9d084ea
PiperOrigin-RevId: 592080195
`AllOk` runs a set of promises concurrently, and like `TryJoin` waits for them all to succeed or one to fail.
Unlike `TryJoin` it returns a single unified status of the composition, so cannot handle member promises that might return `StatusOr` or the like.
Closes#35304
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35304 from ctiller:all-review 30f5f809c6
PiperOrigin-RevId: 591031189
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35306
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35306 from yijiem:dns-empty-hostname 9c7f67c27a
PiperOrigin-RevId: 591028370
Mark D. Roth
AJ Heller
Vignesh Babu
Eugene Ostroukhov
Craig Tiller
apolcyn
Esun Kim
Yijie Ma
ericsalo
Yash Tibrewal
Gregory Cooke
Matthew Stevenson