Allow `CallFilters` to have more than one `CallFilters::Stack` attached to it, and execute all of the stacks when executing.
This permits a trivial implementation of `PassThrough` for `Interceptor`, and allows elimination of an inserted `ForwardCall` if an interception chain terminates in a `CallDestination` preceded by one or more filters.
Built on #36984 which should be merged first.
Closes#36985
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36985 from ctiller:step-by-step 07b14c9d91
PiperOrigin-RevId: 646586483
This test very occasionally needs to have gRPC initialized, so adding the init/shutdown scope just where it's needed.
Closes#37050
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37050 from ctiller:ci 4baad7af76
PiperOrigin-RevId: 646586430
Originally when I was implementing call-v3 I added async handling for server trailing metadata because it was symmetrical with all the other handling we had. Turns out that we really don't need this, and further it's probably harmful to our ability to reason about the stack - so I'm removing that capability.
Closes#36984
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36984 from ctiller:no-infallible-seq f3b3548685
PiperOrigin-RevId: 646146666
Updates to channel init registration ordering:
* Add ability to float filters towards the top of the stack or to sink them towards the bottom. This is a secondary hint to the explicit ordering constraints that we have. To avoid possible ambiguities we enforce that there can never be two tops/two bottoms available to add simultaneously (if this eventuality appears we simply require an explicit ordering between the two filters).
* Add the ability to skip some filters in the V2 stack (so we can start building V3 only filters)
Along the way:
* Add hashability to `UniqueTypeName`
* Remove `UniqueTypeName` copy constructor (this is just the default copy constructor anyway, and its presence started to confuse `clang-tidy`).
* Separate the dependency tracking and logging parts of channel_init.cc. This was necessary for my brain to be able to process how to implement the new ordering constraints. Happily, it also eliminates an O(n**2) loop in initialization!
Closes#36993
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36993 from ctiller:ordering-innit e9d08e7a81
PiperOrigin-RevId: 646144240
There's some tooling floating around that looks for `BM_` prefixed strings and was failing with these benchmarks.
Re-jiggle some macros around so that's no longer the case.
Closes#37024
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37024 from ctiller:bm- 9b2fc76807
PiperOrigin-RevId: 646137939
As we've learned what configuration is needed for our benchmarks the settings have been growing more and more bespoke for each binary. Try to consolidate that into some useful defaults.
Also ensure we always `linkstatic=1`. `cc_binary` defaults to this, so it's reasonable to assume that's the performance our customers see. It also deeply impacts performance for small microbenchmarks, and so enabling it gives us more apples:apples, and saves chasing things that don't matter.
Closes#37012
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37012 from ctiller:benchmark-bin b84cab58c6
PiperOrigin-RevId: 645483183
Previously, metadata mutations were made by the picker directly, which meant that they would be applied even if the channel winds up discarding the pick due to the returned subchannel having been disconnected by the time the pick result is returned. This changes the API such that pickers return metadata mutations along with the pick result, so that the mutations won't get applied unless the pick result is actually used.
Closes#36968
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36968 from markdroth:lb_metadata_api 2765da6121
PiperOrigin-RevId: 645451869
### Problem 1
Context :
gpr_log() internally calls gpr_log_message() .
gpr_log_message() may either call gpr_default_log() or a user provided logging function.
gpr_default_log() uses absl LOG. This is the default. Most users will log this way.
For the small percentage of users who have customized the logging function, gpr_log will log to custom this function.
Problem :
We have converted half the instances of gpr_log to absl LOG().
For users who use the defaults - there will be no issue.
For the users who use a customized logging function
1. All the absl LOGs will log to the absl log sink.
2. All the gpr_log statements will log via this user provided function.
This is in-consistent behaviour and will cause confusion and difficulty in debugging.
Solution:
All logs should go to the same sink.
So we decided to make gpr_set_log_function a no op in this release.
The function will be deleted in the next release.
https://github.com/grpc/proposal/pull/425
### Problem 2
Context :
gpr_should_log is used to avoid computing expensive stuff for logging if the log is not going to be visible.
Problem :
gpr_should_log was referencing the GRPC_VERBOSITY flag and values set by gpr_set_log_verbosity .
However, actual logging happens based on the absl settings.
This is incorrect. Because if the old settings are not honoured, they should not be checked and no decision in code should be made based on settings which are not going to get used.
Solution :
Given the above changes in Problem 1, since all custom logging is disabled, all logging from gRPC with honour the absl LOG settings. Hence we modified the gpr_should_log function to refer to absl settings.
### Problem 3
We still have the issue of php using a custom log sink. We will address this in a separate PR.
### Problem 4
Tests related to test/core/end2end/tests/no_logging.cc are broken . These will be fixed in another PR.
Closes#36961
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36961 from tanvi-jagtap:fix_gpr_should_log 70c3224af1
PiperOrigin-RevId: 645096418
In the client fuzzer, some valid fuzzing scenarios would close the transport (thus deleting the endpoint), while the fuzzer mechanics still attempted to read/write to that endpoint. There was an inherent ownership problem, where both the transport and the fuzzer logic expected to own the endpoint lifetime.
This PR ensures that the transport owns the endpoint, and the fuzzer logic owns an object that can write to some shared endpoint state. This shared object can outlive the endpoint.
Closes#36966
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36966 from drfloob:fuzzer/4908841560506368 a9ea2e795d
PiperOrigin-RevId: 645081665
Specifically:
- use `OrphanablePtr<>` for `grpc_endpoint`
- use `absl::AnyInvocable<>` instead of `grpc_closure`
- use `EventEngine::Run()` instead of `ExecCtx::Run()`
- use `SliceBuffer` instead of `grpc_slice_buffer`
- use `absl::Status` instead of `grpc_error_handle`
- use `absl::string_view` instead of `const char*` for handshaker names
Also pass acceptor via `HandshakerArgs` instead of as a separate parameter.
Also changed chttp2 and httpcli to use `OrphanablePtr<>` for the endpoint.
PiperOrigin-RevId: 644551906
We don't have a guarantee of timeliness for connectivity issues to clear up with resource quota, and so this wait for ready setting is causing very long delays for the test to complete.
Closes#36967
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36967 from ctiller:rq 4e76af9bda
PiperOrigin-RevId: 644502087
Preparation for switching away from `grpc_channel_filter*` to identify channel filters.
Closes#36907
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36907 from ctiller:type-name e7ad4c67a2
PiperOrigin-RevId: 644483948
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36941
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36941 from tanvi-jagtap:src_core_xds 9ab8de800d
PiperOrigin-RevId: 644229971
I noticed that we were unconditionally checking the timer during the first time period. I've added a test to ensure that we no longer do, and improved the code to fix it.
Closes#36932
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36932 from ctiller:periodically c3b612a995
PiperOrigin-RevId: 644193802
This provides better integration with our ecosystem, and allows more types to be automatically printed (yay tuples!).
Closes#36897
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36897 from ctiller:stringify da98d247e2
PiperOrigin-RevId: 644120732
Ensure arena accounting is working, and add a test that a constant call size results in a constant call size estimate.
Closes#36933
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36933 from ctiller:arena-accounts 116c805633
PiperOrigin-RevId: 644102412
This change brings up the direct channel, and inproc promise based transports.
This work exposed a bug that was very difficult to fix with the current call_filters.cc implementation, so I've substantially revamped that - instead of having a pipe-like object per call element, we now have a big ol' combined state machine for the entire call. It's a touch more code, but substantially easier to reason about individual cases, so I much prefer this form (it's also a slight memory improvement: 12 bytes total to track call state, and 10 of those are wakeup bitmasks...).
Closes#36734
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36734 from ctiller:transport-refs-9 3e2a80b40d
PiperOrigin-RevId: 644034593
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36879
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36879 from tanvi-jagtap:test_folder_gpr_log 40eeeaf987
PiperOrigin-RevId: 643593896
Previously we'd enter a wakeup storm if there were too many concurrent senders. Now we allow a small burst over the send limit (up to the number of concurrent senders on the mpsc), and make the wait until that send passes to the receiver. In this way we don't wake all pending senders even if there's not sufficient queue space available.
Closes#36862
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36862 from ctiller:mpsc-quadratic 4d2ad48e2d
PiperOrigin-RevId: 643375554
Also made some minor improvements to the `ConfigSelector` API.
Closes#36877
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36877 from markdroth:client_channel_v3_dynamic_filters 6a539fe320
PiperOrigin-RevId: 642755276
- ensure ordering of `OnAccept` and `Shutdown` callbacks from thready event engine (previously these could be reordered and this caused spurious failures)
- enable thready_tsan for one C++ e2e test
- don't filter thready_tsan for local builds (only CI)
Closes#36886
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36886 from ctiller:erm e3b88e7d86
PiperOrigin-RevId: 642702724
Refines the test in the fuzzer to only check input == output if the frame was re-encodable.
Also tweaks some hpack encoder stuff to not crash but simply report errors where appropriate.
Closes#36821
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36821 from ctiller:f-frame 1b08875c4e
PiperOrigin-RevId: 642311442
[Gpr_To_Absl_Logging] Move function to test header form log.h
This is not really needed in log.h
Closes#36860
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36860 from tanvi-jagtap:move_function_to_test_header e6494bd06f
PiperOrigin-RevId: 642080756
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36729
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36729 from yijiem:per-channel-stats-plugin 4786bed42f
PiperOrigin-RevId: 642030366
All TraceFlags are now configured in `src/core/lib/debug/trace_flags.yaml`. The format is:
```
my_flag:
default: false # the default value; default=false
description: Some Description
debug_only: false # debug_only flags only work in debug builds; default=false
internal: false # internal flags will not show up in documentation; default=false
```
To regenerate the trace flag source code, run `tools/codegen/core/gen_trace_flags.py` (requires mako). This script is also run when sanity checking.
This PR also adds two new features:
### Glob-based flag configuration
Trace flag configuration now supports `?` (single wildcard character) and `*` (one or more wildcard characters). For example, using `GRPC_TRACE='event_engine*'` will enable all flags that match that glob. It expands to:
* event_engine
* event_engine_client_channel_resolver
* event_engine_dns
* event_engine_endpoint
* event_engine_endpoint_data
* event_engine_poller
### A cleaner trace-logging macro in abseil logging format
If your goal is only to add log statements when the `fault_injection_filter` trace flag is enabled, you can use the macro:
```
GRPC_TRACE_LOG(fault_injection, INFO) << "Filtered:" << 42;
```
When the trace flag is enabled, the the log will show something like this:
```
I0000 00:00:1715733657.430042 16 file.cc:174] Filtered:42
```
----
Note: just like with the gpr_log to abseil logging conversion, the pre-existing trace logging usages can be replaced with the new tracing macro across multiple PRs.
Closes#36576
PiperOrigin-RevId: 641295215
It's still possible to define GRPC_IOS_EVENT_ENGINE_CLIENT=0 to disable event engine for iOS.
Closes#36785
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36785 from HannahShiSFB:default-to-event-engine-in-ios 441fe552a6
PiperOrigin-RevId: 640716651
Craig Tiller
Mark D. Roth
Tanvi Jagtap
AJ Heller
Tanvi Jagtap
Nana Pang
Yijie Ma
Vishwanath Sastry
Hannah Shi