Final piece of gRFC A83 (https://github.com/grpc/proposal/pull/438): the GCP authentication filter itself.
Infrastructure changes include:
- Added a general-purpose LRU cache library that can be reused elsewhere.
- Fixed the client channel code to use the channel args returned by the resolver for the dynamic filters. This was necessary so that the GCP auth filter could access the `XdsConfig` object, which is passed via a channel arg.
- Unlike the other xDS HTTP filters we support, the GCP auth filter does not support config overrides, and its configuration includes a cache size parameter that we always need at the channel level, not per-call. As a result, I had to change the xDS HTTP filter API to give it the ability to set top-level fields in the service config, not just per-method fields. (We use the service config as a way of passing configuration down into xDS HTTP filters.) Note that for now, this works only on the client side, because we don't have machinery for a top-level service config on the server side.
- The GCP auth filter is also the first case where the filter needs to know its instance name from the xDS config, so I changed the xDS HTTP filter API to plumb that through.
- Fixed a bug in the HTTP client library that prevented the override functions from declining to override a particular request.
Closes#37550
[Gpr_To_Absl_Logging] Remove gpr_log. Adding absl LOG wrappers
List of changes in this PR
1. Replacing all instances of gpr_log in PHP and RUBY with the new absl wrapper APIs. The replacement mapping is given below
gpr_log(GPR_ERROR, ...)
=> grpc_absl_log_error
gpr_log(GPR_INFO, ...)
=> grpc_absl_log_info - Printing a simple message
=> grpc_absl_log_info_int - Printing a message and a number
=> grpc_absl_log_info_str - Printing 2 strings.
gpr_log(GPR_DEBUG, ...)
=> grpc_absl_vlog - Printing a simple message
=> grpc_absl_vlog_int - Printing a message and a number
=> grpc_absl_vlog_str - Printing 2 strings.
Adding grpc_absl_vlog2_enabled() check around gpr_log(GPR_DEBUG, ...)
2. src/python/grpcio_observability/grpc_observability/observability_util.cc One instance of gpr_log to absl LOG replacement was missed earlier. Fixing that.
3. Deleting deprecated gpr stuff : gpr_log_severity , GPR_DEBUG , GPR_INFO , GPR_ERROR , gpr_log .
4. Adding new APIs for Ruby and PHP. These APIs are very simple wrappers around absl.
5. Removing the legacy functions in platform specific log.cc files. These files are safe to delete now.
6. Fixing the allow list in banned_functions.py . This makes sure that these new wrappers don't get used all over the place by everyone. We carefully only allow list the PHP and RUBY files and allow the use of these wrappers. Everywhere else - using these wrappers should fail Sanity Tests.
Closes#37431
Add validation of the `Audience` cluster metadata type, as per gRFC A83 (https://github.com/grpc/proposal/pull/438).
I had previously changed the metadata to be represented as JSON in #37468
An error occurred
. However, while working on the GCP Authentication filter implementation, I realized that that's not an ideal representation, because it would have required us to validate the JSON on a per-RPC basis, which would be bad for performance. So I've changed the representation of metadata to be an abstract type, and we now store the `Audience` metadata as a simple string. I've also moved metadata into its own type with its own validation code, so that in the future we can use it in places other than CDS (many xDS resource types have metadata fields).
While I was at it, I also add some helper functions for validating the `UInt32Value` and `UInt64Value` wrapper protos.
Closes#37566
This is needed to use `XdsConfig` in an xDS HTTP filter without causing a circular dependency, which will be needed as part of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Closes#37564
Previously, `grpc_oauth2_token_fetcher_credentials` provided functionality for on-demand token-fetching, but it was integrated into the oauth2 code, so it was not possible to use that same code for on-demand fetching of (e.g.) JWT tokens. This PR splits that class into two parts:
1. A base `TokenFetcherCredentials` class that provides a framework for on-demand fetching of any arbitrary type of auth token.
2. An `Oauth2TokenFetcherCredentials` subclass that derives from `TokenFetcherCredentials` and provides handling for oauth2 tokens.
The `grpc_compute_engine_token_fetcher_credentials`, `StsTokenFetcherCredentials`, and `grpc_google_refresh_token_credentials` classes that previously derived from `grpc_oauth2_token_fetcher_credentials` now derive from `Oauth2TokenFetcherCredentials` instead, so there's not much change to those classes (other than a cleaner interface with the base class functionality).
The `ExternalAccountCredentials` class and its subclasses got more extensive changes here. Previously, this class inheritted from `grpc_oauth2_token_fetcher_credentials` and fooled the base class into thinking that it directly fetched the oauth2 token, when in fact it actually performed a number of steps to gather data and then constructed a synthetic HTTP response to pass back to the base class. I have changed this to instead derive directly from `TokenFetcherCredentials` to provide a much cleaner interface with the parent class.
In addition, I have changed `grpc_call_credentials` from `RefCounted<>` to `DualRefCounted<>` to provide a clean way to shut down any in-flight token fetch when the credentials are unreffed.
This PR paves the way for subsequent work that will allow implementing an on-demand JWT token fetcher call credential, as part of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Closes#37510
This moves more code out of the monolithic `grpc_xds_client` BUILD target. We still need more work to split it up completely, but this is a nice step in the right direction -- and it unblocks a subsequent PR that I'm working on for xDS authority rewriting.
Closes#37130
Adds a build config - latent_see - that configures a module and some macros to allow recording of latency information through the stack and across threads. One can then use tools that understand the chromium event trace format (eg `ui.perfetto.dev`) to visualize the traces.
Closes#36964
Previously, metadata mutations were made by the picker directly, which meant that they would be applied even if the channel winds up discarding the pick due to the returned subchannel having been disconnected by the time the pick result is returned. This changes the API such that pickers return metadata mutations along with the pick result, so that the mutations won't get applied unless the pick result is actually used.
Closes#36968
This change brings up the direct channel, and inproc promise based transports.
This work exposed a bug that was very difficult to fix with the current call_filters.cc implementation, so I've substantially revamped that - instead of having a pipe-like object per call element, we now have a big ol' combined state machine for the entire call. It's a touch more code, but substantially easier to reason about individual cases, so I much prefer this form (it's also a slight memory improvement: 12 bytes total to track call state, and 10 of those are wakeup bitmasks...).
Closes#36734
All TraceFlags are now configured in `src/core/lib/debug/trace_flags.yaml`. The format is:
```
my_flag:
default: false # the default value; default=false
description: Some Description
debug_only: false # debug_only flags only work in debug builds; default=false
internal: false # internal flags will not show up in documentation; default=false
```
To regenerate the trace flag source code, run `tools/codegen/core/gen_trace_flags.py` (requires mako). This script is also run when sanity checking.
This PR also adds two new features:
### Glob-based flag configuration
Trace flag configuration now supports `?` (single wildcard character) and `*` (one or more wildcard characters). For example, using `GRPC_TRACE='event_engine*'` will enable all flags that match that glob. It expands to:
* event_engine
* event_engine_client_channel_resolver
* event_engine_dns
* event_engine_endpoint
* event_engine_endpoint_data
* event_engine_poller
### A cleaner trace-logging macro in abseil logging format
If your goal is only to add log statements when the `fault_injection_filter` trace flag is enabled, you can use the macro:
```
GRPC_TRACE_LOG(fault_injection, INFO) << "Filtered:" << 42;
```
When the trace flag is enabled, the the log will show something like this:
```
I0000 00:00:1715733657.430042 16 file.cc:174] Filtered:42
```
----
Note: just like with the gpr_log to abseil logging conversion, the pre-existing trace logging usages can be replaced with the new tracing macro across multiple PRs.
Closes#36576
The following attributes were completely unused:
- kOffset
- kIndex
- kSize
- kFilename
- kKey
- kValue
The following attributes were added but never programmatically accessed, and I've moved them into the status messages themselves, which is another step toward #22883
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36509
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK.
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36468
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK.
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36467
Also begin to eliminate `CallContext` in favor of just exposing `Call` - ultimately there's not really a need to introduce two types here, so I'm going to wind that idea back over a few PRs.
I've avoided making this an experiment as the changes required were quite structural.
Closes#36477
Mark D. Roth
Tanvi Jagtap
Craig Tiller
Vignesh Babu
AJ Heller
Esun Kim
Eugene Ostroukhov
Xuan Wang
Nana Pang
gRPC Team Bot
Yash Tibrewal
Hannah Shi