As per gRFC A83 (https://github.com/grpc/proposal/pull/438).
For now, I am not exposing this new call creds type via the C-core API or in any C++ or wrapped language public APIs, so there's no way to use it externally. We can easily add that in the future if someone asks, but for now the intent is to use it only internally via the xDS GCP authentication filter, which I'll implement in a subsequent PR.
As part of this, I changed the test framework in credentials_test to check the status code in addition to the message on failure. This exposed several places where existing credential types are returnign the wrong status code (unsurprisingly, because of all of the tech debt surrounding grpc_error). I have not fixed this behavior, but I have added TODOs in the test showing which ones I think need to be fixed.
Closes#37544
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37544 from markdroth:gcp_service_account_identity_call_creds 97e0efc48d
PiperOrigin-RevId: 666869692
This adds functionality that is intended to be used for the new GcpServiceAccountIdentityCallCredentials implementation, as per gRFC A83 (https://github.com/grpc/proposal/pull/438). However, it is also a useful improvement for all token-fetching call credentials types, so I am adding it to the base class.
Closes#37531
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37531 from markdroth:token_fetcher_call_creds_prefetch_and_backoff 0fcdb48465
PiperOrigin-RevId: 666809903
This change moves the location for the prebuilt images used in gRPC OSS benchmarks from GCR to Artifact Registry.
Images are all saved to the same repo, instead of being organized by a prefix (`kokoro`, `kokoro-test`,
`cxx_experiment/kokoro-test`, etc.).
Closes#37558
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37558 from paulosjca:ar a767ab5d91
PiperOrigin-RevId: 666643972
Previously, `grpc_oauth2_token_fetcher_credentials` provided functionality for on-demand token-fetching, but it was integrated into the oauth2 code, so it was not possible to use that same code for on-demand fetching of (e.g.) JWT tokens. This PR splits that class into two parts:
1. A base `TokenFetcherCredentials` class that provides a framework for on-demand fetching of any arbitrary type of auth token.
2. An `Oauth2TokenFetcherCredentials` subclass that derives from `TokenFetcherCredentials` and provides handling for oauth2 tokens.
The `grpc_compute_engine_token_fetcher_credentials`, `StsTokenFetcherCredentials`, and `grpc_google_refresh_token_credentials` classes that previously derived from `grpc_oauth2_token_fetcher_credentials` now derive from `Oauth2TokenFetcherCredentials` instead, so there's not much change to those classes (other than a cleaner interface with the base class functionality).
The `ExternalAccountCredentials` class and its subclasses got more extensive changes here. Previously, this class inheritted from `grpc_oauth2_token_fetcher_credentials` and fooled the base class into thinking that it directly fetched the oauth2 token, when in fact it actually performed a number of steps to gather data and then constructed a synthetic HTTP response to pass back to the base class. I have changed this to instead derive directly from `TokenFetcherCredentials` to provide a much cleaner interface with the parent class.
In addition, I have changed `grpc_call_credentials` from `RefCounted<>` to `DualRefCounted<>` to provide a clean way to shut down any in-flight token fetch when the credentials are unreffed.
This PR paves the way for subsequent work that will allow implementing an on-demand JWT token fetcher call credential, as part of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Closes#37510
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37510 from markdroth:token_fetcher_call_creds_refactor 3bd398a762
PiperOrigin-RevId: 666547985
I observed a case where `epoll_wait` would hang after receiving all available bytes (209 total). Note that all epoll events are configured to be edge-triggered (EPOLLET). The message was an HTTP request where the the HTTP headers indicated the connection would be closed. Presumably the peer would close their end of the connection, but the single epoll event did not contain EPOLLHUP. Regardless, the entire payload could be consumed in one read.
*Without* the TCP_INQ logic, the gRPC endpoint assumed that there was more data to read after the first read of 209 bytes. Since there was sufficient buffer space to read again, the endpoint called `recvmsg` a second time and immediately got EAGAIN. This triggered the downstream code to process the received data. Subsequent endpoint Reads would attempt a `recvmsg` first, and only if that failed, would then opt in to be notified of epoll events on their watched socket.
However, the INQ logic caused a nearly constant hang in this one environment, but non-deterministically. After a successful recvmsg (i.e., some bytes were received without any errors), INQ accurately showed there were no more bytes to read, and `recvmsg` was not called again, so the edge was never seen. The data was returned to the caller and the read was complete. The next endpoint Read would *first* ask to be notified of fd events, since the previous read claimed there was nothing left to read on that socket (this is the bug). Due to the nature of EPOLLET[1], the fd was very likely still in a ready state from the previous read, since it didn't call `recvmsg` until the edge was found. So the subsequent read was stuck waiting on a socket change event that may not come.
This fix resets the endpoint's state after consuming all available bytes according to TCP_INQ. This ensures that subsequent Reads will first attempt a read on the fd before requesting to be notified of new fd events.
[1]: https://www.man7.org/linux/man-pages/man7/epoll.7.html
> Receiving an event from epoll_wait(2) should suggest to you that such file descriptor is ready for the requested I/O operation. You must consider it ready until the next (nonblocking) read/write yields EAGAIN.
PiperOrigin-RevId: 666398394
Prior to this PR, the host and path were passed to the override functions, but the query params were unavailable to them. I have replaced the separate `host` and `path` parameters with a single parameter that passes the URI in as a const reference, which provides access to the query params.
While I was at it, I also changed the PUT and POST override methods to pass in the body as a string_view, which is more ergonomic for tests.
Closes#37540
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37540 from markdroth:httpcli_override_uri 902b0d2097
PiperOrigin-RevId: 665950188
Some applications cannot avoid the dependency on `//:grpc++_reflection` because it is buried by several infrastructural layers. For binaries that need to implement their own reflection server via generic handlers, this makes accepting request impossible as they will instead be handled by the default reflection server added by the server reflection plugin.
It would not be feasible to implement this as a build define because this binary is depended upon by many applications and buried beneath several layers of infrastructure.
Further, we cannot use the flag to control whether or not the plugin is installed as the plugin is installed at static initialization time, meaning that flags have not been parsed yet. Instead, we simply disable the functionality of the plugin with this flag.
CC @temawiCloses#37527
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37527 from gnossen:disable_cpp_reflection 8d7e32961a
PiperOrigin-RevId: 665051996
Extend Fuzzing timeouts to 1 year since the fuzzing EE caps delays passed to RunAfter to 1 year. This will prevent pre-mature timeouts of some legitimate fuzzed inputs.
Should fix b/358207194
PiperOrigin-RevId: 665012820
[Gpr_To_Absl_Logging] Remove gpr logging header from src/core headers.
Some of the cc files are using gpr_log_verbosity_init() functions.
So I removed the `#include <grpc/support/log.h>` from all headers and put it selectively only in the cc files that used gpr_log_verbosity_init()
Closes#37512
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37512 from tanvi-jagtap:remove_gpr_headers_from_headers a2f07f45d6
PiperOrigin-RevId: 664002265
[Gpr_To_Absl_Logging] Remove gpr logging header include from other headers
Some of the cc files are using gpr_log_verbosity_init() functions.
So I removed the #include <grpc/support/log.h> from all headers and put it selectively only in the cc files that used gpr_log_verbosity_init()
Closes#37513
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37513 from tanvi-jagtap:remove_gpr_headers_from_headers_01 612ca6d0f7
PiperOrigin-RevId: 663811895
[Gpr_To_Absl_Logging] Using GRPC_TRACE_LOG instead of GRPC_TRACE_FLAG_ENABLED
Closes#37516
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37516 from tanvi-jagtap:GRPC_TRACE_FLAG_ENABLED_09 c21f60f58a
PiperOrigin-RevId: 663811332
[Gpr_To_Absl_Logging] Fix use of GRPC_TRACE_FLAG_ENABLED in gen code
Closes#37514
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37514 from tanvi-jagtap:fix_gen_code_GRPC_TRACE_FLAG_ENABLED 096004c6d7
PiperOrigin-RevId: 663762507
[Gpr_To_Absl_Logging] Using GRPC_TRACE_LOG instead of GRPC_TRACE_FLAG_ENABLED
Closes#37483
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37483 from tanvi-jagtap:GRPC_TRACE_FLAG_ENABLED_08 ab8b98e8c4
PiperOrigin-RevId: 663618882
[Gpr_To_Absl_Logging] Using GRPC_TRACE_LOG instead of GRPC_TRACE_FLAG_ENABLED
Closes#37484
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37484 from tanvi-jagtap:GRPC_TRACE_FLAG_ENABLED_07 087d5fcfb3
PiperOrigin-RevId: 663606075
[Gpr_To_Absl_Logging] Remove logging header from example and test/cpp folder
Closes#37488
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37488 from tanvi-jagtap:remove_header_examples 1423ee77ff
PiperOrigin-RevId: 663597077
[Gpr_To_Absl_Logging] Remove logging header from example and test/core/t folder
Closes#37489
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37489 from tanvi-jagtap:remove_header_test_core_t fd5422b1f5
PiperOrigin-RevId: 663597008
[Gpr_To_Absl_Logging] Remove logging header from example and test/core/t folder
Closes#37491
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37491 from tanvi-jagtap:remove_header_test_core_02 e524a2da80
PiperOrigin-RevId: 663595237
[Gpr_To_Absl_Logging] Remove logging header from example and test/core/ folder
Closes#37492
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37492 from tanvi-jagtap:remove_header_test_core_misc fbcd02ab53
PiperOrigin-RevId: 663593905
[PH2][Refactor][FunctionSplit][chttp2_transport.cc]
Splitting very large function perform_stream_op_locked into smaller functions.
Adding new functions :
1. send_initial_metadata_locked
2. send_message_locked
3. send_trailing_metadata_locked
4. recv_initial_metadata_locked
5. recv_message_locked
6. recv_trailing_metadata_locked
All the above functions get called from perform_stream_op_locked .
Why was this done?
I ran into test failures while working on this : https://github.com/grpc/grpc/pull/37186/files .
The test failed because the function execute_stream_op was too large. So I had to split the function as well in the same PR (which is messy).
Decided to do the same type of splitting here.
Closes#37453
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37453 from tanvi-jagtap:split_function 5e06594e81
PiperOrigin-RevId: 663570972
Split off from https://github.com/grpc/grpc/pull/37425
We are adding and removing callbacks on the OpenTelemetry Async Instruments without synchronization. This opens us to races where we have an AddCallback and RemoveCallback operation happening at the same time. The correct result after these operations is to still have a callback registered with OpenTelemetry at the end, but the two operations could race and we could just decide to remove the OpenTelemetry callback.
The fix delays removing OpenTelemetry callbacks to plugin destruction time.
Closes#37485
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37485 from yashykt:FixRaceOTelGauge 016b0a41b5
PiperOrigin-RevId: 663492598
It is hard to reason about tests if multiple callbacks record values for the same metrics with the same label sets.
Closes#37486
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37486 from yashykt:SimplifyOTelPluginTest 3b4d7f90b3
PiperOrigin-RevId: 663457107
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37097
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37097 from XuanWang-Amos:add_1_65_to_interop 8709662244
PiperOrigin-RevId: 663325868
Mark D. Roth
AJ Heller
Esun Kim
Paulo Castello da Costa
Eugene Ostroukhov
gRPC Team Bot
Yash Tibrewal
Nathan Baulch
Evan Brown
Richard Belleville
Vignesh Babu
Tanvi Jagtap
Tanvi Jagtap
Nana Pang
Hannah Shi
Xuan Wang