This change moves the location for the prebuilt images used in gRPC OSS benchmarks from GCR to Artifact Registry.
Images are all saved to the same repo, instead of being organized by a prefix (`kokoro`, `kokoro-test`,
`cxx_experiment/kokoro-test`, etc.).
Closes#37558
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37558 from paulosjca:ar a767ab5d91
PiperOrigin-RevId: 666643972
Previously, `grpc_oauth2_token_fetcher_credentials` provided functionality for on-demand token-fetching, but it was integrated into the oauth2 code, so it was not possible to use that same code for on-demand fetching of (e.g.) JWT tokens. This PR splits that class into two parts:
1. A base `TokenFetcherCredentials` class that provides a framework for on-demand fetching of any arbitrary type of auth token.
2. An `Oauth2TokenFetcherCredentials` subclass that derives from `TokenFetcherCredentials` and provides handling for oauth2 tokens.
The `grpc_compute_engine_token_fetcher_credentials`, `StsTokenFetcherCredentials`, and `grpc_google_refresh_token_credentials` classes that previously derived from `grpc_oauth2_token_fetcher_credentials` now derive from `Oauth2TokenFetcherCredentials` instead, so there's not much change to those classes (other than a cleaner interface with the base class functionality).
The `ExternalAccountCredentials` class and its subclasses got more extensive changes here. Previously, this class inheritted from `grpc_oauth2_token_fetcher_credentials` and fooled the base class into thinking that it directly fetched the oauth2 token, when in fact it actually performed a number of steps to gather data and then constructed a synthetic HTTP response to pass back to the base class. I have changed this to instead derive directly from `TokenFetcherCredentials` to provide a much cleaner interface with the parent class.
In addition, I have changed `grpc_call_credentials` from `RefCounted<>` to `DualRefCounted<>` to provide a clean way to shut down any in-flight token fetch when the credentials are unreffed.
This PR paves the way for subsequent work that will allow implementing an on-demand JWT token fetcher call credential, as part of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Closes#37510
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37510 from markdroth:token_fetcher_call_creds_refactor 3bd398a762
PiperOrigin-RevId: 666547985
Some applications cannot avoid the dependency on `//:grpc++_reflection` because it is buried by several infrastructural layers. For binaries that need to implement their own reflection server via generic handlers, this makes accepting request impossible as they will instead be handled by the default reflection server added by the server reflection plugin.
It would not be feasible to implement this as a build define because this binary is depended upon by many applications and buried beneath several layers of infrastructure.
Further, we cannot use the flag to control whether or not the plugin is installed as the plugin is installed at static initialization time, meaning that flags have not been parsed yet. Instead, we simply disable the functionality of the plugin with this flag.
CC @temawiCloses#37527
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37527 from gnossen:disable_cpp_reflection 8d7e32961a
PiperOrigin-RevId: 665051996
[Gpr_To_Absl_Logging] Fix use of GRPC_TRACE_FLAG_ENABLED in gen code
Closes#37514
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37514 from tanvi-jagtap:fix_gen_code_GRPC_TRACE_FLAG_ENABLED 096004c6d7
PiperOrigin-RevId: 663762507
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37097
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37097 from XuanWang-Amos:add_1_65_to_interop 8709662244
PiperOrigin-RevId: 663325868
[Gpr_To_Absl_Logging] Removing absl_vlog2_enabled .
@apolcyn : Please review the Ruby code.
@yashykt : Please review the C++ code. And the python sanity test.
Closes#37476
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37476 from tanvi-jagtap:remove_should_fn c05fd6445e
PiperOrigin-RevId: 663193927
Fix for grpc_build_protobuf_at_head -> python_linux_opt_native_buildonly timeout
Found the following error during Python build:
When we mount and reuse the existing repo from host machine inside docker container, the `tools/bazel.rc` file is shared to the docker container and the Bazel override host location written to `tools/bazel.rc` from tools/.../grpc_build_submodule_at_head.sh (outside docker container) forces bazel to look for the same host location inside the docker container, which doesn't exist.
Hence overriding it again with the working directory inside the container should solve this issue.
Closes#37404
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37404 from sreenithi:fix_build_protobuf_at_head_timeout cae74938ff
PiperOrigin-RevId: 662526169
release notes: no
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37439
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37439 from purnesh42H:add-psm-dualstack-config 94b4767c5c
PiperOrigin-RevId: 661450981
[Gpr_To_Absl_Logging] Remove gpr_should_log from the header file
Closes#37420
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37420 from tanvi-jagtap:clean_up_ruby_php_01 ce303fa808
PiperOrigin-RevId: 661118631
Centos7 reached EOL on June 30th
The test was still working until google-protobuf released 3.25.4 a ~week ago, which seems to no longer compile on centos 7 b/c it needs a new header, `stdatomic.h`
Closes#37401
PiperOrigin-RevId: 659992748
The microbenchmarks are tools for us, and it doesn't seem like we're buying much by maintaining them for different build systems.
Closes#37391
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37391 from ctiller:pickthis 24184f2b44
PiperOrigin-RevId: 658861012
Two new benchmarks here-in.
Benchmark 1: `bm_picker`
------
Measures various load balancing policies pick performance. For now we cover `pick_first` and `weighted_round_robin` at 1, 10, 100, 1000, 10000, and 100000 backends.
Today's output:
```
------------------------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------------------------
BM_Pick/pick_first/1 20.4 ns 20.4 ns 68285
BM_Pick/pick_first/10 20.6 ns 20.6 ns 68274
BM_Pick/pick_first/100 20.5 ns 20.5 ns 67817
BM_Pick/pick_first/1000 20.6 ns 20.6 ns 67347
BM_Pick/pick_first/10000 20.7 ns 20.7 ns 67317
BM_Pick/pick_first/100000 20.9 ns 20.9 ns 67385
BM_Pick/weighted_round_robin/1 54.7 ns 54.7 ns 26641
BM_Pick/weighted_round_robin/10 54.2 ns 54.2 ns 25828
BM_Pick/weighted_round_robin/100 55.2 ns 55.2 ns 26210
BM_Pick/weighted_round_robin/1000 54.1 ns 54.1 ns 25678
BM_Pick/weighted_round_robin/10000 77.3 ns 76.6 ns 15776
BM_Pick/weighted_round_robin/100000 148 ns 148 ns 9882
```
Benchmark 2: `bm_load_balanced_call_destination`
-----
This benchmark measures call performance when a call spine passes through a `LoadBalancedCallDestination`, and with `BM_LoadBalancedCallDestination` also the construction/destruction cost of this object.
We do not consider picker performance in this benchmark as it's separately covered by `bm_picker` above.
Today's output:
```
-----------------------------------------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------------------------------------------------------------------------
BM_UnaryWithSpawnPerEnd<UnstartedCallDestinationFixture<LoadBalancedCallDestinationTraits>> 1255 ns 1255 ns 1076
BM_UnaryWithSpawnPerOp<UnstartedCallDestinationFixture<LoadBalancedCallDestinationTraits>> 1459 ns 1459 ns 939
BM_ClientToServerStreaming<UnstartedCallDestinationFixture<LoadBalancedCallDestinationTraits>> 209 ns 209 ns 6775
BM_LoadBalancedCallDestination 92.8 ns 92.8 ns 15063
```
Notes
------
There's some duplicated code between the benchmarks & tests -- this is ok -- as the tests evolve we'll likely want to add more checks to the fixtures, whereas as the benchmarks evolve we may well want to optimize the fixtures so that performance of the systems under test dominate more. That is, the duplicated code is expected to have different evolutionary tracks.
Closes#37052
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37052 from ctiller:moar-benchy 30c7072d87
PiperOrigin-RevId: 658181731
The Ruby artifact build is timing out at 1hr30m, specifically `build_artifact.ruby_native_gem_linux_aarch64-linux` in the `tools/internal_ci/linux/grpc_distribtests_ruby.sh` job. Most of the other Ruby builds take around 1hr15m, so the build time is increasing regardless.
@stanley-cheung this should probably be investigated. In the meantime, to hopefully unblock the v1.66 release, let's increase the build timeout.
Closes#37341
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37341 from drfloob:bump-ruby-artifact-build-timeout 8219cc9a33
PiperOrigin-RevId: 657747546
This setting has no utility in general open source, but is still useful in other environments. This PR ensures that there are no `debug` configurations when the default codegen CI runs. This simplifies the release process as well.
Closes#37277
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37277 from drfloob:gen-exp-no-dbg 6517fec6e4
PiperOrigin-RevId: 654928917
Change was created by the release automation script. See go/grpc-release.
Closes#37279
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37279 from drfloob:bump_dev_version_202407222027 4e6607411e
PiperOrigin-RevId: 654925894
Change was created by the release automation script. See go/grpc-release.
Closes#37276
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37276 from drfloob:bump_core_version_202407221838 7f287d8546
PiperOrigin-RevId: 654867615
The oldest gcc version that gRPC supports as of today is gcc 7 but gcc 7 has an issue with template supports that gRPC already picked up. Recently we managed to fix it in gRPC library code but we still have some in our test code. Given that it's not easy to fix since it requires many trial error approach to find a way to satisfy gcc 7 and eventually gcc 7 will be dropped from our supported compilers, let's have this mitigation where just main grpc++ target is being tested for gcc 7 so that users can use grpc with it without having to fix this hairy issue.
Fixes https://github.com/grpc/grpc/issues/36751Closes#37257
PiperOrigin-RevId: 654076384
I added this support about a year ago experimentally, but we don't need it. Removing it results in much simpler codegen.
Closes#37145
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37145 from ctiller:movie 086620ecc9
PiperOrigin-RevId: 653353272
- add a benchmark for various metadata creation styles
- add factory functions for status + message - these are 3-10x faster than going via absl::Status
- add a `MakePooledForOverwrite` function to Arena, use it everywhere -- this naming matches `std::make_unique_for_overwrite` in C++20, and avoids some language mandated initialization in `Table` (underlying `MetadataMap<>`) - speeding creation of metadata handles by 30%
For `bm_call_spine` we see before:
```
BM_UnaryWithSpawnPerEnd<CallSpineFixture>_median 745 ns 745 ns
```
and after:
```
BM_UnaryWithSpawnPerEnd<CallSpineFixture>_median 699 ns 699 ns
```
Closes#37111
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37111 from ctiller:meta-magic-2 100464fc7e
PiperOrigin-RevId: 652900726
Paulo Castello da Costa
Mark D. Roth
Eugene Ostroukhov
Esun Kim
Nathan Baulch
Richard Belleville
Tanvi Jagtap
Tanvi Jagtap
Nana Pang
Xuan Wang
Sreenithi Sridharan
Purnesh Dixit
apolcyn
Craig Tiller
Vignesh Babu
AJ Heller
Larry Safran
Yash Tibrewal