Prior to this PR, the host and path were passed to the override functions, but the query params were unavailable to them. I have replaced the separate `host` and `path` parameters with a single parameter that passes the URI in as a const reference, which provides access to the query params.
While I was at it, I also changed the PUT and POST override methods to pass in the body as a string_view, which is more ergonomic for tests.
Closes#37540
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37540 from markdroth:httpcli_override_uri 902b0d2097
PiperOrigin-RevId: 665950188
Extend Fuzzing timeouts to 1 year since the fuzzing EE caps delays passed to RunAfter to 1 year. This will prevent pre-mature timeouts of some legitimate fuzzed inputs.
Should fix b/358207194
PiperOrigin-RevId: 665012820
[Gpr_To_Absl_Logging] Remove gpr logging header include from other headers
Some of the cc files are using gpr_log_verbosity_init() functions.
So I removed the #include <grpc/support/log.h> from all headers and put it selectively only in the cc files that used gpr_log_verbosity_init()
Closes#37513
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37513 from tanvi-jagtap:remove_gpr_headers_from_headers_01 612ca6d0f7
PiperOrigin-RevId: 663811895
[Gpr_To_Absl_Logging] Remove logging header from example and test/cpp folder
Closes#37488
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37488 from tanvi-jagtap:remove_header_examples 1423ee77ff
PiperOrigin-RevId: 663597077
[Gpr_To_Absl_Logging] Remove logging header from example and test/core/t folder
Closes#37489
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37489 from tanvi-jagtap:remove_header_test_core_t fd5422b1f5
PiperOrigin-RevId: 663597008
[Gpr_To_Absl_Logging] Remove logging header from example and test/core/t folder
Closes#37491
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37491 from tanvi-jagtap:remove_header_test_core_02 e524a2da80
PiperOrigin-RevId: 663595237
[Gpr_To_Absl_Logging] Remove logging header from example and test/core/ folder
Closes#37492
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37492 from tanvi-jagtap:remove_header_test_core_misc fbcd02ab53
PiperOrigin-RevId: 663593905
Split off from https://github.com/grpc/grpc/pull/37425
We are adding and removing callbacks on the OpenTelemetry Async Instruments without synchronization. This opens us to races where we have an AddCallback and RemoveCallback operation happening at the same time. The correct result after these operations is to still have a callback registered with OpenTelemetry at the end, but the two operations could race and we could just decide to remove the OpenTelemetry callback.
The fix delays removing OpenTelemetry callbacks to plugin destruction time.
Closes#37485
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37485 from yashykt:FixRaceOTelGauge 016b0a41b5
PiperOrigin-RevId: 663492598
It is hard to reason about tests if multiple callbacks record values for the same metrics with the same label sets.
Closes#37486
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37486 from yashykt:SimplifyOTelPluginTest 3b4d7f90b3
PiperOrigin-RevId: 663457107
Includes a few changes to pollset stuff to make it easier to not use pollsets (which I think is going to be generally helpful in the coming months)
Closes#37397
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37397 from ctiller:client-chicken 1099cd500a
PiperOrigin-RevId: 660014128
The microbenchmarks are tools for us, and it doesn't seem like we're buying much by maintaining them for different build systems.
Closes#37391
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37391 from ctiller:pickthis 24184f2b44
PiperOrigin-RevId: 658861012
Two new benchmarks here-in.
Benchmark 1: `bm_picker`
------
Measures various load balancing policies pick performance. For now we cover `pick_first` and `weighted_round_robin` at 1, 10, 100, 1000, 10000, and 100000 backends.
Today's output:
```
------------------------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------------------------
BM_Pick/pick_first/1 20.4 ns 20.4 ns 68285
BM_Pick/pick_first/10 20.6 ns 20.6 ns 68274
BM_Pick/pick_first/100 20.5 ns 20.5 ns 67817
BM_Pick/pick_first/1000 20.6 ns 20.6 ns 67347
BM_Pick/pick_first/10000 20.7 ns 20.7 ns 67317
BM_Pick/pick_first/100000 20.9 ns 20.9 ns 67385
BM_Pick/weighted_round_robin/1 54.7 ns 54.7 ns 26641
BM_Pick/weighted_round_robin/10 54.2 ns 54.2 ns 25828
BM_Pick/weighted_round_robin/100 55.2 ns 55.2 ns 26210
BM_Pick/weighted_round_robin/1000 54.1 ns 54.1 ns 25678
BM_Pick/weighted_round_robin/10000 77.3 ns 76.6 ns 15776
BM_Pick/weighted_round_robin/100000 148 ns 148 ns 9882
```
Benchmark 2: `bm_load_balanced_call_destination`
-----
This benchmark measures call performance when a call spine passes through a `LoadBalancedCallDestination`, and with `BM_LoadBalancedCallDestination` also the construction/destruction cost of this object.
We do not consider picker performance in this benchmark as it's separately covered by `bm_picker` above.
Today's output:
```
-----------------------------------------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
-----------------------------------------------------------------------------------------------------------------------------------------
BM_UnaryWithSpawnPerEnd<UnstartedCallDestinationFixture<LoadBalancedCallDestinationTraits>> 1255 ns 1255 ns 1076
BM_UnaryWithSpawnPerOp<UnstartedCallDestinationFixture<LoadBalancedCallDestinationTraits>> 1459 ns 1459 ns 939
BM_ClientToServerStreaming<UnstartedCallDestinationFixture<LoadBalancedCallDestinationTraits>> 209 ns 209 ns 6775
BM_LoadBalancedCallDestination 92.8 ns 92.8 ns 15063
```
Notes
------
There's some duplicated code between the benchmarks & tests -- this is ok -- as the tests evolve we'll likely want to add more checks to the fixtures, whereas as the benchmarks evolve we may well want to optimize the fixtures so that performance of the systems under test dominate more. That is, the duplicated code is expected to have different evolutionary tracks.
Closes#37052
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37052 from ctiller:moar-benchy 30c7072d87
PiperOrigin-RevId: 658181731
Sample output:
```
➜ grpc git:(otel-metrics-benchmark) ✗ bazel-bin/test/cpp/microbenchmarks/bm_stats_plugin
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721695619.615582 2126186 config.cc:257] gRPC experiments enabled: call_status_override_on_cancellation, call_tracer_in_transport, event_engine_dns, event_engine_listener, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
2024-07-22T17:46:59-07:00
Running bazel-bin/test/cpp/microbenchmarks/bm_stats_plugin
Run on (48 X 2450 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x24)
L1 Instruction 32 KiB (x24)
L2 Unified 512 KiB (x24)
L3 Unified 32768 KiB (x3)
Load Average: 1.16, 0.85, 0.85
***WARNING*** Library was built as DEBUG. Timings may be affected.
---------------------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------------------
BM_AddCounterWithFakeStatsPlugin 1738 ns 1738 ns 404265
BM_AddCounterWithOTelPlugin 757 ns 757 ns 928142
I0000 00:00:1721695621.304593 2126186 test_config.cc:186] TestEnvironment ends
```
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37282
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37282 from yijiem:otel-metrics-benchmark eeba3dfb5e
PiperOrigin-RevId: 655286398
1. Fixing unit test that flags log noise.
2. This test was broken for many months. As a result , a lot of log noise was added. Removing the noise as a part of the PR.
3. If we want to retain any log line as `INFO` instead of `VLOG(2)`, please let me know, I will add it to allow list.
4. In this PR , we replace the old `gpr_set_log_function` mechanism with an `absl LogSink` . So here , `Send` function will do everything that `NoLog` used to do before.
Closes#37177
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37177 from tanvi-jagtap:fix_nologging_tests ad58e2fb79
PiperOrigin-RevId: 655209718
1. Function gpr_default_log has been deprecated. This function will be deleted in a few weeks.
1. This entire unit test is being re-written as a part of another PR. But that PR will take a while to merge. In the mean team I want to delete all instances of this function to prevent further backsliding.
https://github.com/grpc/grpc/pull/37177
PiperOrigin-RevId: 654998772
1. Function gpr_set_log_verbosity has been deprecated. This function will be deleted in a few weeks.
1. gRPC now internally uses absl logging. Earlier gRPC was using its own custom logging mechanism called gpr which had a whole set of functions beginning with gpr_.
1. This entire unit test is being re-written as a part of another PR. But that PR will take a while to merge. In the mean team I want to delete all instances of this function to prevent further backsliding.
https://github.com/grpc/grpc/pull/37177
PiperOrigin-RevId: 654639955
Previously the registered callback's duration is set too low (10ms) that 2 different OTel callbacks get called from OTel consecutively would trigger the registered callback again, making the test flaky.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37243
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37243 from yijiem:otel-plugin-test-flake 70a1572cd2
PiperOrigin-RevId: 653717643
- add a benchmark for various metadata creation styles
- add factory functions for status + message - these are 3-10x faster than going via absl::Status
- add a `MakePooledForOverwrite` function to Arena, use it everywhere -- this naming matches `std::make_unique_for_overwrite` in C++20, and avoids some language mandated initialization in `Table` (underlying `MetadataMap<>`) - speeding creation of metadata handles by 30%
For `bm_call_spine` we see before:
```
BM_UnaryWithSpawnPerEnd<CallSpineFixture>_median 745 ns 745 ns
```
and after:
```
BM_UnaryWithSpawnPerEnd<CallSpineFixture>_median 699 ns 699 ns
```
Closes#37111
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37111 from ctiller:meta-magic-2 100464fc7e
PiperOrigin-RevId: 652900726
[Gpr_To_Absl_Logging] Remove GRPC_API_TRACE .
This will be replaced by GRPC_TRACE_LOG
Closes#37190
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37190 from tanvi-jagtap:grpc_api_trace_fix 6dae72bc6e
PiperOrigin-RevId: 652397141
Instead of passing the transport byte counts back up through the filter
stack to be reported to the `CallTracer`, we now have the transport
pass the transport byte counts directly to the `CallTracer` itself.
This will eventually allow us to avoid unnecessarily storing these byte
counts in cases where no `CallTracer` actually cares about the data, which
will reduce per-call memory. (In the short term, it actually increases
memory usage, but we can separately do some work to avoid the memory
usage in the transport by removing the `grpc_transport_stream_stats`
struct from the legacy filter API.)
This is a prereq for supporting `CallTracer` in the new call v3 stack,
which does not include the transport byte counts as part of the
receieve-trailing-metadata hook, unlike the legacy filter stack.
This change is controlled by the `call_tracer_in_transport` experiment,
which is enabled by default.
As part of this experiment, we also fix a couple of related bugs:
- On the client side, the chttp2 transport was incorrectly adding
annotations to the parent `ClientCallTracer` instead of the
`CallAttemptTracer`.
- The OpenCensus `ServerCallTracer` was incorrectly swapping the values
of sent and received bytes.
PiperOrigin-RevId: 650728181
[Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#37148
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37148 from tanvi-jagtap:src_core_lib_grpp_time 525cf70e24
PiperOrigin-RevId: 650460526
As per gRFC A81 (https://github.com/grpc/proposal/pull/435).
A few small improvements along the way:
- Use `DownCast<>` instead of `static_cast<>` in `ServiceConfigCallData::GetCallAttribute()`.
- Add a convenient templated wrapper for `ClientChannelLbCallState::GetCallAttribute()`, similar to the one on `ServiceConfigCallData`.
- Added a `ParseBoolValue()` helper to xds_common_types_parser, and use it in various places in xDS resource parsing.
Closes#37087
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37087 from markdroth:xds_authority_rewriting 3679d2b061
PiperOrigin-RevId: 650293912
As part of the xDS authority rewriting implementation in #37087, I need to write some tests showing the authority being rewritten. However, the fake security connector currently crashes on unexpected authorities. (As a side note, I think the fake security connector is very cumbersome and should be redesigned, but that's a separate project.) As a result, I need a way to use InsecureCreds on a per-test basis.
I thought about just adding an option to `XdsTestType` to trigger use of InsecureCreds, but the logic we use for determining which creds type to use for what is already very cumbersome, and adding another option there would have just made that worse. Instead, I have switched to a simpler approach where the individual tests can decide what creds type to use directly. This both unblocks my other PR and makes the existing code more maintainable.
Closes#37156
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37156 from markdroth:xds_e2e_test_creds_improvement 2e813834d0
PiperOrigin-RevId: 649225248
Mark D. Roth
Yash Tibrewal
Vignesh Babu
Tanvi Jagtap
Craig Tiller
Alisha Nanda
Yijie Ma
Tanvi Jagtap
alto-ruby
Esun Kim