Implements https://github.com/grpc/proposal/pull/429
Currently, the behavior of `GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA` blocks more pings from being sent if we are sending too many pings without a data/header frame being sent as well. The original intention of this channel arg was to play nice with proxies that have restrictive settings when it comes to pings. This causes awkwardness when configuring keepalive pings for transports with long lived streams with sparse communication. In such a case, gRPC Core would stop sending keepalive pings since no data/header frame is being sent, resulting in a situation where we are unable to detect whether the transport is alive or not.
This change adds an experiment "max_pings_wo_data_throttle" to modify the behavior of `GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA` to throttle pings to a frequency of 1 minute instead of completely blocking pings when too many pings have been sent without data/header frames.
Closes#36374
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36374 from yashykt:ThrottlePings b5bd42a019
PiperOrigin-RevId: 638110795
Introduce "Yodel" - a framework for testing things vaguely related to calls.
This is breaking up some work I did for transport test suites - we've got a nice way of spawning test-only promises and tracking them visually, and support for setting up an environment that can run as a test or a fuzzer. I'm making that piece a little more reusable, and then rebasing the transport test suite atop that infrastructure.
Closes#36635
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36635 from ctiller:transport-refs-6 843a9f4b7e
PiperOrigin-RevId: 637022756
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36701
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36701 from tanvi-jagtap:test_core_gpr_log 1d8c69e9c6
PiperOrigin-RevId: 636850577
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36703
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36703 from tanvi-jagtap:test_core_gpr_log_01 26c4307b08
PiperOrigin-RevId: 636801504
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36678
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36678 from tanvi-jagtap:test_cpp_gpr_log 81b8f4179f
PiperOrigin-RevId: 636410944
In a follow-up PR, we will use these helper APIs to add validation checks when creating TLS credentials objects.
Closes#36357
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36357 from matthewstevenson88:pem-util dca4f2e6c5
PiperOrigin-RevId: 636162540
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK.
These changes have been made using string replacement and regex.
Closes#36657
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36657 from tanvi-jagtap:gpr_assert_mm_tjagtap d8a7f0262a
PiperOrigin-RevId: 635734174
This fixes breakage triggered by #36434.
Changing the order of the data members in `TraceEvent` caused the total size of the structure to be different, which meant that the long string we were adding was enough to cause 3 events to be evicted instead of 2.
b/339066599
Closes#36643
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36643 from markdroth:channel_trace_test_fix 94586faced
PiperOrigin-RevId: 634505085
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36608
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36608 from tanvi-jagtap:regex_test_core_misc e6b9481dac
PiperOrigin-RevId: 634246311
With 1k iterations instead of 10k iterations, this test takes ~90s in the RBE environment. There must have been a regression somewhere, since this test did not used to fail every time.
Closes#36612
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36612 from drfloob:nerf-writes-per-rpc 0ba8af4167
PiperOrigin-RevId: 633677652
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - BUILD
In this CL we are just editing the build and bzl files to add dependencies.
This is done to prevent merge conflict and constantly having to re-make the make files using generate_projects.sh for each set of changes.
Closes#36604
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36604 from tanvi-jagtap:build_test_core_misc_01 8995ba4914
PiperOrigin-RevId: 633519619
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - BUILD
In this CL we are just editing the build and bzl files to add dependencies.
This is done to prevent merge conflict and constantly having to re-make the make files using generate_projects.sh for each set of changes.
Closes#36606
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36606 from tanvi-jagtap:build_test_core_tsi_and_misc 708a724c46
PiperOrigin-RevId: 633518709
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - BUILD
In this CL we are just editing the build and bzl files to add dependencies.
This is done to prevent merge conflict and constantly having to re-make the make files using generate_projects.sh for each set of changes.
Closes#36603
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36603 from tanvi-jagtap:build_test_core_event_engine 8b7188e444
PiperOrigin-RevId: 633517518
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - BUILD
In this CL we are just editing the build and bzl files to add dependencies.
This is done to prevent merge conflict and constantly having to re-make the make files using generate_projects.sh for each set of changes.
Closes#36605
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36605 from tanvi-jagtap:build_test_core_misc_02 f928eb19de
PiperOrigin-RevId: 633516934
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36595
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36595 from tanvi-jagtap:regex_test_core_iomgr_transport b0836fda1c
PiperOrigin-RevId: 633456839
This cleans up the channel trace test to provide better output when tests fail, which will help diagnose the problem with b/339066599.
Closes#36582
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36582 from markdroth:channelz_test_fix fd287182ad
PiperOrigin-RevId: 633358151
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36509
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36509 from ctiller:transport-refs-3 2771a2b0e1
PiperOrigin-RevId: 633240374
Not the fastest implementation possible, but it's a log helper so I'm not particularly fussed either -- but a useful utility that we can iterate on later to help debugging.
Closes#36554
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36554 from ctiller:args 33b4802fcc
PiperOrigin-RevId: 632997320
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36575
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36575 from tanvi-jagtap:test_core_e2e_all ef6f23db8c
PiperOrigin-RevId: 632669009
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36577
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36577 from tanvi-jagtap:test_core_all 9887d74d81
PiperOrigin-RevId: 632441576
This adds compile-time checking that the type of the value and the size of the labels and optional labels passed when recording a metric must match with the type and the size specified when the metric is registered.
The RegistrationBuilder API idea is credited to @ctiller.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36449
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36449 from yijiem:registration-builder-api a727810136
PiperOrigin-RevId: 632271022
Add a test utility to assert there are no memory leaks at an arbitrary point in code (when running under ASAN).
Use that utility to help diagnose why event engine fails to shut down.
.. also moves some preprocessor junk around in build.cc to make that file a little easier to parse
Closes#36520
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36520 from ctiller:leaky-boat 540bfca8c6
PiperOrigin-RevId: 631552535
Notes:
* The special `on_write` callback was never used, all slices were discarded. I removed that functionality.
Closes#36513
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36513 from drfloob:rewrite-mock-endpoint-to-ee e45a964633
PiperOrigin-RevId: 631187792
Fuzzers were OOMing while trying to create 10k errors. This sets a sane default.
Closes#36537
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36537 from drfloob:less-validation-noise db835d948a
PiperOrigin-RevId: 631158900
Also begin to eliminate `CallContext` in favor of just exposing `Call` - ultimately there's not really a need to introduce two types here, so I'm going to wind that idea back over a few PRs.
I've avoided making this an experiment as the changes required were quite structural.
Closes#36477
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36477 from ctiller:deadline-time 9856eeebe6
PiperOrigin-RevId: 629599230
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36389
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36389 from sastryvp:fuzzer_331840428 818112d75d
PiperOrigin-RevId: 629331008
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK.
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36457
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36457 from tanvi-jagtap:tjagtap_misc_test 978d0411b8
PiperOrigin-RevId: 628949744
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36439
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36439 from tanvi-jagtap:tjagtap_iomgr_large fd759fac84
PiperOrigin-RevId: 628564520
Craig Tiller
Mark D. Roth
Yash Tibrewal
Tanvi Jagtap
Esun Kim
Matthew Stevenson
AJ Heller
Luwei Ge
Yijie Ma
Vishwanath Sastry