[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36881
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36881 from tanvi-jagtap:src_core_load_balancing_gpr_log 4c2abe9061
PiperOrigin-RevId: 643593750
Previously we'd enter a wakeup storm if there were too many concurrent senders. Now we allow a small burst over the send limit (up to the number of concurrent senders on the mpsc), and make the wait until that send passes to the receiver. In this way we don't wake all pending senders even if there's not sufficient queue space available.
Closes#36862
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36862 from ctiller:mpsc-quadratic 4d2ad48e2d
PiperOrigin-RevId: 643375554
Fixes the CBF of `src/ruby/end2end/killed_client_thread_test.rb` (failure mode is a hang of the child process that receives the SIGTERM) that has been happening since https://github.com/grpc/grpc/pull/36724
So far grpc-ruby CQ pluck operations have so far used a 20ms-interval busy poll to check interrupts in case we've received a signal, handle process shutdown, etc. This means ongoing RPCs will not terminate their CQ operations if we need to terminate the process (the loop simply exits without waiting for the CQ op to finish), causing a leak. Those RPCs can leave refs over their corresponding channels preventing [this](8564f72e8e/src/ruby/ext/grpc/rb_channel.c (L653)) from terminating (the channels don't reach state SHUTDOWN after being destroyed).
Fix is to unblock CQ pluck operations by cancelling calls, and thus allowing the CQ pluck to actually complete its operation. For server listening CQ operations, we unblock them by shutting down the server.
A side win here is to remove the [20ms-interval busy poll](8564f72e8e/src/ruby/ext/grpc/rb_completion_queue.c (L44)) on CQ operations, which was only needed to handle shutdown.
Closes#36903
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36903 from apolcyn:fix_ruby_interrupt bed1ee2feb
PiperOrigin-RevId: 643046465
Change was created by the release automation script. See go/grpc-release.
Closes#36892
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36892 from XuanWang-Amos:bump_dev_version_202406112301 9b2898d716
PiperOrigin-RevId: 643027546
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with c assert.
These changes have been made using string replacement and regex.
Closes#36817
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36817 from tanvi-jagtap:gpr_assert_removal_in_php 2222f95413
PiperOrigin-RevId: 642855768
Also made some minor improvements to the `ConfigSelector` API.
Closes#36877
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36877 from markdroth:client_channel_v3_dynamic_filters 6a539fe320
PiperOrigin-RevId: 642755276
Seems MSVC has trouble compiling this statically initialized lambda in a template; crash goes away when I split it out to be a named function instead.
Closes#36893
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36893 from ctiller:edward 4bf9b88797
PiperOrigin-RevId: 642714465
- ensure ordering of `OnAccept` and `Shutdown` callbacks from thready event engine (previously these could be reordered and this caused spurious failures)
- enable thready_tsan for one C++ e2e test
- don't filter thready_tsan for local builds (only CI)
Closes#36886
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36886 from ctiller:erm e3b88e7d86
PiperOrigin-RevId: 642702724
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36880
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36880 from tanvi-jagtap:src_cpp_folder_gpr_log e6e795f469
PiperOrigin-RevId: 642436269
Change was created by the release automation script. See go/grpc-release.
Closes#36887
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36887 from XuanWang-Amos:bump_core_version_202406111526 5df15650dd
PiperOrigin-RevId: 642358496
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36875
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36875 from yousukseung:reenable-work-serializer-dispatch da03e7d9df
PiperOrigin-RevId: 642347062
Refines the test in the fuzzer to only check input == output if the frame was re-encodable.
Also tweaks some hpack encoder stuff to not crash but simply report errors where appropriate.
Closes#36821
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36821 from ctiller:f-frame 1b08875c4e
PiperOrigin-RevId: 642311442
Reattempt enabling the Windows EventEngine client experiment, after fixing a race condition in connection establishment vs timeout.
Closes#36876
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36876 from drfloob:wineec 125f3a20d8
PiperOrigin-RevId: 642294760
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36800
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36800 from tanvi-jagtap:src_core_lib cc0411e99d
PiperOrigin-RevId: 642109824
[Gpr_To_Absl_Logging] Move function to test header form log.h
This is not really needed in log.h
Closes#36860
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36860 from tanvi-jagtap:move_function_to_test_header e6494bd06f
PiperOrigin-RevId: 642080756
gRPC shutdown may happen asynchronously with `work_serializer_dispatch`, so wait briefly until shutdown completes in the child's post-fork.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36873
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36873 from yousukseung:python-fork-shutdown-wait 7908a6912a
PiperOrigin-RevId: 642066286
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36729
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36729 from yijiem:per-channel-stats-plugin 4786bed42f
PiperOrigin-RevId: 642030366
[Gpr_To_Absl_Logging] Removing instances of gpr_should_log and replacing it with ABSL_VLOG_IS_ON.
gpr_should_log function will be deleted soon.
**VLOG(2)** is equivalent to gpr_log(**GPR_DEBUG**, ... )
We are replacing two instances of **gpr_log(GPR_INFO**, ... ) with **VLOG(2)** because this code appears to be slightly expensive.
I can also continue to use LOG(INFO) and check current settings using absl::MinLogLevel() . Would you want me to do that?
Closes#36866
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36866 from tanvi-jagtap:fix_secure_endpoint_gpr_should 556006c720
PiperOrigin-RevId: 641939641
[Gpr_To_Absl_Logging] Removing instances of gpr_should_log.
Replacing it with ABSL_VLOG_IS_ON.
VLOG(2) is equivalent to gpr_log(GPR_DEBUG, ... )
This function will be deprecated and deleted soon.
Closes#36861
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36861 from tanvi-jagtap:gpr_should_log_DEBUG_replace 09cd39e05a
PiperOrigin-RevId: 641939617
[Gpr_To_Absl_Logging] Removing instances of gpr_should_log and replacing it with ABSL_VLOG_IS_ON.
gpr_should_log function will be deleted soon.
**VLOG(2)** is equivalent to gpr_log(**GPR_DEBUG**, ... )
We are replacing two instances of **gpr_log(GPR_INFO**, ... ) with **VLOG(2)** because this code appears to be slightly expensive.
I can also continue to use LOG(INFO) and check current settings using absl::MinLogLevel() . Would you want me to do that?
Closes#36868
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36868 from tanvi-jagtap:fix_tcp_windows_gpr_should_log c2b660d4e2
PiperOrigin-RevId: 641938430
[Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_set_log_function
Adding a warning for users of gpr_set_log_function.
This function will be deleted soon.
https://github.com/grpc/proposal/pull/425
Other options considered ( and discarded )
1. Using DFATAL instead of ERROR. Discarded this because we still have one critical unit test to clean up before we delete gpr_set_log_function. This test will fail if we use DFATAL. EXPECT_DFATAL macro is not available to be used.
2. Considered making a new absl log sink , and then that log sink should have a function pointer with the same signature as gpr_log_func. This function pointer can be set from gpr_set_log_function. However if this custom function uses absl from within it, that will cause infinite recursion. Made me drop the idea.
Closes#36844
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36844 from tanvi-jagtap:assert_gpr_set_log_function 805de358a9
PiperOrigin-RevId: 641473969
All TraceFlags are now configured in `src/core/lib/debug/trace_flags.yaml`. The format is:
```
my_flag:
default: false # the default value; default=false
description: Some Description
debug_only: false # debug_only flags only work in debug builds; default=false
internal: false # internal flags will not show up in documentation; default=false
```
To regenerate the trace flag source code, run `tools/codegen/core/gen_trace_flags.py` (requires mako). This script is also run when sanity checking.
This PR also adds two new features:
### Glob-based flag configuration
Trace flag configuration now supports `?` (single wildcard character) and `*` (one or more wildcard characters). For example, using `GRPC_TRACE='event_engine*'` will enable all flags that match that glob. It expands to:
* event_engine
* event_engine_client_channel_resolver
* event_engine_dns
* event_engine_endpoint
* event_engine_endpoint_data
* event_engine_poller
### A cleaner trace-logging macro in abseil logging format
If your goal is only to add log statements when the `fault_injection_filter` trace flag is enabled, you can use the macro:
```
GRPC_TRACE_LOG(fault_injection, INFO) << "Filtered:" << 42;
```
When the trace flag is enabled, the the log will show something like this:
```
I0000 00:00:1715733657.430042 16 file.cc:174] Filtered:42
```
----
Note: just like with the gpr_log to abseil logging conversion, the pre-existing trace logging usages can be replaced with the new tracing macro across multiple PRs.
Closes#36576
PiperOrigin-RevId: 641295215
[grpc][Gpr_To_Absl_Logging]
1. Supporting the legacy GRPC_VERBOSITY environment variable in the new absl logging implementation.
2. Adding a new way to disable VLOG logging.
3. Documenting the recommendations clearly.
4. Editing the init code.
Additional Context :
Check function gpr_default_log() for more context
https://github.com/search?q=repo%3Agrpc%2Fgrpc%20gpr_default_log&type=codeCloses#36798
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36798 from tanvi-jagtap:grpc_verbosity_flag_support ab0d600849
PiperOrigin-RevId: 641092851
Usage: `bazel build --config=clang-cl --build_tag_filters=-no_windows :all`
The highlight, for me anyway: thread safety annotation checking can now be done on Windows-only code.
clang-cl's interpretation of `-Wall` differs from that of clang on linux. This PR uses the set of warnings enabled on linux, and whittles down the list until all builds pass. I left comments in `copts.bzl` describing the warnings, many of which can be removed with targeted code cleanups.
Closes#36831
PiperOrigin-RevId: 641009783
We changed the way flag was passed in this PR: https://github.com/grpc/psm-interop/pull/94
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36827
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36827 from XuanWang-Amos:csm_fix_interop a2c2c47f30
PiperOrigin-RevId: 640908453
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK.
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
Closes#36816
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36816 from tanvi-jagtap:gpr_assert_removal 9db548beb0
PiperOrigin-RevId: 640731171
It's still possible to define GRPC_IOS_EVENT_ENGINE_CLIENT=0 to disable event engine for iOS.
Closes#36785
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36785 from HannahShiSFB:default-to-event-engine-in-ios 441fe552a6
PiperOrigin-RevId: 640716651
This gives grpc_endpoint the same destruction-is-shutdown semantic as
EventEngine::Endpoint, which will make the migration easier.
PiperOrigin-RevId: 639867616
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36713
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36713 from tanvi-jagtap:src_core_lib_misc_gpr_log 0a36bebcbc
PiperOrigin-RevId: 639729711
Also rename the class from `MetadataQuery` to `GcpMetadataQuery`.
Closes#36789
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36789 from markdroth:reorg_gcp_metadata_query d2b84f46be
PiperOrigin-RevId: 639256546
Tanvi Jagtap
Craig Tiller
Mark D. Roth
apolcyn
Xuan Wang
Tanvi Jagtap
Yousuk Seung
AJ Heller
Nana Pang
Yijie Ma
Eugene Ostroukhov
Hannah Shi
Yash Tibrewal
Amy Huang