Fixes the CBF of `src/ruby/end2end/killed_client_thread_test.rb` (failure mode is a hang of the child process that receives the SIGTERM) that has been happening since https://github.com/grpc/grpc/pull/36724
So far grpc-ruby CQ pluck operations have so far used a 20ms-interval busy poll to check interrupts in case we've received a signal, handle process shutdown, etc. This means ongoing RPCs will not terminate their CQ operations if we need to terminate the process (the loop simply exits without waiting for the CQ op to finish), causing a leak. Those RPCs can leave refs over their corresponding channels preventing [this](8564f72e8e/src/ruby/ext/grpc/rb_channel.c (L653)) from terminating (the channels don't reach state SHUTDOWN after being destroyed).
Fix is to unblock CQ pluck operations by cancelling calls, and thus allowing the CQ pluck to actually complete its operation. For server listening CQ operations, we unblock them by shutting down the server.
A side win here is to remove the [20ms-interval busy poll](8564f72e8e/src/ruby/ext/grpc/rb_completion_queue.c (L44)) on CQ operations, which was only needed to handle shutdown.
Closes#36903
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36903 from apolcyn:fix_ruby_interrupt bed1ee2feb
PiperOrigin-RevId: 643046465
Change was created by the release automation script. See go/grpc-release.
Closes#36892
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36892 from XuanWang-Amos:bump_dev_version_202406112301 9b2898d716
PiperOrigin-RevId: 643027546
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with c assert.
These changes have been made using string replacement and regex.
Closes#36817
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36817 from tanvi-jagtap:gpr_assert_removal_in_php 2222f95413
PiperOrigin-RevId: 642855768
See https://github.com/grpc/proposal/pull/426
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36447
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36447 from yousukseung:generic-stub-service-refactor 1cc0cbdc4b
PiperOrigin-RevId: 642774012
Also made some minor improvements to the `ConfigSelector` API.
Closes#36877
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36877 from markdroth:client_channel_v3_dynamic_filters 6a539fe320
PiperOrigin-RevId: 642755276
Seems MSVC has trouble compiling this statically initialized lambda in a template; crash goes away when I split it out to be a named function instead.
Closes#36893
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36893 from ctiller:edward 4bf9b88797
PiperOrigin-RevId: 642714465
- ensure ordering of `OnAccept` and `Shutdown` callbacks from thready event engine (previously these could be reordered and this caused spurious failures)
- enable thready_tsan for one C++ e2e test
- don't filter thready_tsan for local builds (only CI)
Closes#36886
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36886 from ctiller:erm e3b88e7d86
PiperOrigin-RevId: 642702724
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36880
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36880 from tanvi-jagtap:src_cpp_folder_gpr_log e6e795f469
PiperOrigin-RevId: 642436269
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36874
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36874 from XuanWang-Amos:remove_common_proto_dep 51400c5aff
PiperOrigin-RevId: 642424550
Change was created by the release automation script. See go/grpc-release.
Closes#36887
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36887 from XuanWang-Amos:bump_core_version_202406111526 5df15650dd
PiperOrigin-RevId: 642358496
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36875
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36875 from yousukseung:reenable-work-serializer-dispatch da03e7d9df
PiperOrigin-RevId: 642347062
Refines the test in the fuzzer to only check input == output if the frame was re-encodable.
Also tweaks some hpack encoder stuff to not crash but simply report errors where appropriate.
Closes#36821
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36821 from ctiller:f-frame 1b08875c4e
PiperOrigin-RevId: 642311442
Reattempt enabling the Windows EventEngine client experiment, after fixing a race condition in connection establishment vs timeout.
Closes#36876
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36876 from drfloob:wineec 125f3a20d8
PiperOrigin-RevId: 642294760
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36800
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36800 from tanvi-jagtap:src_core_lib cc0411e99d
PiperOrigin-RevId: 642109824
[Gpr_To_Absl_Logging] Move function to test header form log.h
This is not really needed in log.h
Closes#36860
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36860 from tanvi-jagtap:move_function_to_test_header e6494bd06f
PiperOrigin-RevId: 642080756
gRPC shutdown may happen asynchronously with `work_serializer_dispatch`, so wait briefly until shutdown completes in the child's post-fork.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36873
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36873 from yousukseung:python-fork-shutdown-wait 7908a6912a
PiperOrigin-RevId: 642066286
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36729
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36729 from yijiem:per-channel-stats-plugin 4786bed42f
PiperOrigin-RevId: 642030366
[Gpr_To_Absl_Logging] Removing instances of gpr_should_log and replacing it with ABSL_VLOG_IS_ON.
gpr_should_log function will be deleted soon.
**VLOG(2)** is equivalent to gpr_log(**GPR_DEBUG**, ... )
We are replacing two instances of **gpr_log(GPR_INFO**, ... ) with **VLOG(2)** because this code appears to be slightly expensive.
I can also continue to use LOG(INFO) and check current settings using absl::MinLogLevel() . Would you want me to do that?
Closes#36866
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36866 from tanvi-jagtap:fix_secure_endpoint_gpr_should 556006c720
PiperOrigin-RevId: 641939641
[Gpr_To_Absl_Logging] Removing instances of gpr_should_log.
Replacing it with ABSL_VLOG_IS_ON.
VLOG(2) is equivalent to gpr_log(GPR_DEBUG, ... )
This function will be deprecated and deleted soon.
Closes#36861
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36861 from tanvi-jagtap:gpr_should_log_DEBUG_replace 09cd39e05a
PiperOrigin-RevId: 641939617
[Gpr_To_Absl_Logging] Removing instances of gpr_should_log and replacing it with ABSL_VLOG_IS_ON.
gpr_should_log function will be deleted soon.
**VLOG(2)** is equivalent to gpr_log(**GPR_DEBUG**, ... )
We are replacing two instances of **gpr_log(GPR_INFO**, ... ) with **VLOG(2)** because this code appears to be slightly expensive.
I can also continue to use LOG(INFO) and check current settings using absl::MinLogLevel() . Would you want me to do that?
Closes#36868
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36868 from tanvi-jagtap:fix_tcp_windows_gpr_should_log c2b660d4e2
PiperOrigin-RevId: 641938430
[Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_set_log_function
Adding a warning for users of gpr_set_log_function.
This function will be deleted soon.
https://github.com/grpc/proposal/pull/425
Other options considered ( and discarded )
1. Using DFATAL instead of ERROR. Discarded this because we still have one critical unit test to clean up before we delete gpr_set_log_function. This test will fail if we use DFATAL. EXPECT_DFATAL macro is not available to be used.
2. Considered making a new absl log sink , and then that log sink should have a function pointer with the same signature as gpr_log_func. This function pointer can be set from gpr_set_log_function. However if this custom function uses absl from within it, that will cause infinite recursion. Made me drop the idea.
Closes#36844
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36844 from tanvi-jagtap:assert_gpr_set_log_function 805de358a9
PiperOrigin-RevId: 641473969
Verified that when following User Guide, we're able to collect metrics
and export to GCP.
Images are available at:
*
`us-docker.pkg.dev/grpc-testing/examples/csm-o11y-example-python-client:latest`
*
`us-docker.pkg.dev/grpc-testing/examples/csm-o11y-example-python-server:latest`
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
All TraceFlags are now configured in `src/core/lib/debug/trace_flags.yaml`. The format is:
```
my_flag:
default: false # the default value; default=false
description: Some Description
debug_only: false # debug_only flags only work in debug builds; default=false
internal: false # internal flags will not show up in documentation; default=false
```
To regenerate the trace flag source code, run `tools/codegen/core/gen_trace_flags.py` (requires mako). This script is also run when sanity checking.
This PR also adds two new features:
### Glob-based flag configuration
Trace flag configuration now supports `?` (single wildcard character) and `*` (one or more wildcard characters). For example, using `GRPC_TRACE='event_engine*'` will enable all flags that match that glob. It expands to:
* event_engine
* event_engine_client_channel_resolver
* event_engine_dns
* event_engine_endpoint
* event_engine_endpoint_data
* event_engine_poller
### A cleaner trace-logging macro in abseil logging format
If your goal is only to add log statements when the `fault_injection_filter` trace flag is enabled, you can use the macro:
```
GRPC_TRACE_LOG(fault_injection, INFO) << "Filtered:" << 42;
```
When the trace flag is enabled, the the log will show something like this:
```
I0000 00:00:1715733657.430042 16 file.cc:174] Filtered:42
```
----
Note: just like with the gpr_log to abseil logging conversion, the pre-existing trace logging usages can be replaced with the new tracing macro across multiple PRs.
Closes#36576
PiperOrigin-RevId: 641295215
Use the latest golang base image for Go interop tests. Presently the latest image points to Go 1.22.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
## Commands run
To render the Dockerfile from its template
```sh
$ tools/buildgen/generate_projects.sh
```
To build and upload the base docker image.
```sh
$ tools/dockerfile/push_testing_images.sh
```
To build and upload the interop docker image for grpc-go 1.64.0
```sh
$ tools/interop_matrix/create_matrix_images.py --git_checkout --release=v1.64.0 --upload_images --language go
```
To verify the image is uploaded
```sh
$ gcloud container images list-tags gcr.io/grpc-testing/grpc_interop_go1.x
DIGEST TAGS TIMESTAMP
b36be4961ec4 infrastructure-public-image-v1.64.0,v1.64.0 2024-06-03T22:26:48
```
Verify the backward compatability tests pass
```sh
$ export docker_image=gcr.io/grpc-testing/grpc_interop_go1.x:v1.64.0
$ tools/interop_matrix/testcases/go__master
Testing gcr.io/grpc-testing/grpc_interop_go1.x:v1.64.0
```
Fixes: https://github.com/grpc/grpc/issues/36060Closes#36755
PiperOrigin-RevId: 641272572
[grpc][Gpr_To_Absl_Logging]
1. Supporting the legacy GRPC_VERBOSITY environment variable in the new absl logging implementation.
2. Adding a new way to disable VLOG logging.
3. Documenting the recommendations clearly.
4. Editing the init code.
Additional Context :
Check function gpr_default_log() for more context
https://github.com/search?q=repo%3Agrpc%2Fgrpc%20gpr_default_log&type=codeCloses#36798
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36798 from tanvi-jagtap:grpc_verbosity_flag_support ab0d600849
PiperOrigin-RevId: 641092851
This task was parallelized into 16 sub-tasks, but 32 cores are available. Let's see if this speeds up CI, the clang-tidy job usually takes about 30m to 40m.
Closes#36838
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36838 from drfloob:tidy-32 81edd68348
PiperOrigin-RevId: 641014003
Usage: `bazel build --config=clang-cl --build_tag_filters=-no_windows :all`
The highlight, for me anyway: thread safety annotation checking can now be done on Windows-only code.
clang-cl's interpretation of `-Wall` differs from that of clang on linux. This PR uses the set of warnings enabled on linux, and whittles down the list until all builds pass. I left comments in `copts.bzl` describing the warnings, many of which can be removed with targeted code cleanups.
Closes#36831
PiperOrigin-RevId: 641009783
Noticed this feature was missing from the list, so I added it.
@murgatroid99 - is this supported in Node? Let me know what version and I'll include it.
Closes#36714
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36714 from dfawley:a53 a1bb3605d4
PiperOrigin-RevId: 640982395
We changed the way flag was passed in this PR: https://github.com/grpc/psm-interop/pull/94
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36827
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36827 from XuanWang-Amos:csm_fix_interop a2c2c47f30
PiperOrigin-RevId: 640908453
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK.
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
Closes#36816
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36816 from tanvi-jagtap:gpr_assert_removal 9db548beb0
PiperOrigin-RevId: 640731171
apolcyn
Xuan Wang
Tanvi Jagtap
Tanvi Jagtap
Yousuk Seung
Mark D. Roth
Craig Tiller
AJ Heller
Esun Kim
Marcel
Nana Pang
Yijie Ma
Arjan Singh Bal
Eugene Ostroukhov
Doug Fawley
John Cormie
Vishwanath Sastry