Duo crashes occasionally with the stack trace bellow. I wasn't able to reproduce with stress run a modified version of existing tests, however I suspect what happened was that:
cfstream endpoint was shutdown while a callback is being scheduled, so
a. [retain](https://github.com/opensource-apple/CF/blob/master/CFStream.c#L595-L598) is about to be called after null check
b. client callback [context is cleared](https://github.com/opensource-apple/CF/blob/master/CFStream.c#L1322) during shutdown
Since there is no lock in (b), a race can happen and result in retain to be a null pointer and caused the crash.
This PR removed setting the client callback to null so the ongoing event delivery can continue; and added clear the dispatch queue so less likely events are scheduled while connection is shutting down.
----
Crash stack trace:
```
Thread 1 (id: 0x0025a274)CRASHED
Exception infoEXC_BAD_ACCESS / KERN_INVALID_ADDRESS @0x00000008
Stack Quality95%Show frame trust levels
0x00000001034e79a4 (Tachyon -atomic:1014) long std::__1::__cxx_atomic_fetch_add[abi:v160006]<long>(std::__1::__cxx_atomic_base_impl<long>*, long, std::__1::memory_order)
0x00000001034e79a4 (Tachyon -atomic:1649) std::__1::__atomic_base<long, true>::fetch_add[abi:v160006](long, std::__1::memory_order)
0x00000001034e79a4 (Tachyon -ref_counted.h:78) grpc_core::RefCount::Ref(long)
0x00000001034e79a4 (Tachyon -ref_counted.h:379) grpc_core::RefCounted<grpc_event_engine::experimental::CFStreamEndpointImpl, grpc_core::PolymorphicRefCount, grpc_core::UnrefDelete>::IncrementRefCount() const
0x00000001034e79a4 (Tachyon -ref_counted.h:288) grpc_core::RefCounted<grpc_event_engine::experimental::CFStreamEndpointImpl, grpc_core::PolymorphicRefCount, grpc_core::UnrefDelete>::Ref()
0x00000001034e79a4 (Tachyon -cfstream_endpoint.h:75) grpc_event_engine::experimental::CFStreamEndpointImpl::Retain(void*)
0x0000000199d2a814 (CoreFoundation + 0x000b8814) _signalEventSync
0x0000000199db0654 (CoreFoundation + 0x0013e654) ___signalEventQueue_block_invoke
0x00000001a1b6d138 (libdispatch.dylib + 0x00002138) _dispatch_call_block_and_release
0x00000001a1b6edd0 (libdispatch.dylib + 0x00003dd0) _dispatch_client_callout
0x00000001a1b71f68 (libdispatch.dylib + 0x00006f68) _dispatch_queue_override_invoke
0x00000001a1b80890 (libdispatch.dylib + 0x00015890) _dispatch_root_queue_drain
0x00000001a1b81098 (libdispatch.dylib + 0x00016098) _dispatch_worker_thread2
0x00000001f5c54ee0 (libsystem_pthread.dylib + 0x00001ee0) _pthread_wqthread
```
Closes#36415
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36415 from HannahShiSFB:cf-event-engine-crash f8f609f304
PiperOrigin-RevId: 627154676
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36400
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36400 from XuanWang-Amos:python_distribution bd2bd48031
PiperOrigin-RevId: 627114699
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36405
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36405 from tanvi-jagtap:tjagtap_microbenchmarks_01 0dcec5d852
PiperOrigin-RevId: 626522246
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK
These changes have been made using string replacement and regex.
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36407
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36407 from tanvi-jagtap:tjagtap_end2end_fixtures e98c97245b
PiperOrigin-RevId: 626522240
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36396
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36396 from yijiem:xds-cluster-end2end-test-twin 26062ea1b4
PiperOrigin-RevId: 626207011
Looks like this has been broken for things we've moved out of lib and ext.
Closes#36381
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36381 from markdroth:clang_format_fix 41f0283c42
PiperOrigin-RevId: 626137779
grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK
These changes have been made using string replacement
Will not be replacing all instances of CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only ones which are doable using very simple regex with least chance of failure will be replaced.
Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36368
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36368 from tanvi-jagtap:tjagtap_test_02 0072bd8854
PiperOrigin-RevId: 626114257
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36366
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36366 from tanvi-jagtap:tjagtap_test_01 a74c57ea7e
PiperOrigin-RevId: 625963461
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36294
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36294 from ctiller:port_platform-src-core-resolver-clang-format 6418ff82bf
PiperOrigin-RevId: 625877317
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36365
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36365 from tanvi-jagtap:tjagtap_bad_client d769b84024
PiperOrigin-RevId: 625557190
Fix https://github.com/grpc/grpc/issues/36367.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36382
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36382 from yijiem:fix-example-cmake 54521e87a1
PiperOrigin-RevId: 625469934
Eliminate cases where we get unused parameter errors only because of a debug assert, and because of short-circuit evaluation not add any overhead into release builds.
Closes#36373
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36373 from ctiller:ugh123 073ea1236a
PiperOrigin-RevId: 625435672
Currently channel filter construction returns a `StatusOr<T>`, this change makes it return a `StatusOr<P<T>>` where P is `unique_ptr`, `OrphanablePtr`, `RefCountedPtr`, `DualRefCountedPtr`, etc (most of the code really doesn't need to know, so I'm choosing to leave the flexibility).
That smart pointer is then stored in the channel stack instance, and dereferenced when needed.
This means that channel filters no longer need to be movable (which is a nice simplification), and puts these level-1 filters on a similar memory management track as the level-2 filters we have planned.
(this change also converts client load reporting to v3 apis -- it's a bit accidentally picked up, but seems ok to pull through too)
Closes#36355
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36355 from ctiller:objectify-me 0eb054b748
PiperOrigin-RevId: 625390977
First of several PRs to improve the e2e testing for the SSL credentials API.
Closes#35433
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35433 from matthewstevenson88:more-ssl-testing 2a0db7624e
PiperOrigin-RevId: 625326074
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36359
PiperOrigin-RevId: 625183028
Make `CallSpineInterface` describe the operations that we need, and return `Promise<>` types -- these are `std::function` wrappers and involve an allocation.
This ought to be acceptable for the use cases we'll be using `CallSpineInterface` for in the short term - and doing this lets us bring the rest of the v3 stack in concurrently with the "v2.5" stack -- the v2 stack using some of the v3 interfaces I put together to unblock chaotic-good.
Later we'll remove this scaffolding and eventually `CallSpineInterface` in its entirety, in preference to something wrapped around `CallFilters`.
Closes#36321
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36321 from ctiller:cally-2 7f608c36d3
PiperOrigin-RevId: 625096226
`setuptools` made a decision to change the artifact name it builds (from `grpcio-health-checking` to `grpcio_health_checking`) in their latest release (https://github.com/pypa/setuptools/issues/4300).
As a result, we need broaden our regex so that our tests can pickup the correct files.
### Note
* Using `[_-]*` instead of `[_-]?` to match one character since `bash` uses a different flavor of regular expressions called basic regular expressions (BREs) which do not support the optional quantifier `?`.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36352
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36352 from XuanWang-Amos:fix_distribution_test 8dfcc4e9a3
PiperOrigin-RevId: 625083784
The default value of two experiments are set to true because they have already been enabled internally.
Closes#36361
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36361 from Vignesh2208:experiments-expiry e244a4f30e
PiperOrigin-RevId: 625022979
To simplify the build process, let's remove unnecessary Bazel bindings. These bindings were originally used to manage differences between OSS and our internal build environment. However, bindings are now deprecated and introduce unnecessary complexity. Removing them will make the build process more straightforward and easier to maintain.
Closes#36349
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36349 from veblush:upb-link 3eda0c6f92
PiperOrigin-RevId: 624284314
grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with absl CHECK
Will not be replacing CHECK with CHECK_EQ , CHECK_NE etc because there are too many callsites. Only a few - which fit into single - line regex will be changed. This would be small in number just to reduce the load later.
Replacing CHECK with CHECK_EQ , CHECK_NE etc could be done using Cider-V once these changes are submitted if we want to clean up later. Given that we have 5000+ instances of GPR_ASSERT to edit, Doing it manually is too much work for both the author and reviewer.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36326
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36326 from tanvi-jagtap:tjagtap_gpr_assert_event_eng bfe99bec63
PiperOrigin-RevId: 624038836
Since we already finished v26.1 upgrade in https://github.com/grpc/grpc/pull/35796
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36230
PiperOrigin-RevId: 623960260
Updated `push_testing_images.sh` to add an additional tag, `infrastructure-public-image-HASH` to a new image to be excluded from the GCP Vulnerability scanner.
To reviewer:
- Only `tools/dockerfile/push_testing_images.sh` has an interesting change (adding a new tag)
- Clang 6 and 17 don't have an actual change other than clean-up which is to test if the script is working.
Closes#36341
PiperOrigin-RevId: 623944712
Deprecate things that mention channel stacks directly... a future change will mutate the internal data structure.
Closes#36339
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36339 from ctiller:creation-story 79993c8b03
PiperOrigin-RevId: 623929372
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36334
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36334 from yijiem:grpc-metrics-better-compile-time-enforcement 2ae15acfeb
PiperOrigin-RevId: 623909848
The pick_first policy creates a list of subchannels for each resolver update and then iterates over the list, attempting to connect to each subchannel in turn, until one of them succeeds. However, once a subchannel does succeed, the policy unrefs the other subchannels but still retains a bunch of now-unnecessary state in the subchannel list itself. This wastes a bunch of memory, especially now that petiole policies are delegating to pick_first. This PR contains a new pick_first implementation that stops retaining that state, which significantly reduces per-channel memory.
There is one behavior change here, which is that if we have a connected subchannel and we get a resolver update that no longer includes that address, we now go IDLE instead of proactively trying to connect to the new addresses.
Closes#34766
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34766 from markdroth:pick_first_free_memory_after_connecting 7236b4321f
PiperOrigin-RevId: 623887639
Anirudh Ramachandra
Hannah Shi
Xuan Wang
Yash Tibrewal
Tanvi Jagtap
Mark D. Roth
Yijie Ma
gRPC Team Bot
Craig Tiller
AJ Heller
Alisha Nanda
Yousuk Seung
Matthew Stevenson
Vishwanath Sastry
Vignesh Babu
Esun Kim