mirror of https://github.com/grpc/grpc.git
[ObjC] do not set cfstream client to null on shutdown (#36415)
Duo crashes occasionally with the stack trace bellow. I wasn't able to reproduce with stress run a modified version of existing tests, however I suspect what happened was that:
cfstream endpoint was shutdown while a callback is being scheduled, so
a. [retain](https://github.com/opensource-apple/CF/blob/master/CFStream.c#L595-L598) is about to be called after null check
b. client callback [context is cleared](https://github.com/opensource-apple/CF/blob/master/CFStream.c#L1322) during shutdown
Since there is no lock in (b), a race can happen and result in retain to be a null pointer and caused the crash.
This PR removed setting the client callback to null so the ongoing event delivery can continue; and added clear the dispatch queue so less likely events are scheduled while connection is shutting down.
----
Crash stack trace:
```
Thread 1 (id: 0x0025a274)CRASHED
Exception infoEXC_BAD_ACCESS / KERN_INVALID_ADDRESS @0x00000008
Stack Quality95%Show frame trust levels
0x00000001034e79a4 (Tachyon -atomic:1014) long std::__1::__cxx_atomic_fetch_add[abi:v160006]<long>(std::__1::__cxx_atomic_base_impl<long>*, long, std::__1::memory_order)
0x00000001034e79a4 (Tachyon -atomic:1649) std::__1::__atomic_base<long, true>::fetch_add[abi:v160006](long, std::__1::memory_order)
0x00000001034e79a4 (Tachyon -ref_counted.h:78) grpc_core::RefCount::Ref(long)
0x00000001034e79a4 (Tachyon -ref_counted.h:379) grpc_core::RefCounted<grpc_event_engine::experimental::CFStreamEndpointImpl, grpc_core::PolymorphicRefCount, grpc_core::UnrefDelete>::IncrementRefCount() const
0x00000001034e79a4 (Tachyon -ref_counted.h:288) grpc_core::RefCounted<grpc_event_engine::experimental::CFStreamEndpointImpl, grpc_core::PolymorphicRefCount, grpc_core::UnrefDelete>::Ref()
0x00000001034e79a4 (Tachyon -cfstream_endpoint.h:75) grpc_event_engine::experimental::CFStreamEndpointImpl::Retain(void*)
0x0000000199d2a814 (CoreFoundation + 0x000b8814) _signalEventSync
0x0000000199db0654 (CoreFoundation + 0x0013e654) ___signalEventQueue_block_invoke
0x00000001a1b6d138 (libdispatch.dylib + 0x00002138) _dispatch_call_block_and_release
0x00000001a1b6edd0 (libdispatch.dylib + 0x00003dd0) _dispatch_client_callout
0x00000001a1b71f68 (libdispatch.dylib + 0x00006f68) _dispatch_queue_override_invoke
0x00000001a1b80890 (libdispatch.dylib + 0x00015890) _dispatch_root_queue_drain
0x00000001a1b81098 (libdispatch.dylib + 0x00016098) _dispatch_worker_thread2
0x00000001f5c54ee0 (libsystem_pthread.dylib + 0x00001ee0) _pthread_wqthread
```
Closes #36415
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36415 from HannahShiSFB:cf-event-engine-crash f8f609f304
PiperOrigin-RevId: 627154676
pull/36095/head^2
parent
297e22cb5f
commit
10a95e03d9
1 changed files with 3 additions and 3 deletions
Loading…
Reference in new issue