The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#) https://grpc.io/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

47 lines
2.2 KiB

# Background #
7 years ago
In Python, multithreading is ineffective at concurrency for CPU bound tasks
due to the GIL (global interpreter lock). Extension modules can release
the GIL in CPU bound tasks, but that isn't an option in pure Python.
Users use libraries such as multiprocessing, subprocess, concurrent.futures.ProcessPoolExecutor,
7 years ago
etc, to work around the GIL. These modules call ```fork()``` underneath the hood. Various issues have
7 years ago
been reported when using these modules with gRPC Python. gRPC Python wraps
7 years ago
gRPC core, which uses multithreading for performance, and hence doesn't support ```fork()```.
7 years ago
Historically, we didn't support forking in gRPC, but some users seemed
7 years ago
to be doing fine until their code started to break on version 1.6. This was
likely caused by the addition of background c-threads and a background
Python thread.
# Current Status #
7 years ago
## 1.11 ##
The background Python thread was removed entirely. This allows forking
after creating a channel. However, the channel must not have issued any
RPCs prior to the fork. Attempting to fork with an active channel that
has been used can result in deadlocks/corrupted wire data.
## 1.9 ##
A regression was noted in cases where users are doing fork/exec. This
7 years ago
was due to ```pthread_atfork()``` handler that was added in 1.7 to partially
7 years ago
support forking in gRPC. A deadlock can happen when pthread_atfork
handler is running, and an application thread is calling into gRPC.
We have provided a workaround for this issue by allowing users to turn
off the handler using env flag ```GRPC_ENABLE_FORK_SUPPORT=False```.
This should be set whenever a user expects to always call exec
immediately following fork. It will disable the fork handlers.
7 years ago
## 1.7 ##
7 years ago
A ```pthread_atfork()``` handler was added in 1.7 to automatically shut down
7 years ago
the background c-threads when fork was called. This does not shut down the
background Python thread, so users could not have any open channels when
7 years ago
forking.
7 years ago
# Future Work #
7 years ago
7 years ago
## 1.13 ##
The workaround when using fork/exec by setting
7 years ago
```GRPC_ENABLE_FORK_SUPPORT=False``` should no longer be needed. Following
[this PR](https://github.com/grpc/grpc/pull/14647), fork
handlers will not automatically run when multiple threads are calling
into gRPC.