More PR feedback

pull/14870/head
kpayson64 7 years ago
parent 34daf66ddc
commit 93f49753ac
  1. 49
      doc/fork_support.md

@ -1,22 +1,24 @@
# Background # # Background #
In Python multithreading is ineffective at concurrency for CPU bound tasks In Python, multithreading is ineffective at concurrency for CPU bound tasks
due to the GIL. due to the GIL (global interpreter lock). Extension modules can release
(global interpreter lock). Users use multiprocessing, subprocess, the GIL in CPU bound tasks, but that isn't an option in pure Python.
and concurrent.futures.ProcessPoolExecutor, to work around the GIL. Users use libraries such as multiprocessing, subprocess, concurrent.futures.ProcessPoolExecutor,
These modules call fork() underneath the hood. Various issues have etc, to work around the GIL. These modules call fork() underneath the hood. Various issues have
been reported when using these modules with gRPC Python. been reported when using these modules with gRPC Python. gRPC Python wraps
Historically, we didn't support forking in gRPC, but some users seem gRPC core, which uses multithreading for performance, and hence doesn't support fork().
Historically, we didn't support forking in gRPC, but some users seemed
to be doing fine until their code started to break on version 1.6. This was to be doing fine until their code started to break on version 1.6. This was
likely caused by the addition of background c-threads and a background likely caused by the addition of background c-threads and a background
Python thread. Python thread.
# Current Status # # Current Status #
## 1.7 ##
A pthread_atfork() handler was added in 1.7 to automatically shut down ## 1.11 ##
the background c-threads when fork was called. This does not shut down the The background Python thread was removed entirely. This allows forking
background Python thread, so users could not have any open channels when after creating a channel. However, the channel must not have issued any
forking(). RPCs prior to the fork. Attempting to fork with an active channel that
has been used can result in deadlocks/corrupted wire data.
## 1.9 ## ## 1.9 ##
A regression was noted in cases where users are doing fork/exec. This A regression was noted in cases where users are doing fork/exec. This
@ -28,22 +30,17 @@ off the handler using env flag ```GRPC_ENABLE_FORK_SUPPORT=False```.
This should be set whenever a user expects to always call exec This should be set whenever a user expects to always call exec
immediately following fork. It will disable the fork handlers. immediately following fork. It will disable the fork handlers.
## 1.11 ## ## 1.7 ##
The background Python thread was removed entirely. This allows forking A pthread_atfork() handler was added in 1.7 to automatically shut down
after creating a channel. However, the channel cannot be used by both the the background c-threads when fork was called. This does not shut down the
parent and child process after the fork. Additionaly, the process should background Python thread, so users could not have any open channels when
not fork if there are any active RPCs on the channel. forking().
# Future Work # # Future Work #
## 1.13 ## ## 1.13 ##
The workaround when using fork/exec by setting The workaround when using fork/exec by setting
```GRPC_ENABLE_FORK_SUPPORT=False``` should no longer be needed. Now the fork ```GRPC_ENABLE_FORK_SUPPORT=False``` should no longer be needed. Following
handlers are automatically not run when multiple threads are calling [this PR](https://github.com/grpc/grpc/pull/14647), fork
handlers will not automatically run when multiple threads are calling
into gRPC. into gRPC.
## 1.1x ##
We would like to support forking and using the channel from both the parent
and child process. Additionally, we would like to support servers that
use a prefork model, where the child processes accept the connections
and handle requests.

Loading…
Cancel
Save