From 9a22574b0f8da79a6b2cb608b98c80d1eff0fdb1 Mon Sep 17 00:00:00 2001 From: Yash Tibrewal Date: Wed, 11 Jul 2018 18:31:22 -0700 Subject: [PATCH 1/6] Create keepalive.md Document the current knobs for keepalive ping in grpc core --- doc/keepalive.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 doc/keepalive.md diff --git a/doc/keepalive.md b/doc/keepalive.md new file mode 100644 index 00000000000..cb8daffe084 --- /dev/null +++ b/doc/keepalive.md @@ -0,0 +1,36 @@ +# Keepalive Ping User Guide for gRPC Core (and dependants) + +The keepalive ping is a way to check if a channel is currently working by sending HTTP2 pings over the transport. It is sent periodically, and if the ping is not acknowledged by the peer within a certain timeout period, the transport is disconnected. + +This guide documents the knobs within gRPC core to control the current behavior of the keepalive ping. + +The keepalive ping is controlled by two important channel arguments - +* **GRPC_ARG_KEEPALIVE_TIME_MS** + * This channel argument controls the period (in milliseconds) after which a keepalive ping is sent on the transport. +* **GRPC_ARG_KEEPALIVE_TIMEOUT_MS** + * This channel argument controls the amount of time (in milliseconds), the sender of the keepalive ping waits for an acknowledgement. If it does not receive an acknowledgement within this time, it will close the connection. + +The above two channel arguments should be sufficient for most users, but the following arguments can also be useful in certain use cases. +* **GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS** + * This channel argument if set to 1 (0 : false; 1 : true), allows keepalive pings to be sent even if there are no calls in flight. +* **GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA** + * This channel argument controls the maximum number of pings that can be sent when there is no other data (data frame or header frame) to be sent. GRPC Core will not continue sending pings if we run over the limit +* **GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS** + * If there is no data being sent on the transport, this channel argument controls the minimum time (in milliseconds) gRPC Core will wait between successive pings. +**GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS** + * If there is no data being sent on the transport, this channel argument on the server side controls the minimum time (in milliseconds) that gRPC Core would expect between receiving successive pings. If the time between successive pings is less that than this time, then the ping will be considered a bad ping from the peer. Such a ping counts as a ‘ping strike’. +On the client side, this does not have any effect. +* **GRPC_ARG_HTTP2_MAX_PING_STRIKES** + * This arg controls the maximum number of bad pings that the server will tolerate before sending an HTTP2 GOAWAY frame and closing the transport. Setting it to 0 allows the server to accept any number of bad pings. + +### Defaults Values + +Channel Argument| Client|Server +----------------|-------|------ +GRPC_ARG_KEEPALIVE_TIME_MS|INT_MAX (disabled)|7200000 (2 hours) +GRPC_ARG_KEEPALIVE_TIMEOUT_MS|20000 (20 seconds)|20000 (20 seconds) +GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS|0 (false)|0 (false) +GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA|2|2 +GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS|300000 (5 minutes)|300000 (5 minutes) +GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS|N/A|300000 (5 minutes) +GRPC_ARG_HTTP2_MAX_PING_STRIKES|N/A|2 From 5fa20dfcee664823ff7fe0c270dfb21830c729e6 Mon Sep 17 00:00:00 2001 From: Yash Tibrewal Date: Thu, 12 Jul 2018 12:17:53 -0700 Subject: [PATCH 2/6] Update keepalive.md --- doc/keepalive.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/keepalive.md b/doc/keepalive.md index cb8daffe084..c93262a440e 100644 --- a/doc/keepalive.md +++ b/doc/keepalive.md @@ -17,7 +17,7 @@ The above two channel arguments should be sufficient for most users, but the fol * This channel argument controls the maximum number of pings that can be sent when there is no other data (data frame or header frame) to be sent. GRPC Core will not continue sending pings if we run over the limit * **GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS** * If there is no data being sent on the transport, this channel argument controls the minimum time (in milliseconds) gRPC Core will wait between successive pings. -**GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS** +* **GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS** * If there is no data being sent on the transport, this channel argument on the server side controls the minimum time (in milliseconds) that gRPC Core would expect between receiving successive pings. If the time between successive pings is less that than this time, then the ping will be considered a bad ping from the peer. Such a ping counts as a ‘ping strike’. On the client side, this does not have any effect. * **GRPC_ARG_HTTP2_MAX_PING_STRIKES** From 719f52457916f688bac401392fb45e8df0350c51 Mon Sep 17 00:00:00 2001 From: Yash Tibrewal Date: Thu, 12 Jul 2018 12:27:32 -0700 Subject: [PATCH 3/6] Update keepalive.md --- doc/keepalive.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/keepalive.md b/doc/keepalive.md index c93262a440e..a400d021c6f 100644 --- a/doc/keepalive.md +++ b/doc/keepalive.md @@ -14,7 +14,7 @@ The above two channel arguments should be sufficient for most users, but the fol * **GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS** * This channel argument if set to 1 (0 : false; 1 : true), allows keepalive pings to be sent even if there are no calls in flight. * **GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA** - * This channel argument controls the maximum number of pings that can be sent when there is no other data (data frame or header frame) to be sent. GRPC Core will not continue sending pings if we run over the limit + * This channel argument controls the maximum number of pings that can be sent when there is no other data (data frame or header frame) to be sent. GRPC Core will not continue sending pings if we run over the limit. Setting it to 0 allows sending pings without sending data. * **GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS** * If there is no data being sent on the transport, this channel argument controls the minimum time (in milliseconds) gRPC Core will wait between successive pings. * **GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS** From fbc59b299d41bc29772cd320df6be0bc8141e87e Mon Sep 17 00:00:00 2001 From: Yash Tibrewal Date: Fri, 13 Jul 2018 18:21:31 -0700 Subject: [PATCH 4/6] Update keepalive.md --- doc/keepalive.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/doc/keepalive.md b/doc/keepalive.md index a400d021c6f..6939238be00 100644 --- a/doc/keepalive.md +++ b/doc/keepalive.md @@ -34,3 +34,17 @@ GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA|2|2 GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS|300000 (5 minutes)|300000 (5 minutes) GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS|N/A|300000 (5 minutes) GRPC_ARG_HTTP2_MAX_PING_STRIKES|N/A|2 + +### FAQ +* When is the keepalive timer started? + * The keepalive timer is started when a transport is done connecting (after handshake). +* What happens when the keepalive timer fires? + * When the keepalive timer fires, gRPC Core would try to send a keepalive ping on the transport. This ping can be blocked if - + * there is no active call on that transport and GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS is false. + * the number of pings already sent on the transport without any data has already exceeded GRPC_ARG_HTTP2_MAX_PINGS_WITHOUT_DATA. + * the time expired since the previous ping is less than GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS. + * If a keepalive ping is not blocked and is sent on the transport, then the keepalive watchdog timer is started which would close the transport if the ping is not acknowledged before it fires. +* Why am I receiving a GOAWAY with error code ENHANCE_YOUR_CALM? + * A server sends a GOAWAY with ENHANCE_YOUR_CALM if the client sends too many misbehaving pings. For example - + * if a server has GRPC_ARG_KEEPALIVE_PERMIT_WITHOUT_CALLS set to false, and the client sends pings without there being any call in flight. + * if the client's GRPC_ARG_HTTP2_MIN_SENT_PING_INTERVAL_WITHOUT_DATA_MS setting is lower than the server's GRPC_ARG_HTTP2_MIN_RECV_PING_INTERVAL_WITHOUT_DATA_MS. From ae196c90b7f99bbbd81eab69127ba7bce17fa540 Mon Sep 17 00:00:00 2001 From: Yash Tibrewal Date: Fri, 13 Jul 2018 18:22:10 -0700 Subject: [PATCH 5/6] Update keepalive.md --- doc/keepalive.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/keepalive.md b/doc/keepalive.md index 6939238be00..2f9c8bfc9e4 100644 --- a/doc/keepalive.md +++ b/doc/keepalive.md @@ -1,4 +1,4 @@ -# Keepalive Ping User Guide for gRPC Core (and dependants) +# Keepalive User Guide for gRPC Core (and dependants) The keepalive ping is a way to check if a channel is currently working by sending HTTP2 pings over the transport. It is sent periodically, and if the ping is not acknowledged by the peer within a certain timeout period, the transport is disconnected. From 4fdb28bdd282b602cee9bab9decc7fd84ce938de Mon Sep 17 00:00:00 2001 From: Yash Tibrewal Date: Fri, 13 Jul 2018 18:46:57 -0700 Subject: [PATCH 6/6] Run generate_projects.sh --- tools/doxygen/Doxyfile.c++ | 1 + tools/doxygen/Doxyfile.c++.internal | 1 + tools/doxygen/Doxyfile.core | 1 + tools/doxygen/Doxyfile.core.internal | 1 + 4 files changed, 4 insertions(+) diff --git a/tools/doxygen/Doxyfile.c++ b/tools/doxygen/Doxyfile.c++ index 7ac4ed4b3ec..322ab5eb983 100644 --- a/tools/doxygen/Doxyfile.c++ +++ b/tools/doxygen/Doxyfile.c++ @@ -784,6 +784,7 @@ doc/http-grpc-status-mapping.md \ doc/http2-interop-test-descriptions.md \ doc/internationalization.md \ doc/interop-test-descriptions.md \ +doc/keepalive.md \ doc/load-balancing.md \ doc/naming.md \ doc/server-reflection.md \ diff --git a/tools/doxygen/Doxyfile.c++.internal b/tools/doxygen/Doxyfile.c++.internal index c328387efaf..ba322a90a54 100644 --- a/tools/doxygen/Doxyfile.c++.internal +++ b/tools/doxygen/Doxyfile.c++.internal @@ -784,6 +784,7 @@ doc/http-grpc-status-mapping.md \ doc/http2-interop-test-descriptions.md \ doc/internationalization.md \ doc/interop-test-descriptions.md \ +doc/keepalive.md \ doc/load-balancing.md \ doc/naming.md \ doc/server-reflection.md \ diff --git a/tools/doxygen/Doxyfile.core b/tools/doxygen/Doxyfile.core index a85c328b07b..4899eee3ea4 100644 --- a/tools/doxygen/Doxyfile.core +++ b/tools/doxygen/Doxyfile.core @@ -786,6 +786,7 @@ doc/http-grpc-status-mapping.md \ doc/http2-interop-test-descriptions.md \ doc/internationalization.md \ doc/interop-test-descriptions.md \ +doc/keepalive.md \ doc/load-balancing.md \ doc/naming.md \ doc/server-reflection.md \ diff --git a/tools/doxygen/Doxyfile.core.internal b/tools/doxygen/Doxyfile.core.internal index 1a5dfb902b4..576950934ec 100644 --- a/tools/doxygen/Doxyfile.core.internal +++ b/tools/doxygen/Doxyfile.core.internal @@ -786,6 +786,7 @@ doc/http-grpc-status-mapping.md \ doc/http2-interop-test-descriptions.md \ doc/internationalization.md \ doc/interop-test-descriptions.md \ +doc/keepalive.md \ doc/load-balancing.md \ doc/naming.md \ doc/server-reflection.md \