mirror of https://github.com/grpc/grpc.git
Vijay Pai
10 years ago
commit
31e52f6341
99 changed files with 1725 additions and 317 deletions
@ -0,0 +1,147 @@ |
||||
gRPC Connectivity Semantics and API |
||||
=================================== |
||||
|
||||
This document describes the connectivity semantics for gRPC channels and the |
||||
corresponding impact on RPCs. We then discuss an API. |
||||
|
||||
States of Connectivity |
||||
---------------------- |
||||
|
||||
gRPC Channels provide the abstraction over which clients can communicate with |
||||
servers.The client-side channel object can be constructed using little more |
||||
than a DNS name. Channels encapsulate a range of functionality including name |
||||
resolution, establishing a TCP connection (with retries and backoff) and TLS |
||||
handshakes. Channels can also handle errors on established connections and |
||||
reconnect, or in the case of HTTP/2 GO_AWAY, re-resolve the name and reconnect. |
||||
|
||||
To hide the details of all this activity from the user of the gRPC API (i.e., |
||||
application code) while exposing meaningful information about the state of a |
||||
channel, we use a state machine with four states, defined below: |
||||
|
||||
CONNECTING: The channel is trying to establish a connection and is waiting to |
||||
make progress on one of the steps involved in name resolution, TCP connection |
||||
establishment or TLS handshake. This may be used as the initial state for channels upon |
||||
creation. |
||||
|
||||
READY: The channel has successfully established a connection all the way |
||||
through TLS handshake (or equivalent) and all subsequent attempt to communicate |
||||
have succeeded (or are pending without any known failure ). |
||||
|
||||
TRANSIENT_FAILURE: There has been some transient failure (such as a TCP 3-way |
||||
handshake timing out or a socket error). Channels in this state will eventually |
||||
switch to the CONNECTING state and try to establish a connection again. Since |
||||
retries are done with exponential backoff, channels that fail to connect will |
||||
start out spending very little time in this state but as the attempts fail |
||||
repeatedly, the channel will spend increasingly large amounts of time in this |
||||
state. For many non-fatal failures (e.g., TCP connection attempts timing out |
||||
because the server is not yet available), the channel may spend increasingly |
||||
large amounts of time in this state. |
||||
|
||||
IDLE: This is the state where the channel is not even trying to create a |
||||
connection because of a lack of new or pending RPCs. New channels MAY be created |
||||
in this state. Any attempt to start an RPC on the channel will push the channel |
||||
out of this state to connecting. When there has been no RPC activity on a channel |
||||
for a specified IDLE_TIMEOUT, i.e., no new or pending (active) RPCs for this |
||||
period, channels that are READY or CONNECTING switch to IDLE. Additionaly, |
||||
channels that receive a GOAWAY when there are no active or pending RPCs should |
||||
also switch to IDLE to avoid connection overload at servers that are attempting |
||||
to shed connections. We will use a default IDLE_TIMEOUT of 300 seconds (5 minutes). |
||||
|
||||
SHUTDOWN: This channel has started shutting down. Any new RPCs should fail |
||||
immediately. Pending RPCs may continue running till the application cancels them. |
||||
Channels may enter this state either because the application explicitly requested |
||||
a shutdown or if a non-recoverable error has happened during attempts to connect |
||||
communicate . (As of 6/12/2015, there are no known errors (while connecting or |
||||
communicating) that are classified as non-recoverable) |
||||
Channels that enter this state never leave this state. |
||||
|
||||
The following table lists the legal transitions from one state to another and |
||||
corresponding reasons. Empty cells denote disallowed transitions. |
||||
|
||||
<table style='border: 1px solid black'> |
||||
<tr> |
||||
<th>From/To</th> |
||||
<th>CONNECTING</th> |
||||
<th>READY</th> |
||||
<th>TRANSIENT_FAILURE</th> |
||||
<th>IDLE</th> |
||||
<th>SHUTDOWN</th> |
||||
</tr> |
||||
<tr> |
||||
<th>CONNECTING</th> |
||||
<td>Incremental progress during connection establishment</td> |
||||
<td>All steps needed to establish a connection succeeded</td> |
||||
<td>Any failure in any of the steps needed to establish connection</td> |
||||
<td>No RPC activity on channel for IDLE_TIMEOUT</td> |
||||
<td>Shutdown triggered by application.</td> |
||||
</tr> |
||||
<tr> |
||||
<th>READY</th> |
||||
<td></td> |
||||
<td>Incremental successful communication on established channel.</td> |
||||
<td>Any failure encountered while expecting successful communication on |
||||
established channel.</td> |
||||
<td>No RPC activity on channel for IDLE_TIMEOUT <br>OR<br>upon receiving a GOAWAY while there are no pending RPCs.</td> |
||||
<td>Shutdown triggered by application.</td> |
||||
</tr> |
||||
<tr> |
||||
<th>TRANSIENT_FAILURE</th> |
||||
<td>Wait time required to implement (exponential) backoff is over.</td> |
||||
<td></td> |
||||
<td></td> |
||||
<td></td> |
||||
<td>Shutdown triggered by application.</td> |
||||
</tr> |
||||
<tr> |
||||
<th>IDLE</th> |
||||
<td>Any new RPC activity on the channel</td> |
||||
<td></td> |
||||
<td></td> |
||||
<td></td> |
||||
<td>Shutdown triggered by application.</td> |
||||
</tr> |
||||
<tr> |
||||
<th>FATAL_FAILURE</th> |
||||
<td></td> |
||||
<td></td> |
||||
<td></td> |
||||
<td></td> |
||||
<td></td> |
||||
</tr> |
||||
</table> |
||||
|
||||
|
||||
Channel State API |
||||
----------------- |
||||
|
||||
All gRPC libraries will expose a channel-level API method to poll the current |
||||
state of a channel. In C++, this method is called GetCurrentState and returns |
||||
an enum for one of the four legal states. |
||||
|
||||
All libraries should also expose an API that enables the application (user of |
||||
the gRPC API) to be notified when the channel state changes. Since state |
||||
changes can be rapid and race with any such notification, the notification |
||||
should just inform the user that some state change has happened, leaving it to |
||||
the user to poll the channel for the current state. |
||||
|
||||
The synchronous version of this API is: |
||||
|
||||
```cpp |
||||
bool WaitForStateChange(gpr_timespec deadline, ChannelState source_state); |
||||
``` |
||||
|
||||
which returns true when the state changes to something other than the |
||||
source_state and false if the deadline expires. Asynchronous and futures based |
||||
APIs should have a corresponding method that allows the application to be |
||||
notified when the state of a channel changes. |
||||
|
||||
Note that a notification is delivered every time there is a transition from any |
||||
state to any *other* state. On the other hand the rules for legal state |
||||
transition, require a transition from CONNECTING to TRANSIENT_FAILURE and back |
||||
to CONNECTING for every recoverable failure, even if the corresponding |
||||
exponential backoff requires no wait before retry. The combined effect is that |
||||
the application may receive state change notifications that appear spurious. |
||||
e.g., an application waiting for state changes on a channel that is CONNECTING |
||||
may receive a state change notification but find the channel in the same |
||||
CONNECTING state on polling for current state because the channel may have |
||||
spent infinitesimally small amount of time in the TRANSIENT_FAILURE state. |
||||
@ -0,0 +1,202 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2015, Google Inc. |
||||
* All rights reserved. |
||||
* |
||||
* Redistribution and use in source and binary forms, with or without |
||||
* modification, are permitted provided that the following conditions are |
||||
* met: |
||||
* |
||||
* * Redistributions of source code must retain the above copyright |
||||
* notice, this list of conditions and the following disclaimer. |
||||
* * Redistributions in binary form must reproduce the above |
||||
* copyright notice, this list of conditions and the following disclaimer |
||||
* in the documentation and/or other materials provided with the |
||||
* distribution. |
||||
* * Neither the name of Google Inc. nor the names of its |
||||
* contributors may be used to endorse or promote products derived from |
||||
* this software without specific prior written permission. |
||||
* |
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||
* |
||||
*/ |
||||
|
||||
#include "test/core/end2end/end2end_tests.h" |
||||
|
||||
#include <stdio.h> |
||||
#include <string.h> |
||||
|
||||
#include <grpc/byte_buffer.h> |
||||
#include <grpc/support/alloc.h> |
||||
#include <grpc/support/log.h> |
||||
#include <grpc/support/time.h> |
||||
#include <grpc/support/useful.h> |
||||
#include "src/core/transport/stream_op.h" |
||||
#include "test/core/end2end/cq_verifier.h" |
||||
|
||||
enum { TIMEOUT = 200000 }; |
||||
|
||||
static void *tag(gpr_intptr t) { return (void *)t; } |
||||
|
||||
static grpc_end2end_test_fixture begin_test(grpc_end2end_test_config config, |
||||
const char *test_name, |
||||
grpc_channel_args *client_args, |
||||
grpc_channel_args *server_args) { |
||||
grpc_end2end_test_fixture f; |
||||
gpr_log(GPR_INFO, "%s/%s", test_name, config.name); |
||||
f = config.create_fixture(client_args, server_args); |
||||
config.init_client(&f, client_args); |
||||
config.init_server(&f, server_args); |
||||
return f; |
||||
} |
||||
|
||||
static gpr_timespec n_seconds_time(int n) { |
||||
return GRPC_TIMEOUT_SECONDS_TO_DEADLINE(n); |
||||
} |
||||
|
||||
static gpr_timespec five_seconds_time(void) { return n_seconds_time(5); } |
||||
|
||||
static void drain_cq(grpc_completion_queue *cq) { |
||||
grpc_event ev; |
||||
do { |
||||
ev = grpc_completion_queue_next(cq, five_seconds_time()); |
||||
} while (ev.type != GRPC_QUEUE_SHUTDOWN); |
||||
} |
||||
|
||||
static void shutdown_server(grpc_end2end_test_fixture *f) { |
||||
if (!f->server) return; |
||||
grpc_server_shutdown_and_notify(f->server, f->cq, tag(1000)); |
||||
GPR_ASSERT(grpc_completion_queue_pluck(f->cq, tag(1000), GRPC_TIMEOUT_SECONDS_TO_DEADLINE(5)).type == GRPC_OP_COMPLETE); |
||||
grpc_server_destroy(f->server); |
||||
f->server = NULL; |
||||
} |
||||
|
||||
static void shutdown_client(grpc_end2end_test_fixture *f) { |
||||
if (!f->client) return; |
||||
grpc_channel_destroy(f->client); |
||||
f->client = NULL; |
||||
} |
||||
|
||||
static void end_test(grpc_end2end_test_fixture *f) { |
||||
shutdown_server(f); |
||||
shutdown_client(f); |
||||
|
||||
grpc_completion_queue_shutdown(f->cq); |
||||
drain_cq(f->cq); |
||||
grpc_completion_queue_destroy(f->cq); |
||||
} |
||||
|
||||
static void test_invoke_request_with_flags( |
||||
grpc_end2end_test_config config, gpr_uint32 *flags_for_op, |
||||
grpc_call_error call_start_batch_expected_result) { |
||||
grpc_call *c; |
||||
gpr_slice request_payload_slice = gpr_slice_from_copied_string("hello world"); |
||||
grpc_byte_buffer *request_payload = |
||||
grpc_raw_byte_buffer_create(&request_payload_slice, 1); |
||||
gpr_timespec deadline = five_seconds_time(); |
||||
grpc_end2end_test_fixture f = |
||||
begin_test(config, "test_invoke_request_with_flags", NULL, NULL); |
||||
cq_verifier *cqv = cq_verifier_create(f.cq); |
||||
grpc_op ops[6]; |
||||
grpc_op *op; |
||||
grpc_metadata_array initial_metadata_recv; |
||||
grpc_metadata_array trailing_metadata_recv; |
||||
grpc_metadata_array request_metadata_recv; |
||||
grpc_byte_buffer *request_payload_recv = NULL; |
||||
grpc_call_details call_details; |
||||
grpc_status_code status; |
||||
char *details = NULL; |
||||
size_t details_capacity = 0; |
||||
grpc_call_error expectation; |
||||
|
||||
c = grpc_channel_create_call(f.client, f.cq, "/foo", |
||||
"foo.test.google.fr", deadline); |
||||
GPR_ASSERT(c); |
||||
|
||||
grpc_metadata_array_init(&initial_metadata_recv); |
||||
grpc_metadata_array_init(&trailing_metadata_recv); |
||||
grpc_metadata_array_init(&request_metadata_recv); |
||||
grpc_call_details_init(&call_details); |
||||
|
||||
op = ops; |
||||
op->op = GRPC_OP_SEND_INITIAL_METADATA; |
||||
op->data.send_initial_metadata.count = 0; |
||||
op->flags = flags_for_op[op->op]; |
||||
op++; |
||||
op->op = GRPC_OP_SEND_MESSAGE; |
||||
op->data.send_message = request_payload; |
||||
op->flags = flags_for_op[op->op]; |
||||
op++; |
||||
op->op = GRPC_OP_SEND_CLOSE_FROM_CLIENT; |
||||
op->flags = flags_for_op[op->op]; |
||||
op++; |
||||
op->op = GRPC_OP_RECV_INITIAL_METADATA; |
||||
op->data.recv_initial_metadata = &initial_metadata_recv; |
||||
op->flags = flags_for_op[op->op]; |
||||
op++; |
||||
op->op = GRPC_OP_RECV_STATUS_ON_CLIENT; |
||||
op->data.recv_status_on_client.trailing_metadata = &trailing_metadata_recv; |
||||
op->data.recv_status_on_client.status = &status; |
||||
op->data.recv_status_on_client.status_details = &details; |
||||
op->data.recv_status_on_client.status_details_capacity = &details_capacity; |
||||
op->flags = flags_for_op[op->op]; |
||||
op++; |
||||
expectation = call_start_batch_expected_result; |
||||
GPR_ASSERT(expectation == grpc_call_start_batch(c, ops, op - ops, tag(1))); |
||||
|
||||
gpr_free(details); |
||||
grpc_metadata_array_destroy(&initial_metadata_recv); |
||||
grpc_metadata_array_destroy(&trailing_metadata_recv); |
||||
grpc_metadata_array_destroy(&request_metadata_recv); |
||||
grpc_call_details_destroy(&call_details); |
||||
|
||||
grpc_call_destroy(c); |
||||
|
||||
cq_verifier_destroy(cqv); |
||||
|
||||
grpc_byte_buffer_destroy(request_payload); |
||||
grpc_byte_buffer_destroy(request_payload_recv); |
||||
|
||||
end_test(&f); |
||||
config.tear_down_data(&f); |
||||
} |
||||
|
||||
void grpc_end2end_tests(grpc_end2end_test_config config) { |
||||
size_t i; |
||||
gpr_uint32 flags_for_op[GRPC_OP_RECV_CLOSE_ON_SERVER+1]; |
||||
|
||||
{ |
||||
/* check that all grpc_op_types fail when their flag value is set to an
|
||||
* invalid value */ |
||||
int indices[] = {GRPC_OP_SEND_INITIAL_METADATA, GRPC_OP_SEND_MESSAGE, |
||||
GRPC_OP_SEND_CLOSE_FROM_CLIENT, |
||||
GRPC_OP_RECV_INITIAL_METADATA, |
||||
GRPC_OP_RECV_STATUS_ON_CLIENT}; |
||||
for (i = 0; i < GPR_ARRAY_SIZE(indices); ++i) { |
||||
memset(flags_for_op, 0, sizeof(flags_for_op)); |
||||
flags_for_op[indices[i]] = 0xDEADBEEF; |
||||
test_invoke_request_with_flags(config, flags_for_op, |
||||
GRPC_CALL_ERROR_INVALID_FLAGS); |
||||
} |
||||
} |
||||
{ |
||||
/* check valid operation with allowed flags for GRPC_OP_SEND_BUFFER */ |
||||
gpr_uint32 flags[] = {GRPC_WRITE_BUFFER_HINT, GRPC_WRITE_NO_COMPRESS, |
||||
GRPC_WRITE_INTERNAL_COMPRESS}; |
||||
for (i = 0; i < GPR_ARRAY_SIZE(flags); ++i) { |
||||
memset(flags_for_op, 0, sizeof(flags_for_op)); |
||||
flags_for_op[GRPC_OP_SEND_MESSAGE] = flags[i]; |
||||
test_invoke_request_with_flags(config, flags_for_op, GRPC_CALL_OK); |
||||
} |
||||
} |
||||
} |
||||
File diff suppressed because one or more lines are too long
Loading…
Reference in new issue