Transport Explainer

Existing Transports

gRPC transports plug in below the core API (one level below the C++ or other wrapped-language API). You can write your transport in C or C++ though; currently (Nov 2017) all the transports are nominally written in C++ though they are idiomatically C. The existing transports are:

Among these, the in-process is likely the easiest to understand, though arguably also the least similar to a "real" sockets-based transport since it is only used in a single process.

Transport stream ops

In the gRPC core implementation, a fundamental struct is the grpc_transport_stream_op_batch which represents a collection of stream operations sent to a transport. (Note that in gRPC, stream and RPC are used synonymously since all RPCs are actually streams internally.) The ops in a batch can include:

send_initial_metadata
- Client: initiate an RPC
- Server: supply response headers
recv_initial_metadata
- Client: get response headers
- Server: accept an RPC
send_message (zero or more) : send a data buffer
recv_message (zero or more) : receive a data buffer
send_trailing_metadata
- Client: half-close indicating that no more messages will be coming
- Server: full-close providing final status for the RPC
recv_trailing_metadata: get final status for the RPC
- Server extra: This op shouldn't actually be considered complete until the server has also sent trailing metadata to provide the other side with final status
cancel_stream: Attempt to cancel an RPC
collect_stats: Get stats

The fundamental responsibility of the transport is to transform between this internal format and an actual wire format, so the processing of these operations is largely transport-specific.

One or more of these ops are grouped into a batch. Applications can start all of a call's ops in a single batch, or they can split them up into multiple batches. Results of each batch are returned asynchronously via a completion queue.

Internally, we use callbacks to indicate completion. The surface layer creates a callback when starting a new batch and sends it down the filter stack along with the batch. The transport must invoke this callback when the batch is complete, and then the surface layer returns an event to the application via the completion queue. Each batch can have up to 3 callbacks:

recv_initial_metadata_ready (called by the transport when the recv_initial_metadata op is complete)
recv_message_ready (called by the transport when the recv_message op is complete)
on_complete (called by the transport when the entire batch is complete)

Timelines of transport stream op batches

The transport's job is to sequence and interpret various possible interleavings of the basic stream ops. For example, a sample timeline of batches would be:

Client send_initial_metadata: Initiate an RPC with a path (method) and authority
Server recv_initial_metadata: accept an RPC
Client send_message: Supply the input proto for the RPC
Server recv_message: Get the input proto from the RPC
Client send_trailing_metadata: This is a half-close indicating that the client will not be sending any more messages
Server recv_trailing_metadata: The server sees this from the client and knows that it will not get any more messages. This won't complete yet though, as described above.
Server send_initial_metadata, send_message, send_trailing_metadata: A batch can contain multiple ops, and this batch provides the RPC response headers, response content, and status. Note that sending the trailing metadata will also complete the server's receive of trailing metadata.
Client recv_initial_metadata: The number of ops in one side of the batch has no relation with the number of ops on the other side of the batch. In this case, the client is just collecting the response headers.
Client recv_message, recv_trailing_metadata: Get the data response and status

There are other possible sample timelines. For example, for client-side streaming, a "typical" sequence would be:

Server: recv_initial_metadata
- At API-level, that would be the server requesting an RPC
Server: recv_trailing_metadata
- This is for when the server wants to know the final completion of the RPC through an AsyncNotifyWhenDone API in C++
Client: send_initial_metadata, recv_message, recv_trailing_metadata
- At API-level, that's a client invoking a client-side streaming call. The send_initial_metadata is the call invocation, the recv_message collects the final response from the server, and the recv_trailing_metadata gets the grpc::Status value that will be returned from the call
Client: send_message / Server: recv_message
- Repeat the above step numerous times; these correspond to a client issuing Write in a loop and a server doing Read in a loop until Read fails
Client: send_trailing_metadata / Server: recv_message that indicates doneness (NULL)
- These correspond to a client issuing WritesDone which causes the server's Read to fail
Server: send_message, send_trailing_metadata
- These correspond to the server doing Finish

The sends on one side will call their own callbacks when complete, and they will in turn trigger actions that cause the other side's recv operations to complete. In some transports, a send can sometimes complete before the recv on the other side (e.g., in HTTP/2 if there is sufficient flow-control buffer space available)

Other transport duties

In addition to these basic stream ops, the transport must handle cancellations of a stream at any time and pass their effects to the other side. For example, in HTTP/2, this triggers a RST_STREAM being sent on the wire. The transport must perform operations like pings and statistics that are used to shape transport-level characteristics like flow control (see, for example, their use in the HTTP/2 transport).

Putting things together with detail: Sending Metadata

API layer: map<string, string> that is specific to this RPC
Core surface layer: array of {slice, slice} pairs where each slice references an underlying string
Core transport layer: list of {slice, slice} pairs that includes the above plus possibly some general metadata (e.g., Method and Authority for initial metadata)
Specific transport layer:
- Either send it to the other side using transport-specific API (e.g., Cronet)
- Or have it sent through the iomgr/endpoint layer (e.g., HTTP/2)
- Or just manipulate pointers to get it from one side to the other (e.g., In-process)

Requirements for any transport

Each transport implements several operations in a vtbl (may change to actual virtual functions as transport moves to idiomatic C++).

The most important and common one is perform_stream_op. This function processes a single stream op batch on a specific stream that is associated with a specific transport:

Gets the 6 ops/cancel passed down from the surface
Pass metadata from one side to the other as described above
Transform messages between slice buffer structure and stream of bytes to pass to other side
- May require insertion of extra bytes (e.g., per-message headers in HTTP/2)
React to metadata to preserve expected orderings (*)
Schedule invocation of completion callbacks

There are other functions in the vtbl as well.

perform_transport_op
- Configure the transport instance for the connectivity state change notifier or the server-side accept callback
- Disconnect transport or set up a goaway for later streams
init_stream
- Starts a stream from the client-side
- (*) Server-side of the transport must call accept_stream_cb when a new stream is available
  - Triggers request-matcher
destroy_stream, destroy_transport
- Free up data related to a stream or transport
set_pollset, set_pollset_set, get_endpoint
- Map each specific instance of the transport to FDs being used by iomgr (for HTTP/2)
- Get a pointer to the endpoint structure that actually moves the data (wrapper around a socket for HTTP/2)

Book-keeping responsibilities of the transport layer

A given transport must keep all of its transport and streams ref-counted. This is essential to make sure that no struct disappears before it is done being used.

A transport must also preserve relevant orders for the different categories of ops on a stream, as described above. A transport must also make sure that all relevant batch operations have completed before scheduling the on_complete closure for a batch. Further examples include the idea that the server logic expects to not complete recv_trailing_metadata until after it actually sends trailing metadata since it would have already found this out by seeing a NULL’ed recv_message. This is considered part of the transport's duties in preserving orders.

9.3 KiB Raw Blame History