You can find more detailed documentation and examples in the [doc](doc) and [examples](examples) directories respectively.
You can find more detailed documentation and examples in the [doc](doc) and [examples](examples) directories respectively.
#Installation& Testing
# Installation& Testing
See [INSTALL](INSTALL.md) for installation instructions for various platforms.
See [INSTALL](INSTALL.md) for installation instructions for various platforms.
@ -19,32 +19,31 @@ See [tools/run_tests](tools/run_tests) for more guidance on how to run various t
See [Performance dashboard](http://performance-dot-grpc-testing.appspot.com/explore?dashboard=5712453606309888) for the performance numbers for v1.0.x.
See [Performance dashboard](http://performance-dot-grpc-testing.appspot.com/explore?dashboard=5712453606309888) for the performance numbers for v1.0.x.
#Repository Structure & Status
# Repository Structure & Status
This repository contains source code for gRPC libraries for multiple languages written on top of shared C core library [src/core] (src/core).
This repository contains source code for gRPC libraries for multiple languages written on top of shared C core library [src/core](src/core).
Libraries in different languages may be in different states of development. We are seeking contributions for all of these libraries.
Libraries in different languages may be in different states of development. We are seeking contributions for all of these libraries.
@ -91,7 +91,7 @@ A **Compressed-Flag** value of 1 indicates that the binary octet sequence of **M
For requests, **EOS** (end-of-stream) is indicated by the presence of the END_STREAM flag on the last received DATA frame. In scenarios where the **Request** stream needs to be closed but no data remains to be sent implementations MUST send an empty DATA frame with this flag set.
For requests, **EOS** (end-of-stream) is indicated by the presence of the END_STREAM flag on the last received DATA frame. In scenarios where the **Request** stream needs to be closed but no data remains to be sent implementations MUST send an empty DATA frame with this flag set.
While the protocol does not require a user-agent to function it is recommended that clients provide a structured user-agent string that provides a basic description of the calling library, version & platform to facilitate issue diagnosis in heterogeneous environments. The following structure is recommended to library developers
While the protocol does not require a user-agent to function it is recommended that clients provide a structured user-agent string that provides a basic description of the calling library, version & platform to facilitate issue diagnosis in heterogeneous environments. The following structure is recommended to library developers
Unless explicitly defined to be, gRPC Calls are not assumed to be idempotent. Specifically:
Unless explicitly defined to be, gRPC Calls are not assumed to be idempotent. Specifically:
@ -186,15 +186,15 @@ Unless explicitly defined to be, gRPC Calls are not assumed to be idempotent. S
* Calls that are marked as idempotent may be sent multiple times.
* Calls that are marked as idempotent may be sent multiple times.
####HTTP2 Transport Mapping
####HTTP2 Transport Mapping
#####Stream Identification
#####Stream Identification
All GRPC calls need to specify an internal ID. We will use HTTP2 stream-ids as call identifiers in this scheme. NOTE: These id’s are contextual to an open HTTP2 session and will not be unique within a given process that is handling more than one HTTP2 session nor can they be used as GUIDs.
All GRPC calls need to specify an internal ID. We will use HTTP2 stream-ids as call identifiers in this scheme. NOTE: These id’s are contextual to an open HTTP2 session and will not be unique within a given process that is handling more than one HTTP2 session nor can they be used as GUIDs.
#####Data Frames
#####Data Frames
DATA frame boundaries have no relation to **Length-Prefixed-Message** boundaries and implementations should make no assumptions about their alignment.
DATA frame boundaries have no relation to **Length-Prefixed-Message** boundaries and implementations should make no assumptions about their alignment.
#####Errors
#####Errors
When an application or runtime error occurs during an RPC a **Status** and **Status-Message** are delivered in **Trailers**.
When an application or runtime error occurs during an RPC a **Status** and **Status-Message** are delivered in **Trailers**.
INADEQUATE_SECURITY| PERMISSION_DENIED … with additional detail indicating that permission was denied as protocol is not secure enough for call.
INADEQUATE_SECURITY| PERMISSION_DENIED … with additional detail indicating that permission was denied as protocol is not secure enough for call.
#####Security
#####Security
The HTTP2 specification mandates the use of TLS 1.2 or higher when TLS is used with HTTP2. It also places some additional constraints on the allowed ciphers in deployments to avoid known-problems as well as requiring SNI support. It is also expected that HTTP2 will be used in conjunction with proprietary transport security mechanisms about which the specification can make no meaningful recommendations.
The HTTP2 specification mandates the use of TLS 1.2 or higher when TLS is used with HTTP2. It also places some additional constraints on the allowed ciphers in deployments to avoid known-problems as well as requiring SNI support. It is also expected that HTTP2 will be used in conjunction with proprietary transport security mechanisms about which the specification can make no meaningful recommendations.
#####Connection Management
#####Connection Management
######GOAWAY Frame
######GOAWAY Frame
Sent by servers to clients to indicate that they will no longer accept any new streams on the associated connections. This frame includes the id of the last successfully accepted stream by the server. Clients should consider any stream initiated after the last successfully accepted stream as UNAVAILABLE and retry the call elsewhere. Clients are free to continue working with the already accepted streams until they complete or the connection is terminated.
Sent by servers to clients to indicate that they will no longer accept any new streams on the associated connections. This frame includes the id of the last successfully accepted stream by the server. Clients should consider any stream initiated after the last successfully accepted stream as UNAVAILABLE and retry the call elsewhere. Clients are free to continue working with the already accepted streams until they complete or the connection is terminated.
Servers should send GOAWAY before terminating a connection to reliably inform clients which work has been accepted by the server and is being executed.
Servers should send GOAWAY before terminating a connection to reliably inform clients which work has been accepted by the server and is being executed.
######PING Frame
######PING Frame
Both clients and servers can send a PING frame that the peer must respond to by precisely echoing what they received. This is used to assert that the connection is still live as well as providing a means to estimate end-to-end latency. If a server initiated PING does not receive a response within the deadline expected by the runtime all outstanding calls on the server will be closed with a CANCELLED status. An expired client initiated PING will cause all calls to be closed with an UNAVAILABLE status. Note that the frequency of PINGs is highly dependent on the network environment, implementations are free to adjust PING frequency based on network and application requirements.
Both clients and servers can send a PING frame that the peer must respond to by precisely echoing what they received. This is used to assert that the connection is still live as well as providing a means to estimate end-to-end latency. If a server initiated PING does not receive a response within the deadline expected by the runtime all outstanding calls on the server will be closed with a CANCELLED status. An expired client initiated PING will cause all calls to be closed with an UNAVAILABLE status. Note that the frequency of PINGs is highly dependent on the network environment, implementations are free to adjust PING frequency based on network and application requirements.
######Connection failure
######Connection failure
If a detectable connection failure occurs on the client all calls will be closed with an UNAVAILABLE status. For servers open calls will be closed with a CANCELLED status.
If a detectable connection failure occurs on the client all calls will be closed with an UNAVAILABLE status. For servers open calls will be closed with a CANCELLED status.
The log format is described in [this proto file](src/proto/grpc/binary_log/v1alpha/log.proto). It is intended that multiple parts of the call will be logged in separate files, and then correlated by analysis tools using the rpc\_id.
The log format is described in [this proto file](/src/proto/grpc/binary_log/v1alpha/log.proto). It is intended that multiple parts of the call will be logged in separate files, and then correlated by analysis tools using the rpc\_id.
If you have two threads calling combiner, there will be some kind of
queuing in place. It's called `combiner` because you can pass in more
than one do_stuff at once and they will run under a common `mu`.
The implementation described above has the issue that you're blocking a thread
for a period of time, and this is considered harmful because it's an application thread that you're blocking.
Instead, get a new property:
* Keep things running in serial execution
* Don't ever sleep the thread
* But maybe allow things to end up running on a different thread from where they were started
* This means that `do_stuff` doesn't necessarily run to completion when `combiner.run` is invoked
```
class combiner {
mpscq q; // multi-producer single-consumer queue can be made non-blocking
state s; // is it empty or executing
run(f) {
if (q.push(f)) {
// q.push returns true if it's the first thing
while (q.pop(&f)) { // modulo some extra work to avoid races
f();
}
}
}
}
```
The basic idea is that the first one to push onto the combiner
executes the work and then keeps executing functions from the queue
until the combiner is drained.
Our combiner does some additional work, with the motivation of write-batching.
We have a second tier of `run` called `run_finally`. Anything queued
onto `run_finally` runs after we have drained the queue. That means
that there is essentially a finally-queue. This is not guaranteed to
be final, but it's best-effort. In the process of running the finally
item, we might put something onto the main combiner queue and so we'll
need to re-enter.
`chttp2` runs all ops in the run state except if it sees a write it puts that into a finally. That way anything else that gets put into the combiner can add to that write.
```
class combiner {
mpscq q; // multi-producer single-consumer queue can be made non-blocking
state s; // is it empty or executing
queue finally; // you can only do run_finally when you are already running something from the combiner
run(f) {
if (q.push(f)) {
// q.push returns true if it's the first thing
loop:
while (q.pop(&f)) { // modulo some extra work to avoid races
f();
}
while (finally.pop(&f)) {
f();
}
goto loop;
}
}
}
```
So that explains how combiners work in general. In gRPC, there is
`start_batch(..., tag)` and then work only gets activated by somebody
calling `cq::next` which returns a tag. This gives an API-level
guarantee that there will be a thread doing polling to actually make
work happen. However, some operations are not covered by a poller
thread, such as cancellation that doesn't have a completion. Other
callbacks that don't have a completion are the internal work that gets
done before the batch gets completed. We need a condition called
`covered_by_poller` that means that the item will definitely need some
thread at some point to call `cq::next` . This includes those
callbacks that directly cause a completion but also those that are
indirectly required before getting a completion. If we can't tell for
sure for a specific path, we have to assumed it is not covered by
poller.
The above combiner has the problem that it keeps draining for a
potentially infinite amount of time and that can lead to a huge tail
latency for some operations. So we can tweak it by returning to the application
if we know that it is valid to do so:
```
while (q.pop(&f)) {
f();
if (control_can_be_returned && some_still_queued_thing_is_covered_by_poller) {
offload_combiner_work_to_some_other_thread();
}
}
```
`offload` is more than `break`; it does `break` but also causes some
other thread that is currently waiting on a poll to break out of its
poll. This is done by setting up a per-polling-island work-queue
(distributor) wakeup FD. The work-queue is the converse of the combiner; it
tries to spray events onto as many threads as possible to get as much concurrency as possible.
So `offload` really does:
```
workqueue.run(continue_from_while_loop);
break;
```
This needs us to add another class variable for a `workqueue`
(which is really conceptually a distributor).
```
workqueue::run(f) {
q.push(f)
eventfd.wakeup()
}
workqueue::readable() {
eventfd.consume();
q.pop(&f);
f();
if (!q.empty()) {
eventfd.wakeup(); // spray across as many threads as are waiting on this workqueue
}
}
```
In principle, `run_finally` could get starved, but this hasn't
happened in practice. If we were concerned about this, we could put a
limit on how many things come off the regular `q` before the `finally`
@ -13,7 +13,7 @@ The contents of the *auth properties* are populated by an *auth interceptor*. Th
WARNING: AuthContext is the only reliable source of truth when it comes to authenticating RPCs. Using any other call/context properties for authentication purposes is wrong and inherently unsafe.
WARNING: AuthContext is the only reliable source of truth when it comes to authenticating RPCs. Using any other call/context properties for authentication purposes is wrong and inherently unsafe.
####Example AuthContext contents
####Example AuthContext contents
For secure channel using mutual TLS authentication with both client and server certificates (test certificates from this repository are used).
For secure channel using mutual TLS authentication with both client and server certificates (test certificates from this repository are used).
@ -45,7 +45,7 @@ gRPC comes with some basic "interceptors" already built-in.
WARNING: While there is a public API that allows anyone to write their own custom interceptor, please think twice before using it.
WARNING: While there is a public API that allows anyone to write their own custom interceptor, please think twice before using it.
There are legitimate uses for custom interceptors but you should keep in mind that as auth interceptors essentially decide which RPCs are authenticated and which are not, their code is very sensitive from the security perspective and getting things wrong might have serious consequences. If unsure, we strongly recommend to rely on official & proven interceptors that come with gRPC.
There are legitimate uses for custom interceptors but you should keep in mind that as auth interceptors essentially decide which RPCs are authenticated and which are not, their code is very sensitive from the security perspective and getting things wrong might have serious consequences. If unsure, we strongly recommend to rely on official & proven interceptors that come with gRPC.
####Available auth interceptors
####Available auth interceptors
- TLS/SSL certificate authentication (built into gRPC's security layer, automatically used whenever you use a secure connection)
- TLS/SSL certificate authentication (built into gRPC's security layer, automatically used whenever you use a secure connection)
gRPC uses a set of well defined status codes as part of the RPC API. All RPCs started at a client return a `status` object composed of an integer `code` and a string `message`. The server-side can choose the status it returns for a given RPC.
gRPC uses a set of well defined status codes as part of the RPC API. All
RPCs started at a client return a `status` object composed of an integer
`code` and a string `message`. The server-side can choose the status it
returns for a given RPC.
The gRPC client and server-side implementations may also generate and return `status` on their own when errors happen.
The gRPC client and server-side implementations may also generate and
Only a subset of the pre-defined status codes are generated by the gRPC libraries. The following table lists these codes and summarizes the situations in which they are generated, either by the client or the server-side library implementation.
return `status` on their own when errors happen. Only a subset of
the pre-defined status codes are generated by the gRPC libraries. This
allows applications to be sure that any other code it sees was actually
returned by the application (although it is also possible for the
server-side to return one of the codes generated by the gRPC libraries).
The following table lists the codes that may be returned by the gRPC
libraries (on either the client-side or server-side) and summarizes the
situations in which they are generated.
| Case | Code | Generated at Client or Server |
| Case | Code | Generated at Client or Server |
| ------------- |:-------------| :-----:|
| ------------- |:-------------| :-----:|
@ -26,7 +37,7 @@ Only a subset of the pre-defined status codes are generated by the gRPC librarie
| Response cardinality violation (method requires exactly one response but server sent some other number of responses) | UNIMPLEMENTED | Client|
| Response cardinality violation (method requires exactly one response but server sent some other number of responses) | UNIMPLEMENTED | Client|
| Error parsing response proto | INTERNAL | Client|
| Error parsing response proto | INTERNAL | Client|
| Error parsing request proto | INTERNAL | Server|
| Error parsing request proto | INTERNAL | Server|
| Sent or received message was larger than configured limit | RESOURCE_EXHAUSTED | Both |
The following status codes are never generated by the library:
The following status codes are never generated by the library:
# Issuer: CN=TUBITAK Kamu SM SSL Kok Sertifikasi - Surum 1 O=Turkiye Bilimsel ve Teknolojik Arastirma Kurumu - TUBITAK OU=Kamu Sertifikasyon Merkezi - Kamu SM
# Subject: CN=TUBITAK Kamu SM SSL Kok Sertifikasi - Surum 1 O=Turkiye Bilimsel ve Teknolojik Arastirma Kurumu - TUBITAK OU=Kamu Sertifikasyon Merkezi - Kamu SM
# Label: "TUBITAK Kamu SM SSL Kok Sertifikasi - Surum 1"
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
This is the static code generation variant of the Node examples. Code in these examples is pre-generated using protoc and the Node gRPC protoc plugin, and the generated code can be found in various `*_pb.js` files. The command line sequence for generating those files is as follows (assuming that `protoc` and `grpc_node_plugin` are present, and starting in the base directory of this package):
This is the static code generation variant of the Node examples. Code in these examples is pre-generated using protoc and the Node gRPC protoc plugin, and the generated code can be found in various `*_pb.js` files. The command line sequence for generating those files is as follows (assuming that `protoc` and `grpc_node_plugin` are present, and starting in the directory which contains this README.md file):
Jan Tattermusch
8 years ago
committed by
GitHub