Chiebot-Mirror
/
grpc
8
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge pull request #9106 from markdroth/service_config_doc

Browse Source 
Add service config doc.  Update naming and grpclb docs.
pull/9389/head
Mark D. Roth 8 years ago committed by GitHub
parent f8b268fb25 e860359f23
commit 73b20e6f94
10 changed files with 298 additions and 105 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. BIN
      doc/images/load-balancing.png
  2. 4
      doc/images/load-balancing.svg
  3. BIN
      doc/images/load_balancing_design.png
  4. 189
      doc/load-balancing.md
  5. 59
      doc/naming.md
  6. 147
      doc/service_config.md
  7. 1
      tools/doxygen/Doxyfile.c++
  8. 1
      tools/doxygen/Doxyfile.c++.internal
  9. 1
      tools/doxygen/Doxyfile.core
  10. 1
      tools/doxygen/Doxyfile.core.internal

BIN
doc/images/load-balancing.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

4
doc/images/load-balancing.svg
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 68 KiB

BIN
doc/images/load_balancing_design.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 39 KiB

189
doc/load-balancing.md
Unescape View File

@ -1,13 +1,21 @@
Load Balancing in gRPC
=======================
======================
# Objective
# Scope
To design a load balancing API between a gRPC client and a Load Balancer to
instruct the client how to send load to multiple backend servers.
This document explains the design for load balancing within gRPC.
# Background
## Per-Call Load Balancing
It is worth noting that load-balancing within gRPC happens on a per-call
basis, not a per-connection basis. In other words, even if all requests
come from a single client, we still want them to be load-balanced across
all servers.
## Approaches to Load Balancing
Prior to any gRPC specifics, we explore some usual ways to approach load
balancing.
@ -44,30 +52,31 @@ list of servers to the client.
### External Load Balancing Service
The client load balancing code is kept simple and portable, implementing
well-known algorithms (ie, Round Robin) for server selection.
Complex load balancing algorithms are instead provided by the load balancer. The
client relies on the load balancer to provide _load balancing configuration_ and
_the list of servers_ to which the client should send requests. The balancer
updates the server list as needed to balance the load as well as handle server
well-known algorithms (e.g., Round Robin) for server selection.
Complex load balancing algorithms are instead provided by the load
balancer. The client relies on the load balancer to provide _load
balancing configuration_ and _the list of servers_ to which the client
should send requests. The balancer updates the server list as needed
to balance the load as well as handle server unavailability or health
issues. The load balancer will make any necessary complex decisions and
inform the client. The load balancer may communicate with the backend
servers to collect load and health information.
# Requirements
## Simple API and client
The gRPC client load balancing code must be simple and portable. The
client should only contain simple algorithms (e.g., Round Robin) for
server selection. For complex algorithms, the client should rely on
a load balancer to provide load balancing configuration and the list of
servers to which the client should send requests. The balancer will update
the server list as needed to balance the load as well as handle server
unavailability or health issues. The load balancer will make any necessary
complex decisions and inform the client. The load balancer may communicate with
the backend servers to collect load and health information.
## Requirements
#### Simple API and client
The gRPC client load balancing code must be simple and portable. The client
should only contain simple algorithms (ie Round Robin) for server selection. For
complex algorithms, the client should rely on a load balancer to provide load
balancing configuration and the list of servers to which the client should send
requests. The balancer will update the server list as needed to balance the load
as well as handle server unavailability or health issues. The load balancer will
make any necessary complex decisions and inform the client. The load balancer
may communicate with the backend servers to collect load and health information.
complex decisions and inform the client. The load balancer may communicate
with the backend servers to collect load and health information.
#### Security
## Security
The load balancer may be separate from the actual server backends and a
compromise of the load balancer should only lead to a compromise of the
@ -75,70 +84,64 @@ loadbalancing functionality. In other words, a compromised load balancer should
not be able to cause a client to trust a (potentially malicious) backend server
any more than in a comparable situation without loadbalancing.
# Proposed Architecture
The gRPC load balancing implements the external load balancing server approach:
an external load balancer provides simple clients with an up-to-date list of
servers.
![image](images/load_balancing_design.png)
1. On startup, the gRPC client issues a name resolution request for the service.
The name will resolve to one or more IP addresses to gRPC servers, a hint on
whether the IP address(es) point to a load balancer or not, and also return a
client config.
2. The gRPC client connects to a gRPC Server.
1. If the name resolution has hinted that the endpoint is a load balancer,
the client's gRPC LB policy will attempt to open a stream to the load
balancer service. The server may respond in only one of the following
ways.
1. `status::UNIMPLEMENTED`. There is no loadbalancing in use. The client
call will fail.
2. "I am a Load Balancer and here is the server list." (Goto Step 4.)
3. "Please contact Load Balancer X" (See Step 3.) The client will close
this connection and cancel the stream.
4. If the server fails to respond, the client will wait for some timeout
and then re-resolve the name (process to Step 1 above).
2. If the name resolution has not hinted that the endpoint is a load
balancer, the client connects directly to the service it wants to talk to.
3. The gRPC client's gRPC LB policy opens a separate connection to the Load
Balancer. If this fails, it will go back to step 1 and try another address.
1. During channel initialization to the Load Balancer, the client will
attempt to open a stream to the Load Balancer service.
2. The Load Balancer will return a server list to the gRPC client. If the
server list is empty, the call will wait until a non-empty one is
received. Optional: The Load Balancer will also open channels to the gRPC
servers if load reporting is needed.
4. The gRPC client will send RPCs to the gRPC servers contained in the server
list from the Load Balancer.
5. Optional: The gRPC servers may periodically report load to the Load Balancer.
## Client
When establishing a gRPC _stream_ to the balancer, the client will send an initial
request to the load balancer (via a regular gRPC message). The load balancer
will respond with client config (including, for example, settings for flow
control, RPC deadlines, etc.) or a redirect to another load balancer. If the
balancer did not redirect the client, it will then send a list of servers to the
client. The client will contain simple load balancing logic for choosing the
next server when it needs to send a request.
## Load Balancer
The Load Balancer is responsible for providing the client with a list of servers
and client RPC parameters. The balancer chooses when to update the list of
servers and can decide whether to provide a complete list, a subset, or a
specific list of “picked” servers in a particular order. The balancer can
optionally provide an expiration interval after which the server list should no
longer be trusted and should be updated by the balancer.
The load balancer may open reporting streams to each server contained in the
server list. These streams are primarily used for load reporting. For example,
Weighted Round Robin requires that the servers report utilization to the load
balancer in order to compute the next list of servers.
## Server
The gRPC Server is responsible for answering RPC requests and providing
responses to the client. The server will also report load to the load balancer
if a reporting stream was opened for this purpose.
# Architecture
## Overview
The primary mechanism for load-balancing in gRPC is external
load-balancing, where an external load balancer provides simple clients
with an up-to-date list of servers.
The gRPC client does support an API for built-in load balancing policies.
However, there are only a small number of these (one of which is the
`grpclb` policy, which implements external load balancing), and users
are discouraged from trying to extend gRPC by adding more. Instead, new
load balancing policies should be implemented in external load balancers.
## Workflow
Load-balancing policies fit into the gRPC client workflow in between
name resolution and the connection to the server. Here's how it all
works:
![image](images/load-balancing.png)
1. On startup, the gRPC client issues a [name resolution](naming.md) request
for the server name. The name will resolve to one or more IP addresses,
each of which will indicate whether it is a server address or
a load balancer address, and a [service config](service_config.md)
that indicates which client-side load-balancing policy to use (e.g.,
`round_robin` or `grpclb`).
2. The client instantiates the load balancing policy.
- Note: If all addresses returned by the resolver are balancer
addresses, then the client will use the `grpclb` policy, regardless
of what load-balancing policy was requested by the service config.
Otherwise, the client will use the load-balancing policy requested
by the service config. If no load-balancing policy is requested
by the service config, then the client will default to a policy
that picks the first available server address.
3. The load balancing policy creates a subchannel to each server address.
- For all policies *except* `grpclb`, this means one subchannel for each
address returned by the resolver. Note that these policies
ignore any balancer addresses returned by the resolver.
- In the case of the `grpclb` policy, the workflow is as follows:
1. The policy opens a stream to one of the balancer addresses returned
by the resolver. It asks the balancer for the server addresses to
use for the server name originally requested by the client (i.e.,
the same one originally passed to the name resolver).
- Note: The `grpclb` policy currently ignores any non-balancer
addresses returned by the resolver. However, in the future, it
may be changed to use these addresses as a fallback in case no
balancers can be contacted.
2. The gRPC servers to which the load balancer is directing the client
may report load to the load balancers, if that information is needed
by the load balancer's configuration.
3. The load balancer returns a server list to the gRPC client's `grpclb`
policy. The `grpclb` policy will then create a subchannel to each of
server in the list.
4. For each RPC sent, the load balancing policy decides which
subchannel (i.e., which server) the RPC should be sent to.
- In the case of the `grpclb` policy, the client will send requests
to the servers in the order in which they were returned by the load
balancer. If the server list is empty, the call will block until a
non-empty one is received.

59
doc/naming.md
Unescape View File

@ -1,30 +1,65 @@
#gRPC Naming and Discovery Support
# gRPC Name Resolution
## Overview
gRPC supports DNS as the default name-system. A number of alternative name-systems are used in various deployments. We propose an API that is general enough to support a range of name-systems and the corresponding syntax for names. The gRPC client library in various languages will provide a plugin mechanism so resolvers for different name-systems can be plugged in.
gRPC supports DNS as the default name-system. A number of alternative
name-systems are used in various deployments. We support an API that is
general enough to support a range of name-systems and the corresponding
syntax for names. The gRPC client library in various languages will
provide a plugin mechanism so resolvers for different name-systems can
be plugged in.
## Detailed Proposal
## Detailed Design
A fully qualified, self contained name used for gRPC channel construction uses the syntax:
### Name Syntax
A fully qualified, self contained name used for gRPC channel construction
uses the syntax:
```
scheme://authority/endpoint_name
```
Here, scheme indicates the name-system to be used. Example schemes to be supported include:
Here, `scheme` indicates the name-system to be used. Currently, we
support the following schemes:
- `dns`
- `ipv4` (IPv4 address)
- `ipv6` (IPv6 address)
- `unix` (path to unix domain socket -- unix systems only)
* `dns`
In the future, additional schemes such as `etcd` could be added.
* `etcd`
The `authority` indicates some scheme-specific bootstrap information, e.g.,
for DNS, the authority may include the IP[:port] of the DNS server to
use. Often, a DNS name may be used as the authority, since the ability to
resolve DNS names is already built into all gRPC client libraries.
Authority indicates some scheme-specific bootstrap information, e.g., for DNS, the authority may include the IP[:port] of the DNS server to use. Often, a DNS name may used as the authority, since the ability to resolve DNS names is already built into all gRPC client libraries.
Finally, the `endpoint_name` indicates a concrete name to be looked up
in a given name-system identified by the scheme and the authority. The
syntax of the endpoint name is dictated by the scheme in use.
Finally, the endpoint_name indicates a concrete name to be looked up in a given name-system identified by the scheme and the authority. The syntax of endpoint name is dictated by the scheme in use.
### Resolver Plugins
### Plugins
The gRPC client library will use the specified scheme to pick the right
resolver plugin and pass it the fully qualified name string.
The gRPC client library will switch on the scheme to pick the right resolver plugin and pass it the fully qualified name string.
Resolvers should be able to contact the authority and get a resolution
that they return back to the gRPC client library. The returned contents
include:
Resolvers should be able to contact the authority and get a resolution that they return back to the gRPC client library. The returned contents include a list of IP:port, an optional config and optional auth config data to be used for channel authentication. The plugin API allows the resolvers to continuously watch an endpoint_name and return updated resolutions as needed.
- A list of resolved addresses, each of which has three attributes:
- The address itself, including both IP address and port.
- A boolean indicating whether the address is a backend address (i.e.,
the address to use to contact the server directly) or a balancer
address (for cases where [external load balancing](load-balancing.md)
is in use).
- The name of the balancer, if the address is a balancer address.
This will be used to perform peer authorization.
- A [service config](service_config.md).
The plugin API allows the resolvers to continuously watch an endpoint
and return updated resolutions as needed.

147
doc/service_config.md
Unescape View File

@ -0,0 +1,147 @@
Service Config in gRPC
======================
# Objective
The service config is a mechanism that allows service owners to publish
parameters to be automatically used by all clients of their service.
# Format
The service config is a JSON string of the following form:
```
{
# Load balancing policy name.
# Supported values are 'round_robin' and 'grpclb'.
# Optional; if unset, the default behavior is pick the first available
# backend.
# Note that if the resolver returns only balancer addresses and no
# backend addresses, gRPC will always use the 'grpclb' policy,
# regardless of what this field is set to.
'loadBalancingPolicy': string,
# Per-method configuration. Optional.
'methodConfig': [
{
# The names of the methods to which this method config applies. There
# must be at least one name. Each name entry must be unique across the
# entire service config. If the 'method' field is empty, then this
# method config specifies the defaults for all methods for the specified
# service.
#
# For example, let's say that the service config contains the following
# method config entries:
#
# 'methodConfig': [
# { 'name': [ { 'service': 'MyService' } ] ... },
# { 'name': [ { 'service': 'MyService', 'method': 'Foo' } ] ... }
# ]
#
# For a request for MyService/Foo, we will use the second entry, because
# it exactly matches the service and method name.
# For a request for MyService/Bar, we will use the first entry, because
# it provides the default for all methods of MyService.
'name': [
{
# RPC service name. Required.
# If using gRPC with protobuf as the IDL, then this will be of
# the form "pkg.service_name", where "pkg" is the package name
# defined in the proto file.
'service': string,
# RPC method name. Optional (see above).
'method': string,
}
],
# Whether RPCs sent to this method should wait until the connection is
# ready by default. If false, the RPC will abort immediately if there
# is a transient failure connecting to the server. Otherwise, gRPC will
# attempt to connect until the deadline is exceeded.
#
# The value specified via the gRPC client API will override the value
# set here. However, note that setting the value in the client API will
# also affect transient errors encountered during name resolution,
# which cannot be caught by the value here, since the service config
# is obtained by the gRPC client via name resolution.
'waitForReady': bool,
# The default timeout in seconds for RPCs sent to this method. This can
# be overridden in code. If no reply is received in the specified amount
# of time, the request is aborted and a deadline-exceeded error status
# is returned to the caller.
#
# The actual deadline used will be the minimum of the value specified
# here and the value set by the application via the gRPC client API.
# If either one is not set, then the other will be used.
# If neither is set, then the request has no deadline.
#
# The format of the value is that of the 'Duration' type defined here:
# https://developers.google.com/protocol-buffers/docs/proto3#json
'timeout': string,
# The maximum allowed payload size for an individual request or object
# in a stream (client->server) in bytes. The size which is measured is
# the serialized, uncompressed payload in bytes. This applies both
# to streaming and non-streaming requests.
#
# The actual value used is the minimum of the value specified here and
# the value set by the application via the gRPC client API.
# If either one is not set, then the other will be used.
# If neither is set, then the built-in default is used.
#
# If a client attempts to send an object larger than this value, it
# will not be sent and the client will see an error.
# Note that 0 is a valid value, meaning that the request message must
# be empty.
#
# The format of the value is that of the 'uint64' type defined here:
# https://developers.google.com/protocol-buffers/docs/proto3#json
'maxRequestMessageBytes': string,
# The maximum allowed payload size for an individual response or object
# in a stream (server->client) in bytes. The size which is measured is
# the serialized, uncompressed payload in bytes. This applies both
# to streaming and non-streaming requests.
#
# The actual value used is the minimum of the value specified here and
# the value set by the application via the gRPC client API.
# If either one is not set, then the other will be used.
# If neither is set, then the built-in default is used.
#
# If a server attempts to send an object larger than this value, it
# will not be sent, and the client will see an error.
# Note that 0 is a valid value, meaning that the response message must
# be empty.
#
# The format of the value is that of the 'uint64' type defined here:
# https://developers.google.com/protocol-buffers/docs/proto3#json
'maxResponseMessageBytes': string
}
]
}
```
Note that new per-method parameters may be added in the future as new
functionality is introduced.
# Architecture
A service config is associated with a server name. The [name
resolver](naming.md) plugin, when asked to resolve a particular server
name, will return both the resolved addresses and the service config.
TODO(roth): Design how the service config will be encoded in DNS.
# APIs
The service config is used in the following APIs:
- In the resolver API, used by resolver plugins to return the service
config to the gRPC client.
- In the gRPC client API, where users can query the channel to obtain
the service config associated with the channel (for debugging
purposes).
- In the gRPC client API, where users can set the service config
explicitly. This is intended for use in unit tests.

1
tools/doxygen/Doxyfile.c++
Unescape View File

@ -785,6 +785,7 @@ doc/naming.md \
doc/negative-http2-interop-test-descriptions.md \
doc/server-reflection.md \
doc/server_reflection_tutorial.md \
doc/service_config.md \
doc/statuscodes.md \
doc/stress_test_framework.md \
doc/wait-for-ready.md \

1
tools/doxygen/Doxyfile.c++.internal
Unescape View File

@ -785,6 +785,7 @@ doc/naming.md \
doc/negative-http2-interop-test-descriptions.md \
doc/server-reflection.md \
doc/server_reflection_tutorial.md \
doc/service_config.md \
doc/statuscodes.md \
doc/stress_test_framework.md \
doc/wait-for-ready.md \

1
tools/doxygen/Doxyfile.core
Unescape View File

@ -784,6 +784,7 @@ doc/naming.md \
doc/negative-http2-interop-test-descriptions.md \
doc/server-reflection.md \
doc/server_reflection_tutorial.md \
doc/service_config.md \
doc/statuscodes.md \
doc/stress_test_framework.md \
doc/wait-for-ready.md \

1
tools/doxygen/Doxyfile.core.internal
Unescape View File

@ -784,6 +784,7 @@ doc/naming.md \
doc/negative-http2-interop-test-descriptions.md \
doc/server-reflection.md \
doc/server_reflection_tutorial.md \
doc/service_config.md \
doc/statuscodes.md \
doc/stress_test_framework.md \
doc/wait-for-ready.md \

Write Preview
Loading…
Cancel
Save