Merge pull request #9106 from markdroth/service_config_doc

Add service config doc.  Update naming and grpclb docs.

doc/load-balancing.md

@@ -1,13 +1,21 @@
 Load Balancing in gRPC
-=======================
+======================
 
-# Objective
+# Scope
 
-To design a load balancing API between a gRPC client and a Load Balancer to
+This document explains the design for load balancing within gRPC.
-instruct the client how to send load to multiple backend servers.
 
 # Background
 
+## Per-Call Load Balancing
+
+It is worth noting that load-balancing within gRPC happens on a per-call
+basis, not a per-connection basis.  In other words, even if all requests
+come from a single client, we still want them to be load-balanced across
+all servers.
+
+## Approaches to Load Balancing
+
 Prior to any gRPC specifics, we explore some usual ways to approach load
 balancing.
 
@@ -44,30 +52,31 @@ list of servers to the client.
 ### External Load Balancing Service
 
 The client load balancing code is kept simple and portable, implementing
-well-known algorithms (ie, Round Robin) for server selection.
+well-known algorithms (e.g., Round Robin) for server selection.
-Complex load balancing algorithms are instead provided by the load balancer. The
+Complex load balancing algorithms are instead provided by the load
-client relies on the load balancer to provide _load balancing configuration_ and
+balancer. The client relies on the load balancer to provide _load
-_the list of servers_ to which the client should send requests. The balancer
+balancing configuration_ and _the list of servers_ to which the client
-updates the server list as needed to balance the load as well as handle server
+should send requests. The balancer updates the server list as needed
+to balance the load as well as handle server unavailability or health
+issues. The load balancer will make any necessary complex decisions and
+inform the client. The load balancer may communicate with the backend
+servers to collect load and health information.
+
+# Requirements
+
+## Simple API and client
+
+The gRPC client load balancing code must be simple and portable. The
+client should only contain simple algorithms (e.g., Round Robin) for
+server selection.  For complex algorithms, the client should rely on
+a load balancer to provide load balancing configuration and the list of
+servers to which the client should send requests. The balancer will update
+the server list as needed to balance the load as well as handle server
 unavailability or health issues. The load balancer will make any necessary
-complex decisions and inform the client. The load balancer may communicate with
+complex decisions and inform the client. The load balancer may communicate
-the backend servers to collect load and health information.
+with the backend servers to collect load and health information.
-
-
-## Requirements
-
-#### Simple API and client
-
-The gRPC client load balancing code must be simple and portable. The client
-should only contain simple algorithms (ie Round Robin) for server selection. For
-complex algorithms, the client should rely on a load balancer to provide load
-balancing configuration and the list of servers to which the client should send
-requests. The balancer will update the server list as needed to balance the load
-as well as handle server unavailability or health issues. The load balancer will
-make any necessary complex decisions and inform the client. The load balancer
-may communicate with the backend servers to collect load and health information.
 
-#### Security
+## Security
 
 The load balancer may be separate from the actual server backends and a
 compromise of the load balancer should only lead to a compromise of the
@@ -75,70 +84,64 @@ loadbalancing functionality. In other words, a compromised load balancer should
 not be able to cause a client to trust a (potentially malicious) backend server
 any more than in a comparable situation without loadbalancing.
 
-# Proposed Architecture
+# Architecture
-
+
-The gRPC load balancing implements the external load balancing server approach:
+## Overview
-an external load balancer provides simple clients with an up-to-date list of
+
-servers.
+The primary mechanism for load-balancing in gRPC is external
-
+load-balancing, where an external load balancer provides simple clients
-![image](images/load_balancing_design.png)
+with an up-to-date list of servers.
-
+
-1. On startup, the gRPC client issues a name resolution request for the service.
+The gRPC client does support an API for built-in load balancing policies.
-   The name will resolve to one or more IP addresses to gRPC servers, a hint on
+However, there are only a small number of these (one of which is the
-   whether the IP address(es) point to a load balancer or not, and also return a
+`grpclb` policy, which implements external load balancing), and users
-   client config.
+are discouraged from trying to extend gRPC by adding more.  Instead, new
-2. The gRPC client connects to a gRPC Server.
+load balancing policies should be implemented in external load balancers.
-   1. If the name resolution has hinted that the endpoint is a load balancer,
+
-      the client's gRPC LB policy will attempt to open a stream to the load
+## Workflow
-      balancer service. The server may respond in only one of the following
+
-      ways.
+Load-balancing policies fit into the gRPC client workflow in between
-      1. `status::UNIMPLEMENTED`. There is no loadbalancing in use. The client
+name resolution and the connection to the server.  Here's how it all
-         call will fail.
+works:
-      2. "I am a Load Balancer and here is the server list." (Goto Step 4.)
+
-      3. "Please contact Load Balancer X" (See Step 3.) The client will close
+![image](images/load-balancing.png)
-         this connection and cancel the stream.
+
-      4. If the server fails to respond, the client will wait for some timeout
+1. On startup, the gRPC client issues a [name resolution](naming.md) request
-         and then re-resolve the name (process to Step 1 above).
+   for the server name.  The name will resolve to one or more IP addresses,
-   2. If the name resolution has not hinted that the endpoint is a load
+   each of which will indicate whether it is a server address or
-      balancer, the client connects directly to the service it wants to talk to.
+   a load balancer address, and a [service config](service_config.md)
-3. The gRPC client's gRPC LB policy opens a separate connection to the Load
+   that indicates which client-side load-balancing policy to use (e.g.,
-   Balancer. If this fails, it will go back to step 1 and try another address.
+   `round_robin` or `grpclb`).
-   1. During channel initialization to the Load Balancer, the client will
+2. The client instantiates the load balancing policy.
-      attempt to open a stream to the Load Balancer service.
+   - Note: If all addresses returned by the resolver are balancer
-   2. The Load Balancer will return a server list to the gRPC client. If the
+     addresses, then the client will use the `grpclb` policy, regardless
-      server list is empty, the call will wait until a non-empty one is
+     of what load-balancing policy was requested by the service config.
-      received. Optional: The Load Balancer will also open channels to the gRPC
+     Otherwise, the client will use the load-balancing policy requested
-      servers if load reporting is needed.
+     by the service config.  If no load-balancing policy is requested
-4. The gRPC client will send RPCs to the gRPC servers contained in the server
+     by the service config, then the client will default to a policy
-   list from the Load Balancer.
+     that picks the first available server address.
-5. Optional: The gRPC servers may periodically report load to the Load Balancer.
+3. The load balancing policy creates a subchannel to each server address.
-
+   - For all policies *except* `grpclb`, this means one subchannel for each
-## Client
+     address returned by the resolver. Note that these policies
-
+     ignore any balancer addresses returned by the resolver.
-When establishing a gRPC _stream_ to the balancer, the client will send an initial
+   - In the case of the `grpclb` policy, the workflow is as follows:
-request to the load balancer (via a regular gRPC message). The load balancer
+     1. The policy opens a stream to one of the balancer addresses returned
-will respond with client config (including, for example, settings for flow
+        by the resolver. It asks the balancer for the server addresses to
-control, RPC deadlines, etc.) or a redirect to another load balancer. If the
+        use for the server name originally requested by the client (i.e.,
-balancer did not redirect the client, it will then send a list of servers to the
+        the same one originally passed to the name resolver).
-client. The client will contain simple load balancing logic for choosing the
+        - Note: The `grpclb` policy currently ignores any non-balancer
-next server when it needs to send a request.
+          addresses returned by the resolver. However, in the future, it
-
+          may be changed to use these addresses as a fallback in case no
-## Load Balancer
+          balancers can be contacted.
-
+     2. The gRPC servers to which the load balancer is directing the client
-The Load Balancer is responsible for providing the client with a list of servers
+        may report load to the load balancers, if that information is needed
-and client RPC parameters. The balancer chooses when to update the list of
+        by the load balancer's configuration.
-servers and can decide whether to provide a complete list, a subset, or a
+     3. The load balancer returns a server list to the gRPC client's `grpclb`
-specific list of “picked” servers in a particular order. The balancer can
+        policy. The `grpclb` policy will then create a subchannel to each of
-optionally provide an expiration interval after which the server list should no
+        server in the list.
-longer be trusted and should be updated by the balancer.
+4. For each RPC sent, the load balancing policy decides which
-
+   subchannel (i.e., which server) the RPC should be sent to.
-The load balancer may open reporting streams to each server contained in the
+   - In the case of the `grpclb` policy, the client will send requests
-server list. These streams are primarily used for load reporting. For example,
+     to the servers in the order in which they were returned by the load
-Weighted Round Robin requires that the servers report utilization to the load
+     balancer.  If the server list is empty, the call will block until a
-balancer in order to compute the next list of servers.
+     non-empty one is received.
-
-## Server
-
-The gRPC Server is responsible for answering RPC requests and providing
-responses to the client. The server will also report load to the load balancer
-if a reporting stream was opened for this purpose.

doc/naming.md

@@ -1,30 +1,65 @@
-#gRPC Naming and Discovery Support
+# gRPC Name Resolution
 
 ## Overview
 
-gRPC supports DNS as the default name-system. A number of alternative name-systems are used in various deployments. We propose an API that is general enough to support a range of name-systems and the corresponding syntax for names. The gRPC client library in various languages will provide a plugin mechanism so resolvers for different name-systems can be plugged in.
+gRPC supports DNS as the default name-system. A number of alternative
+name-systems are used in various deployments. We support an API that is
+general enough to support a range of name-systems and the corresponding
+syntax for names. The gRPC client library in various languages will
+provide a plugin mechanism so resolvers for different name-systems can
+be plugged in.
 
-## Detailed Proposal
+## Detailed Design
 
- A fully qualified, self contained name used for gRPC channel construction uses the syntax:
+### Name Syntax
+
+A fully qualified, self contained name used for gRPC channel construction
+uses the syntax:
 
 ```
 scheme://authority/endpoint_name
 ```
 
-Here, scheme indicates the name-system to be used. Example schemes to be supported include: 
+Here, `scheme` indicates the name-system to be used. Currently, we
+support the following schemes:
+
+- `dns`
+
+- `ipv4` (IPv4 address)
+
+- `ipv6` (IPv6 address)
+
+- `unix` (path to unix domain socket -- unix systems only)
 
-* `dns`
+In the future, additional schemes such as `etcd` could be added.
 
-* `etcd`
+The `authority` indicates some scheme-specific bootstrap information, e.g.,
+for DNS, the authority may include the IP[:port] of the DNS server to
+use. Often, a DNS name may be used as the authority, since the ability to
+resolve DNS names is already built into all gRPC client libraries.
 
-Authority indicates some scheme-specific bootstrap information, e.g., for DNS, the authority may include the IP[:port] of the DNS server to use. Often, a DNS name may used as the authority, since the ability to resolve DNS names is already built into all gRPC client libraries.
+Finally, the `endpoint_name` indicates a concrete name to be looked up
+in a given name-system identified by the scheme and the authority. The
+syntax of the endpoint name is dictated by the scheme in use.
 
-Finally, the  endpoint_name indicates a concrete name to be looked up in a given name-system identified by the scheme and the authority. The syntax of endpoint name is dictated by the scheme in use.
+### Resolver Plugins
 
-### Plugins
+The gRPC client library will use the specified scheme to pick the right
+resolver plugin and pass it the fully qualified name string.
 
-The gRPC client library will switch on the scheme to pick the right resolver plugin and pass it the fully qualified name string.
+Resolvers should be able to contact the authority and get a resolution
+that they return back to the gRPC client library. The returned contents
+include:
 
-Resolvers should be able to contact the authority and get a resolution that they return back to the gRPC client library. The returned contents include a list of IP:port, an optional config and optional auth config data to be used for channel authentication. The plugin API allows the resolvers to continuously watch an endpoint_name and return updated resolutions as needed. 
+- A list of resolved addresses, each of which has three attributes:
+  - The address itself, including both IP address and port.
+  - A boolean indicating whether the address is a backend address (i.e.,
+    the address to use to contact the server directly) or a balancer
+    address (for cases where [external load balancing](load-balancing.md)
+    is in use).
+  - The name of the balancer, if the address is a balancer address.
+    This will be used to perform peer authorization.
+- A [service config](service_config.md).
 
+The plugin API allows the resolvers to continuously watch an endpoint
+and return updated resolutions as needed.

doc/service_config.md

@@ -0,0 +1,147 @@
+Service Config in gRPC
+======================
+
+# Objective
+
+The service config is a mechanism that allows service owners to publish
+parameters to be automatically used by all clients of their service.
+
+# Format
+
+The service config is a JSON string of the following form:
+
+```
+{
+  # Load balancing policy name.
+  # Supported values are 'round_robin' and 'grpclb'.
+  # Optional; if unset, the default behavior is pick the first available
+  # backend.
+  # Note that if the resolver returns only balancer addresses and no
+  # backend addresses, gRPC will always use the 'grpclb' policy,
+  # regardless of what this field is set to.
+  'loadBalancingPolicy': string,
+
+  # Per-method configuration.  Optional.
+  'methodConfig': [
+    {
+      # The names of the methods to which this method config applies. There
+      # must be at least one name. Each name entry must be unique across the
+      # entire service config. If the 'method' field is empty, then this
+      # method config specifies the defaults for all methods for the specified
+      # service.
+      #
+      # For example, let's say that the service config contains the following
+      # method config entries:
+      #
+      # 'methodConfig': [
+      #   { 'name': [ { 'service': 'MyService' } ] ... },
+      #   { 'name': [ { 'service': 'MyService', 'method': 'Foo' } ] ... }
+      # ]
+      #
+      # For a request for MyService/Foo, we will use the second entry, because
+      # it exactly matches the service and method name.
+      # For a request for MyService/Bar, we will use the first entry, because
+      # it provides the default for all methods of MyService.
+      'name': [
+        {
+          # RPC service name.  Required.
+          # If using gRPC with protobuf as the IDL, then this will be of
+          # the form "pkg.service_name", where "pkg" is the package name
+          # defined in the proto file.
+          'service': string,
+
+          # RPC method name.  Optional (see above).
+          'method': string,
+        }
+      ],
+
+      # Whether RPCs sent to this method should wait until the connection is
+      # ready by default. If false, the RPC will abort immediately if there
+      # is a transient failure connecting to the server. Otherwise, gRPC will
+      # attempt to connect until the deadline is exceeded.
+      #
+      # The value specified via the gRPC client API will override the value
+      # set here. However, note that setting the value in the client API will
+      # also affect transient errors encountered during name resolution,
+      # which cannot be caught by the value here, since the service config
+      # is obtained by the gRPC client via name resolution.
+      'waitForReady': bool,
+
+      # The default timeout in seconds for RPCs sent to this method. This can
+      # be overridden in code. If no reply is received in the specified amount
+      # of time, the request is aborted and a deadline-exceeded error status
+      # is returned to the caller.
+      #
+      # The actual deadline used will be the minimum of the value specified
+      # here and the value set by the application via the gRPC client API.
+      # If either one is not set, then the other will be used.
+      # If neither is set, then the request has no deadline.
+      #
+      # The format of the value is that of the 'Duration' type defined here:
+      # https://developers.google.com/protocol-buffers/docs/proto3#json
+      'timeout': string,
+
+      # The maximum allowed payload size for an individual request or object
+      # in a stream (client->server) in bytes. The size which is measured is
+      # the serialized, uncompressed payload in bytes. This applies both
+      # to streaming and non-streaming requests.
+      #
+      # The actual value used is the minimum of the value specified here and
+      # the value set by the application via the gRPC client API.
+      # If either one is not set, then the other will be used.
+      # If neither is set, then the built-in default is used.
+      #
+      # If a client attempts to send an object larger than this value, it
+      # will not be sent and the client will see an error.
+      # Note that 0 is a valid value, meaning that the request message must
+      # be empty.
+      #
+      # The format of the value is that of the 'uint64' type defined here:
+      # https://developers.google.com/protocol-buffers/docs/proto3#json
+      'maxRequestMessageBytes': string,
+
+      # The maximum allowed payload size for an individual response or object
+      # in a stream (server->client) in bytes. The size which is measured is
+      # the serialized, uncompressed payload in bytes. This applies both
+      # to streaming and non-streaming requests.
+      #
+      # The actual value used is the minimum of the value specified here and
+      # the value set by the application via the gRPC client API.
+      # If either one is not set, then the other will be used.
+      # If neither is set, then the built-in default is used.
+      #
+      # If a server attempts to send an object larger than this value, it
+      # will not be sent, and the client will see an error.
+      # Note that 0 is a valid value, meaning that the response message must
+      # be empty.
+      #
+      # The format of the value is that of the 'uint64' type defined here:
+      # https://developers.google.com/protocol-buffers/docs/proto3#json
+      'maxResponseMessageBytes': string
+    }
+  ]
+}
+```
+
+Note that new per-method parameters may be added in the future as new
+functionality is introduced.
+
+# Architecture
+
+A service config is associated with a server name.  The [name
+resolver](naming.md) plugin, when asked to resolve a particular server
+name, will return both the resolved addresses and the service config.
+
+TODO(roth): Design how the service config will be encoded in DNS.
+
+# APIs
+
+The service config is used in the following APIs:
+
+- In the resolver API, used by resolver plugins to return the service
+  config to the gRPC client.
+- In the gRPC client API, where users can query the channel to obtain
+  the service config associated with the channel (for debugging
+  purposes).
+- In the gRPC client API, where users can set the service config
+  explicitly.  This is intended for use in unit tests.

tools/doxygen/Doxyfile.c++

@@ -785,6 +785,7 @@ doc/naming.md \
 doc/negative-http2-interop-test-descriptions.md \
 doc/server-reflection.md \
 doc/server_reflection_tutorial.md \
+doc/service_config.md \
 doc/statuscodes.md \
 doc/stress_test_framework.md \
 doc/wait-for-ready.md \

tools/doxygen/Doxyfile.c++.internal

@@ -785,6 +785,7 @@ doc/naming.md \
 doc/negative-http2-interop-test-descriptions.md \
 doc/server-reflection.md \
 doc/server_reflection_tutorial.md \
+doc/service_config.md \
 doc/statuscodes.md \
 doc/stress_test_framework.md \
 doc/wait-for-ready.md \

tools/doxygen/Doxyfile.core

@@ -784,6 +784,7 @@ doc/naming.md \
 doc/negative-http2-interop-test-descriptions.md \
 doc/server-reflection.md \
 doc/server_reflection_tutorial.md \
+doc/service_config.md \
 doc/statuscodes.md \
 doc/stress_test_framework.md \
 doc/wait-for-ready.md \

tools/doxygen/Doxyfile.core.internal

@@ -784,6 +784,7 @@ doc/naming.md \
 doc/negative-http2-interop-test-descriptions.md \
 doc/server-reflection.md \
 doc/server_reflection_tutorial.md \
+doc/service_config.md \
 doc/statuscodes.md \
 doc/stress_test_framework.md \
 doc/wait-for-ready.md \