mirror of https://github.com/grpc/grpc.git
Craig Tiller
9 years ago
commit
dd4531fd33
55 changed files with 583 additions and 201 deletions
@ -0,0 +1,97 @@ |
|||||||
|
Load Balancing in gRPC |
||||||
|
======================= |
||||||
|
|
||||||
|
# Objective |
||||||
|
|
||||||
|
To design a load balancing API between a gRPC client and a Load Balancer to |
||||||
|
instruct the client how to send load to multiple backend servers. |
||||||
|
|
||||||
|
# Background |
||||||
|
|
||||||
|
Prior to any gRPC specifics, we explore some usual ways to approach load |
||||||
|
balancing. |
||||||
|
|
||||||
|
### Proxy Model |
||||||
|
|
||||||
|
Using a proxy provides a solid trustable client that can report load to the load |
||||||
|
balancing system. Proxies typically require more resources to operate since they |
||||||
|
have temporary copies of the RPC request and response. This model also increases |
||||||
|
latency to the RPCs. |
||||||
|
|
||||||
|
The proxy model was deemed inefficient when considering request heavy services |
||||||
|
like storage. |
||||||
|
|
||||||
|
### Balancing-aware Client |
||||||
|
|
||||||
|
This thicker client places more of the load balancing logic in the client. For |
||||||
|
example, the client could contain many load balancing policies (Round Robin, |
||||||
|
Random, etc) used to select servers from a list. In this model, a list of |
||||||
|
servers would be either statically configured in the client, provided by the |
||||||
|
name resolution system, an external load balancer, etc. In any case, the client |
||||||
|
is responsible for choosing the preferred server from the list. |
||||||
|
|
||||||
|
One of the drawbacks of this approach is writing and maintaining the load |
||||||
|
balancing policies in multiple languages and/or versions of the clients. These |
||||||
|
policies can be fairly complicated. Some of the algorithms also require client |
||||||
|
to server communication so the client would need to get thicker to support |
||||||
|
additional RPCs to get health or load information in addition to sending RPCs |
||||||
|
for user requests. |
||||||
|
|
||||||
|
It would also significantly complicate the client's code: the new design hides |
||||||
|
the load balancing complexity of multiple layers and presents it as a simple |
||||||
|
list of servers to the client. |
||||||
|
|
||||||
|
### External Load Balancing Service |
||||||
|
|
||||||
|
The client load balancing code is kept simple and portable, implementing |
||||||
|
well-known algorithms (ie, Round Robin) for server selection. |
||||||
|
Complex load balancing algorithms are instead provided by the load balancer. The |
||||||
|
client relies on the load balancer to provide _load balancing configuration_ and |
||||||
|
_the list of servers_ to which the client should send requests. The balancer |
||||||
|
updates the server list as needed to balance the load as well as handle server |
||||||
|
unavailability or health issues. The load balancer will make any necessary |
||||||
|
complex decisions and inform the client. The load balancer may communicate with |
||||||
|
the backend servers to collect load and health information. |
||||||
|
|
||||||
|
# Proposed Architecture |
||||||
|
|
||||||
|
The gRPC load balancing approach follows the third approach, by having an |
||||||
|
external load balancer which provides simple clients with a list of servers. |
||||||
|
|
||||||
|
## Client |
||||||
|
|
||||||
|
When establishing a gRPC stream to the balancer, the client will send an initial |
||||||
|
request to the load balancer (via a regular gRPC message). The load balancer |
||||||
|
will respond with client config (including, for example, settings for flow |
||||||
|
control, RPC deadlines, etc.) or a redirect to another load balancer. If the |
||||||
|
balancer did not redirect the client, it will then send a list of servers to the |
||||||
|
client. The client will contain simple load balancing logic for choosing the |
||||||
|
next server when it needs to send a request. |
||||||
|
|
||||||
|
## Load Balancer |
||||||
|
|
||||||
|
The Load Balancer is responsible for providing the client with a list of servers |
||||||
|
and client RPC parameters. The balancer chooses when to update the list of |
||||||
|
servers and can decide whether to provide a complete list, a subset, or a |
||||||
|
specific list of “picked” servers in a particular order. The balancer can |
||||||
|
optionally provide an expiration interval after which the server list should no |
||||||
|
longer be trusted and should be updated by the balancer. |
||||||
|
|
||||||
|
The load balancer may open reporting streams to each server contained in the |
||||||
|
server list. These streams are primarily used for load reporting. For example, |
||||||
|
Weighted Round Robin requires that the servers report utilization to the load |
||||||
|
balancer in order to compute the next list of servers. |
||||||
|
|
||||||
|
## Server |
||||||
|
|
||||||
|
The gRPC Server is responsible for answering RPC requests and providing |
||||||
|
responses to the client. The server will also report load to the load balancer |
||||||
|
if a reporting stream was opened for this purpose. |
||||||
|
|
||||||
|
### Security |
||||||
|
|
||||||
|
The load balancer may be separate from the actual server backends and a |
||||||
|
compromise of the load balancer should only lead to a compromise of the |
||||||
|
loadbalancing functionality. In other words, a compromised load balancer should |
||||||
|
not be able to cause a client to trust a (potentially malicious) backend server |
||||||
|
any more than in a comparable situation without loadbalancing. |
||||||
@ -0,0 +1,4 @@ |
|||||||
|
Google Summer of Code 2016 gRPC Ideas |
||||||
|
===================================== |
||||||
|
|
||||||
|
(Skeleton for now.) |
||||||
@ -0,0 +1,82 @@ |
|||||||
|
|
||||||
|
/*
|
||||||
|
* |
||||||
|
* Copyright 2016, Google Inc. |
||||||
|
* All rights reserved. |
||||||
|
* |
||||||
|
* Redistribution and use in source and binary forms, with or without |
||||||
|
* modification, are permitted provided that the following conditions are |
||||||
|
* met: |
||||||
|
* |
||||||
|
* * Redistributions of source code must retain the above copyright |
||||||
|
* notice, this list of conditions and the following disclaimer. |
||||||
|
* * Redistributions in binary form must reproduce the above |
||||||
|
* copyright notice, this list of conditions and the following disclaimer |
||||||
|
* in the documentation and/or other materials provided with the |
||||||
|
* distribution. |
||||||
|
* * Neither the name of Google Inc. nor the names of its |
||||||
|
* contributors may be used to endorse or promote products derived from |
||||||
|
* this software without specific prior written permission. |
||||||
|
* |
||||||
|
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||||
|
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||||
|
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||||
|
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||||
|
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||||
|
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||||
|
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||||
|
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||||
|
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||||
|
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||||
|
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||||
|
* |
||||||
|
*/ |
||||||
|
|
||||||
|
#include "test/cpp/util/test_credentials_provider.h" |
||||||
|
|
||||||
|
#include "test/core/end2end/data/ssl_test_data.h" |
||||||
|
|
||||||
|
namespace grpc { |
||||||
|
namespace testing { |
||||||
|
|
||||||
|
const char kTlsCredentialsType[] = "TLS_CREDENTIALS"; |
||||||
|
|
||||||
|
std::shared_ptr<ChannelCredentials> GetChannelCredentials( |
||||||
|
const grpc::string& type, ChannelArguments* args) { |
||||||
|
if (type == kInsecureCredentialsType) { |
||||||
|
return InsecureChannelCredentials(); |
||||||
|
} else if (type == kTlsCredentialsType) { |
||||||
|
SslCredentialsOptions ssl_opts = {test_root_cert, "", ""}; |
||||||
|
args->SetSslTargetNameOverride("foo.test.google.fr"); |
||||||
|
return SslCredentials(ssl_opts); |
||||||
|
} else { |
||||||
|
gpr_log(GPR_ERROR, "Unsupported credentials type %s.", type.c_str()); |
||||||
|
} |
||||||
|
return nullptr; |
||||||
|
} |
||||||
|
|
||||||
|
std::shared_ptr<ServerCredentials> GetServerCredentials( |
||||||
|
const grpc::string& type) { |
||||||
|
if (type == kInsecureCredentialsType) { |
||||||
|
return InsecureServerCredentials(); |
||||||
|
} else if (type == kTlsCredentialsType) { |
||||||
|
SslServerCredentialsOptions::PemKeyCertPair pkcp = {test_server1_key, |
||||||
|
test_server1_cert}; |
||||||
|
SslServerCredentialsOptions ssl_opts; |
||||||
|
ssl_opts.pem_root_certs = ""; |
||||||
|
ssl_opts.pem_key_cert_pairs.push_back(pkcp); |
||||||
|
return SslServerCredentials(ssl_opts); |
||||||
|
} else { |
||||||
|
gpr_log(GPR_ERROR, "Unsupported credentials type %s.", type.c_str()); |
||||||
|
} |
||||||
|
return nullptr; |
||||||
|
} |
||||||
|
|
||||||
|
std::vector<grpc::string> GetSecureCredentialsTypeList() { |
||||||
|
std::vector<grpc::string> types; |
||||||
|
types.push_back(kTlsCredentialsType); |
||||||
|
return types; |
||||||
|
} |
||||||
|
|
||||||
|
} // namespace testing
|
||||||
|
} // namespace grpc
|
||||||
@ -0,0 +1,63 @@ |
|||||||
|
/*
|
||||||
|
* |
||||||
|
* Copyright 2016, Google Inc. |
||||||
|
* All rights reserved. |
||||||
|
* |
||||||
|
* Redistribution and use in source and binary forms, with or without |
||||||
|
* modification, are permitted provided that the following conditions are |
||||||
|
* met: |
||||||
|
* |
||||||
|
* * Redistributions of source code must retain the above copyright |
||||||
|
* notice, this list of conditions and the following disclaimer. |
||||||
|
* * Redistributions in binary form must reproduce the above |
||||||
|
* copyright notice, this list of conditions and the following disclaimer |
||||||
|
* in the documentation and/or other materials provided with the |
||||||
|
* distribution. |
||||||
|
* * Neither the name of Google Inc. nor the names of its |
||||||
|
* contributors may be used to endorse or promote products derived from |
||||||
|
* this software without specific prior written permission. |
||||||
|
* |
||||||
|
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||||
|
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||||
|
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||||
|
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||||
|
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||||
|
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||||
|
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||||
|
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||||
|
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||||
|
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||||
|
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||||
|
* |
||||||
|
*/ |
||||||
|
|
||||||
|
#ifndef GRPC_TEST_CPP_UTIL_TEST_CREDENTIALS_PROVIDER_H |
||||||
|
#define GRPC_TEST_CPP_UTIL_TEST_CREDENTIALS_PROVIDER_H |
||||||
|
|
||||||
|
#include <memory> |
||||||
|
|
||||||
|
#include <grpc++/security/credentials.h> |
||||||
|
#include <grpc++/security/server_credentials.h> |
||||||
|
#include <grpc++/support/channel_arguments.h> |
||||||
|
|
||||||
|
namespace grpc { |
||||||
|
namespace testing { |
||||||
|
|
||||||
|
const char kInsecureCredentialsType[] = "INSECURE_CREDENTIALS"; |
||||||
|
|
||||||
|
// Provide channel credentials according to the given type. Alter the channel
|
||||||
|
// arguments if needed.
|
||||||
|
std::shared_ptr<ChannelCredentials> GetChannelCredentials( |
||||||
|
const grpc::string& type, ChannelArguments* args); |
||||||
|
|
||||||
|
// Provide server credentials according to the given type.
|
||||||
|
std::shared_ptr<ServerCredentials> GetServerCredentials( |
||||||
|
const grpc::string& type); |
||||||
|
|
||||||
|
// Provide a list of secure credentials type.
|
||||||
|
std::vector<grpc::string> GetSecureCredentialsTypeList(); |
||||||
|
|
||||||
|
} // namespace testing
|
||||||
|
} // namespace grpc
|
||||||
|
|
||||||
|
#endif // GRPC_TEST_CPP_UTIL_TEST_CREDENTIALS_PROVIDER_H
|
||||||
Loading…
Reference in new issue