mirror of https://github.com/grpc/grpc.git
Craig Tiller
9 years ago
commit
dd4531fd33
55 changed files with 583 additions and 201 deletions
@ -0,0 +1,97 @@ |
||||
Load Balancing in gRPC |
||||
======================= |
||||
|
||||
# Objective |
||||
|
||||
To design a load balancing API between a gRPC client and a Load Balancer to |
||||
instruct the client how to send load to multiple backend servers. |
||||
|
||||
# Background |
||||
|
||||
Prior to any gRPC specifics, we explore some usual ways to approach load |
||||
balancing. |
||||
|
||||
### Proxy Model |
||||
|
||||
Using a proxy provides a solid trustable client that can report load to the load |
||||
balancing system. Proxies typically require more resources to operate since they |
||||
have temporary copies of the RPC request and response. This model also increases |
||||
latency to the RPCs. |
||||
|
||||
The proxy model was deemed inefficient when considering request heavy services |
||||
like storage. |
||||
|
||||
### Balancing-aware Client |
||||
|
||||
This thicker client places more of the load balancing logic in the client. For |
||||
example, the client could contain many load balancing policies (Round Robin, |
||||
Random, etc) used to select servers from a list. In this model, a list of |
||||
servers would be either statically configured in the client, provided by the |
||||
name resolution system, an external load balancer, etc. In any case, the client |
||||
is responsible for choosing the preferred server from the list. |
||||
|
||||
One of the drawbacks of this approach is writing and maintaining the load |
||||
balancing policies in multiple languages and/or versions of the clients. These |
||||
policies can be fairly complicated. Some of the algorithms also require client |
||||
to server communication so the client would need to get thicker to support |
||||
additional RPCs to get health or load information in addition to sending RPCs |
||||
for user requests. |
||||
|
||||
It would also significantly complicate the client's code: the new design hides |
||||
the load balancing complexity of multiple layers and presents it as a simple |
||||
list of servers to the client. |
||||
|
||||
### External Load Balancing Service |
||||
|
||||
The client load balancing code is kept simple and portable, implementing |
||||
well-known algorithms (ie, Round Robin) for server selection. |
||||
Complex load balancing algorithms are instead provided by the load balancer. The |
||||
client relies on the load balancer to provide _load balancing configuration_ and |
||||
_the list of servers_ to which the client should send requests. The balancer |
||||
updates the server list as needed to balance the load as well as handle server |
||||
unavailability or health issues. The load balancer will make any necessary |
||||
complex decisions and inform the client. The load balancer may communicate with |
||||
the backend servers to collect load and health information. |
||||
|
||||
# Proposed Architecture |
||||
|
||||
The gRPC load balancing approach follows the third approach, by having an |
||||
external load balancer which provides simple clients with a list of servers. |
||||
|
||||
## Client |
||||
|
||||
When establishing a gRPC stream to the balancer, the client will send an initial |
||||
request to the load balancer (via a regular gRPC message). The load balancer |
||||
will respond with client config (including, for example, settings for flow |
||||
control, RPC deadlines, etc.) or a redirect to another load balancer. If the |
||||
balancer did not redirect the client, it will then send a list of servers to the |
||||
client. The client will contain simple load balancing logic for choosing the |
||||
next server when it needs to send a request. |
||||
|
||||
## Load Balancer |
||||
|
||||
The Load Balancer is responsible for providing the client with a list of servers |
||||
and client RPC parameters. The balancer chooses when to update the list of |
||||
servers and can decide whether to provide a complete list, a subset, or a |
||||
specific list of “picked” servers in a particular order. The balancer can |
||||
optionally provide an expiration interval after which the server list should no |
||||
longer be trusted and should be updated by the balancer. |
||||
|
||||
The load balancer may open reporting streams to each server contained in the |
||||
server list. These streams are primarily used for load reporting. For example, |
||||
Weighted Round Robin requires that the servers report utilization to the load |
||||
balancer in order to compute the next list of servers. |
||||
|
||||
## Server |
||||
|
||||
The gRPC Server is responsible for answering RPC requests and providing |
||||
responses to the client. The server will also report load to the load balancer |
||||
if a reporting stream was opened for this purpose. |
||||
|
||||
### Security |
||||
|
||||
The load balancer may be separate from the actual server backends and a |
||||
compromise of the load balancer should only lead to a compromise of the |
||||
loadbalancing functionality. In other words, a compromised load balancer should |
||||
not be able to cause a client to trust a (potentially malicious) backend server |
||||
any more than in a comparable situation without loadbalancing. |
||||
@ -0,0 +1,4 @@ |
||||
Google Summer of Code 2016 gRPC Ideas |
||||
===================================== |
||||
|
||||
(Skeleton for now.) |
||||
@ -0,0 +1,82 @@ |
||||
|
||||
/*
|
||||
* |
||||
* Copyright 2016, Google Inc. |
||||
* All rights reserved. |
||||
* |
||||
* Redistribution and use in source and binary forms, with or without |
||||
* modification, are permitted provided that the following conditions are |
||||
* met: |
||||
* |
||||
* * Redistributions of source code must retain the above copyright |
||||
* notice, this list of conditions and the following disclaimer. |
||||
* * Redistributions in binary form must reproduce the above |
||||
* copyright notice, this list of conditions and the following disclaimer |
||||
* in the documentation and/or other materials provided with the |
||||
* distribution. |
||||
* * Neither the name of Google Inc. nor the names of its |
||||
* contributors may be used to endorse or promote products derived from |
||||
* this software without specific prior written permission. |
||||
* |
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||
* |
||||
*/ |
||||
|
||||
#include "test/cpp/util/test_credentials_provider.h" |
||||
|
||||
#include "test/core/end2end/data/ssl_test_data.h" |
||||
|
||||
namespace grpc { |
||||
namespace testing { |
||||
|
||||
const char kTlsCredentialsType[] = "TLS_CREDENTIALS"; |
||||
|
||||
std::shared_ptr<ChannelCredentials> GetChannelCredentials( |
||||
const grpc::string& type, ChannelArguments* args) { |
||||
if (type == kInsecureCredentialsType) { |
||||
return InsecureChannelCredentials(); |
||||
} else if (type == kTlsCredentialsType) { |
||||
SslCredentialsOptions ssl_opts = {test_root_cert, "", ""}; |
||||
args->SetSslTargetNameOverride("foo.test.google.fr"); |
||||
return SslCredentials(ssl_opts); |
||||
} else { |
||||
gpr_log(GPR_ERROR, "Unsupported credentials type %s.", type.c_str()); |
||||
} |
||||
return nullptr; |
||||
} |
||||
|
||||
std::shared_ptr<ServerCredentials> GetServerCredentials( |
||||
const grpc::string& type) { |
||||
if (type == kInsecureCredentialsType) { |
||||
return InsecureServerCredentials(); |
||||
} else if (type == kTlsCredentialsType) { |
||||
SslServerCredentialsOptions::PemKeyCertPair pkcp = {test_server1_key, |
||||
test_server1_cert}; |
||||
SslServerCredentialsOptions ssl_opts; |
||||
ssl_opts.pem_root_certs = ""; |
||||
ssl_opts.pem_key_cert_pairs.push_back(pkcp); |
||||
return SslServerCredentials(ssl_opts); |
||||
} else { |
||||
gpr_log(GPR_ERROR, "Unsupported credentials type %s.", type.c_str()); |
||||
} |
||||
return nullptr; |
||||
} |
||||
|
||||
std::vector<grpc::string> GetSecureCredentialsTypeList() { |
||||
std::vector<grpc::string> types; |
||||
types.push_back(kTlsCredentialsType); |
||||
return types; |
||||
} |
||||
|
||||
} // namespace testing
|
||||
} // namespace grpc
|
||||
@ -0,0 +1,63 @@ |
||||
/*
|
||||
* |
||||
* Copyright 2016, Google Inc. |
||||
* All rights reserved. |
||||
* |
||||
* Redistribution and use in source and binary forms, with or without |
||||
* modification, are permitted provided that the following conditions are |
||||
* met: |
||||
* |
||||
* * Redistributions of source code must retain the above copyright |
||||
* notice, this list of conditions and the following disclaimer. |
||||
* * Redistributions in binary form must reproduce the above |
||||
* copyright notice, this list of conditions and the following disclaimer |
||||
* in the documentation and/or other materials provided with the |
||||
* distribution. |
||||
* * Neither the name of Google Inc. nor the names of its |
||||
* contributors may be used to endorse or promote products derived from |
||||
* this software without specific prior written permission. |
||||
* |
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||
* |
||||
*/ |
||||
|
||||
#ifndef GRPC_TEST_CPP_UTIL_TEST_CREDENTIALS_PROVIDER_H |
||||
#define GRPC_TEST_CPP_UTIL_TEST_CREDENTIALS_PROVIDER_H |
||||
|
||||
#include <memory> |
||||
|
||||
#include <grpc++/security/credentials.h> |
||||
#include <grpc++/security/server_credentials.h> |
||||
#include <grpc++/support/channel_arguments.h> |
||||
|
||||
namespace grpc { |
||||
namespace testing { |
||||
|
||||
const char kInsecureCredentialsType[] = "INSECURE_CREDENTIALS"; |
||||
|
||||
// Provide channel credentials according to the given type. Alter the channel
|
||||
// arguments if needed.
|
||||
std::shared_ptr<ChannelCredentials> GetChannelCredentials( |
||||
const grpc::string& type, ChannelArguments* args); |
||||
|
||||
// Provide server credentials according to the given type.
|
||||
std::shared_ptr<ServerCredentials> GetServerCredentials( |
||||
const grpc::string& type); |
||||
|
||||
// Provide a list of secure credentials type.
|
||||
std::vector<grpc::string> GetSecureCredentialsTypeList(); |
||||
|
||||
} // namespace testing
|
||||
} // namespace grpc
|
||||
|
||||
#endif // GRPC_TEST_CPP_UTIL_TEST_CREDENTIALS_PROVIDER_H
|
||||
Loading…
Reference in new issue