Enable load balancing policy extensions (#17400)

Enables `LOAD_BALANCING_POLICY_CONFIG` enum value in `LbPolicy` and supports typed load balancers specified in `load_balancing_policy`. Continues work done by Charlie Getzen <charliegetzenlc@gmail.com> in #15827.

Custom load balancers specified by `load_balancing_policy` are created as implementations of `ThreadAwareLoadBalancer`. Thread-local load balancers can be implemented as thread-aware load balancers that contain no logic at the thread-aware level, i.e. the purpose of the thread-aware LB is solely to contain the factory used to instantiate the thread-local LBs. (In the future it might be appropriate to provide a construct that abstracts away thread-aware aspects of `ThreadAwareLoadBalancer` for LBs that don't need to be thread-aware.)

A cluster that uses `LOAD_BALANCING_POLICY_CONFIG` may not also set a subset LB configuration. If the load balancer type makes use of subsetting, it should include a subset configuration in its own configuration message. Future work on load balancing extensions should include moving the subset LB to use load balancing extensions.

Similarly, a cluster that uses `LOAD_BALANCING_POLICY_CONFIG` may not set the `CommonLbConfig`, and it is not passed into load balancer creation (mostly owing to its dubious applicability as a top level configuration message to hierarchical load balancing policy). If the load balancer type makes use of the `CommonLbConfig`, it should include a `CommonLbConfig` in the configuration message for the load balancing policy.

Considerations for migration of existing load balancers:

- pieces of the `ThreadAwareLoadBalancerBase` implementation are specific to the built-in hashing load balancers and should be moved into a base class specifically for hashing load balancers. As it stands, custom load balancing policies are required to implement a `createLoadBalancer()` method even if the architecture of the LB policy does not require a hashing load balancer. I think we would also benefit from disentangling `ThreadAwareLoadBalancerBase` from `LoadBalancerBase`, as the former never actually does any host picking.

- as we convert existing thread-local load balancers to thread-aware load balancers, new local LBs will be re-created upon membership changes. We should provide a mechanism allowing load balancers to control whether this rebuild should occur, e.g. a callback that calls `create()` for thread-aware LBs by default, which can be overridden to do nothing for thread-local LBs.

Risk Level: low
Testing: brought up a cluster with a custom load balancer specified by `load_balancing_policy`; new unit tests included
Docs Changes: n/a
Release Notes: Enable load balancing policy extensions
Platform Specific Features: n/a
Fixes #5598

Signed-off-by: Eugene Chan <eugenechan@google.com>

Mirrored from https://github.com/envoyproxy/envoy @ e31bc1edec3bb80760a1442869cc442d123e9adc
pull/624/head
data-plane-api(Azure Pipelines) 4 years ago
parent f4f80dbcd2
commit ccd5246d84
  1. 18
      envoy/config/cluster/v3/cluster.proto
  2. 18
      envoy/config/cluster/v4alpha/cluster.proto

@ -110,7 +110,7 @@ message Cluster {
// this option or not.
CLUSTER_PROVIDED = 6;
// [#not-implemented-hide:] Use the new :ref:`load_balancing_policy
// Use the new :ref:`load_balancing_policy
// <envoy_v3_api_field_config.cluster.v3.Cluster.load_balancing_policy>` field to determine the LB policy.
// [#next-major-version: In the v3 API, we should consider deprecating the lb_policy field
// and instead using the new load_balancing_policy field as the one and only mechanism for
@ -720,8 +720,7 @@ message Cluster {
// The :ref:`load balancer type <arch_overview_load_balancing_types>` to use
// when picking a host in the cluster.
// [#comment:TODO: Remove enum constraint :ref:`LOAD_BALANCING_POLICY_CONFIG<envoy_v3_api_enum_value_config.cluster.v3.Cluster.LbPolicy.LOAD_BALANCING_POLICY_CONFIG>` when implemented.]
LbPolicy lb_policy = 6 [(validate.rules).enum = {defined_only: true not_in: 7}];
LbPolicy lb_policy = 6 [(validate.rules).enum = {defined_only: true}];
// Setting this is required for specifying members of
// :ref:`STATIC<envoy_v3_api_enum_value_config.cluster.v3.Cluster.DiscoveryType.STATIC>`,
@ -1008,7 +1007,7 @@ message Cluster {
// servers of this cluster.
repeated Filter filters = 40;
// [#not-implemented-hide:] New mechanism for LB policy configuration. Used only if the
// New mechanism for LB policy configuration. Used only if the
// :ref:`lb_policy<envoy_v3_api_field_config.cluster.v3.Cluster.lb_policy>` field has the value
// :ref:`LOAD_BALANCING_POLICY_CONFIG<envoy_v3_api_enum_value_config.cluster.v3.Cluster.LbPolicy.LOAD_BALANCING_POLICY_CONFIG>`.
LoadBalancingPolicy load_balancing_policy = 41;
@ -1073,7 +1072,7 @@ message Cluster {
bool connection_pool_per_downstream_connection = 51;
}
// [#not-implemented-hide:] Extensible load balancing policy configuration.
// Extensible load balancing policy configuration.
//
// Every LB policy defined via this mechanism will be identified via a unique name using reverse
// DNS notation. If the policy needs configuration parameters, it must define a message for its
@ -1099,14 +1098,11 @@ message LoadBalancingPolicy {
option (udpa.annotations.versioning).previous_message_type =
"envoy.api.v2.LoadBalancingPolicy.Policy";
reserved 2;
reserved "config";
reserved 2, 1, 3;
// Required. The name of the LB policy.
string name = 1;
reserved "config", "name", "typed_config";
google.protobuf.Any typed_config = 3;
core.v3.TypedExtensionConfig typed_extension_config = 4;
}
// Each client will iterate over the list in order and stop at the first policy that it

@ -110,7 +110,7 @@ message Cluster {
// this option or not.
CLUSTER_PROVIDED = 6;
// [#not-implemented-hide:] Use the new :ref:`load_balancing_policy
// Use the new :ref:`load_balancing_policy
// <envoy_v3_api_field_config.cluster.v3.Cluster.load_balancing_policy>` field to determine the LB policy.
// [#next-major-version: In the v3 API, we should consider deprecating the lb_policy field
// and instead using the new load_balancing_policy field as the one and only mechanism for
@ -730,8 +730,7 @@ message Cluster {
// The :ref:`load balancer type <arch_overview_load_balancing_types>` to use
// when picking a host in the cluster.
// [#comment:TODO: Remove enum constraint :ref:`LOAD_BALANCING_POLICY_CONFIG<envoy_v3_api_enum_value_config.cluster.v3.Cluster.LbPolicy.LOAD_BALANCING_POLICY_CONFIG>` when implemented.]
LbPolicy lb_policy = 6 [(validate.rules).enum = {defined_only: true not_in: 7}];
LbPolicy lb_policy = 6 [(validate.rules).enum = {defined_only: true}];
// Setting this is required for specifying members of
// :ref:`STATIC<envoy_v3_api_enum_value_config.cluster.v3.Cluster.DiscoveryType.STATIC>`,
@ -916,7 +915,7 @@ message Cluster {
// servers of this cluster.
repeated Filter filters = 40;
// [#not-implemented-hide:] New mechanism for LB policy configuration. Used only if the
// New mechanism for LB policy configuration. Used only if the
// :ref:`lb_policy<envoy_v3_api_field_config.cluster.v3.Cluster.lb_policy>` field has the value
// :ref:`LOAD_BALANCING_POLICY_CONFIG<envoy_v3_api_enum_value_config.cluster.v3.Cluster.LbPolicy.LOAD_BALANCING_POLICY_CONFIG>`.
LoadBalancingPolicy load_balancing_policy = 41;
@ -968,7 +967,7 @@ message Cluster {
bool connection_pool_per_downstream_connection = 51;
}
// [#not-implemented-hide:] Extensible load balancing policy configuration.
// Extensible load balancing policy configuration.
//
// Every LB policy defined via this mechanism will be identified via a unique name using reverse
// DNS notation. If the policy needs configuration parameters, it must define a message for its
@ -995,14 +994,11 @@ message LoadBalancingPolicy {
option (udpa.annotations.versioning).previous_message_type =
"envoy.config.cluster.v3.LoadBalancingPolicy.Policy";
reserved 2;
reserved "config";
reserved 2, 1, 3;
// Required. The name of the LB policy.
string name = 1;
reserved "config", "name", "typed_config";
google.protobuf.Any typed_config = 3;
core.v4alpha.TypedExtensionConfig typed_extension_config = 4;
}
// Each client will iterate over the list in order and stop at the first policy that it

Loading…
Cancel
Save