This is a follow up to 2c60632.
This forces all callers to think about multiple header values. There may be places that we want
to support multiple values, but none of them are security critical and this change should be
functionally equivalent to what exists today.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4d77fc802c3bc1c517e66c54e9c9507ed7ae8d9b
Risk Level: LOW
Testing: Unit and format
Fixes#10535
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ 6321e5d95f7e435625d762ea82316b7a9f7071a4
This does predictive (peekahead) prefetch for WRR, RR, and Random schedulers, and plumbing it up to the connection manager, which will currently only use it to prefetch 1 connection (see TODO for follow-up plans)
Risk Level: Medium (refactors LBs somewhat - should be no-op)
Testing: new unit tests
Docs Changes: n/a
Release Notes: n/a
Part of #2755
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 5875f23b60aeeb49e038596d8de36446c6728f5d
Currently, the maglev hash algorithm default to table size to 65537.
It is the recommended size by a paper but it is better if the user
can set this value.
This patch introduces a new MaglevLbConfig that contains table
size of maglev.
So, now, the user can set the table size of maglev by their situation.
Signed-off-by: DongRyeol Cha <dr83.cha@samsung.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5fd73ca889aa12618c626c96bb33fde4707f8bf0
This can be used in similar situations to maglev/ring, but can be used when the host selection must remain the same for existing sessions even if more hosts are added to the cluster.
In a test with 1500 endpoints, updating 50 (remove 50, add 50 different ones) took:
without new setting: ~500ms
with new setting (this PR): ~4ms
Signed-off-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 724df08c329c8270509e2a8568be90fd43d597c1
This is as described in the paper https://arxiv.org/abs/1608.01350; Logic followed is similar to what's in haproxy.
Signed-off-by: Santosh Cheler <santosh.cheler@appdynamics.com>
Mirrored from https://github.com/envoyproxy/envoy @ 760a164640acf75a8009ac89b5ff06f78a733221
Add connect_pool_per_downstream_connection flag to the cluster config (disabled by default)
Add a hashKey method to Connection in order to be able to hash on the connection ID
Signed-off-by: Craig Radcliffe <craig.radcliffe@broadcom.com>
Mirrored from https://github.com/envoyproxy/envoy @ 76a1b24dd511bb5b3a38da3e84c9003ccadc64ce
Implementing per-upstream prefetching, useful for high QPS or latency-sensitive services.
Risk Level: low (off by default)
Testing: new unit tests, integration test
Docs Changes: n/a
Release Notes: not yet
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ f6e90f2966887b25a8b6d5dd7f13ffb32865d66b
Created a new struct for optional cluster stats. Moved timeout budget stats and added request response headers and body stats in the new struct.
Risk Level: Low
Testing: Added test cases
Docs Changes: added
Release Notes: added
Fixes#10308 , Fixes#3621
Signed-off-by: Ranjith Kumar <ranjith.dakshana2015@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a24c95e4fe035ae5d05691ee2b2bd7011093a9e9
* Add support for making Least Requests LB behave like Round Robin in weighted hosts case
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Perf/logging improvements
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback and cleanup BUILD file
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Make active requests exponent configurable via CDS/runtime
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Validate log message
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Update cluster memory test golden values
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Fix method name
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Explicitly initialize active_request_bias_
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Try to make clang-tidy happy
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Use unique_ptr instead of optional
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Update stats integration test
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Check whether memory footprint is reduced without LB changes
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Use plain double for active request bias
Use a plain double instead of a runtime double to store the per-cluster
active request bias.
Note: The goal of this commit is to evaluate the memory overhead of this
approach. A commit with te Least Requests LB changes might follow if we
deem the memory overhead of this approach acceptable.
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Revert back to approved implementation using RuntimeDouble
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Add extra fields to CDS cluster proto to check memory usage
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Revert "Add extra fields to CDS cluster proto to check memory usage"
This reverts commit a6a285dcee9e0fe618286d3dfbfab98c957dd9c7.
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Add changelog entry
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9f7d44850310999ecbd0cfbe8fadb70db6f258f9
These are currently set as [#not-implemented-hide:]. There should be enough plumbing here to start
implementing the first roadmap steps in #11264, i.e. some simple delta discovery flows with core
LDS/CDS/RDS/EDS/SDS.
I've punted on VHDS and SRDS until later, as these will require special case mapping of their
resource keys into context parameters.
Risk level: Low (Unused API changes).
Testing: Built docs and proto_format.sh.
Part of #11264.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 99471fd8f4ef7406f50cf41cf1cfa22bbdfeacc7
Mainly moving code, but finishes up the series of pluggable upstream PRs.
Additional Description: This unhides the configurable extension point to the cluster for selecting a connection pool and creating an upstream, which can be used for custom business logic in upstream creation.
Risk Level: medium (router refactor)
Testing: with prior PRs
Docs Changes: inline with APIs
Release Notes: added
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 8a9d6155601846d2d256defd09a3c408c6eb4386
split out from #11327
There's a bit of transitive ugliness: declaring the extensions requires security posture, requires stub build files, requires codeowners before the code move, but it'll be pretty short lived.
Risk Level: Low (mostly only APIs)
Testing: n/a
Docs Changes: some of the new docs
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ e8dc25ecec277c0b94d02151de79353a9ba07b4e
Fixes a mismatch in metadata attribute name used by transport_socket_matches
Risk Level: Low
Testing: Manual review
Docs Changes: Not applicable
Fixes#10660
Signed-off-by: Raghavendra Balgi <rkbalgi@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ cb08d542389d455069f83f20dc543176ffb484fb
Users can specify metadata in a health check's config that will be
used to select a matching transport socket from those configured in a
cluster's transport_socket_matches. This allows users to configure a
different transport socket for health check connections than the one
that is used to create a connection to an endpoint for proxying.
Risk Level: low; small optional feature
Testing: added unit tests
Docs Changes: updated health check and cluster proto docs with an explanation and example.
Release Notes: added
Fixes#10575
Signed-off-by: Spencer Lewis <slewis@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ b6c8bb3a4ac6bcce221643a4924befd5eefd6815
This patch performs a major version freeze and bump by modifying
package_version_status, using the tooling developed in #10636.
Specifically:
v2 APIs are frozen (except for where they are the latest in their
package history and still active)
v3 APIs are transitioned to be active
Candidate v4alpha APIs are generated (not used by Envoy yet)
Fixes#10355
Risk level: medium (entire API's files are modified, visually verified
to ensure things look sane, all tests pass)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 549164c42cae84b59154ca4c36009e408aa10b52
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
Modifies the well-known-names of the built-in http filters
to use the same names as the extension build system.
Handles transition of per-filter-configs.
Risk Level: low, previous name is still accepted
Testing: existing tests + deprecated tests for old names
Docs Changes: updated names
Release Notes: updated
Deprecated: old names are logged as deprecated
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ c5c1e5bb6570f81f30dd48d8b379392c84e81fe0
This PR introduces a parallel v3 API reference documentation tree to the
existing v2 one.
The docs/root/api-v3/ tree was copied from docs/root/api-v2 and the
necessary package path fixups were made manually. As a result, the tree
largely resembles the v2 docs. Long term this is likely to evolve to
reflect the shape of the new extensions tree.
The message type, field and enum anchors are sed'ed to be distinct and
self-consistent inside api-v3/.
There were a number of API proto changes that were made to obtain a
successful Sphinx build:
* References to deprecated fields were replaced by references to the replacement field.
* clang-format line wrapping in protoxform was removed, this breaks RST in some v3 protos.
* Some packages (type/metadata/v2, data/cluster/v2alpha) were force upgraded to v3, to deal with references to types that are distinct in v2/v3. This is OK as these packages probably make sense to bump for v3, in general we're going to have to think about how to do this more
cleanly, supporting dual v2/v3 references alongside each other.
* Some evil hacks for field renaming added to migrate.py for RouteAction.
There's also some additional machinery added to compute distinct v3/v3
build targets to point protodoc at.
Risk level: Low
Testing: Docs build, manual inspection.
Fixes#8087
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ac88316892cd47b6a9b58e3736e20e8863cd0d27
In which we convert every v3alpha reference to v3. In future revs of the
stable API versioning policy, we will develop better tooling to support
> 2 alpha and stable versions. For v3, it seems reasonable to just mv
v3alpha to v3, since there should be no external consumers yet.
Risk level: Low
Testing: bazel test //test/..., CI.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5248a4fb7d4c2a3d1fa151f944d3a63f6b7a06cf
This PR makes the following fatal by default:
from cluster.proto: ORIGINAL_DST_LB, tls_context, extension_protocol_options
from health_check.proto: use_http2
from route_components.proto: allow_origin regex, pattern, method, regex_match, value
from http_connection_manager.proto: operation_name
from trace.proto: HTTP_JSON_V1
from string.proto: regex
Risk Level: Medium (who knows who is using them)
Testing: test framework updates
Docs Changes: n/a
Release Notes: n/a
Originally #8847
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 33100e8bbeca057e23d07e46e42c51376700f872
Now that we have stable Envoy API versioning, deprecate in v2, with the
understanding that we won't ever make this a runtime controlled field.
Risk level: Low
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ab965195f6e238b5b66312497b19e9503f22cfb2
This change adds a pair of histograms to track timeout budget usage: one for the
per-request timeout and one for the global timeout. The histograms are scaled
such that 100% of the timeout budget is a value of 10000 so that four digits of
the percentage used can be recorded.
Risk Level: Low (new stats only).
Testing: Unit tests added/modified.
Docs Changes: Added notes about historgrams.
Release Notes: Added comment with link to docs.
Fixes#6122
Signed-off-by: Matthew Gumport <mgumport@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ ebab713091fc21818d229e1f9f99b1ee8334eb87
Description: add auto_sni flag on cluster that executes to change sni field on sni field automatically.
Risk Level: Low
Testing: Unit Test
Docs Changes: Added
Release Notes: Added
Signed-off-by: shikugawa <rei@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 18ec59136e7a265556ca1d660dd7eb103182ea3a
This allows for a clean separation of config/service in v3. This is a
continuation of #9548.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3bddaee1912fcd1fedc4786aee830b2e4a7c599
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
In order to get file level move annotation, import has to be before options.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 062c895f499382ae61dead16db2a7e78b9146525
Instead of formatting options heuristically, which will erase new annotations without changing protoxform, use proto descriptor to format options, and enforce its order as well.
Risk Level: Low
Testing: CI
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ dfe687d49574ef7eb1bf84867bf571e805a2bf97
* api: link to previous message type package in API BUILD files.
We need to include the descriptors from the previous message version in
the build. We opt to do this transitively; when you include v3 of a
package, you get the v2 via a transitive dep. This should work based on
alwayslink semantics for cc_library.
The computation of the deps is based on the previous_message_type
annotation, which will allow cross package migrations.
Part of #8082.
Risk level: Low
Testing: Disabled ip_tagging v2 descriptor hack, observed
version_integration_test. After the BUILD changes, this now passes
again.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7f8fb9509d3189819dd253e25ec76e939ae106e7
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
* Add an explicit threat model to the end user facing docs, link to this from SECURITY.md
* Switch all Envoy extensions to use a new macro `envoy_cc_extension`, mandating that extensions declare a security posture. Extensions can also optionally declare `alpha` or `wip` status.
* Tag all documentation sites with their well-known Envoy names.
* Introduce tooling to automagically populate a list of known trusted/untrusted extensions in the threat model docs.
* Generate API docs for extensions that depend on `google.protobuf.Empty`. This pattern is deprecated as per https://github.com/envoyproxy/envoy/issues/8933, but we need these for tooling support meanwhile.
This work was motivated by oss-fuzz issue https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18370
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 90d1094b32aa017f90cc8efcd379aeb143acabfc
This provides canonical BUILD formatting and puts protoxform in charge
of being able to determine import paths, without having to worry about
Bazel implications.
Part of #8082.
Risk level: Low
Testing: tools/proto_sync.py, visual inspection of diffs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e53f40f0e5ccc84fca5cd350416fe0f2accf8229
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
data-plane-api(CircleCI)