This is a follow-up to:
#14432 (comment)
After that PR, it's no longer possible (unless you do a dynamic_cast)
to set the remote address from a filter. This is something that we
need to do because we have specialized logic for this (XFF doesn't
work for us).
So this adds an extension point which will allow us to push that logic
down to ConnectionManagerUtility::mutateRequestHeaders() where it
belongs.
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ beac1ece7512e6e39b4f1c29490e247996a0f51c
canonical must include the hashed payload for most services. The prior
behavior of using UNSIGNED-PAYLOAD is an exception to the rule, which
select services like s3 support, since hashing the payload may be
impractical if the payload is very large.
A new filter option is introduced, so that the filter may be explicitly
configured to use the UNSIGNED-PAYLOAD string literal as specified
in the S3 signing docs:
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.htmlfixes#13904
Additional Description:
The original implementation was seemingly very specific to S3 and was subsequently amended to extend the same niche singing behaviors for ES and Glacier. This changes the filter's default behavior to match the general SigV4 guidelines while providing a configuration option to enable the specialized UNSIGNED-PAYLOAD behavior.
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
Risk Level: Medium?
Deployments using the filter will now buffer requests by default, which could result in 413 responses for requests with bodies exceeding the buffer limit. These users can mitigate buffering by enabling the `unsigned_payload` option.
Testing:
I tested locally with a filter config. I anticipate updating the automated tests based on feedback from maintainers.
Docs Changes: Added
Signed-off-by: Jonathan Stewmon <jstewmon@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 80e1ca899e3170c8f87dfd300c9c5dacc76d100b
This fixes the refs in protos and removes the sed operations for build
The benefits are:
- less indirection in proto refs
- (small) speedup of build
- simplify docs build
- simplify reuse of proto comments (eg generating jsonschemas)
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ 45ec050f91407147ed53a999434b09ef77590177
This allows self-contained bootstrap without extra token files.
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ a5020ce885592e8e3a5d9bb644554a30c0f79a18
Commit Message: Support the clear_route_cache parameter on responses
from the remote server
Risk Level: Low. Only enabled if the flag is set.
Testing: New unit test to ensure that the method is called.
Docs Changes: Marked "clear_route_cache" in the API as no longer
"not-implemented".
Release Notes: If the clear_route_cache flag is set on a
response from the external processing server, then the filter will
call the "clearRouteCache" method on the filter state. Processors
should set this flag if they have changed any headers, such as
":path", which may affect routing after the filter runs.
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b7ce74e8cc8d882ef23146c5039712e84796950e
Add support for starttls on the client side. This is exactly the same
code as used for the server implementation, just reused.
Risk level: Low
Signed-off-by: Bryce Anderson <banderson@twitter.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9f2bb17be1fe970119e18dbe507dc2ddd31928e6
Commit Message: Add initial stream and connection level flow control windows to envoy.config.core.v3.QuicProtocolOptions which is be used in QUIC listener config and Http3 upstream cluster config.
Risk Level: low
Testing: re-enable more Http3 downstream protocol test.
Part of #2557#12930#14829
Signed-off-by: Dan Zhang <danzh@google.com>
Co-authored-by: Dan Zhang <danzh@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ fe62d976a26faa46efc590a48a734f11ee6545f9
Config proto for Secure Session Agent (S2A) transport socket extension (#16110).
Signed-off-by: Tavish Vaidya <tavish@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ abe2f9b4af4b3c697ee8b3fda934aa2e181bc5ef
Port stripping the host for CONNECT requests for the duration of the filter chain (reinstating before sending upstream)
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 8c714008ed9d98c15ee18d645fdd59e18db2d9c2
If listener1 redirects the connection to listener2, the balancer field in listener2 decides whether to rebalance.
Previously we rely on the rebalancing at listener1, however, the rebalance is weak because listener1 is likely to
not own any connection and the rebalance is no-op.
Risk Level: MID. Rebalance may introduce latency. User needs to clear rebalancer field of listener2 to recover the original behavior.
Fix#15146#16113
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3e9678019e4d6244d9d066f1e6c8336360281f77
Introduce the ability to add request headers to forwarded upstream requests when the local HTTP rate limit filter determines (through its configured ruleset) that a request should be rate limited, but is not hard enforcing the incoming traffic.
Additional Description:
This is useful when a backend service should take customized action when a particular traffic volume is reached, rather than forcing request shedding by the proxy.
Risk Level: Low, adding a new feature
Testing: Modified unit tests
Docs Changes: Updated associated doc
Signed-off-by: William Fu <wfu@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ ce1cb0ed18b6a6b8f6ddecb820424cec63b1cc31
Existing trailers will be sent to the processing
server if the processing mode is set to enable them. If the
processing mode is set to sent trailers, but there are no trailers
present, then empty trailers will be sent to the server for
modification.
However, trailers may only be added in the end of the data callback
in Envoy, which may come in before a previous gRPC reply returns.
Filters that need to be able to consistently add trailers where none
existed should enable trailer processing in the Envoy filter
configuration instead of relying on being able to turn it on
dynamically.
Risk Level: Low. Trailers only enabled if a service called by the filter
is configured to ask for them.
Testing: New integration and unit tests added.
Docs Changes: API docs updated in .proto files.
Release Notes:
When the processing mode is changed to SEND
for request or response trailers, a corresponding message will be
sent to the server, which can respond with trailer mutations as desired.
In addition, if trailer processing is enabled in the filter
configuration, then trailer messages will be sent to the server
even if trailers are not present. This makes it possible for the server
to add trailers where none exist.
Finally, at the moment Envoy only implements trailers for the HTTP/2
protocol. Nothing will happen if trailer processing is enabled and
Envoy is using HTTP/1 until Envoy implements trailers for HTTP/1.
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 29eb17096e50fc9385edbf260c03da7adc4ca24c
Commit Message:
Raise max configurable max_request_headers_kb limit to 8192 KiB (8MiB) from 96 KiB in http connection manager protobuf.
Additional Description:
Added/Updated relevant unit tests and integration tests. This change will allow increasing max_request_headers_kb to 8MiB from http connection manager's configuration. The change is mainly updating the limit in a validation check in the protobuf. Also, the old (merged) PR #5859 is read, no nghttp2 library-related issues are observed on raising the max_request_headers_kb beyond 96 KiB.
Risk Level: Low
Testing: Unit, Integration, Manual
Docs Changes: Inline in proto file.
Signed-off-by: Anirudha Singh <ps.anirudha@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ d20dbc4be73d2e185ecc3a8159ca95150e204aaa
Adds a new configuration flag that makes the metrics service use
Labels to express tags instead of sending the full stats name
that might include embedded tag keys/values.
When configured, tags will be sent as labels while the reported
name will be the tag extracted name instead of the full name.
Risk Level: Low, new configuration flag
Testing: UTs
Docs Changes: Inline proto docs
Release Notes: Added
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 286ff81935a8a6f68a2ef8fa9d4dac7acb7becee
Adds a new filter that makes use of the new matching API to allow specifying a filter configuration to use for specific match
results. Once a configuration is selected, the desired filter is created and all callbacks delegated to the specific filter.
Risk Level: Low, new filter
Testing: IT + UT
Docs Changes: Inline proto comments
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ d31ba23d494c310973ceaf1e053393697821e4d3
Commit Message: allow HCM to config Http3Options and use it with other HCM configs, i.e. max_request_headers_kb and headers_with_underscores_action, to setup QuicHttpServerConnectionImpl. And support these configurations in QUIC. Currently the only Http3Options configuration is override_stream_error_on_invalid_http_message.
Additional Description: added Http3 codec stats. Pass it along with Http3Options and other HCM configs.
Risk Level: low
Testing: enable related tests in quic_protocol_integration_test
Docs Changes: updated docs/root/configuration/http/http_conn_man/response_code_details.rst
Part of #12930#2557
Signed-off-by: Dan Zhang <danzh@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c93d486c2d303dbf2cc9f88717de0bfd66e1afdb
Adds some basic structure for validating match tree constructions, only
with support for validating the data input. Updates the HTTP filter
factory to be able to specify data input requirements and wires it up.
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ afb751344c93f6eb02ea1bd1630a282c55c7aef2
Fixes a few issues with the generated Go protobuf code:
a) the OT protos are not copied over, which means that their importpath
needs to reference a path where it can pull the protos down from. The
"logs" importpath was nonsensical and couldn't be resolved
b) Bunching several proto packages into one target did not work well
with the go package system. This splits the common protos out into its
own target so that it can reference a different importpath than the
other logs protos
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8037370ce45d8c10654114b2b46e27377525c156
Adds support for a "generic input" extension point that allows specifying inputs that are not dependent on protocol data.
Adds an environment variable generic input that allows matching on the value of an environment variable.
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 758a9a90c15331fbfd093f23170454c5a2fbf903
Adding a deprecated API version annotation to deprecated fields and enum values in proto files.
This is part of the work on adding minor/patch versioning work.
Risk Level: Low (adding annotation to existing protos).
Testing: Added and modified tests for the tooling (in tools/testdata).
Signed-off-by: Adi Suissa-Peleg <adip@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5a8bfa20dc3c85ecb61826d122696ecaa75dffa0
Part of https://github.com/envoyproxy/envoy/issues/12829. The transport
extension will stay a built-in extension since it fits well. UDP
listener and UDP writer extension points have been removed. GSO is
still only enabled for QUIC because it currently depends on QUICHE, has
some obvious perf issues, and is failing non-QUIC integration tests.
Futher work is needed to remove codec extension factories.
Part of https://github.com/envoyproxy/envoy/issues/12829
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8e8a21d20ef0e90ef31ea16f5b5c85ac08d922ae
This adds the foundations of support for glob collections in CDS/SRDS
(in addition to LDS from #14311) and singleton collections in RDS/EDS.
xdstp:// URLs are not normalized across the internal resource graph yet,
so this is only safe to use for sorted context parameters (followup PRs
planned on this).
Part of #11264.
Risk level: Low (opt-in)
Testing: ADS/SRDS integration tests added, various unit tests in
//test/common/config/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ fbb7f9ae4171b89a5b622587fec0564820e400ce
Fixes#15579
Risk Level: n/a (no functional change)
Testing: Unit test to verify case added to docs
Docs Changes: Added definition of default value
Signed-off-by: Eric Anderson <ejona@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 54fc5dc459c06d15494fd759ac3e4c4785f6a7b5
some of the category annotations are not in the correct place - this is one of them
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ 5b875969d4571fc5bda77d6340d100142ed98797
Alas I didn't end up moving all of the protocol options over, as the cluster has its own timeout params so most of the QuicProtocolOptions are listener-specific.
Commit Message: adding quic protocol config to quic upstream
Risk Level: n/a (quic only)
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a
part of #14829
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 255f78b3cc66d9ed3aa86cb491d98759314294b9
This PR adds an OTLP logger to Envoy, which allows users to flexibly configure their log structure (according to OpenTelemetry protos).
The API users the common gRPC configuration, used in ALS, for gRPC settings. The flexible OT fields (body, attributes) are completely configurable and are filled with request/response information based on Envoy access log formatting.
There are still a couple of TODOs -
The 'filter_state_objects_to_log' from the common gRPC config isn't added to the log
Resource attributes are not configurable, and only Envoy node info is added.
Note: this extension was added as v3alpha API.
Risk Level: Low - a new extension, so unlikely to break any existing setups (marked as v3alpha API).
Testing: Unit tests.
Docs Changes:
Release Notes: access log: added a new :ref:OpenTelemetry access logger <envoy_v3_api_msg_extensions.access_loggers.open_telemetry.v3alpha.OpenTelemetryAccessLogConfig> extension, allowing a flexible log structure with native Envoy access log formatting.
Signed-off-by: Itamar Kaminski <itamark@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4f52f9ef554e1522e763c0eaf6ef6833aab283e5
data-plane-api(Azure Pipelines)