Commit Message: make quic proof source and crypto streams extensions. Add config for default ones. If not specified in config, the default ones will be used.
Risk Level: low
Testing: existing tests passed
Part of #2557
Co-authored-by: Dan Zhang <danzh@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ beb5a93b08bd0c48a2a7dd2f40ca13bcdb0ed40e
The HTTP Gzip filter has been disabled in v1.17 by default. Now we can remove it completely. Also the code common for the gzip filter and for the generic compressor filter is moved to source/extensions/filters/http/compressor.
Risk Level: Low
Testing: unit tests
Docs Changes: removed the page about the Gzip filter
Release Notes: added a note about the removal to current.rst
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ dc2cddeef3ef06febe4c8f9f40b3dd98e1b273bc
alternate_protocols_cache: Create a filter for parsing Alt-Svc headers and populating the AlternateProtocolsCache.
Risk Level: Low
Testing: Unit and Integration
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Ryan Hamilton <rch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e35a790bc3619fe304a3904ef81112db0a7e7b08
WebAssembly Micro Runtime (WAMR) is a standalone WebAssembly (WASM) runtime with a small footprint. It includes a few parts as below:
The "iwasm" VM core, supporting WebAssembly interpreter, ahead of time compilation (AoT) and Just-in-Time compilation (JIT)
The application framework and the supporting API's for the WASM applications
The dynamic management of the WASM applications
Risk Level: Medium
Testing: Runtime unit testing, integration testing and manual testing
Signed-off-by: Le Yao <le.yao@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ a7fa3311ad1f847d80e139970d5325a4efcbdbc1
It is to fix: https://github.com/envoyproxy/envoy/issues/14556#issuecomment-754203164
Currently, remote Jwks is fetched on-demand, in the worker thread after the requests come. The first few requests need to pause to wait for the Jwks.
Add a new feature to fetch remote Jwks in the main thread, before the listener is activated.
Detail changes:
* Change the filter config to add async_fetch field inside RemoteJwks message
* Add a new class: JwksAsyncFetcher class to handle this new config.
* Add two new statistics counters `jwks_fetch_success` and `jwks_fetch_fail`.
Risk Level: Low since new feature is guarded by the new config.
Testing: unit-tested and integration tested
Docs Changes: None
Release Notes: Yes
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ db16625415e0e5e3486854e0c993b178a80bab58
Currently, envoy ignores the value of present match. This commit adjust
envoy's behavior. When the value is True, envoy will check the header
present. When the value is False, envoy will check the header absent.
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9a7a7ede474780c2bd8fe59e7a93e537204cfe31
the filter smoothens the flow of data in both direction up to the specified bandwidth limit.
Risk Level: Low, new filter
Testing: UTs added. Adding more UTs and integration tests.
Docs Changes: Added
Release Notes: Added
Fixes#13604
Signed-off-by: Nitin Goyal <nigoyal@microsoft.com>
Mirrored from https://github.com/envoyproxy/envoy @ a467b0ef41864459f4b4ad76fad7e1efb06b0bf8
Add an option in connection manager which allow to strip trailing host dot from Host/authority header
(e.g transforms "example.com." to "example.com").
Signed-off-by: maheshkurund <mahesh.kurund@oneconvergence.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7cc10b03bcbff0367daaa8bd9b228eee9871d0e3
This is a PR for scoping token buckets in the local rate limiting flow on a per connection basis as opposed to scoping it on the entire envoy instance. More details in #15637
Currently, the HTTP local rate limiter's token bucket is shared across all workers, thus causing the rate limits to be applied per Envoy instance/process. This could potentially result in bad actors quickly exhausting limits on a given envoy instance before legitimate users have had a fair chance. We achieve this by adding an instance of the LocalRateLimit::LocalRateLimiterImpl to each connection object, if there isn't one already, via FilterState data
Risk Level: Low
Testing:
Added unit tests to local_ratelimit
Manually tested via curl'ing against a locally patched envoy instance.
One can send multiple requests on the same connection via curl using the following:
curl -vI example.com example.com
Docs Changes:
Added new toggle to local rate limit configuration to enable per connection local rate limiting
// Specifies the scope of the rate limiter's token bucket.
// If set to false, the token bucket is shared across all worker threads
// thus the rate limits are applied per Envoy process.
// If set to true, a token bucket is allocated for each connection.
// Thus the rate limits are applied per connection thereby allowing
// one to rate limit requests on a per connection basis.
// If unspecified, the default value is false.
bool local_rate_limit_per_downstream_connection
Sample configuration
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
stat_prefix: http_local_rate_limiter
token_bucket:
max_tokens: 10000
tokens_per_fill: 1000
fill_interval: 1s
filter_enabled:
runtime_key: local_rate_limit_enabled
default_value:
numerator: 100
denominator: HUNDRED
filter_enforced:
runtime_key: local_rate_limit_enforced
default_value:
numerator: 100
denominator: HUNDRED
response_headers_to_add:
- append: false
header:
key: x-local-rate-limit
value: 'true'
local_rate_limit_per_downstream_connection: true
Fixes#15637
Signed-off-by: Gokul Nair <gnair@twitter.com>
Mirrored from https://github.com/envoyproxy/envoy @ 151aa0ce902071c9ad07e78f5f62dc22119bab39
This extension simply allows to add user-defined dynamic metadata. This is intended to be used with matchers, so that the "result" of a matching process can imply specific metadata.
See issue #16266
Risk Level: Medium
Testing: Unit tests for the extension are added
Docs Changes: A small documentation for the extension is added in docs/root/configuration/http/http_filters
Signed-off-by: Adrien Guinet <adrien@reblaze.com>
Mirrored from https://github.com/envoyproxy/envoy @ cbe08bbe33005c1832964f7c9e657d6dbe105bbb
Commit Message: To prevent long event loop when too many UDP packets are in the queue, limit how many packets to read in each event loop. If haven't finished reading, artifacts a READ event to continue in the next event loop.
Additional Description:
Add numPacketsExpectedPerEventLoop() callback to UdpListenerCallback, so that QUIC listener can tell how many packets it wants to read in each loop. The actually number of packets read are still bound by MAX_NUM_PACKETS_PER_EVENT_LOOP (6000).
Quic listener returns numPacketsExpectedPerEventLoop() based on number of connections it has at the moment and the configured envoy::config::listener::QuicProtocolOptions.packets_to_read_to_connection_count_ratio.
Made InjectableSingleton really thread safe.
Risk Level: medium, other than quic listener, other UdpListenerCallbacks return max size_t for numPacketsExpectedPerEventLoop(). This will cause those callbacks to read 6000 packets per READ event.
Testing: added udp listener unit tests.
Fixes#16335#16278
Part of #16198#16493
Signed-off-by: Dan Zhang <danzh@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8db01a4a7ab41268d28c323a0f3f80efae8af977
These changes clarify that as of v1.16 the default cipher suite is different for client and servers.
Risk Level: Low
Testing: N/A
Docs Changes: Yes
Release Notes: N/A
Platform Specific Features: N/A
Fixes#16469
Signed-off-by: Bobby DeSimone <bobbydesimone@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ c94e646e0280e4c521f8e613f1ae2a02b274dbbf
Marks the matching API more clearly as experimental and updates the composite filter security posture to reflect this.
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ b6039234e526eeccdf332a7eb041729aaa1bc286
This makes it possible for a processor to add a body to a request or response that does
not have one, or replace the entire body in the response from a header callback
without otherwise touching it.
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c307494627e6f52154c3437fc7aea4c47dce07f0
Enable `is_optional` field for `HttpFilter`. The default value is `false`, it will keep the same behavior with the current implementation. The envoy will reject the unsupported HTTP filter, also will reject the unsupported HTTP filter in typed per filter config. When the value is `true`, the unsupported HTTP filter will be ignored by the envoy, also will be ignored by typed per filter config, with a warning log.
Risk Level: low
Testing: unit tests and integration tests are added
Docs Changes: API doc is added
Release Notes: added as new feature
Fixes#15770Fixes#15025
Signed-off-by: He Jie Xu <hejie.xu@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ ea4cadcd0e33da0c0e8272131866874b0ea377fb
grid: Plumb the AlternateProtocolCache down to the grid from the UpstreamClusterManager.
Create a new AlternateProtocolsCacheOptions proto message for configuring an AlternateProtocolCache, and add this message to AutoHttpConfig.
Create a new AlternateProtocolCacheManager class for fetching/creating a Cache based on the config.
Modify the AlternateProtocolCache to store state per thread. (Does not yet synchronize this state across threads)
Risk Level: Low
Testing: Unit tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Ryan Hamilton <rch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 964de6c2a166bb7f66a9293a2e2e8cb6040dfbf2
Allow http route and cluster metadata to contain typed metadata in Any in addition to Struct format
#13986.
Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 178c0886ff5e4f61067843080260de590c522e64
This is a follow-up to:
#14432 (comment)
After that PR, it's no longer possible (unless you do a dynamic_cast)
to set the remote address from a filter. This is something that we
need to do because we have specialized logic for this (XFF doesn't
work for us).
So this adds an extension point which will allow us to push that logic
down to ConnectionManagerUtility::mutateRequestHeaders() where it
belongs.
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ beac1ece7512e6e39b4f1c29490e247996a0f51c
canonical must include the hashed payload for most services. The prior
behavior of using UNSIGNED-PAYLOAD is an exception to the rule, which
select services like s3 support, since hashing the payload may be
impractical if the payload is very large.
A new filter option is introduced, so that the filter may be explicitly
configured to use the UNSIGNED-PAYLOAD string literal as specified
in the S3 signing docs:
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.htmlfixes#13904
Additional Description:
The original implementation was seemingly very specific to S3 and was subsequently amended to extend the same niche singing behaviors for ES and Glacier. This changes the filter's default behavior to match the general SigV4 guidelines while providing a configuration option to enable the specialized UNSIGNED-PAYLOAD behavior.
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
Risk Level: Medium?
Deployments using the filter will now buffer requests by default, which could result in 413 responses for requests with bodies exceeding the buffer limit. These users can mitigate buffering by enabling the `unsigned_payload` option.
Testing:
I tested locally with a filter config. I anticipate updating the automated tests based on feedback from maintainers.
Docs Changes: Added
Signed-off-by: Jonathan Stewmon <jstewmon@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 80e1ca899e3170c8f87dfd300c9c5dacc76d100b
This fixes the refs in protos and removes the sed operations for build
The benefits are:
- less indirection in proto refs
- (small) speedup of build
- simplify docs build
- simplify reuse of proto comments (eg generating jsonschemas)
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ 45ec050f91407147ed53a999434b09ef77590177
This allows self-contained bootstrap without extra token files.
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ a5020ce885592e8e3a5d9bb644554a30c0f79a18
Commit Message: Support the clear_route_cache parameter on responses
from the remote server
Risk Level: Low. Only enabled if the flag is set.
Testing: New unit test to ensure that the method is called.
Docs Changes: Marked "clear_route_cache" in the API as no longer
"not-implemented".
Release Notes: If the clear_route_cache flag is set on a
response from the external processing server, then the filter will
call the "clearRouteCache" method on the filter state. Processors
should set this flag if they have changed any headers, such as
":path", which may affect routing after the filter runs.
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b7ce74e8cc8d882ef23146c5039712e84796950e
Add support for starttls on the client side. This is exactly the same
code as used for the server implementation, just reused.
Risk level: Low
Signed-off-by: Bryce Anderson <banderson@twitter.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9f2bb17be1fe970119e18dbe507dc2ddd31928e6
Commit Message: Add initial stream and connection level flow control windows to envoy.config.core.v3.QuicProtocolOptions which is be used in QUIC listener config and Http3 upstream cluster config.
Risk Level: low
Testing: re-enable more Http3 downstream protocol test.
Part of #2557#12930#14829
Signed-off-by: Dan Zhang <danzh@google.com>
Co-authored-by: Dan Zhang <danzh@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ fe62d976a26faa46efc590a48a734f11ee6545f9
Config proto for Secure Session Agent (S2A) transport socket extension (#16110).
Signed-off-by: Tavish Vaidya <tavish@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ abe2f9b4af4b3c697ee8b3fda934aa2e181bc5ef
Port stripping the host for CONNECT requests for the duration of the filter chain (reinstating before sending upstream)
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 8c714008ed9d98c15ee18d645fdd59e18db2d9c2
If listener1 redirects the connection to listener2, the balancer field in listener2 decides whether to rebalance.
Previously we rely on the rebalancing at listener1, however, the rebalance is weak because listener1 is likely to
not own any connection and the rebalance is no-op.
Risk Level: MID. Rebalance may introduce latency. User needs to clear rebalancer field of listener2 to recover the original behavior.
Fix#15146#16113
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3e9678019e4d6244d9d066f1e6c8336360281f77
Introduce the ability to add request headers to forwarded upstream requests when the local HTTP rate limit filter determines (through its configured ruleset) that a request should be rate limited, but is not hard enforcing the incoming traffic.
Additional Description:
This is useful when a backend service should take customized action when a particular traffic volume is reached, rather than forcing request shedding by the proxy.
Risk Level: Low, adding a new feature
Testing: Modified unit tests
Docs Changes: Updated associated doc
Signed-off-by: William Fu <wfu@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ ce1cb0ed18b6a6b8f6ddecb820424cec63b1cc31
Existing trailers will be sent to the processing
server if the processing mode is set to enable them. If the
processing mode is set to sent trailers, but there are no trailers
present, then empty trailers will be sent to the server for
modification.
However, trailers may only be added in the end of the data callback
in Envoy, which may come in before a previous gRPC reply returns.
Filters that need to be able to consistently add trailers where none
existed should enable trailer processing in the Envoy filter
configuration instead of relying on being able to turn it on
dynamically.
Risk Level: Low. Trailers only enabled if a service called by the filter
is configured to ask for them.
Testing: New integration and unit tests added.
Docs Changes: API docs updated in .proto files.
Release Notes:
When the processing mode is changed to SEND
for request or response trailers, a corresponding message will be
sent to the server, which can respond with trailer mutations as desired.
In addition, if trailer processing is enabled in the filter
configuration, then trailer messages will be sent to the server
even if trailers are not present. This makes it possible for the server
to add trailers where none exist.
Finally, at the moment Envoy only implements trailers for the HTTP/2
protocol. Nothing will happen if trailer processing is enabled and
Envoy is using HTTP/1 until Envoy implements trailers for HTTP/1.
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 29eb17096e50fc9385edbf260c03da7adc4ca24c
Commit Message:
Raise max configurable max_request_headers_kb limit to 8192 KiB (8MiB) from 96 KiB in http connection manager protobuf.
Additional Description:
Added/Updated relevant unit tests and integration tests. This change will allow increasing max_request_headers_kb to 8MiB from http connection manager's configuration. The change is mainly updating the limit in a validation check in the protobuf. Also, the old (merged) PR #5859 is read, no nghttp2 library-related issues are observed on raising the max_request_headers_kb beyond 96 KiB.
Risk Level: Low
Testing: Unit, Integration, Manual
Docs Changes: Inline in proto file.
Signed-off-by: Anirudha Singh <ps.anirudha@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ d20dbc4be73d2e185ecc3a8159ca95150e204aaa
Adds a new configuration flag that makes the metrics service use
Labels to express tags instead of sending the full stats name
that might include embedded tag keys/values.
When configured, tags will be sent as labels while the reported
name will be the tag extracted name instead of the full name.
Risk Level: Low, new configuration flag
Testing: UTs
Docs Changes: Inline proto docs
Release Notes: Added
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 286ff81935a8a6f68a2ef8fa9d4dac7acb7becee
data-plane-api(Azure Pipelines)