This PR implements a new retry back off strategy that uses values from response headers like Retry-After or X-RateLimit-Reset (the headers are configurable) to decide the back off interval before retrying a request.
Signed-off-by: Martin Matusiak <numerodix@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ bd2b989c578b2472faaff44902573e5b187f671f
Establish an extension point for actions to run based on Watch Dog Events.
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8c312f2c5b40b5ce7f6f68a1f9d4b0e98ef3829e
Change the Overload Manager API to extend the overload action state with non-binary values. This will allow future overload actions to take effect in response to increasing load, instead of to the existing inactive/active binary values. This PR also adds a range field to the overload trigger config, though no actions currently trigger on the 'scaling' state.
The existing behavior is preserved by replacing all places where OverloadActionState::Active was being used with OverloadActionState::saturated(). This is a minimal re-hashing of #11697 for ##11427.
Risk Level: medium
Testing: ran unit and integration tests
Docs Changes: none
Release Notes: add "scaling" trigger for OverloadManager actions
Signed-off-by: Alex Konradi <akonradi@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28df333572e71fc6850e82437a59e9e378d7b00b
This implements a host_rewrite_path option for rewriting the Host header based on path. See rational in the linked issue.
Note: the regex is executed on the path with query/fragment stripped. This is analogues to what regex_rewrite option is doing.
Risk Level: Low
Testing: added unit tests
Docs Changes: document the new option in proto file
Release Notes: added to current.rst
Fixes#12430
Signed-off-by: Petr Pchelko <ppchelko@wikimedia.org>
Mirrored from https://github.com/envoyproxy/envoy @ 374dca7905fc048be74169a7655d0462606555ad
The log action will be used to set the dynamic metadata key "envoy.log", which can be used to decide whether to log a request.
Signed-off-by: davidraskin <draskin@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 89b594e09e2f4ebaac43c6b38d778a3e0dc13a3a
This is the 1st PR for #11832 that factors out the TAP filter matcher to prepare for reuse in other filters.
Signed-off-by: Yangmin Zhu <ymzhu@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 520389e677cdcd4a85df769deb40f6cdd2f4f6f8
Introduces UdpPacketWriter Interface that can be used to perform writes in Batched/PassThrough modes by using QuicGsoBatchWriter implementation from QUICHE extension.
**Additional Description:**
UDP GSO (Generic Segmentation Offload) was introduced in Linux at version 4.18. It allows batch-writing of multiple messages into a single payload and sending these messages along as a batch in a single sendmsg syscall.
Currently, Envoy performs the sending of messages using simple sendmsg implementation in pass-through mode, i.e. no support for batch writing. With this change, UdpListener can use UdpPacketWriter interface as a DefaultWriter or a GsoBatchWriter to perform pass-through or batched writes respectively.
Detailed description of the changes can be found in the design document, [here](https://docs.google.com/document/d/16ePbgkfrzQ6v-cOVMSnKDja3dUdZvX-mxT9jw29rx4g/edit?usp=sharing).
**Risk Level:** Low, not in use
**Testing:**
- Added udp_listener_impl_batched_writes_test, to verify that multiple packets of varying sizes are batched/flushed as per gso specifications while using UdpGsoBatchWriter.
- Modified existing tests, to verify that UdpDefaultWriter performs writes in pass-through mode.
- Ran all tests. All 677 tests passed successfully.
```
**Docs Changes:** None
**Release Notes:** None
**Fixes:** #11925
Signed-off-by: Yugant <yugant@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 933e267db60961e76549ae6e65ac4872cb78db4d
Add authority field in envoy grpc message to override the default host name as cluster name.
Risk Level: Low
Testing:
Docs Changes:
Release Notes:
Fix#12116
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 29b49276db0812b566d143a79996848819248428
Adding a filter for access logs that will decide whether to log based on dynamic metadata.
Signed-off-by: davidraskin <draskin@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d7c7e9a79eed80afa56a0b05cf6adf0516750d14
Implementing per-upstream prefetching, useful for high QPS or latency-sensitive services.
Risk Level: low (off by default)
Testing: new unit tests, integration test
Docs Changes: n/a
Release Notes: not yet
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ f6e90f2966887b25a8b6d5dd7f13ffb32865d66b
This ensures that the configuration is valid.
Note that this adds stricter validation to an existing proto field, but the field was only added 5 days ago, so only someone running HEAD, using this new config, with invalid/non-sensical config should be affected.
Signed-off-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 61c2816f704d11f66a507b76abc8f01c1816fe57
Previously, a hard-coded set of buckets were used, which were not appropriate for the range or required precision of some histograms.
Fixes#7599
Signed-off-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0a0fc51e25599665d772f274c54addf3891ef5c7
Created a new struct for optional cluster stats. Moved timeout budget stats and added request response headers and body stats in the new struct.
Risk Level: Low
Testing: Added test cases
Docs Changes: added
Release Notes: added
Fixes#10308 , Fixes#3621
Signed-off-by: Ranjith Kumar <ranjith.dakshana2015@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a24c95e4fe035ae5d05691ee2b2bd7011093a9e9
Optional buffer on statsd udp
Signed-off-by: John Murray <murray@stripe.com>
Mirrored from https://github.com/envoyproxy/envoy @ 90c6a27bb28b4f8288392a53af8d2e88ef059a8b
WatchDog will now kill if max(2, registered_threads * multi_kill_threshold) threads have gone above the multikill_timeout.
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7f78581116ecdc9dcca319ebe68d4c8ac1d817ba
* fix filter fuzzer crash from OSS fuzz by checking for non-implemented proto fields
This will raise an exception that will be caught and abort the test run when an unimplemented oneof field is hit by the fuzzer
Signed-off-by: Sam Flattery <samflattery@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e6c57fab8022d9df228052f1f517a57584030654
Pulling out version to a separate package
Risk Level: Low
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 9a5de98bf26280a3b57882413dfb9084c5dbb2a6
This unifies HTTP/1.1 and HTTP/2 stream error on invalid messaging. Previously HTTP/1.1 defaulted permissive and HTTP/2 defaulted to strict. This defaults both to strict, resetting connections on invalid requests. This will have a major latency impact if downstream is sending a mix of valid and invalid requests over HTTP/1.1
Additional Description:
This change is runtime guarded per default behavioral change rules. It can also be reverted by setting the default to permissive (for prior HTTP/1 behvior) then overriding HTTP/2 to struct (for prior HTTP/2 behavior).
This works in conjunction with #11714, as the HTTP connection manager enforces the strictness, so the responses need to be sent via the HTTP connection manager to have strictness applied correctly.
Risk Level: High (HCM changes)
Testing: new unit tests, updated integration tests
Docs Changes: n/a
Release Notes: inline
Runtime guard: envoy.reloadable_features.hcm_stream_error_on_invalid_message
Fixes#9846
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 88dcb292817946510bb87f8f379a954962cece79
* Add support for making Least Requests LB behave like Round Robin in weighted hosts case
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Perf/logging improvements
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback and cleanup BUILD file
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Make active requests exponent configurable via CDS/runtime
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Validate log message
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Update cluster memory test golden values
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Fix method name
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Explicitly initialize active_request_bias_
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Try to make clang-tidy happy
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Use unique_ptr instead of optional
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Update stats integration test
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Check whether memory footprint is reduced without LB changes
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Use plain double for active request bias
Use a plain double instead of a runtime double to store the per-cluster
active request bias.
Note: The goal of this commit is to evaluate the memory overhead of this
approach. A commit with te Least Requests LB changes might follow if we
deem the memory overhead of this approach acceptable.
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Revert back to approved implementation using RuntimeDouble
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Add extra fields to CDS cluster proto to check memory usage
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Revert "Add extra fields to CDS cluster proto to check memory usage"
This reverts commit a6a285dcee9e0fe618286d3dfbfab98c957dd9c7.
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Add changelog entry
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9f7d44850310999ecbd0cfbe8fadb70db6f258f9
Modified dynamic_metadata action to now accept an optional default
value for instances where no value is queried from the dynamic metadata.
Signed-off-by: Clara Andrew-Wani <candrewwani@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3afa3b50eacfa39fa5b3518b05b03689dc56ef42
Add CheckedExpr as oneof expression in RBAC proto
Additional Description: CheckedExpr is type check CEL expression. It is future proof and offers better evaluation performance (not for C++ yet, but for other languages). There will be a separate PR to plumb checked expr to RBAC filter.
Risk Level: low
Testing: existing tests
Signed-off-by: jiangtaoli2016 <jiangtao@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3b8c61b5e5eb4dfddf3c4c0cae9e04eafe1ac61a
Added GenericBodyMatcher to tap filter to parse HTTP requests and responses. The matcher may be configured to look for text string or hex bytes.
Risk Level:
Low for tap filter.
Med - modified buffer::search method to limit search to specified number of bytes. This method is used in other parts of the code. Added unit test cases to make sure that there are no side effects.
Testing:
Added unit tests.
Docs Changes:
Yes - updated section with example configs.
Release Notes:
Yes.
Fixes#6107
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 356fe40edd67ddb8181442548241664424d3ac05
Extend hashing via header to support regex substitution.
This is useful when you need to hash on a transformation
of a header value (e.g.: extract a part of `:path`).
Fixes#11811
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ 73fc620a34135a16070083f3c94b93d074f6e59f
These are currently set as [#not-implemented-hide:]. There should be enough plumbing here to start
implementing the first roadmap steps in #11264, i.e. some simple delta discovery flows with core
LDS/CDS/RDS/EDS/SDS.
I've punted on VHDS and SRDS until later, as these will require special case mapping of their
resource keys into context parameters.
Risk level: Low (Unused API changes).
Testing: Built docs and proto_format.sh.
Part of #11264.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 99471fd8f4ef7406f50cf41cf1cfa22bbdfeacc7
This commit adds a new stream flush timeout to guard against a
remote server that does not open window once an entire stream has
been buffered for flushing. Additional stats have also been added
to better understand the codecs view of active streams as well as
amount of data buffered.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0e49a495826ea9e29134c1bd54fdeb31a034f40c
Provides a way to specify dynamic rate limit override in the rate limit descriptor from static value or from dynamic metadata. New type, RateLimitUnit was created to share across config protocol and rate limit service protocol. A PR for the reference implementation of the rate limit service will follow after the API changes are discussed and accepted.
Signed-off-by: Petr Pchelko <ppchelko@wikimedia.org>
Mirrored from https://github.com/envoyproxy/envoy @ 7ea1f24bd53522408d9bc55316dff9ed1701bc73
Modifies ratelimit filter to be able to use information
from the dynamic metadata as one of its actions.
Signed-off-by: Clara Andrew-Wani <candrewwani@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 471e2394b79d2b5b13a270626738d4807a51e34a
This will allow configuring the version of LRS independently from the version of xDS in the lrs_server field in Cluster messages.
Risk Level: Low
Testing: N/A
Docs Changes: Inline in PR
Signed-off-by: Mark D. Roth <roth@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ edaad6d02b3e9265418196f96b4fd6b3689cc2b7
This PR proposes to rename the RetryPolicy field num_retries to max_retries.
This parameter exists in two places: 1) the RetryPolicy message in the route configuration and 2) the header x-envoy-max-retries. The naming inconsistency is a UX papercut. max_retries feels like right name for what this field is for ie. the maximum number of retries that are permitted.
There is also a stripped down RetryPolicy message which is used by RemoteDataSource which has a num_retries field. I'm including a matching rename of that for consistency.
Risk Level: Low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Martin Matusiak <numerodix@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 286ca92afa3eaa586ad60fe20d4f5541e77b5d5e
Mainly moving code, but finishes up the series of pluggable upstream PRs.
Additional Description: This unhides the configurable extension point to the cluster for selecting a connection pool and creating an upstream, which can be used for custom business logic in upstream creation.
Risk Level: medium (router refactor)
Testing: with prior PRs
Docs Changes: inline with APIs
Release Notes: added
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 8a9d6155601846d2d256defd09a3c408c6eb4386
This PR fixes a TODO in the health check filter by translating the floating point arithmetic to integer arithmetic instead.
Signed-off-by: Martin Matusiak <numerodix@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ b8e9a3599541084efe44c6c91651c6308f90e671
This patch allows configuring the API version for gRPC external services. This specifies the version of the endpoint and message to be used.
Affected extensions:
- filters/http/ext_authz
- filters/network/ext_authz
- filters/http/ratelimit
- filters/network/ratelimit
- access_loggers/grpc
- stat_sinks/metrics_service
Affected upstream services:
- upstream/health_discovery_service
- upstream/load_stats_reporter
Additional Description:
tap
- TapSinkService is not yet implemented.
- TapDiscoveryService uses theconfig.core.v3.ApiConfigSource and is not yet implemented.
trace
- StreamTraces has no implemented client inside the code-base (implemented externally?)
Risk Level: Low
Testing: Added.
Docs Changes: Added.
Release Notes: Added.
Fixes#10609
Signed-off-by: Dhi Aurrahman <dio@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 96920250a05e62095a68fbaf4f77612179e5af32
Note need to install a sufficiently recent libstdc++ version when
setting up clang builds. For example, Ubuntu gcc packages install
libstdc++, but depending on the gcc version libstdc++ might not be
new enough to compile Envoy.
Additional Description:
c.f., https://envoyproxy.slack.com/archives/C7E6C71QB/p1586307917037300
Risk Level: low, docs only
Testing: n/a
Docs Changes: yes
Release Notes: n/a
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ c33e1a4f2882e7494bee424aec8a1dba8671b556
Notice that identifier is defined in api/envoy/config/core/v3/protocol.proto as:
google.protobuf.UInt32Value identifier = 1
[ (validate.rules).uint32 = {lte: 65536 gte: 1}, (validate.rules).message = {required: true} ];
The value should range from 0 to 65535 instead of from 1 to 65536.
Because 65536 is greater than uint16::max, it triggers the assert to terminate the program.
The fix is to set it to {lte:65535 gte:0}, instead of {lte: 65536 gte: 1}
Signed-off-by: jianwen <jianwendong@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e5ccf1af01ec3c6c70726f8e12495ecfdcfdc2ca
split out from #11327
There's a bit of transitive ugliness: declaring the extensions requires security posture, requires stub build files, requires codeowners before the code move, but it'll be pretty short lived.
Risk Level: Low (mostly only APIs)
Testing: n/a
Docs Changes: some of the new docs
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ e8dc25ecec277c0b94d02151de79353a9ba07b4e
data-plane-api(CircleCI)