Add log control (list and modify log level) in admin interface for Fancy Logger, a new fine-grained logger for Envoy, and provide command line option --enable-fine-grain-logging for developers.
Additional Description: A doc of overview is provided here: source/docs/fancy_logger.md.
Risk Level: Medium
Testing: Unit tests.
Docs Changes: Added a new option --enable-fine-grain-logging and doc it.
Release Notes: Added to current.rst.
Signed-off-by: Jinhui Song <jinhuisong@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b7138814dde530b8c2957e806ea40879a4fdce32
This PR implements a new retry back off strategy that uses values from response headers like Retry-After or X-RateLimit-Reset (the headers are configurable) to decide the back off interval before retrying a request.
Signed-off-by: Martin Matusiak <numerodix@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ bd2b989c578b2472faaff44902573e5b187f671f
Establish an extension point for actions to run based on Watch Dog Events.
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8c312f2c5b40b5ce7f6f68a1f9d4b0e98ef3829e
Added an node field to csds request to identify the CSDS client to the CSDS server, and removed the [#not-implemented-hide:] for the endpoint_config since it has been implemented in #11577
Risk Level: Low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Yutong Li <yutongli@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 90a97c7ef3d67e213980db6b0ebeff284a938837
Un-Reverts 048583b, with fix for high cpu consumption.
This PR implements fault injection for Redis; specifically delay and error faults (which themselves can have delays added). I chose not to implement a separate filter after discussing with Henry; we concluded that the faults we felt were useful didn't need many levels- just a delay on top of the original fault, if any. In addition, as the Redis protocol doesn't support headers that makes it a bit different again from Envoy's http fault injection. This PR fixes the issue previously seen with redis fault injection where excessive cpu was used on connection creation.
Faults record metrics on the original request- and the delay fault adds extra latency which is included in the command latency for that request. Also, faults can apply only to certain commands.
Future work: Add several other faults, including cache misses and connection failures.
Signed-off-by: FAYiEKcbD0XFqF2QK2E4viAHg8rMm2VbjYKdjTg <nflacco@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ e319b7c2a93783cd8bde45fa4c002ae02a4cb13f
Change the Overload Manager API to extend the overload action state with non-binary values. This will allow future overload actions to take effect in response to increasing load, instead of to the existing inactive/active binary values. This PR also adds a range field to the overload trigger config, though no actions currently trigger on the 'scaling' state.
The existing behavior is preserved by replacing all places where OverloadActionState::Active was being used with OverloadActionState::saturated(). This is a minimal re-hashing of #11697 for ##11427.
Risk Level: medium
Testing: ran unit and integration tests
Docs Changes: none
Release Notes: add "scaling" trigger for OverloadManager actions
Signed-off-by: Alex Konradi <akonradi@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28df333572e71fc6850e82437a59e9e378d7b00b
This implements a host_rewrite_path option for rewriting the Host header based on path. See rational in the linked issue.
Note: the regex is executed on the path with query/fragment stripped. This is analogues to what regex_rewrite option is doing.
Risk Level: Low
Testing: added unit tests
Docs Changes: document the new option in proto file
Release Notes: added to current.rst
Fixes#12430
Signed-off-by: Petr Pchelko <ppchelko@wikimedia.org>
Mirrored from https://github.com/envoyproxy/envoy @ 374dca7905fc048be74169a7655d0462606555ad
The log action will be used to set the dynamic metadata key "envoy.log", which can be used to decide whether to log a request.
Signed-off-by: davidraskin <draskin@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 89b594e09e2f4ebaac43c6b38d778a3e0dc13a3a
This is the 1st PR for #11832 that factors out the TAP filter matcher to prepare for reuse in other filters.
Signed-off-by: Yangmin Zhu <ymzhu@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 520389e677cdcd4a85df769deb40f6cdd2f4f6f8
Introduces UdpPacketWriter Interface that can be used to perform writes in Batched/PassThrough modes by using QuicGsoBatchWriter implementation from QUICHE extension.
**Additional Description:**
UDP GSO (Generic Segmentation Offload) was introduced in Linux at version 4.18. It allows batch-writing of multiple messages into a single payload and sending these messages along as a batch in a single sendmsg syscall.
Currently, Envoy performs the sending of messages using simple sendmsg implementation in pass-through mode, i.e. no support for batch writing. With this change, UdpListener can use UdpPacketWriter interface as a DefaultWriter or a GsoBatchWriter to perform pass-through or batched writes respectively.
Detailed description of the changes can be found in the design document, [here](https://docs.google.com/document/d/16ePbgkfrzQ6v-cOVMSnKDja3dUdZvX-mxT9jw29rx4g/edit?usp=sharing).
**Risk Level:** Low, not in use
**Testing:**
- Added udp_listener_impl_batched_writes_test, to verify that multiple packets of varying sizes are batched/flushed as per gso specifications while using UdpGsoBatchWriter.
- Modified existing tests, to verify that UdpDefaultWriter performs writes in pass-through mode.
- Ran all tests. All 677 tests passed successfully.
```
**Docs Changes:** None
**Release Notes:** None
**Fixes:** #11925
Signed-off-by: Yugant <yugant@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 933e267db60961e76549ae6e65ac4872cb78db4d
Followup for a new feature introduced by #12410 Apologies for not noticing that a later draft was introduced recently. I think we should start with supporting the latest available spec draft, so update version 2 to version 3. The change is technically backwards-incompatible, but the new feature was introduced one day ago, nobody could have been so fast to depend on it.
Signed-off-by: Petr Pchelko <ppchelko@wikimedia.org>
Mirrored from https://github.com/envoyproxy/envoy @ 8452f4b749674deb5f4b963d0da78797fb7b1278
Adds support for X-RateLimit-* headers described in the draft RFC. The X-RateLimit-Limit header contains the quota-policy per RFC. The descriptor name is included in the quota policy under the name key. X-RateLimit-Reset header is emitted, but it would need a followup in the ratelimit service, which I will do once this is merged.
Signed-off-by: Petr Pchelko <ppchelko@wikimedia.org>
Mirrored from https://github.com/envoyproxy/envoy @ 9f405633b63e71c623d4de81698807fca17debe9
Add authority field in envoy grpc message to override the default host name as cluster name.
Risk Level: Low
Testing:
Docs Changes:
Release Notes:
Fix#12116
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 29b49276db0812b566d143a79996848819248428
An assert failure will occur when `config.route_config.route.cluster_header` contains invalid characters for the HTTP header. Added validation for this field to avoid assert failure and a regression test case in the unit test.
```
[assert] [bazel-out/k8-fastbuild/bin/include/envoy/http/_virtual_includes/header_map_interface/envoy/http/header_map.h:54] assert failure: valid().
```
Risk Level: Low
Testing: Added a regression test case
Fixes#12361
Signed-off-by: jianwen <jianwendong@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 71e9049eb3b22bd53d1651b8a435ed3b4142ee9f
Adding a filter for access logs that will decide whether to log based on dynamic metadata.
Signed-off-by: davidraskin <draskin@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d7c7e9a79eed80afa56a0b05cf6adf0516750d14
Description: Upstream Wasm proto files from envoy-wasm.
Risk Level: Low
Testing: Unit tests in envoy-wasm, integration tests in istio/proxy.
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: John Plevyak <jplevyak@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 26eaa2e85cee69e5c32ab6bf4c5ae3d338fa462f
header-to-metadata filter supports adding a header's value to a metadata key which is later used for subset load balancing.
This PR adds support for extracting a specific cookie value before it's added as metadata.
Signed-off-by: Radha Kumari <rkumari@slack-corp.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4dfa844f56d637b6ec26f415f65035707cdf01c8
Create metadata similar to MySQL based on SQL query sent by Postgres client. The metadata may be used by other filters like RBAC.
Risk Level: Low.
Testing: Added unit tests.
Docs Changes: Yes - updated Postgres section.
Release Notes: Yes.
Fixes#11065
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 7287fad48f49cae45ce897643622fbf989c3c6c8
Implementing per-upstream prefetching, useful for high QPS or latency-sensitive services.
Risk Level: low (off by default)
Testing: new unit tests, integration test
Docs Changes: n/a
Release Notes: not yet
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ f6e90f2966887b25a8b6d5dd7f13ffb32865d66b
This is a docs only change, mostly on how to refer to LuaPerRoute info.
Signed-off-by: Dhi Aurrahman <dio@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 3dedf1693f45239c670c5ba7598db44ff2e32c2f
This ensures that the configuration is valid.
Note that this adds stricter validation to an existing proto field, but the field was only added 5 days ago, so only someone running HEAD, using this new config, with invalid/non-sensical config should be affected.
Signed-off-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 61c2816f704d11f66a507b76abc8f01c1816fe57
Commit Message: Add proxy proto transport socket
Additional Description: This is the part 1 PR described in #10682. It adds the transports socket / unit tests, a transport socket options struct for the proxy proto header, and does a refactor to make the listener filter use the common proxy proto constants (potentially want to move these now since the proxy proto config api type is not in extensions?)
Risk Level: Small
Testing: Unit
Docs Changes: None
Release Notes: None
Part Of: #1031
Signed-off-by: Weston Carlson <wez470@gmail.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 8972b478e6c9f1e7342e3dbfb57b35317c0cc009
* added generic freamework for testing filters.
This is a fuzzer for testing network-layer(L3/L4) filters.
Now Envoy has 20 network-layer filters which will deal with raw bytes from untrusted networks and thus they are security-critical to some extent. The idea of this is to write a fuzzer which can be applied to different kinds of network filters(potentially cover all the filters), and when new filters are added to Envoy, we won't need to write dedicated fuzzers one by one to give them fuzz coverage.
Signed-off-by: jianwen <jianwendong@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ce26fe19e0f8cd033e16aa3a9145281b0d03b748
Previously, a hard-coded set of buckets were used, which were not appropriate for the range or required precision of some histograms.
Fixes#7599
Signed-off-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0a0fc51e25599665d772f274c54addf3891ef5c7
We will want the same DNS configuration options from the Cluster in
the Dynamic Forward Proxy’s DNS configuration.
Signed-off-by: Justin Mazzola Paluska <justinmp@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ af8aa37c2148c7dc9a064f36e8d14d0859d35cb2
Currently, the HDS specifier proto gives endpoint information organized by their locality and grouped by cluster. This information is retained when doing health checks, however when responding with health statuses from each endpoint locality and cluster information is dropped. This puts all endpoint information into a flat list when returning, making it the responsibility of the receiver to re-structure data by cluster and locality.
This change follows a similar format to the specifier coming into HDS, organizing it in the same way. The proposed changed adds consistency to how messages are flowing in and out of HDS.
Note that his PR only contains the protos changes, and does not include implementation. This is done to get approval from api-shepherds for the changes before continuing.
Risk Level: Low
Testing: No new tests. New tests will be added upon implementation, but for now the fields remain empty.
Docs Changes:
Inline comments in hds.proto.
Release Notes:
N/A - will append to release notes upon implementation completion.
Deprecated:
The endpoints_health field in EndpointHealthResponse should be deprecated in favor of the new cluster_endpoints_health field.
Signed-off-by: Drew S. Ortega <drewortega@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4f1ee95e36ad233aac0d37a8b1d7c4536d4972c7
Created a new struct for optional cluster stats. Moved timeout budget stats and added request response headers and body stats in the new struct.
Risk Level: Low
Testing: Added test cases
Docs Changes: added
Release Notes: added
Fixes#10308 , Fixes#3621
Signed-off-by: Ranjith Kumar <ranjith.dakshana2015@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a24c95e4fe035ae5d05691ee2b2bd7011093a9e9
Optional buffer on statsd udp
Signed-off-by: John Murray <murray@stripe.com>
Mirrored from https://github.com/envoyproxy/envoy @ 90c6a27bb28b4f8288392a53af8d2e88ef059a8b
WatchDog will now kill if max(2, registered_threads * multi_kill_threshold) threads have gone above the multikill_timeout.
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7f78581116ecdc9dcca319ebe68d4c8ac1d817ba
* fix filter fuzzer crash from OSS fuzz by checking for non-implemented proto fields
This will raise an exception that will be caught and abort the test run when an unimplemented oneof field is hit by the fuzzer
Signed-off-by: Sam Flattery <samflattery@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e6c57fab8022d9df228052f1f517a57584030654
This PR implements fault injection for Redis; specifically delay and error faults (which themselves can have delays added). I chose not to implement a separate filter after discussing with Henry; we concluded that the faults we felt were useful didn't need many levels- just a delay on top of the original fault, if any. In addition, as the Redis protocol doesn't support headers that makes it a bit different again from Envoy's http fault injection.
Signed-off-by: FAYiEKcbD0XFqF2QK2E4viAHg8rMm2VbjYKdjTg <nflacco@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 048583b924e6c5c7812af56ec344ae210c168b3b
Pulling out version to a separate package
Risk Level: Low
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 9a5de98bf26280a3b57882413dfb9084c5dbb2a6
This unifies HTTP/1.1 and HTTP/2 stream error on invalid messaging. Previously HTTP/1.1 defaulted permissive and HTTP/2 defaulted to strict. This defaults both to strict, resetting connections on invalid requests. This will have a major latency impact if downstream is sending a mix of valid and invalid requests over HTTP/1.1
Additional Description:
This change is runtime guarded per default behavioral change rules. It can also be reverted by setting the default to permissive (for prior HTTP/1 behvior) then overriding HTTP/2 to struct (for prior HTTP/2 behavior).
This works in conjunction with #11714, as the HTTP connection manager enforces the strictness, so the responses need to be sent via the HTTP connection manager to have strictness applied correctly.
Risk Level: High (HCM changes)
Testing: new unit tests, updated integration tests
Docs Changes: n/a
Release Notes: inline
Runtime guard: envoy.reloadable_features.hcm_stream_error_on_invalid_message
Fixes#9846
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 88dcb292817946510bb87f8f379a954962cece79
This patch enables ext_authz backed with gRPC service to emit metadata.
The authorization can set the dynamic metadata (an opaque
google.protobuf.Struct) as part of the `CheckResponse` when it is
successful (i.e. when `http_response` is `OkHttpResponse`).
Signed-off-by: Dhi Aurrahman <dio@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ ff44d731ecc8d744b971b31aec816ad0b56bad42
* Add support for making Least Requests LB behave like Round Robin in weighted hosts case
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Perf/logging improvements
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback and cleanup BUILD file
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Make active requests exponent configurable via CDS/runtime
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Address feedback
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Validate log message
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Update cluster memory test golden values
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Fix method name
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Explicitly initialize active_request_bias_
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Try to make clang-tidy happy
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Use unique_ptr instead of optional
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Update stats integration test
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Check whether memory footprint is reduced without LB changes
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Use plain double for active request bias
Use a plain double instead of a runtime double to store the per-cluster
active request bias.
Note: The goal of this commit is to evaluate the memory overhead of this
approach. A commit with te Least Requests LB changes might follow if we
deem the memory overhead of this approach acceptable.
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Revert back to approved implementation using RuntimeDouble
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Add extra fields to CDS cluster proto to check memory usage
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Revert "Add extra fields to CDS cluster proto to check memory usage"
This reverts commit a6a285dcee9e0fe618286d3dfbfab98c957dd9c7.
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
* Add changelog entry
Signed-off-by: Gastón Kleiman <gkleiman@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9f7d44850310999ecbd0cfbe8fadb70db6f258f9
data-plane-api(CircleCI)