Currently health check failure events are only log if the HealthFlag for a host transition from non-FAILED_ACTIVE_HC to FAILED_ACTIVE_HC. However, since hosts are initialized in the FAILED_ACTIVE_HC state, hosts that never became healthy have no events associated with it.
Since the current health check events only log transitions, we'll have to scan the entire log in order to find the hosts in a current failing state. Then we'll still have to filter the hosts permanently removed from the cluster by the discovery service. This makes the events very difficult to use in operations.
Proposed solution
Both of these 2 issues can be solved by emitting a health check failure event if either of these conditions are true:
If the active health check failed and it's the first health check for a host. This ensures we have events for hosts that never became healthy.
If the active health check failed and a AlwaysLogFailures configuration is set to true, by default this flag is set to false. This makes it very easy to find the hosts currently failing by looking at the last few seconds of logs.
Signed-off-by: Henry Yang <hyang@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 11e196b67ee9124f33c45f5adf542841386e3c39
Add a field in listener proto to be able to reverse the order of TCP write filters. The field is set false by default, indicating write filters have the same order as configured in the filter chain. If true, their order will be reversed.
Risk Level: Low
Testing: bazel test //test/...
Part of #4599
Signed-off-by: Qi (Anna) Wang <qiwang@qiwang-macbookpro.roam.corp.google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5da782c8503aa2664ceac1995628d161bbaa6441
We've been using this in production for over 3 months now and it's
been very useful to prevent CPU spikes when we get a stream of
updates.
This enables update merging every 1s.
Fixes#4018.
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ fad993e5aed40fba95897e9017afd19bdf170ed0
Implement a new certificate validation context type CombinedCertificateValidationContext, which has a default CertificateValidationContextoption and SDS config. This default CertificateValidationContext will be merged with dynamic CertificateValidationContext into a new secret to serve. This is option 4 in https://docs.google.com/document/d/12gdjGN5m3v4vxUnDAglCP6pyyMoeuVGAGo7D_jc27jw/edit?usp=sharing
Risk Level: Low
Testing:
Docs Changes: NONE
Release Notes: NONE
Fixes: #4763
Signed-off-by: JimmyCYJ <jimmychen.0102@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7a651dc4d09ed07d6a5b5a40cc0809e3cf2e700e
This commit enables the configuration of the mapping that translates 429
response code to a gRPC status code. By default, the Rate Limit filter
in Envoy translates a 429 HTTP response code to UNAVAILABLE as specified
in the gRPC mapping document. Google, however, recommends translating a
429 response to RESOURCE_EXHAUSTED. This commit provides a flag named
rate_limited_as_resource_exhausted in the RateLimit config which allows
users to explicitly specify whether they want 429 responses to be mapped
to RESOURCE_EXHAUSTED, while UNAVAILABLE remains the default.
References:
* https://github.com/grpc/grpc/blob/master/doc/http-grpc-status-mapping.md
* https://cloud.google.com/apis/design/errors#generating_errors
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ f71a883b557a18cc418d4103b2f07a6780fc6576
Added an ability to add context extensions on a per virtualhost
oute\weighted-cluster to the ext auth filter.
This will allow adding custom extra data to the check request on a per-route basis. This can be used to create a more sophisticated authorization policy.
Risk Level: Low-Medium (opt-in, no impact for existing users)
Testing: Added unit tests to new code; manual testing.
Docs Changes: added usage example in docs/root/configuration/http_filters/ext_authz_filter.rst
Release Notes: added notes to version_history.rst
Signed-off-by: Yuval Kohavi <yuval.kohavi@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 15c5befd43fb9ee9b145cc87e507beb801726316
API for #4475.
Risk Level: Low (not implemented)
Testing: CI
Docs Changes: Added but hided
Release Notes: N/A, will add when adding impl.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 45a460fabf34698a875060482de96f7f618bdc9f
We no longer allow running v1 config, and users can look at old
docs if they want to see the v1 docs still.
Also fix a doc issue in the /server_info admin endpoint.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8aba108eda816ea29dc80b973a1387257cd209c6
Converts the existing /server_info admin endpoint to be represented by a protobuf. This will make it easier to extend with new values in the future.
Risk Level: Low
Testing: Updated the existing unit test
Docs Changes: n/a
Release Notes: n/a
Part of #4405
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ 71bd095297ba64712bfad30d0aee1f019fbd32d8
*Description*: PGV picks up unused imports in `api/envoy/data/core/v2alpha/health_check_event.proto`.
Error message is:
```
INFO: From ProtoGenValidateCcGenerate external/envoy_api/envoy/data/core/v2alpha/health_check_event.pb.validate.h:
envoy/data/core/v2alpha/health_check_event.proto: warning: Import envoy/api/v2/core/base.proto but not used.
envoy/data/core/v2alpha/health_check_event.proto: warning: Import google/protobuf/wrappers.proto but not used.
envoy/data/core/v2alpha/health_check_event.proto: warning: Import google/protobuf/duration.proto but not used.
```
*Risk Level*: Low
*Testing*: `bazel test //test/...` and running on local instances
*Docs Changes*: none required
*Release Notes*: none required
Signed-off-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ c951e6088a5e1214c864448b0ccfd104bf2131ee
When the redirect action changes the scheme (https_redirect or scheme_redirect), remove the default port if it is set in the request. I.e. if the request is http://192.168.0.1:80/path redirected to https, the resulting URI will be https://192.168.0.1/path.
Risk Level: Low
Testing: unit and integration tests.
Docs Changes: the proto documentation.
Release Notes:
Signed-off-by: Nikolay Nikolaev <nnikolay@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0f7120968e60da62feb59f00170078611dffc18a
Implements rate limiting for discovery requests
Risk Level: Medium. This changes the way DiscoveryRequests are processed today (queues them) and adds rate limiting behaviour. While we have good test coverage (and also additional tests have been added), there is some risk.
Testing: Automated tests
Docs Changes: N/A
Release Notes: Added
Fixes#4718
Signed-off-by: Rama <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 455714694bb930729c32a1f92c0f9c4f083a3bdb
In preparation for removing std::hash for LB (a deprecated v1 option)
Risk Level: Medium (changing existing code where default config was used)
Testing: Tweaked existing unit tests
Docs Changes: updated API docs
Release Notes: Noted in release notes
Deprecated*: std::hash in LB (already deprecated, but might as well get the bugs auto-filed for 1.9.0)
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ db793ca15cfa9a500e76172c1011fa7baa4327ef
This was supposed to work already, but it wasn't due to a missing
call to X509_STORE_set_flags() and lack of test coverage.
*Risk Level*: Low
*Testing*: bazel test //test/...
*Docs Changes*: Added
*Release Notes*: Added
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 76278101ee854550cc29d8ba87db05e544b1f585
This fixes a bug in the other priority plugin that would cause a crash
when retries were attempted when the upstream had no healthy hosts. The
existing check for no healthy was ineffective due to the "everything is
terrible" fallback in the LoadBalancerBase which sets P0 to 100 when all
the priorities are unhealthy.
The fix is to check for healthy % based on the loads computed in the
plugin, not the ones returned by LoadBalancerBase. When all hosts are
unhealthy, we return the original priority load. This ensures that we
maintain whatever fallback the default LB uses when there are no
unhealthy hosts.
Signed-off-by: Snow Pettersen snowp@squareup.com
Risk Level: Medium
Testing: Added regression test for no unhealthy hosts
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ 59816a486c64cd05e9e0c0f08194b121690d6632
The main difference between what we had and official nghttp2 support is lack of support for upgrade-with-bodies (on request or response path). Adjusted header munging, tests, and docs accordingly.
Risk Level: Low (changes code on a "hidden" code path)
Testing: updated tests, new unit tests
Docs Changes: updated
Release Notes: noted H2 websocket support
Fixes#1630
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 5b9de64f2858439b7c3b6ddabc08f50f4a752b90
This makes marking filters as encoder/decoder/both illegal.
Risk Level: Medium (breaking change for old configs)
Testing: existing tests pass including legacy json tests (with modified config)
Docs Changes: No
Release Notes: Not currently
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ dcb4f39ba103062472f4f94f3f39c4900750763f
Use dynamicMetadata in the StreamInfo to pass all successfully verified JWT payloads to other HTTP filters.
Risk Level: Low
Testing: Add unit-tests
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2399402297831bf7c2c24284a287fd6c1e74115f
To use --distdir option of Bazel (which allows to use previously
fetched tarballs instead of downloading dependencies during
build), all dependencies should use http instead of git and need
to have sha256 sums specified.
Risk Level: low
Testing: n/a
Docs Changes: updated bazel/EXTERNAL_DEPS.md
Release Notes: n/a
Signed-off-by: Michal Rostecki <mrostecki@suse.de>
Mirrored from https://github.com/envoyproxy/envoy @ dc0d35b9ed93b4f5d2ad83f6a7e78832504aeeea
Add scheme and port redirects which replace the respective
URI components when applied.
Fixes issue #3060.
Signed-off-by: Nikolay Nikolaev <nnikolay@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 057edf16474df8f1ed834dbfa8ceefb45613b3a4
Added a field in HCM proto to be able to reverse the order of HTTP encoder filters. The field is set false by default, indicating HTTP encoder filters have the same order as configured in the filter
chain. If true, their order will be reversed.
Risk Level: low
Testing: bazel test //test/...
Part of #4599
Signed-off-by: Qi (Anna) Wang <qiwang@qiwang-macbookpro.roam.corp.google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0ccc70ae77909baadcb07dd0c9ca2ef583dde3b5
Add a new config option under access_log called json_format. This is a single level dictionary that contains strings as keys, and envoy access log format specifiers (such as %PROTOCOL%) as values.
The specifiers will be replaced with actual values at logging time. I call this dictionary the "format dictionary" (as opposed to "format string").
You can specify only one of format (format string) or json_format (format dictionary). If neither are there, we fall back to the default string format.
Add the correct plumbing inside the configuration parsing to handle this.
Add a new access log formatter class that is instantiated with the format dictionary. It maintains the mapping of dictionary keys to loggers
Create a new class called FormatterProvider, to distinguish things that actually extract the information from a request. The things that combine together a bunch of FormatterProviders are still called Formatters. This is primarily a semantic/naming difference, but imo these are two conceptually separate things. There is, however no API difference, and if people are truly opposed to this, I could just merge them back into one Formatter class. This also provides a better foundation for adding more log formats in the future.
At present, only one specifier per key in the format dictionary is allowed. This is because the whole point of JSON logging is to make logs easily machine-parseable. If you can include multiple formats in the same field, then you'll be right back to parsing those manually
At present, only top-level keys are allowed in the format dictionary. This is validated at config load time. In the future, we can expand this to have nested dictionaries.
Risk Level: Low. It's an optional feature that has to be explicitly enabled.
Testing: Unit testing for the actual formatter, and config load. Also manually tested using an example config file.
Docs:
Amended Access Log docs to create a notion of "Format Strings" and "Format Dictionaries".
Put things that are common to access logs in general under "Format Rules", and then distinguished how strings and dictionaries are different.
Called out restrictions on format dictionaries
Added protobuf comments for format and json_format
Signed-off-by: Aaltan Ahmad <aa@stripe.com>
Mirrored from https://github.com/envoyproxy/envoy @ de039269f54aa21aa0da21da89a5075aa3db3bb9
This is a follow up to #4726. In #4726, the access log path became optional, but the admin field
was not itself marked optional. This then led to server_fuzz_test trivially passing due to an early
PGV validation exception, and ~20 bugs being closed out by oss-fuzz. This PR completes the admin
optionality changes.
Risk Level: Low
Testing: Unit tests updated.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 86790c2367558160282d8b0afa1c5e4698e2fed3
A fuzz failure indicated a divide-by-zero in HealthCheckerImplBase::interval(). Beyond this, on
inspection, it seems we allow zero retry intervals, which can live lock the main thread. This PR
fixes these issues.
Fixes oss-fuzz issue https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10786.
Risk Level: Low
Testing: Additional unit test and a corpus entry added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b61b28fe7ae9927452477a0f4e337c53830a7070
Introduces a rate limiter filter for thrift requests, implemented as a
decoder filter that works in conjunction with the thrift router filter.
Risk Level: low
Testing: unit tests
Doc Changes: inline with proto definitions
Release notes: updated
Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>
Mirrored from https://github.com/envoyproxy/envoy @ 6ac936f2750c39a8b4fb232d6ddc4802f4e6aeee
data-plane-api(CircleCI)