This is a MVP for the HTTP tap filter. It includes minimal
infrastructure for the following:
1. Generic tap configuration which in the future will be used for
static config, XDS config, etc. In this MVP the tap can be
configured via a /tap admin endpoint.
2. Generic output configuration which in the future will be used for
different output sinks such as files, gRPC API, etc. In this MVP
the tap results are streamed back out the /tap admin endpoint.
3. Matching infrastructure. In this MVP only matching on request and
response headers are implemented. Both logical AND and logical OR
matches are possible.
4. In this MVP request/response body is not considered at all.
5. All docs are included and with all the caveats the filter is ready
to use for the limited cases it supports (which are likely still to
be useful).
There is a lot of follow on work which I will do in subsequent PRs.
This includes:
1. Merging the existing capture transport socket into this framework.
2. Implementing body support, both for matching on body contents as
well as outputting body data.
3. Tap rate limiting so too many streams do not get tapped.
4. gRPC matching. Using reflection and loaded proto definitions, it will
be possible to match on gRPC fields.
5. JSON matching. If the body parses as JSON, we can allow matching on
JSON fields.
Part of https://github.com/envoyproxy/envoy/issues/1413.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ cf80045587240d494e54e9772949bc9af5eda61f
This filter contains the logic to decode the mysql wire protocol and SQL queries (SQL99 only).
The code is based on our internal version at VMware. The SQL parser can be found at https://github.com/rshriram/sql-parser. Its a cleaned up version of Hyrise SQL parser. I am keeping the code as a separate library as importing the sources into envoy will cause a lot of changes to the code.
Signed-off-by: Giorgio Valentini <gvalentini@vmware.com>
Signed-off-by: Deepa Kalani <dkalani@vmware.com>
Signed-off-by: Shriram Rajagopalan <shriramr@vmware.com>
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ b3be5713f2100ab5c40316e73ce34581245bd26a
Add support of Any as opaque config for extensions. Deprecates Struct configs. Fixes#4475.
Risk Level: Low
Testing: CI
Docs Changes: Added.
Release Notes: Added.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 851f591f4ed84594e5e5041e7ada4167a4f3a273
Adds a filter that allows converting a gRPC request into an HTTP/1.1
request with a custom content-type. Allows a vanilla HTTP/1.1 upstream
to handle incoming requests by reading and responding with protobuf
messages in binary octet format.
For now this shields the upstream from any gRPC association: the filter
removes the gRPC specific message prefix and manages the conversion of
the HTTP status code into grpc-status.
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ a97e138ffc58c13dc5be277b877c6a5949083023
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
Description: The new request_timeout in http connection manager covers the filter chain.
Risk Level: N/A
Fixes#4830
Signed-off-by: Auni Ahsan <auni@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 92e932a9a26ca178ad77a8e3616fadd57d3f807d
This identifies downstream disconnects from within the HTTP Connection
Manager and eventually echoes it out via the access log.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 988f7d81f973e17c76db3f900cf145c371d6e8b3
Allowing the HCM upgrades to be on or off by default, and adding per-route overrides to turn it off or on.
Risk Level: Medium (refactors existing code)
Testing: new unit and e2e tests
Docs Changes: proto docs
Release Notes: inline
Fixes#4921
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ d72eaaf6d1905f7d478ab80cc7163684fc271fd9
This adds dynamic metadata to the stream info while processing data in
the mongo_proxy filter.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 16843c193af26d3eb838aa83034096fe6d132b05
This commit enables the configuration of the mapping that translates 429
response code to a gRPC status code. By default, the Rate Limit filter
in Envoy translates a 429 HTTP response code to UNAVAILABLE as specified
in the gRPC mapping document. Google, however, recommends translating a
429 response to RESOURCE_EXHAUSTED. This commit provides a flag named
rate_limited_as_resource_exhausted in the RateLimit config which allows
users to explicitly specify whether they want 429 responses to be mapped
to RESOURCE_EXHAUSTED, while UNAVAILABLE remains the default.
References:
* https://github.com/grpc/grpc/blob/master/doc/http-grpc-status-mapping.md
* https://cloud.google.com/apis/design/errors#generating_errors
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ f71a883b557a18cc418d4103b2f07a6780fc6576
Added an ability to add context extensions on a per virtualhost
oute\weighted-cluster to the ext auth filter.
This will allow adding custom extra data to the check request on a per-route basis. This can be used to create a more sophisticated authorization policy.
Risk Level: Low-Medium (opt-in, no impact for existing users)
Testing: Added unit tests to new code; manual testing.
Docs Changes: added usage example in docs/root/configuration/http_filters/ext_authz_filter.rst
Release Notes: added notes to version_history.rst
Signed-off-by: Yuval Kohavi <yuval.kohavi@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 15c5befd43fb9ee9b145cc87e507beb801726316
API for #4475.
Risk Level: Low (not implemented)
Testing: CI
Docs Changes: Added but hided
Release Notes: N/A, will add when adding impl.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 45a460fabf34698a875060482de96f7f618bdc9f
This fixes a bug in the other priority plugin that would cause a crash
when retries were attempted when the upstream had no healthy hosts. The
existing check for no healthy was ineffective due to the "everything is
terrible" fallback in the LoadBalancerBase which sets P0 to 100 when all
the priorities are unhealthy.
The fix is to check for healthy % based on the loads computed in the
plugin, not the ones returned by LoadBalancerBase. When all hosts are
unhealthy, we return the original priority load. This ensures that we
maintain whatever fallback the default LB uses when there are no
unhealthy hosts.
Signed-off-by: Snow Pettersen snowp@squareup.com
Risk Level: Medium
Testing: Added regression test for no unhealthy hosts
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ 59816a486c64cd05e9e0c0f08194b121690d6632
This makes marking filters as encoder/decoder/both illegal.
Risk Level: Medium (breaking change for old configs)
Testing: existing tests pass including legacy json tests (with modified config)
Docs Changes: No
Release Notes: Not currently
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ dcb4f39ba103062472f4f94f3f39c4900750763f
Use dynamicMetadata in the StreamInfo to pass all successfully verified JWT payloads to other HTTP filters.
Risk Level: Low
Testing: Add unit-tests
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2399402297831bf7c2c24284a287fd6c1e74115f
Added a field in HCM proto to be able to reverse the order of HTTP encoder filters. The field is set false by default, indicating HTTP encoder filters have the same order as configured in the filter
chain. If true, their order will be reversed.
Risk Level: low
Testing: bazel test //test/...
Part of #4599
Signed-off-by: Qi (Anna) Wang <qiwang@qiwang-macbookpro.roam.corp.google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0ccc70ae77909baadcb07dd0c9ca2ef583dde3b5
Add a new config option under access_log called json_format. This is a single level dictionary that contains strings as keys, and envoy access log format specifiers (such as %PROTOCOL%) as values.
The specifiers will be replaced with actual values at logging time. I call this dictionary the "format dictionary" (as opposed to "format string").
You can specify only one of format (format string) or json_format (format dictionary). If neither are there, we fall back to the default string format.
Add the correct plumbing inside the configuration parsing to handle this.
Add a new access log formatter class that is instantiated with the format dictionary. It maintains the mapping of dictionary keys to loggers
Create a new class called FormatterProvider, to distinguish things that actually extract the information from a request. The things that combine together a bunch of FormatterProviders are still called Formatters. This is primarily a semantic/naming difference, but imo these are two conceptually separate things. There is, however no API difference, and if people are truly opposed to this, I could just merge them back into one Formatter class. This also provides a better foundation for adding more log formats in the future.
At present, only one specifier per key in the format dictionary is allowed. This is because the whole point of JSON logging is to make logs easily machine-parseable. If you can include multiple formats in the same field, then you'll be right back to parsing those manually
At present, only top-level keys are allowed in the format dictionary. This is validated at config load time. In the future, we can expand this to have nested dictionaries.
Risk Level: Low. It's an optional feature that has to be explicitly enabled.
Testing: Unit testing for the actual formatter, and config load. Also manually tested using an example config file.
Docs:
Amended Access Log docs to create a notion of "Format Strings" and "Format Dictionaries".
Put things that are common to access logs in general under "Format Rules", and then distinguished how strings and dictionaries are different.
Called out restrictions on format dictionaries
Added protobuf comments for format and json_format
Signed-off-by: Aaltan Ahmad <aa@stripe.com>
Mirrored from https://github.com/envoyproxy/envoy @ de039269f54aa21aa0da21da89a5075aa3db3bb9
This is a follow up to #4726. In #4726, the access log path became optional, but the admin field
was not itself marked optional. This then led to server_fuzz_test trivially passing due to an early
PGV validation exception, and ~20 bugs being closed out by oss-fuzz. This PR completes the admin
optionality changes.
Risk Level: Low
Testing: Unit tests updated.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 86790c2367558160282d8b0afa1c5e4698e2fed3
Introduces a rate limiter filter for thrift requests, implemented as a
decoder filter that works in conjunction with the thrift router filter.
Risk Level: low
Testing: unit tests
Doc Changes: inline with proto definitions
Release notes: updated
Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>
Mirrored from https://github.com/envoyproxy/envoy @ 6ac936f2750c39a8b4fb232d6ddc4802f4e6aeee
Now that Envoy can function without an admin listener (#4486), there's no need to specify path or
even an empty admin message, so relax this requirement.
Risk Level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 570a403b8882ea34309bbe73209fee41417fa10e
This commit adds support for SNI based permissions by matching over a
connection's requested server name.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 811ee0dc52951acf2f66190587f92394473c245c
This commit removes support for the previously deprecated integer
percentages in the fault subsystem.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7ccf67db45db8c4e9bc406488c8f1f2a8566f506
Re-enable the changes reverted in 9d32e5c2a14cd9ab96b6e77fb04f7bd77b2c0d71, which were originally merged as part of #4382.
Signed-off-by: Andres Guedez <aguedez@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b16f5299e45ca71a36ec6f7fc006ed311b58a843
Implements a RetryPriority which will keep track of attempted
priorities and attempt to route retry requests to other priorities. The
update frequency is configurable, allowing multiple requests to hit each
priority if desired.
As a fallback, when no healthy priorities remain, the list of attempted
priorities will be reset and a host will selected again using the
original priority load.
Extracts out the recalculatePerPriorityState from LoadBalancerBase to
recompute the priority load with the same code used by the LB.
Signed-off-by: Snow Pettersen snowp@squareup.com
Risk Level: Medium, new extension
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ ba5d3f0c130bb21958cf093c368af0526a4740b7
update doc for jwt_authn http filter
format config.proto comment for doc
add a new rst file: docs/root/configuration/http_filters/jwt_authn_filter.rst
Risk Level: None
Docs Changes: Yes
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d101ae7decfdae148f53ea9d2220444c726dfbfd
This PR reverts #4382. When deploying at Lyft we noticed crashes on here where we might be derefencing the connection_stats_ pointer after the point has been reset.
Note: this PR keeps the changes to the API made in the original PR but tags the field as not implemented. This is what we have done in the past for reverts that involve changes that change the API.
Signed-off-by: Jose Nino <jnino@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9d32e5c2a14cd9ab96b6e77fb04f7bd77b2c0d71
Correct commonly misspelled English words
Signed-off-by: zyfjeff <tianqian.zyf@alibaba-inc.com>
Mirrored from https://github.com/envoyproxy/envoy @ 31d0611d0ffb982414974adaf6940afd1863fa95
data-plane-api(CircleCI)