1) Add request/response body tapping
2) Add buffered body limits (TBI for transport socket)
3) Add the JSON_BODY_AS_BYTES and JSON_BODY_AS_STRING output
formats for convenience when the body is known to be human
readable.
4) Add JSON output for the file per tap sink.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9a06dc0777d2809195cb1fc414b05ae7c0660193
This commit refactors the tap transport socket to use the common
tap extension configuration and tap matching infrastructure. More
match conditions will be added in a future PR as well as additional
cleanups that have been marked with TODOs.
One result of this PR is that the HTTP tap filter can now have a static
configuration as well as write to a file per tap sink.
All future tap PRs should be smaller and more targeted after this one.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ f37ebdc14f4c0adf0e90aabddae833355c0cec1b
This is a rename PR only. It renames the capture transport socket
and associated tools to the tap transport socket. It also updates
some documentation. In a subsequent PR I'm going to refactor the
tap transport socket to use the new common tap framework so that
the tap transport socket can be configured via admin, the HTTP
tap filter can write to a file, the tap transport socket can have
matching, etc.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7a5849f2a8bcc55fa16da3eaee94d9c99a11147c
This is a MVP for the HTTP tap filter. It includes minimal
infrastructure for the following:
1. Generic tap configuration which in the future will be used for
static config, XDS config, etc. In this MVP the tap can be
configured via a /tap admin endpoint.
2. Generic output configuration which in the future will be used for
different output sinks such as files, gRPC API, etc. In this MVP
the tap results are streamed back out the /tap admin endpoint.
3. Matching infrastructure. In this MVP only matching on request and
response headers are implemented. Both logical AND and logical OR
matches are possible.
4. In this MVP request/response body is not considered at all.
5. All docs are included and with all the caveats the filter is ready
to use for the limited cases it supports (which are likely still to
be useful).
There is a lot of follow on work which I will do in subsequent PRs.
This includes:
1. Merging the existing capture transport socket into this framework.
2. Implementing body support, both for matching on body contents as
well as outputting body data.
3. Tap rate limiting so too many streams do not get tapped.
4. gRPC matching. Using reflection and loaded proto definitions, it will
be possible to match on gRPC fields.
5. JSON matching. If the body parses as JSON, we can allow matching on
JSON fields.
Part of https://github.com/envoyproxy/envoy/issues/1413.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ cf80045587240d494e54e9772949bc9af5eda61f
We no longer allow running v1 config, and users can look at old
docs if they want to see the v1 docs still.
Also fix a doc issue in the /server_info admin endpoint.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8aba108eda816ea29dc80b973a1387257cd209c6
Introduces a rate limiter filter for thrift requests, implemented as a
decoder filter that works in conjunction with the thrift router filter.
Risk Level: low
Testing: unit tests
Doc Changes: inline with proto definitions
Release notes: updated
Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>
Mirrored from https://github.com/envoyproxy/envoy @ 6ac936f2750c39a8b4fb232d6ddc4802f4e6aeee
*Description*: Adds support for displaying Subject Alternate Names as a comma separated list of string in `/certs` admin end point
*Risk Level*: Low
*Testing*: Automated
*Docs Changes*: Updated
*Release Notes*: Updated
Signed-off-by: Rama <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7403314b0b255a89ab284b7ed0eead5a5f4aecfd
Description: While working on certs changes, I realized that memory proto doc links were missing. This PR adds them.
Risk Level: Low
Signed-off-by: Rama <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7600419d74aaa0d52c5f56f1a2b35c7ce05a76a0
update doc for jwt_authn http filter
format config.proto comment for doc
add a new rst file: docs/root/configuration/http_filters/jwt_authn_filter.rst
Risk Level: None
Docs Changes: Yes
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d101ae7decfdae148f53ea9d2220444c726dfbfd
This allows injecting a synthetic resource pressure from a file, primarily intended for use in integration tests to force envoy into an overloaded state.
Testing: unit tests
Docs Changes: docs updated to reference new resource monitor
Signed-off-by: Elisha Ziskind eziskind@google.com
Mirrored from https://github.com/envoyproxy/envoy @ 12957a29bd2627bc0bba921603bd5869a449be47
We use the new extension_protocol_options field on Cluster to allow clusters
to be configured with a transport and/or protocol. Downstream requests are
automatically translated to the upstream dialect and upstream responses are
translated back to the downstream's dialect.
Moves the TransportType and ProtocolType protobuf enums out of the
ThriftProxy message to allow their re-use in ThriftProtocolOptions.
*Risk Level*: low
*Testing*: integration test
*Docs Changes*: added thrift filter docs
*Release Notes*: n/a
Signed-off-by: Stephan Zuercher <stephan@turbinelabs.io>
Mirrored from https://github.com/envoyproxy/envoy @ c91625ed829a4ec4123bab8b4e6b223f67d88e4a
Added the /clusters?format=json admin endpoint along with a proto representation of /clusters.
Risk Level: Low
Testing: Added a unit test for the new format.
Docs Changes: Added a brief description on the admin docs and linked to the more detailed proto definition.
Release Notes: Added release notes.
Fixes#2020
Signed-off-by: Matt Rice <mattrice@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 64605338ef040a949c5ea205bdd472a8fe42306c
Add support for extracting dynamic metadata from requests. This can then
be used as static metadata would be used (e.g.: for subset load balancer
metadata matches, logging, etc).
Risk Level: Low
Testing: unit-test
Docs Changes: Basic docs.
Release Notes: N/A
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ 827c0a548ab38d55debe00587ee27253786befad
Adds a file based grpc credentials extension. See issue #3392 for more details.
Risk Level: Low: extension for grpc credentials loaded by explicit configuration options
Testing: tests included in PR
Docs Changes: Inline docs via comments and proto docs
Release Notes: N/A
Fixes#3392
Signed-off-by: Michael Wozniak <wozz@koh.ms>
Mirrored from https://github.com/envoyproxy/envoy @ 230d2216fdd520a182dea9b5152522756853cd90
Fixes https://github.com/envoyproxy/envoy/issues/743
This is a general cleanup of all of the access logging documentation.
I have reorganized a bunch of things and hidden the various gRPC logging
fields that are not implemented yet.
I've also moved the existing tap protos into a new "output" directory. This
is the best name I could come up for cleanly separating output data that might
be stored outside of any service or configuration.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ c15019e79c832d9f0a09468affaadabc4be3e115
* tap/fuzz: transport socket extension for traffic capture.
This PR introduces a transport socket extension that wraps a given transport socket, interposes on its
plain text traffic and records it into a proto trace file on the filesystem. This can be used for a
number of purposes:
1. As a corpus for fuzzing the data plane.
2. Converted to PCAP using a soon-to-be-written utility, allowing existing tools such as Wireshark
to be used to decode L4/L7 protocol history in the trace. Essentially this lets us take advantage
of the PCAP ecosystem.
Relates to #1413 and #508.
Risk Level: Low (opt-in).
Testing: New SSL integration tests, demonstrating plain text intercept.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 6c7a91733469f76381487f9ca78bdece6825c8c9
This PR includes the necessary modifications in support of envoyproxy/envoy#2828.
Added additional configuration to ext_authz.proto so that the filter is able to call an HTTP/1.1 authorization service.
In external_auth.proto, added a nested message to CheckResponse that allows the authorization service to pass additional HTTP response attributes back to the authz filter.
Signed-off-by: Gabriel <gsagula@gmail.com>
1) Add ability to runtime filter to configure default, divisor, and
whether independent randomness is used.
2) Also add LE to the comparison filter.
Signed-off-by: Matt Klein <mklein@lyft.com>
Updates the L7 IP tagging filter proto, v1 configuration documentation and overview documentation.
Signed-off-by: Constance Caramanolis <ccaramanolis@lyft.com>
v2 api changes: Use oneof to specify header match options, based on value(exact_match), regex or range.
The existing value and regex fields will be deprecated. Use the header_match_specfier oneof instead.
Add a new range.proto (envoy.type.v2) for the range type definition.
The SInt64Range message is defined in envoy.type.v2 range.proto.
It consists of start and end (inclusive, exclusive) sint64 values.
v1 api: Add a range_match object to the route headers json. Presence of this object indicates range based route match.
Example: For the below route config:
{
"prefix": "/",
"cluster": "PartitionB",
"name": "PartitionKey",
"range_match": { "start": 0, "end": 10}
}
In the incoming request, if the PartitionKey header value = 0, route match succeeds. Route match fails if the header value = 10, -10, "somestring".
This feature can be used for request routing with Service Fabric stateful services, to route to the desired partition with the [ranged partitioning scheme](https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-concepts-partitioning#ranged-partitioning-scheme)
Signed-off-by: Kavya Kotacherry <kavyako@microsoft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
data-plane-api(CircleCI)
Dhi Aurrahman
Gabriel Sagula
Matt Klein
Constance Caramanolis
Kavya Kotacherry - MSFT
Kuat