This PR adds an OTLP logger to Envoy, which allows users to flexibly configure their log structure (according to OpenTelemetry protos).
The API users the common gRPC configuration, used in ALS, for gRPC settings. The flexible OT fields (body, attributes) are completely configurable and are filled with request/response information based on Envoy access log formatting.
There are still a couple of TODOs -
The 'filter_state_objects_to_log' from the common gRPC config isn't added to the log
Resource attributes are not configurable, and only Envoy node info is added.
Note: this extension was added as v3alpha API.
Risk Level: Low - a new extension, so unlikely to break any existing setups (marked as v3alpha API).
Testing: Unit tests.
Docs Changes:
Release Notes: access log: added a new :ref:OpenTelemetry access logger <envoy_v3_api_msg_extensions.access_loggers.open_telemetry.v3alpha.OpenTelemetryAccessLogConfig> extension, allowing a flexible log structure with native Envoy access log formatting.
Signed-off-by: Itamar Kaminski <itamark@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4f52f9ef554e1522e763c0eaf6ef6833aab283e5
This introduces a new matcher that allows matching on an input value by
computing a hash value and matching if the value % (configured value) is
greater than a configured threshold. This is useful in being able to
define match criteria that should match for a certain % of input values
in a way that is consistent between independent Envoy instances (e.g. it
does not rely on a random input).
Risk Level: Low, new extension
Testing: UTs
Docs Changes: Inline proto docs
Release Notes: n/a
Platform Specific Features: n/a
Fixes#14782
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7fe3d358f7e7e4d4c7282d50c498a7ab0e759a36
* Configuration for path normalization so that normalization can be configured for internal only use and/or forwarding.
Signed-off-by: Asra Ali <asraa@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 236107df5d15381ccf5040c0cb0f163ece9627a2
1) Promote the default UUID request_id implementation to
an actual extension that is required in the build and wire
up all documentation.
2) Add a configuration option to the extension that allows trace
reason packing to be disabled (the default continues to be for it
to be enabled to match existing behavior).
3) Update all documentation for the new behavior.
4) Substantial cleanup of these code paths for clarity and robustness.
Fixes https://github.com/envoyproxy/envoy/issues/11532
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 07c4c17be61c77d87d2c108b0775f2e606a7ae12
* Add v3 equivalents of v2 configs that were included in v3 due to no
transitive deprecation. This increases consistency and reduces user
confusion. We will continue to support these straggler v2 configs
beyond the v2 turndown due to the late addition of v3 counterparts,
special case code is added to utility.cc to handle this.
* There were two extensions, //envoy/config/cluster/redis and
//envoy/config/retry/previous_priorities, that for some reason were
not upgraded to use v3 config. This is now fixed and I've grepped for
other v2 in //source, none remain.
Risk level: Medium (changes to extension config types and deprecated
config handling).
Testing: Additional unit test added for utility.cc handling, upgraded
configs to v3 for other tests.
Fixes#14735Fixes#12841
Signed-off-by: Harvey Tuch <htuch@google.com>
Co-authored-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ c04a75efe9d601ebcb9650f274b5adde60181f1c
Commit Message: compression: add brotli compressor and decompressor
Additional Description: Add new brotli compression extensions in addition to gzip.
Risk Level: Low, no existing functionality is touched
Testing: uni tests, manual tests with curl.
Docs Changes: updated docs for compression and decompression HTTP filters to refer the new available encoder/decoder.
Release Notes: updated current.rst
Fixes#4429
The PR adds a new dependency on https://github.com/google/brotli. Here's the current criteria answers:
| Criteria | Answer |
|---------|---------|
| Cloud Native Computing Foundation (CNCF) approved license | MIT |
| Dependencies must not substantially increase the binary size unless they are optional | brotli's binary size built with `-c opt` is 752K |
| No duplication of existing dependencies | no other dep provides Brotli |
| Hosted on a git repository and the archive fetch must directly reference this repository. | https://github.com/google/brotli |
| CVE history appears reasonable, no pathological CVE arcs | so far 4 CVEs related to brotli have been registered |
| Code review (ideally PRs) before merge | PRs are reviewed before merge |
| Security vulnerability process exists, with contact details and reporting/disclosure process | no policy exists, submitted https://github.com/google/brotli/issues/878 |
| > 1 contributor responsible for a non-trivial number of commits | 75 contributors |
| Tests run in CI | CI set up with AppVeyor and Github actions |
| High test coverage (also static/dynamic analysis, fuzzing) | Fuzzers are run in CI |
| Envoy can obtain advanced notification of vulnerabilities or of security releases | brotli is registered in CPE |
| Do other significant projects have shared fate by using this dependency? | Google Chrome is using the library |
| Releases (with release notes) | https://github.com/google/brotli/releases |
| Commits/releases in last 90 days | last commit 9 days ago |
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 127aa55306c9e4366e10b3a6e26d01df826e2fc0
part of #12841
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ f1638fbed36a1416d3b44783eedfb7ef6fb39f17
Introduces the FilterDependency proto. This isn't quite an extension, but it's a common proto to be used by all filter extensions.
Risk Level: Low (proto addition only)
#14470
Signed-off-by: Auni Ahsan <auni@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2da224f88afe5ec84ba802bd932f73ed13308a92
part of #12841
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ 77d7cecb7e68cb41b2f5f7c2aa7f4294c88089ce
Risk Level: LOW
Testing: unit ( proto_format and docs )
part of #12841
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ c15193642432291f13c6358ed8c4850fba3a8969
Adds support for associating a match tree with a HTTP filter, supporting a single "Skip" operation that will have the FM
ignore the filter for the duration of the stream once matched.
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 935a6598cd01324f03608ca77ebffc9608f7af81
This introduces a new filter called the "external processing filter." It is intended to allow an external service to be able to operate as if it were part of the filter chain using a gRPC stream. It is intended to support a variety of use cases in which processing of HTTP requests and responses by an external service is desired.
A document that describes the filter can be found here:
https://docs.google.com/document/d/1IZqm5IUnG9gc2VqwGaN5C2TZAD9_QbsY9Vvy5vr9Zmw/edit#heading=h.3zlthggr9vvv
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 98d2f3b553b87c3e935f57ba15b4faf68b45d7f0
Replacing the http-protocol-specific fields in the cluster config with a new plugin
Risk Level: medium
Testing: updated tests to use the new config
Docs Changes: updated docs to use the new config
Release Notes: deprecation notes in the PR
Deprecated: all http-specific cluster config.
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 7554d61bccf136638bdfb383c10d049dc8bd3790
Add a KillRequest HTTP filter which can crash Envoy when receiving a Kill request. It will be used to fault inject kill request to Envoy and measure the blast radius.
Risk Level: Low, new feature.
Testing: Unit/integration tests.
Docs Changes: Added
Release Notes: Added
Issue: #13978
Signed-off-by: Qin Qin <qqin@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 237b29d6399953f22a47c6e4d19df74b4fbcee8d
Use abort action as a default if killing is enabled and we're on a supported platform.
Risk Level: low
Testing: unit tests
Docs Changes: Included
Release Notes: Included
See PR #13208 for context as the reason it's part of core envoy and not an extension.
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 415af040e09a1f6993c15ffc022793d39ecf5e8e
Signed-off-by: John Plevyak <jplevyak@gmail.com>
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 319a9a647f19036d831a75f68350c90e98bfc365
The CdnLoopFilter implements an HTTP filter that detects and prevents
CDN loops using the RFC 8586 CDN-Loop header. The filter can be
configured with the CDN identifier to look for as well as the number
of times the CDN identifier can be seen before responding with an
error.
Signed-off-by: Justin Mazzola Paluska <justinmp@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c71ec2729cc3c0708223d303e0f24e3bf9a5d0eb
Added a watchdog extension that triggers profiling.
Risk Level: Medium (new extension that is optional)
Testing: Unit tests
Docs Changes: Included (added a reference to the generated extension proto.rst)
Release Notes: Included
Fixes#11388
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c88515fc0c8a291992732947671190b37949bbbd
This was written by Derek Argueta originally. Some more
work might be needed to make it more generic.
Risk Level: low, new filter
Testing: unit tests included
Docs Changes: filter docs added
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Co-authored-by: Derek Argueta <darguetap@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ c6bfd7f9f52468d576781a9b1fe9ea5d3f9086c9
This is the 1st PR for #11832 that factors out the TAP filter matcher to prepare for reuse in other filters.
Signed-off-by: Yangmin Zhu <ymzhu@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 520389e677cdcd4a85df769deb40f6cdd2f4f6f8
Description: Upstream Wasm proto files from envoy-wasm.
Risk Level: Low
Testing: Unit tests in envoy-wasm, integration tests in istio/proxy.
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: John Plevyak <jplevyak@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 26eaa2e85cee69e5c32ab6bf4c5ae3d338fa462f
Commit Message: Add proxy proto transport socket
Additional Description: This is the part 1 PR described in #10682. It adds the transports socket / unit tests, a transport socket options struct for the proxy proto header, and does a refactor to make the listener filter use the common proxy proto constants (potentially want to move these now since the proxy proto config api type is not in extensions?)
Risk Level: Small
Testing: Unit
Docs Changes: None
Release Notes: None
Part Of: #1031
Signed-off-by: Weston Carlson <wez470@gmail.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 8972b478e6c9f1e7342e3dbfb57b35317c0cc009
split out from #11327
There's a bit of transitive ugliness: declaring the extensions requires security posture, requires stub build files, requires codeowners before the code move, but it'll be pretty short lived.
Risk Level: Low (mostly only APIs)
Testing: n/a
Docs Changes: some of the new docs
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ e8dc25ecec277c0b94d02151de79353a9ba07b4e
This extension is used in production and we should treat it as such.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 86caf439d6cae2c8173b19fd4fdc95361565a72d
Commit Message: add generic decompressor filter
Risk Level: low - low as it is an extension, med - for users as this is a brand new filter.
Testing: unit tests, integration tests
Docs Changes: added docs
Release Notes: added release notes
Signed-off-by: Jose Nino <jnino@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 48a5b21d9483e7eddac79aeff7daac178d7b7462
Description: router: Create InternalRedirectPolicy to capture all internal redirect related options and extend it with pluggable predicates similar to retry plugins. The previous_routes and whitelisted_routes predicate allow creating a DAG of routes for internal redirects. Each node in the DAG is a route. whitelisted_routes defines the edges of each node. previous_routes serves as visited status keeper for each of the edge. This prevents infinite loop, while allowing loop to exist in the DAG.
Risk Level: Medium
Testing: Unit tests. Integration tests.
Docs Changes: Updated HCM architecture overview page. Added toctree for the predicates.
Release Notes: Updated version history.
Signed-off-by: pengg <pengg@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1ce010929d4d283fce977bc65558da71ffe6bf7c
creates decompressors as an extension point and moves the zlib based gzip decompressor.
Signed-off-by: Jose Nino <jnino@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 520e0c7050564ac7886129a87290e9e433470efd
Currently the generic HTTP compressor filter isn't exposed to users
even though it's used internally by `envoy.filters.http.gzip` and can be
used by external filter extensions.
Expose the compressor's config API to users. For example the filter
can be configured as follows:
...
filter_chains:
filters:
- name: envoy.http_connection_manager
config:
http_filters:
- name: envoy.filters.http.compressor
config:
disable_on_etag_header: true
content_length: 100
content_type:
- text/html
- application/json
compressor_library:
name: envoy.filters.http.compressor.gzip
config:
memory_level: 3
window_bits: 10
compression_level: best
compression_strategy: rle
...
Multiple compressor filters using different compressor libraries,
e.g. gzip and brotli, can be stacked in one filter chain.
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 49efb9841a58ebdc43a666f55c445911c8e4181c
This change adds DNS Request Parsing to the DNS filter. The filter will parse and decode DNS requests for A and AAAA records. Tests simply validate that the filter can consume queries.
Signed-off-by: Alvin Baptiste <alvinsb@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 54cd4d49e895befb8ecb10ebb14585cd8fc71ee7
Description:
The filter implements decoding postgres wire protocol and parses messages exchanged between postgres server and client. Based on the decoded messages the filter generates statistics (counters) indicating how many messages of a specific type were exchanged. #9107
Risk Level: Low: The filter is implemented as extension and the code is not executed unless inserted into filter chain.
Testing: Added unit and integration tests.
Docs Changes: Yes - added architecture overview chapter and configuration specific sections
Release Notes: Yes
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Co-authored-by: Dhi Aurrahman <dio@tetrate.io>
Co-authored-by: Fabrízio de Royes Mello <fabrizio@ongres.com>
Mirrored from https://github.com/envoyproxy/envoy @ f599ad7c05824a2cdbcde60ab2c035d264cd4247
This commit is this base structure and api definition
for the DNS filter. The code itself takes no action
on packets. Tests will be added later.
Signed-off-by: Alvin Baptiste <alvinsb@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ b3949eaf2080809b8a3a6cf720eba2cfdf864472
This filter transform HTTP requests to AWS Lambda invocations.
The filter supports pass-through only. Meaning, the request body
is passed to Lambda as is. Note: Lambda requires the request to be in
JSON format.
In a later iteration, we'll wrap the headers the body in a JSON string
before passing it to Lambda.
The filter requires the ARN of the Lambda function and supports
per-filter-config. When the per-filter configuration is used, the target
cluster must be tagged with specific metadata. This indicates to the
filter whether to process the request or to skip it.
Lambda supports two invocation modes:
- Synchronous (Request-Response)
- Asynchronous (Event)
This initial version of the filter supports the synchronous mode only.
In a later iteration I'll add support for the asynchronous (Event-based)
version.
Signed-off-by: Marco Magdy <mmagdy@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 807401004d500899e9aa4c78fce007cf83b538cd
data-plane-api(Azure Pipelines)