Define filter config discovery. Add FDS for HTTP filters (HTTP extensions is where the pain is felt the most). Modelled after RDS with a twist of config override for re-use.
Risk Level: low (not implemented)
Testing:
Docs Changes:
Release Notes:
Issue: #7867
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ bda15c0a5a0390daed991ed4c96a4873ab3ba7d8
split out from #11327
There's a bit of transitive ugliness: declaring the extensions requires security posture, requires stub build files, requires codeowners before the code move, but it'll be pretty short lived.
Risk Level: Low (mostly only APIs)
Testing: n/a
Docs Changes: some of the new docs
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ e8dc25ecec277c0b94d02151de79353a9ba07b4e
This extension is used in production and we should treat it as such.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 86caf439d6cae2c8173b19fd4fdc95361565a72d
Commit Message: add generic decompressor filter
Risk Level: low - low as it is an extension, med - for users as this is a brand new filter.
Testing: unit tests, integration tests
Docs Changes: added docs
Release Notes: added release notes
Signed-off-by: Jose Nino <jnino@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 48a5b21d9483e7eddac79aeff7daac178d7b7462
Description: router: Create InternalRedirectPolicy to capture all internal redirect related options and extend it with pluggable predicates similar to retry plugins. The previous_routes and whitelisted_routes predicate allow creating a DAG of routes for internal redirects. Each node in the DAG is a route. whitelisted_routes defines the edges of each node. previous_routes serves as visited status keeper for each of the edge. This prevents infinite loop, while allowing loop to exist in the DAG.
Risk Level: Medium
Testing: Unit tests. Integration tests.
Docs Changes: Updated HCM architecture overview page. Added toctree for the predicates.
Release Notes: Updated version history.
Signed-off-by: pengg <pengg@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1ce010929d4d283fce977bc65558da71ffe6bf7c
creates decompressors as an extension point and moves the zlib based gzip decompressor.
Signed-off-by: Jose Nino <jnino@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 520e0c7050564ac7886129a87290e9e433470efd
Currently the generic HTTP compressor filter isn't exposed to users
even though it's used internally by `envoy.filters.http.gzip` and can be
used by external filter extensions.
Expose the compressor's config API to users. For example the filter
can be configured as follows:
...
filter_chains:
filters:
- name: envoy.http_connection_manager
config:
http_filters:
- name: envoy.filters.http.compressor
config:
disable_on_etag_header: true
content_length: 100
content_type:
- text/html
- application/json
compressor_library:
name: envoy.filters.http.compressor.gzip
config:
memory_level: 3
window_bits: 10
compression_level: best
compression_strategy: rle
...
Multiple compressor filters using different compressor libraries,
e.g. gzip and brotli, can be stacked in one filter chain.
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 49efb9841a58ebdc43a666f55c445911c8e4181c
This change adds DNS Request Parsing to the DNS filter. The filter will parse and decode DNS requests for A and AAAA records. Tests simply validate that the filter can consume queries.
Signed-off-by: Alvin Baptiste <alvinsb@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 54cd4d49e895befb8ecb10ebb14585cd8fc71ee7
Description:
The filter implements decoding postgres wire protocol and parses messages exchanged between postgres server and client. Based on the decoded messages the filter generates statistics (counters) indicating how many messages of a specific type were exchanged. #9107
Risk Level: Low: The filter is implemented as extension and the code is not executed unless inserted into filter chain.
Testing: Added unit and integration tests.
Docs Changes: Yes - added architecture overview chapter and configuration specific sections
Release Notes: Yes
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Co-authored-by: Dhi Aurrahman <dio@tetrate.io>
Co-authored-by: Fabrízio de Royes Mello <fabrizio@ongres.com>
Mirrored from https://github.com/envoyproxy/envoy @ f599ad7c05824a2cdbcde60ab2c035d264cd4247
This patch performs a major version freeze and bump by modifying
package_version_status, using the tooling developed in #10636.
Specifically:
v2 APIs are frozen (except for where they are the latest in their
package history and still active)
v3 APIs are transitioned to be active
Candidate v4alpha APIs are generated (not used by Envoy yet)
Fixes#10355
Risk level: medium (entire API's files are modified, visually verified
to ensure things look sane, all tests pass)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 549164c42cae84b59154ca4c36009e408aa10b52
This patch introduces the tools/docs for a new flow for protoxform: freeze.
This performs a major version freeze and bump by modifying
package_version_status. Tooling now needs to deal with 3 possible versions (v2,
v3, v4alpha). The following contributions are made in the patch:
* Tooling added to support v2 APIs freeze, transitioning v3 to active and generating v4alpha.
* A new tool, merge_active_shadow.py, is introduced to combine hand-edited active v3 API protos with the v2 implied shadows for deprecated fields.
* The type database now spans v2 and v3 (and implied v4alpha files)
* api_proto_plugins can now take extra args on the Bazel CLI; this is used to plumb in the freeze arg to protoxform.
Docs are updated to reflect the v3 flow that developers should now
follow.
Part of #10355
This is the tools/docs component of #10601. See #10601 for the resulting
API protos generated by a freeze/fix cycle.
Risk level: Low (tooling only, no API changes yet)
Testing: new golden protos tests for protoxform, unit tests for merge_active_shadow.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ de222c3acf1fbf124ca44f50495fc4b8e9aba525
This commit is this base structure and api definition
for the DNS filter. The code itself takes no action
on packets. Tests will be added later.
Signed-off-by: Alvin Baptiste <alvinsb@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ b3949eaf2080809b8a3a6cf720eba2cfdf864472
* Automatically derive active protos target used in proto_format.sh from the packages with files
annotated as active.
* Replace some sites with hardcoded v2/v3 with active/next_major_version_candidate.
* Move //docs:BUILD to //versioning:BUILD in API tree. This has already ceased to be used directly
in the docs build, so this makes things a bit clearer.
Risk level: Low (API tooling and build only)
Testing: fix_format (mostly idempotent)
Part of #10355.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4403003b9c0888ce81606a2f45fcc1da126514e7
This filter transform HTTP requests to AWS Lambda invocations.
The filter supports pass-through only. Meaning, the request body
is passed to Lambda as is. Note: Lambda requires the request to be in
JSON format.
In a later iteration, we'll wrap the headers the body in a JSON string
before passing it to Lambda.
The filter requires the ARN of the Lambda function and supports
per-filter-config. When the per-filter configuration is used, the target
cluster must be tagged with specific metadata. This indicates to the
filter whether to process the request or to skip it.
Lambda supports two invocation modes:
- Synchronous (Request-Response)
- Asynchronous (Event)
This initial version of the filter supports the synchronous mode only.
In a later iteration I'll add support for the asynchronous (Event-based)
version.
Signed-off-by: Marco Magdy <mmagdy@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 807401004d500899e9aa4c78fce007cf83b538cd
This new alpha filter injects authentication headers for requests
directed at AWS services that require authentication.
Note:
Requests over plain HTTP aren't handled yet, since the message body
needs to be signed.
Fixes#9708
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ ee2306673b79215641be02893cb4d8b2b256c466
Add Client Status Discovery Service (CSDS) API definition. This can be used by debug tools to obtain config information for specific clients from control plane.
Risk Level: Low
Testing: N/A
Signed-off-by: Fuqiang Gao <fuqianggao@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 10f756efa17e56c8d4d1033be7b4286410db4e01
Currently supported retry host predicates only allow omitting either previously tried or canary hosts. This enhancement is to add a new host predicate that looks at the endpoint metadata match and omits the host in case of a match with the host metadata. See #9531
Risk Level: Low
Testing: Unit testing
Docs Changes: Added new proto for omit_hosts
Signed-off-by: Prakhar Gautam <prakhag@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ e2fdf70f0fca0f9a9a66046fd80b280981b3f0ed
Implements on-demand resolution of VirtualHosts via VHDS
Signed-off-by: Dmitri Dolguikh <ddolguik@redhat.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8e2d909ad22f84d9eb055f06890924a5879bad76
Define empty config protos for all filters expecting google::protobuf::Empty
Risk Level: medium (change of config type)
Testing: unit
Docs Changes: done
Release Notes: define config protos for all extensions
Co-authored-by: Derek Argueta <dereka@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2d5a4e94720cc195324f79ca68f0e7a7dc83ee9e
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a60f6853a2c2ebbbfed79dfff0b5b644fd735980
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
This is a beachhead PR for a Clang Libtooling based workflow that automagically updates Envoy's
source tree to the latest API version for every referenced package. So far, ths tool is only capable
of inferring types and performing header fixups, later PRs will expand this.
Risk level: Low
Testing: Manual cleanup of all headers in source/ test/ and include/, all tests pass.
Part of #8082
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b54d1c366744314dda38aed5f14e7e93323acb41
* Add an explicit threat model to the end user facing docs, link to this from SECURITY.md
* Switch all Envoy extensions to use a new macro `envoy_cc_extension`, mandating that extensions declare a security posture. Extensions can also optionally declare `alpha` or `wip` status.
* Tag all documentation sites with their well-known Envoy names.
* Introduce tooling to automagically populate a list of known trusted/untrusted extensions in the threat model docs.
* Generate API docs for extensions that depend on `google.protobuf.Empty`. This pattern is deprecated as per https://github.com/envoyproxy/envoy/issues/8933, but we need these for tooling support meanwhile.
This work was motivated by oss-fuzz issue https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18370
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 90d1094b32aa017f90cc8efcd379aeb143acabfc
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
These were missed in #8125.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1b3b4ae1180b67bee6395fab5c075896fb1964ec
This avoids having to list new docs protos in both docs/build.sh and
api/docs/BUILD. This technical debt cleanup is helpful in v3 proto work
to simplify collecting proto artifacts from a Bazel aspect.
Risk level: Low
Testing: docs/build.sh, visual inspection of docs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b06e2b5c473b4b88e23ed66abd501e29f6bddbee
The libstdc++ std::regex implementation is not safe in all cases
for user provided input. This change deprecates the used of std::regex
in all user facing paths and introduces a new safe regex matcher with
an explicitly configurable engine, right now limited to Google's re2
regex engine. This is not a drop in replacement for std::regex as all
language features are not supported. As such we will go through a
deprecation period for the old regex engine.
Fixes https://github.com/envoyproxy/envoy/issues/7728
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ eff020170c6267e6c8dc235473f7fc85c5b1e07d
This allows using Envoy as a generic HTTP proxy without any
prior configuration of DNS targets. See the included documentation
for more information.
Part of https://github.com/envoyproxy/envoy/issues/1606
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 79e53f21c680b4a02695b1761e7d91868c3481d4
data-plane-api(CircleCI)