Framed was the only transport that supported payload passthrough. However the payload
of Header transport is the same as Framed. Therefore its possible to also use payload
passthrough with Header to Header, Framed to Header and Header to Framed. Therefore
allow those extra three combinations and add integration tests. Note that in future if Header
transforms become supported then passthroughData will need to undo any transforms.
New metrics request_passthrough and response_passthrough are added to show when payload
passthrough occurs. Note that previously response_success was always incremented when
performing payload passthrough, and that is corrected to only occur when parsing the
payload.
This will enable payload passthrough for combinations of downstream/upstream transports that did not
previously perform passthrough. If a private filter does not implement passthroughEnabled or passthroughData
correctly, has enabled payload passthrough and is using Header to Header, Framed to Header or Header to Framed
then the filter may have issues.
Risk Level: Medium
Testing: Extended existing integration tests and verified on local deployment.
Docs Changes: Updated ThriftProxy proto docs.
Release Notes: Minor behavior change.
Platform Specific Features: N/A
Signed-off-by: James Fish <jfish@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0f31648808783b1057c78ff0c90721b81d32e19c
To fix: #13965
grpc_json envoy filter adds a new field 'query_param_unescape_plus` to enable the support of converting "+" to space in query parameters.
The actual unescape code is implemented in grpc-httpjson-transcoding.
Risk Level: None
Testing: integration tests
Docs Changes: None
Release Notes: Yes
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1ba355b5e9f00cda595d5b52e3390d473fa5ecce
Based on the dicsussion in the issue https://github.com/envoyproxy/envoy/issues/17410, this PR adds
implementation for ability to add rbac policy to filter upstream ip addresses for dynamically
resolved domains (by dynamic proxy filter).
Signed-off-by: Jojy George Varghese <jojy_varghese@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 53f95570a7e9d7fa5414a4bc145852524fa017ed
Intel's IPP (Integrated Performance Primitives) crypto library has support for multi-buffer crypto operations. Briefly, multi-buffer
cryptography is implemented with AVX-512 instructions using a SIMD (single instruction, multiple data) mechanism. Up to eight RSA or ECDSA operations are gathered together into a buffer and processed at the same time, providing potentially improved performance. The AVX-512 instructions are available on recently launched 3rd generation Xeon Scalable server processors (Ice Lake server) processors.
This commit adds a private key provider to accelerate RSA and ECDSA crypto operations on recent Intel Xeon processors. Every worker thread has a queue of up-to-eight crypto operations. When the queue is full or when the timer is triggered, the queue is processed and all the pending handshakes are notified.
The potential performance benefit depends on many factors: the size of the cpuset Envoy is running on, incoming traffic pattern, encryption type (RSA or ECDSA), and key size. In my own testing I saw the biggest performance increase when long RSA keys were used on an Envoy running in a fairly limited environment serving lots of new incoming TLS requests. For more details, see this Intel whitepaper which contains some more information about the AVX-512 instructions and potential performance increase: https://www.intel.com/content/www/us/en/architecture-and-technology/crypto-acceleration-in-xeon-scalable-processors-wp.html
Additional Description:
One new dependency is introduced: Intel’s ipp-crypto library. Currently the PR is using a development version of ipp-crypto because BoringSSL support is not yet part of any release. The ipp-crypto team has indicated that BoringSSL version will be included in future ipp-crypto releases.
Basic tests are provided, and a fake library interface is included for testing on systems without the required AVX-512 instruction set.
Risk Level: Medium (TLS security feature, not enabled by default)
Testing: Unit tests
Docs Changes: API interface is documented
Release Notes: Added CryptoMB private key provider to contrib.
Platform Specific Features: Requires Intel 3rd generation Xeon Scalable server processor for the AVX-512 IFMA instruction set.
Fixes: #15871
Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com>
Co-authored-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2144166ca7a3f100ecae16700bc82920b2de4871
This is a first step towards https://github.com/envoyproxy/envoy/issues/17920
A single proto (kafka mesh) has been swapped from using the udpa
file_status annotation to the xds file_status annotation to avoid a
large amount of churn and a forthcoming migration of many alpha/wip
protos to non alpha/wip. The rest will be audited and swapped in
future PRs. This single one was done to make sure the doc machinary
works properly.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ f0f17a3caa75106a9e28b99edc27dd09c1bed488
This allows for dynamic metadata when using an HTTP authorization service. Currently, it is only possible to set dynamic metadata using a gRPC authorization service.
Risk Level: low, new opt-in feature in an extension
Testing: unit tests
Docs Changes: proto definitions documented
Release Notes: ext_authz: added :ref:dynamic_metadata_from_headers <envoy_v3_api_field_extensions.filters.http.ext_authz.v3.AuthorizationResponse.dynamic_metadata_from_headers> to support emitting dynamic metadata from headers returned by an external authorization service via HTTP.
Signed-off-by: John Esmet <john.esmet@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 31f48ee7a3c32ed383b46c8940263af9dc9bdcd4
This PR adds some more description around max_connection_duration in the Timeout FAQs.
Commit Message: add more description for max_connection_duration in timeout faqs.
Additional Description: -
Risk Level: N/A
Testing: N/A
Docs Changes: Added some more description around max_connection_duration in the Timeout FAQs.
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2e8efdefef86f7749aee3ca30d0752218a07ca8a
Risk Level: low
Testing: new unit tests. integration test TODO in a follow up
Docs Changes: n/a
Release Notes: will land with integration test.
Fixes#18034
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 7f63632be6b08193550041c4069e63f557b16c5d
upstream: Make the Alt-Svc cache configuration required if HTTP/3 is enabled with AutoHttpConfig.
Risk Level: Low
Testing: Unit tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Ryan Hamilton <rch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 19102ffc814204d2d2f98747afa3af3a84a4024c
This patch adds header_to_metadata field to JwtProvider config to allow
setting the extracted header of a successfully verified JWT to dynamic
metadata.
Signed-off-by: Dhi Aurrahman <dio@rockybars.com>
Mirrored from https://github.com/envoyproxy/envoy @ 25622ce26532ad42d93a383d48b6c86d9e28d3d2
Removal of params or cookies after authentication is not implemented as of today.
authenticator.cc calls the removeJwt(...) if forward is set to false (default)
and this leads to an assertion failures caused by NOT_IMPLEMENTED_GCOVR_EXCL_LINE.
Changed removeJwt(...) for JwtCookieLocation to be empty, added test coverage
and updated proto doc to call-out this caveat.
Signed-off-by: Shubham Patil <theshubhamp@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5591dbc0f6d663641241f1919dc6135668c1da75
http: Unhide the AlternateProtocolCache configuration
and move some docs from http3_upstream.md to connection_pooling.rst
Risk Level: N/A - Docs only
Testing: N/A - Docs only
Docs Changes: Unhide the AlternateProtocolCache configuration and move some docs from http3_upstream.md to connection_pooling.rst
Release Notes: Added
Platform Specific Features: N/A
Signed-off-by: Ryan Hamilton <rch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d7fdbbc567b5e3175240c1fa9827255275c53c36
This was built for Envoy Mobile, but will allow generic modification
of router behavior between retries. Currently it only supports modifying
upstream socket options (to in practice impact interface binding), but
in the future is likely to be extended to modify timeouts, retry back
off times, request headers, etc.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5b1015cffbf6e02a7ccec9c12a6b7f246f8f3d30
Allows ensuring continual progress of individual request attempts.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ b500a0a655809f518ed30a05ceb7262a02dbc537
Adds a new optional param called override_auto_sni_header which can be used to populate the upstream SNI value from an arbitrary header other than Host/Authority.
Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5c146696351be2bac731abd501333fbac824d919
This includes validation for upgrade connects per Ryan's offline advice.
n.b. this should be a no-op for HTTP (where there is no mechanism to send both) and HTTP/2 (where nghttp2 validates) so not currently calling out in release notes.
Risk Level: low
Testing: new integration tests
Docs Changes: inline
Release Notes: n/a (quic alpha)
co-author: @DavidSchinazi
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 2d934e1e618714bab0ae7f2dcfd6a0531212dd6c
Before this, when a gRPC server sends out DeniedResponse as a check
response for a request but without setting the
HttpResponse.DeniedResponse.Status, HTTP ext_authz filter translates
that as "0" (empty/unknown HTTP status code). This patch makes sure we
reply with a valid 403 Forbidden HTTP status code (the current default
status code for denied response).
Signed-off-by: Dhi Aurrahman <dio@rockybars.com>
Mirrored from https://github.com/envoyproxy/envoy @ e3af0943fff8444660a797cce6a78fd956e3c2cf
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0c8942dd82e3991d5162c4c53ec5b2e5c1d71010
Today "disabled" and the processing mode are the two things supported
because that is what is currently supported in ext_proc.
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 01ea37b280d980c25bb9e2c8069d62e4aef37961
Field 2 in the `DnsFilterConfig` message was released in Envoy 1.18,
and repurposed in Envoy 1.19. Renumber the field, while leaving a
compatibility field so that control planes can gracefully migrate to a
subsequent 1.19 release.
This fixes#17921.
Signed-off-by: James Peach <jpeach@apache.org>
Mirrored from https://github.com/envoyproxy/envoy @ ff9704e502184c4415d38351795f0d5d11627fcd
Support extraction of JWT from Cookies in JWT Extension
Added "from_cookies" config directive to jwt_authn that enables JWT extraction from request cookies.
Risk Level: low
Testing: unit tests
Docs Changes: Updated `docs/root/configuration/http/http_filters/jwt_authn_filter.rst`
Release Notes: Updated `docs/root/version_history/current.rst`
Platform Specific Features: None
Fixes#17424
Signed-off-by: Shubham Patil <theshubhamp@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 143083c3c3e78c9252b644648ef5be5cf9158419
- Removed the JsonV1Serializer and added exception for HTTP_JSON_V1version in Serializer creation.
- Specified collector_endpoint_version in the test
- Removed the default value for endpoint_, it is redundant here because in the zipkin.proto, it has PGV rule [(validate.rules).string = {min_len: 1}], so we always need to provide the config for this field.
- Modified the ConstructBuffer test to create the buffer using ZipkinConfig::HTTP_JSON
Risk Level: LOW
Testing: CI
Signed-off-by: Tianyu Xia <tyxia@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ db74e313b3651588e59c671af45077714ac32cef
Update the 14 day API compatibility exception to make it clear that
whether a field has already been included in an Envoy release may
affect whether incompatible changes should be made.
This updates #17920.
Signed-off-by: James Peach <jpeach@apache.org>
Mirrored from https://github.com/envoyproxy/envoy @ 702cadcd8ba887744f5eb5541d6560ddef984976
`context_extenstions` map has the potential to contain secrets which need to be redacted from config dumps.
Risk Level: Low
Signed-off-by: Paul Gallagher <pgal@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5d9b58b4342448031f1343b670613a88c05da8b9
This PR is a continuation to #17515 - it adds a script that uses buf to check for breaking changes on proto files in the api folder. It does so by comparing the current api folder against the api folder at the git commit computed by tools/git/last_github_commit.sh - that should hopefully represent the most recent commit on main (if there is a better method to obtain the base branch commit, let me know!).
Adding the script also required re-organizing some of the breaking change detector logic from the previous pr: some levels of abstraction were added, and the detector now expects a git repository and ref as the input for initial state (rather than a proto file).
The script is invoked in do_ci.sh if bazel.api_compat is specified as the CI_TARGET.
This PR also bumps the buf bazel dependency to 0.53.0. If this is preferred to be in a separate PR, let me know and I would be happy to do so
Risk Level: low (hopefully) - the CI script will be invoked in a separate CI pipeline job that can be set to be optional on github. The azure pipeline has been added but needs to be set to optional by a CI maintainer
Testing: New scripts and logic were tested manually; also ran tests from the previous PR and they still pass. hoping to observe more output of this tool through reading CI logs of other PRs once this is merged (this PR should not affect the existing PR workflow - refer to Risk Level)
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: CI script uses a linux binary for buf so it cannot be run outside of docker on non-linux systems
Fixes#3368
Signed-off-by: Yaseen Alkhafaji <yaseena@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ f30c289e456df11d549d7236c56f22da00500e9e
- Add cluster header to weighted cluster proto. This enables the ability of routing to the cluster specified in the request_header for weighted cluster (i.e. multiple upstreams) case. name and cluster_header will be oneof cluster_specifiter field in the next release. Currently, they can only be regular fields due to compatibility issue.
- Add the validation logic (PGV annotation + runtime check)
- Add 3 unit tests for invalid weighted cluster configs.
Note: I have implemented the core implementation and tested it with end-to-end
integration test/example locally. Those changes will be in a separate PR once this API change is approved.
Risk Level: Low
Testing: CI, added an integration test for implementation of this API locally
Signed-off-by: Tianyu Xia <tyxia@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ef70185f3ac241d09eabfe15c65b0d631134f74f
Stop generating v4alpha protos as it won't land in foreseeable future.
This fixes go-control-plane sync because of it fails to generate contrib API correctly.
Risk Level: Medium
Testing: CI
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ e453c6c613206da749e3ff645e2d92f534535f3e
* config/router/tcp_proxy/options: v2 API, boosting and --boostrap-verison CLI removal.
This PR cleans up a bunch of v2 tech debt, including version conversion and API oracle
code:
* Remove all non-trivial field references to hidden_envoy_deprecated_*. These were primarily
in tcp_proxy and router.
* Remove boosting, hidden type annotation and the API type oracle. We are now only v3 for
both resource and transport API version.
* Remove --boostrap-version flag (all boostraps already must be v3).
Risk level: High (It's possible there were some runtime guarded or not dependencies on
boosting).
Testing: Validated test functional coverage when removing hidden_envoy_deprecated_* field tests.
Releasing notes: TODO
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d142c9d55ae9aab34e9924aa25f20bd27635e060
Follow-up to #17375 where it was agreed that protolock is not actively maintained enough to depend on. This PR "migrates" the tests from that PR to use buf instead, and also cleans some of the code per a few of the review comments. Still a few outstanding points:
- buf build on the envoy/api folder requires several protobuf dependencies such as udpa to be available to buf to consume. Suggested solution by buf is to point buf's config to necessary BSR modules that the buf team is hosting.
- These lines are commented out in this PR as I had some trouble automating it for the tests, and it is not necessary for the tests to pass
- May introduce issues if buf is not pointing to the same version of modules that bazel builds for envoy. May need to introduce some way to couple them, or (ideally) find a way to run the breaking change detector without building the dependencies
- Currently bazel is using a binary release of buf (for linux). Goal is to move to building it from source in the near future
- It may be in our interest to expand the list of API-breaking-change rules (buf provides an extensive list of rules we could adopt)
Risk Level: Low
Testing: Tests that evaluate buf against "allowed" and "breaking" protobuf API changes. Currently 4 tests are skipped - 3 of them are PGV-related (we need to communicate our desired PGV rules to the buf team so they may add them in the near future). The 4th is a test I had originally written to evaluate protolock but may not apply to buf ("forcing" a breaking change) - refer to comments
Docs Changes:
Release Notes:
Platform Specific Features: buf binary imported by bazel is linux-only. Hopefully the ["manual"] tags attribute prevents any issues for non-linux users
Signed-off-by: Yaseen Alkhafaji <yaseena@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c74cebb3cc7dfb26488f7cac8eb9258cf9a7588e
data-plane-api(Azure Pipelines)