Intel's IPP (Integrated Performance Primitives) crypto library has support for multi-buffer crypto operations. Briefly, multi-buffer
cryptography is implemented with AVX-512 instructions using a SIMD (single instruction, multiple data) mechanism. Up to eight RSA or ECDSA operations are gathered together into a buffer and processed at the same time, providing potentially improved performance. The AVX-512 instructions are available on recently launched 3rd generation Xeon Scalable server processors (Ice Lake server) processors.
This commit adds a private key provider to accelerate RSA and ECDSA crypto operations on recent Intel Xeon processors. Every worker thread has a queue of up-to-eight crypto operations. When the queue is full or when the timer is triggered, the queue is processed and all the pending handshakes are notified.
The potential performance benefit depends on many factors: the size of the cpuset Envoy is running on, incoming traffic pattern, encryption type (RSA or ECDSA), and key size. In my own testing I saw the biggest performance increase when long RSA keys were used on an Envoy running in a fairly limited environment serving lots of new incoming TLS requests. For more details, see this Intel whitepaper which contains some more information about the AVX-512 instructions and potential performance increase: https://www.intel.com/content/www/us/en/architecture-and-technology/crypto-acceleration-in-xeon-scalable-processors-wp.html
Additional Description:
One new dependency is introduced: Intel’s ipp-crypto library. Currently the PR is using a development version of ipp-crypto because BoringSSL support is not yet part of any release. The ipp-crypto team has indicated that BoringSSL version will be included in future ipp-crypto releases.
Basic tests are provided, and a fake library interface is included for testing on systems without the required AVX-512 instruction set.
Risk Level: Medium (TLS security feature, not enabled by default)
Testing: Unit tests
Docs Changes: API interface is documented
Release Notes: Added CryptoMB private key provider to contrib.
Platform Specific Features: Requires Intel 3rd generation Xeon Scalable server processor for the AVX-512 IFMA instruction set.
Fixes: #15871
Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com>
Co-authored-by: Greg Greenway <ggreenway@apple.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2144166ca7a3f100ecae16700bc82920b2de4871
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0c8942dd82e3991d5162c4c53ec5b2e5c1d71010
This PR is a continuation to #17515 - it adds a script that uses buf to check for breaking changes on proto files in the api folder. It does so by comparing the current api folder against the api folder at the git commit computed by tools/git/last_github_commit.sh - that should hopefully represent the most recent commit on main (if there is a better method to obtain the base branch commit, let me know!).
Adding the script also required re-organizing some of the breaking change detector logic from the previous pr: some levels of abstraction were added, and the detector now expects a git repository and ref as the input for initial state (rather than a proto file).
The script is invoked in do_ci.sh if bazel.api_compat is specified as the CI_TARGET.
This PR also bumps the buf bazel dependency to 0.53.0. If this is preferred to be in a separate PR, let me know and I would be happy to do so
Risk Level: low (hopefully) - the CI script will be invoked in a separate CI pipeline job that can be set to be optional on github. The azure pipeline has been added but needs to be set to optional by a CI maintainer
Testing: New scripts and logic were tested manually; also ran tests from the previous PR and they still pass. hoping to observe more output of this tool through reading CI logs of other PRs once this is merged (this PR should not affect the existing PR workflow - refer to Risk Level)
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: CI script uses a linux binary for buf so it cannot be run outside of docker on non-linux systems
Fixes#3368
Signed-off-by: Yaseen Alkhafaji <yaseena@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ f30c289e456df11d549d7236c56f22da00500e9e
This PR addresses several issues:
1. MetadataFormatter base class takes a pointer to a function to access specific type of metadata. All other methods operating on internals of metadata structure are moved to base class. This simplified adding new types of metadata.
2. new type of access_log METADATA tag is introduced to handle all types of metadata. It is implemented as formatter extension. It handles existing tags DYNAMIC_METADATA and CLUSTER_METADATA, so two ways of accessing dynamic and cluster metadata are allowed now. Maybe we should obsolete DYNAMIC_METADATA and CLUSTER_METADATA in the future?
3. Adds handler for ROUTE metadata. See issue #3006.
Follow up to #16121
Risk Level: Low
Testing: Unit tests.
Docs Changes: Yes.
Release Notes: Yes.
Platform Specific Features: No
Signed-off-by: Christoph Pakulski <christoph@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 43b597e719230c8266998394301c5d0b3c862688
Input matcher that checks that an IP{v4,v6} belongs to a list of CIDR ranges
Signed-off-by: Adrien Guinet <adrien@reblaze.com>
Mirrored from https://github.com/envoyproxy/envoy @ 6cd6a0cab68c757c785e62e0d73d881f57c97839
Commit Message: make quic proof source and crypto streams extensions. Add config for default ones. If not specified in config, the default ones will be used.
Risk Level: low
Testing: existing tests passed
Part of #2557
Co-authored-by: Dan Zhang <danzh@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ beb5a93b08bd0c48a2a7dd2f40ca13bcdb0ed40e
alternate_protocols_cache: Create a filter for parsing Alt-Svc headers and populating the AlternateProtocolsCache.
Risk Level: Low
Testing: Unit and Integration
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
Signed-off-by: Ryan Hamilton <rch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e35a790bc3619fe304a3904ef81112db0a7e7b08
the filter smoothens the flow of data in both direction up to the specified bandwidth limit.
Risk Level: Low, new filter
Testing: UTs added. Adding more UTs and integration tests.
Docs Changes: Added
Release Notes: Added
Fixes#13604
Signed-off-by: Nitin Goyal <nigoyal@microsoft.com>
Mirrored from https://github.com/envoyproxy/envoy @ a467b0ef41864459f4b4ad76fad7e1efb06b0bf8
This extension simply allows to add user-defined dynamic metadata. This is intended to be used with matchers, so that the "result" of a matching process can imply specific metadata.
See issue #16266
Risk Level: Medium
Testing: Unit tests for the extension are added
Docs Changes: A small documentation for the extension is added in docs/root/configuration/http/http_filters
Signed-off-by: Adrien Guinet <adrien@reblaze.com>
Mirrored from https://github.com/envoyproxy/envoy @ cbe08bbe33005c1832964f7c9e657d6dbe105bbb
This is a follow-up to:
#14432 (comment)
After that PR, it's no longer possible (unless you do a dynamic_cast)
to set the remote address from a filter. This is something that we
need to do because we have specialized logic for this (XFF doesn't
work for us).
So this adds an extension point which will allow us to push that logic
down to ConnectionManagerUtility::mutateRequestHeaders() where it
belongs.
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ beac1ece7512e6e39b4f1c29490e247996a0f51c
Config proto for Secure Session Agent (S2A) transport socket extension (#16110).
Signed-off-by: Tavish Vaidya <tavish@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ abe2f9b4af4b3c697ee8b3fda934aa2e181bc5ef
Adds a new filter that makes use of the new matching API to allow specifying a filter configuration to use for specific match
results. Once a configuration is selected, the desired filter is created and all callbacks delegated to the specific filter.
Risk Level: Low, new filter
Testing: IT + UT
Docs Changes: Inline proto comments
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ d31ba23d494c310973ceaf1e053393697821e4d3
Adds support for a "generic input" extension point that allows specifying inputs that are not dependent on protocol data.
Adds an environment variable generic input that allows matching on the value of an environment variable.
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 758a9a90c15331fbfd093f23170454c5a2fbf903
This PR adds an OTLP logger to Envoy, which allows users to flexibly configure their log structure (according to OpenTelemetry protos).
The API users the common gRPC configuration, used in ALS, for gRPC settings. The flexible OT fields (body, attributes) are completely configurable and are filled with request/response information based on Envoy access log formatting.
There are still a couple of TODOs -
The 'filter_state_objects_to_log' from the common gRPC config isn't added to the log
Resource attributes are not configurable, and only Envoy node info is added.
Note: this extension was added as v3alpha API.
Risk Level: Low - a new extension, so unlikely to break any existing setups (marked as v3alpha API).
Testing: Unit tests.
Docs Changes:
Release Notes: access log: added a new :ref:OpenTelemetry access logger <envoy_v3_api_msg_extensions.access_loggers.open_telemetry.v3alpha.OpenTelemetryAccessLogConfig> extension, allowing a flexible log structure with native Envoy access log formatting.
Signed-off-by: Itamar Kaminski <itamark@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4f52f9ef554e1522e763c0eaf6ef6833aab283e5
This introduces a new matcher that allows matching on an input value by
computing a hash value and matching if the value % (configured value) is
greater than a configured threshold. This is useful in being able to
define match criteria that should match for a certain % of input values
in a way that is consistent between independent Envoy instances (e.g. it
does not rely on a random input).
Risk Level: Low, new extension
Testing: UTs
Docs Changes: Inline proto docs
Release Notes: n/a
Platform Specific Features: n/a
Fixes#14782
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7fe3d358f7e7e4d4c7282d50c498a7ab0e759a36
* Configuration for path normalization so that normalization can be configured for internal only use and/or forwarding.
Signed-off-by: Asra Ali <asraa@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 236107df5d15381ccf5040c0cb0f163ece9627a2
1) Promote the default UUID request_id implementation to
an actual extension that is required in the build and wire
up all documentation.
2) Add a configuration option to the extension that allows trace
reason packing to be disabled (the default continues to be for it
to be enabled to match existing behavior).
3) Update all documentation for the new behavior.
4) Substantial cleanup of these code paths for clarity and robustness.
Fixes https://github.com/envoyproxy/envoy/issues/11532
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 07c4c17be61c77d87d2c108b0775f2e606a7ae12
* Add v3 equivalents of v2 configs that were included in v3 due to no
transitive deprecation. This increases consistency and reduces user
confusion. We will continue to support these straggler v2 configs
beyond the v2 turndown due to the late addition of v3 counterparts,
special case code is added to utility.cc to handle this.
* There were two extensions, //envoy/config/cluster/redis and
//envoy/config/retry/previous_priorities, that for some reason were
not upgraded to use v3 config. This is now fixed and I've grepped for
other v2 in //source, none remain.
Risk level: Medium (changes to extension config types and deprecated
config handling).
Testing: Additional unit test added for utility.cc handling, upgraded
configs to v3 for other tests.
Fixes#14735Fixes#12841
Signed-off-by: Harvey Tuch <htuch@google.com>
Co-authored-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ c04a75efe9d601ebcb9650f274b5adde60181f1c
Commit Message: compression: add brotli compressor and decompressor
Additional Description: Add new brotli compression extensions in addition to gzip.
Risk Level: Low, no existing functionality is touched
Testing: uni tests, manual tests with curl.
Docs Changes: updated docs for compression and decompression HTTP filters to refer the new available encoder/decoder.
Release Notes: updated current.rst
Fixes#4429
The PR adds a new dependency on https://github.com/google/brotli. Here's the current criteria answers:
| Criteria | Answer |
|---------|---------|
| Cloud Native Computing Foundation (CNCF) approved license | MIT |
| Dependencies must not substantially increase the binary size unless they are optional | brotli's binary size built with `-c opt` is 752K |
| No duplication of existing dependencies | no other dep provides Brotli |
| Hosted on a git repository and the archive fetch must directly reference this repository. | https://github.com/google/brotli |
| CVE history appears reasonable, no pathological CVE arcs | so far 4 CVEs related to brotli have been registered |
| Code review (ideally PRs) before merge | PRs are reviewed before merge |
| Security vulnerability process exists, with contact details and reporting/disclosure process | no policy exists, submitted https://github.com/google/brotli/issues/878 |
| > 1 contributor responsible for a non-trivial number of commits | 75 contributors |
| Tests run in CI | CI set up with AppVeyor and Github actions |
| High test coverage (also static/dynamic analysis, fuzzing) | Fuzzers are run in CI |
| Envoy can obtain advanced notification of vulnerabilities or of security releases | brotli is registered in CPE |
| Do other significant projects have shared fate by using this dependency? | Google Chrome is using the library |
| Releases (with release notes) | https://github.com/google/brotli/releases |
| Commits/releases in last 90 days | last commit 9 days ago |
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 127aa55306c9e4366e10b3a6e26d01df826e2fc0
part of #12841
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ f1638fbed36a1416d3b44783eedfb7ef6fb39f17
Introduces the FilterDependency proto. This isn't quite an extension, but it's a common proto to be used by all filter extensions.
Risk Level: Low (proto addition only)
#14470
Signed-off-by: Auni Ahsan <auni@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2da224f88afe5ec84ba802bd932f73ed13308a92
part of #12841
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ 77d7cecb7e68cb41b2f5f7c2aa7f4294c88089ce
Risk Level: LOW
Testing: unit ( proto_format and docs )
part of #12841
Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Mirrored from https://github.com/envoyproxy/envoy @ c15193642432291f13c6358ed8c4850fba3a8969
Adds support for associating a match tree with a HTTP filter, supporting a single "Skip" operation that will have the FM
ignore the filter for the duration of the stream once matched.
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 935a6598cd01324f03608ca77ebffc9608f7af81
This introduces a new filter called the "external processing filter." It is intended to allow an external service to be able to operate as if it were part of the filter chain using a gRPC stream. It is intended to support a variety of use cases in which processing of HTTP requests and responses by an external service is desired.
A document that describes the filter can be found here:
https://docs.google.com/document/d/1IZqm5IUnG9gc2VqwGaN5C2TZAD9_QbsY9Vvy5vr9Zmw/edit#heading=h.3zlthggr9vvv
Signed-off-by: Gregory Brail <gregbrail@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 98d2f3b553b87c3e935f57ba15b4faf68b45d7f0
data-plane-api(Azure Pipelines)