*Ability to add custom upstream headers from ratelimit service/filter.
*For LimitStatus::OK, custom upstream headers are added if RLS service sends upstream headers.
Risk Level: Low
Testing:
Unit and integration tests added.
Verified with modified github.com/lyft/ratelimit service.
Passes "bazel test //test/..." in Linux
Docs Changes: protobuf documentation updated
Release Notes: ratelimit: support for adding custom headers to upstream server
from ratelimit service
** Issues: #6141
Signed-off-by: Nandu Vinodan <nandu.vinodan@freshworks.com>
Mirrored from https://github.com/envoyproxy/envoy @ c2a396c84cc97711eef0231f04d81ce680652abb
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
This provides canonical BUILD formatting and puts protoxform in charge
of being able to determine import paths, without having to worry about
Bazel implications.
Part of #8082.
Risk level: Low
Testing: tools/proto_sync.py, visual inspection of diffs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e53f40f0e5ccc84fca5cd350416fe0f2accf8229
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
* [#not-implemented-warn:] was barely used and its purposes are better
served by [#not-implemented-hide:].
* [#proto-status:] was there for an earlier style of versioning, where
APIs were "frozen" or "draft", etc. Now we have semantic versioning
and a regular API clock as per #6271.
Part of #8371.
Risk level: Low (docs only).
Testing: Docs rebuild.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2c4b6f2b3d614a15f312e34f4664ebeb96d07d12
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
This patch introduces a new tool, protoxform, that will be the basis of
the v2 -> v3 migration tooling. It operates as a Python protoc plugin,
within the same framework as protodoc, and provides the ability to
operate on protoc AST input and generate proto output.
As a first step, the tool is applied reflexively on v2, and functions as
a formatting tool. In later patches, this will be added to
check_format/fix_format scripts and CI.
Part of #8082.
Risk level: medium (it's possible that some inadvertent wire changes
occur, if they do, this patch should be rolled back).
Testing: manual inspection of diff, bazel test //test/..., some
grep/diff scripts to ensure we haven't lost any comments.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 08b123a8321d359ea66cbbc0e2926545798dabd3
* Move trailing detached comments that protoc ignores to some anchored place. In the near future,
this will be enforced by check_format.
* Remove comments on reserved fields; we won't be using reserved in v3 and it's a bit of a pain
preserving them.
Risk level: Low
Testing: Docs build, diff inspection, comparison against output of proto transform tool.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 22712d26c427091a47324b5f05fd0b493ee4be01
This change defines C++ grpc bindings for the external AuthZ interface. In the istio Security WG we're planning on using this interface to provide transparent authentication of requests. As this interface lies on the data plane we wish to implement it in C++ to maintain consistent and predictable performance
The changes allow a C++ implementation built using bazel to import Envoy as a workspace dependency and generate the C++ gRPC bindings directly without the need for complex import and generation scripts.
Signed-off-by: Nick A. Smith <nickrmc83@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 41932e9e6f3c932f37f77ae0a5191d65bb7ec8eb
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
These were missed in #8125.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1b3b4ae1180b67bee6395fab5c075896fb1964ec
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Adds support for DNS SAN in ext authz peer validation
Risk Level: Low
Testing: Added
Docs Changes: Added
Release Notes: N/A
Signed-off-by: Rama Chavali <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ d4dc0a5b74acd42aafaaef8bb9c4ebed832674bd
This adds the ability to specify dynamic metadata (by namespace) to
send with the ext_authz check request. This allows one filter to
specify information that can be then used in evaluating an
authorization decision.
Risk Level: Medium. Optional feature/extension of existing filter
Testing: Unit testing
Docs Changes: Inline in attribute_context.proto and ext_authz.proto
Fixes#7699
Signed-off-by: Ben Plotnick <plotnick@yelp.com>
Mirrored from https://github.com/envoyproxy/envoy @ d2e2cd6fa3dd7669f406ae11736f2e179213b449
Signed-off-by: Daniel Hochman <danielhochman@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ c586af9d3f5940acc73d6a5fd0fdc2c521b6243d
Description: Remove an unused dependency from tap. This makes a build warning go away.
Risk Level: low
Testing: build
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 101f8157fe06d78262d0b3c07cf1c2b7c8e72c98
When building protos using the Java protoc, multiple input files mapping
to the same output file causes an erorr. This updates the name of the
generated files for the tap proto files to be unique.
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3c8a1ef9d128bf11ccc179b2e171e180a0861332
Delta services other than DeltaClusters were missing. Also added those services to proto_descriptors.cc. Also added some other things that proto_descriptors.cc was missing. #4991
Risk Level: low
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 40d8b7f68c64591f11d4e62ac429af5e074ed3a7
This PR provides the TDS API config. The implementation and docs will be
added in a followup, the idea is to unblock some projects that depend on
the details here.
Risk level: Low
Testing: @envoy_api build
Part of #6708
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad2cacc4215477526dc61f838db8a7ddf55f4cc6
Description: PR #6211 updated the documentation of CheckResponse.status to reflect Envoy's actual behavior at the time. Later, PR #6505 changed that behavior to be in-line with the pre-6211 docs. So, revert that part of PR #6211.
Risk Level: Low
Testing: None
Docs Changes: Inline in API protos
Release Notes: none
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Mirrored from https://github.com/envoyproxy/envoy @ 5fef1519f4ce3fa2dd1389bfa390e7294182b1e7
Fix a build warning:
INFO: From ProtoGenValidateCcGenerate external/envoy_api/envoy/service/auth/v2/attribute_context.pb.h:
envoy/service/auth/v2/attribute_context.proto: warning: Import envoy/api/v2/core/base.proto but not used.
Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com>
Mirrored from https://github.com/envoyproxy/envoy @ 16e85b1ffc4fceb3c1c3857b96862937126468a8
This PR adds support to `ext_authz` filter for buffering the request data. This is useful when the authorization server needs to check the request body, e.g. HMAC validation.
Fixes#5676
*Risk Level*: low
*Testing*: unit
*Docs Changes*: yes
*Release Notes*: yes
Signed-off-by: Gabriel <gsagula@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 075edf802b9ef94bdf26fddc9e775f8bb89df92d
Update some documentation comments in api/envoy/service/auth/v2/*.proto to
more accurately describe the *current* behavior (without making any
judgment on whether that behavior is "correct" or desirable).
Signed-off-by: Luke Shumaker <lukeshu@datawire.io>
Mirrored from https://github.com/envoyproxy/envoy @ 0ac3706ddf0fda091521bb90cffac7bca8530197
Using proto.MarshalAny results in unstable output due to non-deterministic map ordering. This in turn causes Envoy's diff to reload a config since the hash of the structure changes.
Enable stable marshaler for gogoproto to avoid this problem. See #6252
Risk Level: low
Testing: n/a
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 15a19b9cb1cc8bd5a5ec71d125177b3f6c9a3cf5
Description: Code for Envoy to speak delta CDS with a management server. DELTA_GRPC added to config_source.proto's ApiTypes, to allow bootstrap configs to ask for incremental xDS.
Part of #4991. Was #5466; giving up on broken DCO craziness.
Risk Level: medium
Testing: new integration test
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8116b6ddb53409c3cea6f55fb00367aa43d7e845
Users can now choose between buffered tapping (simpler) and
streaming tapping (more flexible but harder to work with).
Streaming tapping for the transport socket will be added in a
follow-up.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ e2e4baaa85a98b14f2bee6ea5aa16dd79cb832d4
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
Mirrored from https://github.com/envoyproxy/envoy @ 15f1c56397ff404fc98fb3381725bbe48c205234
1) Add request/response body tapping
2) Add buffered body limits (TBI for transport socket)
3) Add the JSON_BODY_AS_BYTES and JSON_BODY_AS_STRING output
formats for convenience when the body is known to be human
readable.
4) Add JSON output for the file per tap sink.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9a06dc0777d2809195cb1fc414b05ae7c0660193
1) Add request/response trailer matching
2) Output request/response trailers
3) Refactor matchers to reduce boilerplate and make it harder
to make mistakes when adding new update functions.
4) Split out a match configuration for HTTP request and response
into individual things like request headers, request trailers,
etc. This will make it easier and more logical to add various
types of body matching and wire it up using the existing and/or
logic.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ fbcf6bcaac6ac3c2b13f8c51fb1470a25b240abd
This commit refactors the tap transport socket to use the common
tap extension configuration and tap matching infrastructure. More
match conditions will be added in a future PR as well as additional
cleanups that have been marked with TODOs.
One result of this PR is that the HTTP tap filter can now have a static
configuration as well as write to a file per tap sink.
All future tap PRs should be smaller and more targeted after this one.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ f37ebdc14f4c0adf0e90aabddae833355c0cec1b
This is a MVP for the HTTP tap filter. It includes minimal
infrastructure for the following:
1. Generic tap configuration which in the future will be used for
static config, XDS config, etc. In this MVP the tap can be
configured via a /tap admin endpoint.
2. Generic output configuration which in the future will be used for
different output sinks such as files, gRPC API, etc. In this MVP
the tap results are streamed back out the /tap admin endpoint.
3. Matching infrastructure. In this MVP only matching on request and
response headers are implemented. Both logical AND and logical OR
matches are possible.
4. In this MVP request/response body is not considered at all.
5. All docs are included and with all the caveats the filter is ready
to use for the limited cases it supports (which are likely still to
be useful).
There is a lot of follow on work which I will do in subsequent PRs.
This includes:
1. Merging the existing capture transport socket into this framework.
2. Implementing body support, both for matching on body contents as
well as outputting body data.
3. Tap rate limiting so too many streams do not get tapped.
4. gRPC matching. Using reflection and loaded proto definitions, it will
be possible to match on gRPC fields.
5. JSON matching. If the body parses as JSON, we can allow matching on
JSON fields.
Part of https://github.com/envoyproxy/envoy/issues/1413.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ cf80045587240d494e54e9772949bc9af5eda61f
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
Part of #2456
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b475977f649c3614371940a819611f7d17e3a4e
- Ability to add custom response headers from ratelimit
service/filter
- For both (LimitStatus::OK and LimitStatus::OverLimit) custom
headers are added if RLS service sends headers
- For LimitStatus:OK, we temporarily store the headers and add
them to the response (via Filter::encodeHeaders())
*Risk Level*: Low
*Testing*: unit and integration tests added. Verified with modified
github.com/lyft/ratelimit service. Passes "bazel test //test/..." in
Linux
Signed-off-by: Suresh Kumar <suresh@freshdesk.com>
Mirrored from https://github.com/envoyproxy/envoy @ 71152b710e3543732464fca57c8f07b7395de68d
Renaming message types in api/envoy/service/discovery/v2/hds.proto to improve readability
Risk Level:
Low
This is for #1310.
Signed-off-by: Lilika Markatou <lilika@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9b33c49d1ca32e73d761849a617b988ed6b596a7
data-plane-api(CircleCI)