As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
* [#not-implemented-warn:] was barely used and its purposes are better
served by [#not-implemented-hide:].
* [#proto-status:] was there for an earlier style of versioning, where
APIs were "frozen" or "draft", etc. Now we have semantic versioning
and a regular API clock as per #6271.
Part of #8371.
Risk level: Low (docs only).
Testing: Docs rebuild.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2c4b6f2b3d614a15f312e34f4664ebeb96d07d12
We should start tagging messages with "option deprecated = true;" if we
want them to be auto-deprecated by protoxform going forward.
Risk level: Low (v3alpha is not used yet)
Testing: bazel test @envoy_api//..., manual diff inspection.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e3247539bcc39f3111287b479a85e51655a6f1db
This takes protoxform from being a v2 format tool to being able to
generate something similar to what ./api/migration/v3alpha.sh generates.
Risk level: Low (v3alpha not used yet)
Testing: fix_format, manual verification that the delta from
./api/migration/v3alpha.sh makes sense (for now).
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 89539d31e9fe749418d499352897c7acb474fab2
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
This patch introduces a new tool, protoxform, that will be the basis of
the v2 -> v3 migration tooling. It operates as a Python protoc plugin,
within the same framework as protodoc, and provides the ability to
operate on protoc AST input and generate proto output.
As a first step, the tool is applied reflexively on v2, and functions as
a formatting tool. In later patches, this will be added to
check_format/fix_format scripts and CI.
Part of #8082.
Risk level: medium (it's possible that some inadvertent wire changes
occur, if they do, this patch should be rolled back).
Testing: manual inspection of diff, bazel test //test/..., some
grep/diff scripts to ensure we haven't lost any comments.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 08b123a8321d359ea66cbbc0e2926545798dabd3
This PR does a few things:
1) Adds per-worker listener stats, useful for viewing worker
connection imbalance.
2) Adds per-worker watchdog miss stats, useful for viewing per
worker event loop latency.
3) Misc connection handling cleanups.
Part of https://github.com/envoyproxy/envoy/issues/4602
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 483aa09545a55853fa41710f80ceff23fcad290d
* Move trailing detached comments that protoc ignores to some anchored place. In the near future,
this will be enforced by check_format.
* Remove comments on reserved fields; we won't be using reserved in v3 and it's a bit of a pain
preserving them.
Risk level: Low
Testing: Docs build, diff inspection, comparison against output of proto transform tool.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 22712d26c427091a47324b5f05fd0b493ee4be01
When trailer indicates a gRPC error and there was no HTTP body, with the `convert_grpc_status option` enabled, take `google.rpc.Status` from the `grpc-status-details-bin` header and use it as a JSON body.
If there was no such header, make `google.rpc.Status` out of the `grpc-status` and `grpc-message` headers.
This also adds `google.rpc.Status` to user-provided protobuf descriptor.
Risk Level: Small-medium
Testing: Added unit and integration tests tests, tested manually.
Docs Changes:
Added field description in api/envoy/config/filter/http/transcoder/v2/transcoder.proto
Release Notes:
Fixes#3383
Signed-off-by: Anatoly Scheglov <ascheglov@yandex-team.ru>
Mirrored from https://github.com/envoyproxy/envoy @ 219e8b91809d14cfea2c064c59a5ac98647840b1
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
This PR adds a configuration option for controlling the length of the request path that is included in the HttpUrl span tag. Currently, this length is hard-coded to 256. With this PR, that length will be configurable (defaulting to the old value).
Risk Level: Low
Testing: Unit
Docs Changes: Inline with the API proto. Documented new field.
Release Notes: Added in the PR.
Related issue: istio/istio#14563
Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 277e7175091cdf9d6c8732bc40381ff4e0fee48c
Use the listener-level field for the tracing direction over the per-filter field. Unfortunately, the per filter did not provide an "unspecified" default, so this appears to be the right approach to deprecate the per-filter field with minimal impact.
Risk Level: low (uses a newly introduce field traffic_direction)
Testing: unit test
Docs Changes: proto docs
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4478c1984d17146b1ff78d0babedae2a4752b027
These were missed in #8125.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1b3b4ae1180b67bee6395fab5c075896fb1964ec
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Default behavior remains unchanged, but now Envoy can override, override iff there's no server header from upstream, or always leave the server header (or lack thereof) unmodified.
Risk Level: low (config guarded change)
Testing: new unit tests
Docs Changes: n/a
Release Notes: inline
Fixes#6716
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ b8966cbbfff4566b08eaed7b2b060a6c12eec168
Add PGV bounds to the stats flush interval (greater than 1ms and less than 5000ms) to prevent Envoy from hanging from too small of a flush time.
Risk Level: Low
Testing: Corpus Entry added
Fixes OSS-Fuzz issue
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16300
Signed-off-by: Asra Ali <asraa@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0a3fc6a091a768ba868ff4dd94cf526af4ffc0ff
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 816d6f14fd65329a8511351e74b385fafa3160a8
Some BUILD files are missing build rules to generate go protos. envoyproxy/go-control-plane depends on these protos, so they should be exposed publicly. Added build rules to generate *.pb.go files.
Risk Level: Low
Testing: These rules were copied to google3 and tested internally. Unfortunately, I am having a bit of trouble with bazel build directly on these targets ("Package is considered deleted due to --deleted_packages"). Please let me know if there is a better way to test this change.
Signed-off-by: Teju Nareddy <nareddyt@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c7110f8d10d928a3b18ee6a05439d7b4e49e595c
This adds the ability to specify dynamic metadata (by namespace) to
send with the ext_authz check request. This allows one filter to
specify information that can be then used in evaluating an
authorization decision.
Risk Level: Medium. Optional feature/extension of existing filter
Testing: Unit testing
Docs Changes: Inline in attribute_context.proto and ext_authz.proto
Fixes#7699
Signed-off-by: Ben Plotnick <plotnick@yelp.com>
Mirrored from https://github.com/envoyproxy/envoy @ d2e2cd6fa3dd7669f406ae11736f2e179213b449
Introduces a generic expression-based admission filter using https://github.com/google/cel-cpp.
This is a follow-up to discussion in https://github.com/envoyproxy/envoy/issues/6751.
The advantage of this approach is:
1. Un-opinionated about the policy structure since the only config is an expression. This is friendly towards control planes which can bear the complexity of translation, analysis, and evolution of policies.
2. Multi-language, CEL supports go, java, and c++ runtimes.
3. Inter-operability with other filters using request `metadata`. Companion filters can populate metadata about requests and resources that affect policy decisions.
4. Generic utility, it can be used for custom metric labels, access log entries, etc.
The dis-advantage of this approach is that its performance is lower than domain-optimized interpreters. On a fair example, the interpreter evaluates in around 1ms (see https://github.com/google/cel-cpp/blob/master/eval/tests/benchmark_test.cc#L591) vs ~150ns for hand-written C++ native code. There is space for improvement (especially if WASM can be used as a compilation target), but ultimately the generic expression form carries a cost.
Conditions are added to support RBAC filter for complementing the existing principal/permission model. They add support for the extended checks (e.g. time of query, resource-bound), but add no cost unless used.
Description: add expression-based admission filter
Risk Level: low
Testing:
Docs Changes:
Release Notes:
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ f90e1b08ac5b4973c45a6529780ebdd211ff901f
Description: Adds support for DNS SAN as Principal in RBAC filter.
Risk Level: Low
Testing: Added automated tests
Docs Changes: Updated
Release Notes: Added
Fixes#7836
Signed-off-by: Rama Chavali <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 882a30677619856446f7e1b9d28c6ab319b21d1b
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3a75316f2fa495fc7be36efd4f291445ac7b857
For the first point in #7771 for converting arbitrary protobuf value from header to metadata.
Risk Level: Low
Testing: Unit tests
Docs Changes: Updated version_history.rst
Release Notes: Updated version_history.rst
Signed-off-by: Yangmin Zhu <ymzhu@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9e552c5f0a86542eaae897d8f27bd34be85ad1df
Let the config override the Stackdriver address. This can be used for
proxying and testing.
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3e63182c3e74e0d517d2f0d1fc2ee950bbfe21e1
Promote tracing operation field to listener level. This expands the scope of the field to support two use cases:
Tracing TCP connections: istio can send connection events to create a service communication graph. Network filters can benefit from the common knowledge about the intent of the listener/filter chain (client-side vs server-side).
Using ingress/egress designation for other telemetry. The direction of the traffic is a useful label on metrics, and it is not explicit at the moment, unless depending on tracing configuration in HTTP connection manager or naming convention. Both workarounds are not ideal.
Risk Level: low
Testing: all unit tests continue to pass
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ca2af9723598fab4f511b59407396cc5cff9ed94
Due to a seg fault issue with the gogo protobuf library
[https://github.com/gogo/protobuf/issues/568], non nullable repeated
fields in a proto will cause proto.Merge(dst, src) to panic.
The nullable field setting was first added by @kyessenov when he was
re-organizing the protos. Unfortunately, people have been copy pasting it
across several areas in the Envoy proto. To keep the impact radius to a minimum,
I have updated only the fields that are currently causing the segfault
(in go-control-plane) for us.
Its also partly against proto principles. You should be able to determine if
a field is set or not. This non-nullable setting in gogo will insist on initializing
the field to default values.
Risk Level: to go control plane users
Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b22d2b5cf09f779962cfedaaab24969f384cbc48
* grpc-json: add support for ignoring unknown query parameters
Current behavior is not to transcode a request that contains a query parameter
that cannot be mapped. In cases where there is a specific set of parameters to
ignore one can use ignored_query_parameters. If there is a proxy or client
adding query parameters that one cannot control this won't work and the
transcoder becomes useless. ignore_unknown_query_parameters solves this problem.
Risk Level: low to medium
Testing: Added a unit test for the transcoder.
Tested manually, locally.
Docs Changes:
Added field description in api/envoy/config/filter/http/transcoder/v2/transcoder.proto
Release Notes:
Signed-off-by: Neri Marschik <codesuki@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 6af53be976c79a2b51a3f55825b722b58686c8a0
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ ea3ebca3b6d84a8b29c35ca03fa3666a0f4951c9
All http filters have build rules to generate cc protobufs, but not go protobufs. Added build rules (to a few filters) to generate go protobuf files. Emulates the rules in the health_check http filter.
Risk Level: Low
Testing: These rules were copied to google3 and tested internally. Unfortunately, I am having a bit of trouble with bazel build directly on these targets ("Package is considered deleted due to --deleted_packages"). Please let me know if there is a better way to test this change.
Signed-off-by: Teju Nareddy <nareddyt@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 23d82b9d14a6cf9f49ebcd3ae584fe3079f597d1
data-plane-api(CircleCI)