See envoyproxy/go-control-plane#824 for more information
This PR adds the vtprotobuf protoc plugin for Go. This works on top of the existing protoc-gen-go, to add optimized marshal functions that callers can opt in to using. This is not like gogo, which was a very invasive change -- everything is layered and opt-in. See issue for benchmarks, etc.
Additionally, to avoid possible binary size increase, the entire new code is protected under a go build tag. Users will need to opt-in at build time (-tags=vtprotobuf). By default, there is no impact for users at all.
Risk Level: Low - only additional opt-in code
Testing: Manually tested in Istio codebase
Signed-off-by: John Howard <howardjohn@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 21b52ba73d8ebbb51834d529a68f55ea2ec5e614
* Update googleapis to 114a745 and xDS API to 46e39c7
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8319b9287e5324a4f24bfd2072493c1a208cfa83
This cycles proto deps and creates a descriptor file from them.
This will allow other proto plugins to call protoc with the descriptor
file and just their `direct_sources` and not all of the `transitive_sources`
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ 00aab5c9c89b16435633613e5e57374f28cd7e26
The plans for bazel to move to rules_cc have been postponed without any
communication. There's no value to us in using this right now, but it
will be trivial to re-adopt in the future if needed. But it has the
downside of using a fork of bazel's crosstool, that has to be updated
independently of bazel, which doesn't always happen as improvements are
made.
More details: https://github.com/bazelbuild/rules_cc/issues/86https://github.com/bazelbuild/bazel/issues/14150
This also required a `--host_action_env` addition to mirror the variables we
pass through to actions in general. This is required because C++ toolchain
setup which discovers linkers in the cc configuration which uses PATH directly,
but then host actions didn't use PATH because of
--incompatible_strict_action_env, so gcc couldn't discover the path of lld even
though `-fuse-ld=lld` was passed.
Fixes https://github.com/envoyproxy/envoy/issues/16608
Signed-off-by: Keith Smiley <keithbsmiley@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1edd65f80bd8ded5cb2c0ebeec818784ebd76bfb
There is an unfortunate change in 3.14 that changed go_package for WKT, which necessitates updating several go dependencies as well:
protoc-gen-validate to 2020-11-30
transitive dependencies from protoc-gen-validate
gazelle to 0.22.2
rules_go to 0.25.0
go to 1.15.5
Risk Level: low
Signed-off-by: Kuat Yessenov <kuat@google.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b3bb0f9847497bc97951172d248fad3c88096829
Add CheckedExpr as oneof expression in RBAC proto
Additional Description: CheckedExpr is type check CEL expression. It is future proof and offers better evaluation performance (not for C++ yet, but for other languages). There will be a separate PR to plumb checked expr to RBAC filter.
Risk Level: low
Testing: existing tests
Signed-off-by: jiangtaoli2016 <jiangtao@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3b8c61b5e5eb4dfddf3c4c0cae9e04eafe1ac61a
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
This is intended to simplify the internal handling of deprecations during API boosting.
See https://docs.google.com/document/d/1mGO9LtVo7t4Lph7WlmyGCxXye3h6j29z3JZvIBbs_D0/edit
Ultimately the plan is to hide this all as a build artifact in Bazel cache, but due to the technical complexity of the pure Bazel solution (involving changes spanning PGV, protoxform, API build rules), we will use checked-in artifacts for 1.13.0.
Risk level: Low
Testing: Additional API test and protoxform golden test.
Part of #8082
This should be unwound in the future with #9479
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ da5fb5ce6ec7cc6e4693656a5ba2161107f6ac4a
This provides canonical BUILD formatting and puts protoxform in charge
of being able to determine import paths, without having to worry about
Bazel implications.
Part of #8082.
Risk level: Low
Testing: tools/proto_sync.py, visual inspection of diffs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e53f40f0e5ccc84fca5cd350416fe0f2accf8229
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
Adds a script to create a go module from the generated protobufs as part of #8151.
The module appears to build with the following module declaration:
module github.com/envoyproxy/data-plane-api/api
go 1.12
require (
github.com/census-instrumentation/opencensus-proto v0.2.1
github.com/envoyproxy/protoc-gen-validate v0.1.0
github.com/gogo/protobuf v1.3.0
github.com/golang/protobuf v1.3.2
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
google.golang.org/grpc v1.23.0
)
Add CI automation to trigger the script after the merge to master in envoyproxy.
Risk Level: low
Testing: local build
Docs Changes: none
Release Notes: none
Fixes#8151
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 32e6018f10f9d6de58fd6c3d6844a68a5be7723b
Support tags[] arg for more specific build control.
Where the underlying bazel primitives support tags[], envoy_() should support them.
Risk Level: Low
Testing: Local on Windows and Linux CI
Signed-off-by: Yechiel Kalmenson <ykalmenson@pivotal.io>
Mirrored from https://github.com/envoyproxy/envoy @ 0bce3a14ab8ac0dbb9e9c3f291b9fc5e4f25ea8d
This change defines C++ grpc bindings for the external AuthZ interface. In the istio Security WG we're planning on using this interface to provide transparent authentication of requests. As this interface lies on the data plane we wish to implement it in C++ to maintain consistent and predictable performance
The changes allow a C++ implementation built using bazel to import Envoy as a workspace dependency and generate the C++ gRPC bindings directly without the need for complex import and generation scripts.
Signed-off-by: Nick A. Smith <nickrmc83@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 41932e9e6f3c932f37f77ae0a5191d65bb7ec8eb
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This is a one-time movement of all UDPA content from envoyproxy/envoy to
cncf/udpa. The permanent home of UDPA will be
https://github.com/cncf/udpa.
Risk level: Low
Testing: Added UDPA service entry to build_test.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e67923fc0b60d0cdf3935fcb30af61d131301fad
Introduces a generic expression-based admission filter using https://github.com/google/cel-cpp.
This is a follow-up to discussion in https://github.com/envoyproxy/envoy/issues/6751.
The advantage of this approach is:
1. Un-opinionated about the policy structure since the only config is an expression. This is friendly towards control planes which can bear the complexity of translation, analysis, and evolution of policies.
2. Multi-language, CEL supports go, java, and c++ runtimes.
3. Inter-operability with other filters using request `metadata`. Companion filters can populate metadata about requests and resources that affect policy decisions.
4. Generic utility, it can be used for custom metric labels, access log entries, etc.
The dis-advantage of this approach is that its performance is lower than domain-optimized interpreters. On a fair example, the interpreter evaluates in around 1ms (see https://github.com/google/cel-cpp/blob/master/eval/tests/benchmark_test.cc#L591) vs ~150ns for hand-written C++ native code. There is space for improvement (especially if WASM can be used as a compilation target), but ultimately the generic expression form carries a cost.
Conditions are added to support RBAC filter for complementing the existing principal/permission model. They add support for the extended checks (e.g. time of query, resource-bound), but add no cost unless used.
Description: add expression-based admission filter
Risk Level: low
Testing:
Docs Changes:
Release Notes:
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ f90e1b08ac5b4973c45a6529780ebdd211ff901f
Description:
Un-pin opencensus and googleapis to use master versions
Use SetName span method to set route operation names (aligning with other tracers).
Risk Level: low
Testing: Unit tests
Docs Changes: None
Release Notes: None
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ef054f08695b8c883c94674904ad27210aa9ba38
They include fixes for future Bazel changes. In particular, the
repository can now build with --incompatible_depset_is_not_iterable,
which will be included in Bazel 0.27.
Description:
Risk Level: low
Testing: bazel build --nobuild ... --incompatible_depset_is_not_iterable
#6995
Signed-off-by: Laurent Le Brun <laurentlb@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0739cd6c9e33d77df2c9ff30c6f43fbb0676e023
Removes golang/protobuf dependency. This is an old @sesmith177 TODO and now the version of rules_go we import has the requisite well known types (WKT). Bumping rules_go to the latest version 0.18.4 (release notes) aligns with PGV which is on 0.18.3.
Risk Level: low (I think but there are WKT changes)
Testing: bazel test //test/...
Signed-off-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ 7bf53149d137dcb2a7b4eddb4f8c23eda23d66e5
This should unblock Windows, fix the long CLI issue and also fuzz weak link problems.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 92971047ca0b8a97c1004e3b5196916203f9185b
In the latest iteration of
https://github.com/envoyproxy/envoy/pull/4220, it was necessary to use
PGV constraints on fuzzer inputs. To do this would require PGV
generation in envoy_build_system.bzl.
There is also quite a bit of mess in
how we were doing envoy_proto_library() today. So, this PR allows us to
throw away the custom envoy_proto_library() and benefit from leveraging
a single source of Envoy proto build truth.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28d5f4118d60f828b1453cd8ad25033f2c8e38ab
This patch prepares HTTP response when an upstream gRPC service method
uses google.api.HttpBody proto as its message output type.
Risk Level: Low
Testing: Unit
Docs Changes:
Added notes on sending arbitrary content as HTTP response by using google.api.HttpBody.
Release Notes:
Added support for encoding google.api.HttpBody when building HTTP response.
Fixes#3205
Signed-off-by: Dhi Aurrahman <dio@rockybars.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0cf947f44247e83dad4274a39067e738a17b2994
adds the required visibility rules and delegates the rest to the generic
api_proto_library. I tested the change by doing the following without
getting errors.
./ci/run_envoy_docker.sh './ci/do_ci.sh docs'
I changed the BUILD files using the following commands.
/envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/api_proto_library(/api_proto_library_internal(/g'
envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/"api_proto_library"/"api_proto_library_internal"/g'
Signed-off-by: mickey <mickeyju@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b871c0ab9350882271a490adcee44e613ed9807
This change makes it possible to create custom Google gRPC channel credentials to be used for communication with the control plane. The current implementation only supports mutual TLS.
Risk Level: Low: optional feature, no functional change unless specifically overridden
Testing: additional unit tests.
Mirrored from https://github.com/envoyproxy/envoy @ b31452e724b024742719f16cd9adc7c8915f8c36
Fixes issue #396. Once this is approved, will make Envoy side changes.
Signed-off-by: Rama rama.rao@salesforce.com
Signed-off-by: Rama <rama.rao@salesforce.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
Created new definitions in api_build_system.bzl that wrap
go_proto_library, go_grpc_library, and go_test. Changed rules in api/BUILD and
test/build/BUILD to use these new definitions. In the future these
definitions could be expanded upon for auto generation in api_proto_library.
Signed-off-by: Kyle Myerson <kmyerson@google.com>
* Added PGV C++ generation support. This (hopefully temporarily)
abandons using native proto_library in favor of pgv_cc_proto_library.
We maintain build support for proto_library for the glorious future in
which we write a Bazel aspect to run PGV against the native
proto_library shadow graph.
* Replace min_len with min_bytes on strings, until PGV gets not-empty or
min_len support for C++.
* Various fixups for places where the PGV plugin objected to
annotations.
Signed-off-by: Harvey Tuch <htuch@google.com>
update-envoy[bot]
data-plane-api(Azure Pipelines)
Daniel Hochman
Rama Chavali
Kuat
kmyerson
William Chang
mandarjog
alyssawilk
htuch