Previously, type_to_endpoint.cc had a lot of hardcoding, which doesn't scale well with multiple API
versions. See https://github.com/envoyproxy/envoy/pull/9526 for an example of the issues
encountered.
This patch switches to using explicit resource type annotations on service descriptors, which is
great for documentation (previously this was sometimes given in comments, sometimes not), and allows
for a reflection driven reverse map from resource type URL to endpoints to be built at runtime.
Risk level: Low
Testing: New unit tests for type_to_endpoint.cc and golden protoxform tests for the new annotations.
Fixes#9454.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ cceab393664429a3063d787cf28cade3c8ab01c7
This allows for a clean separation of config/service in v3. This is a
continuation of #9548.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3bddaee1912fcd1fedc4786aee830b2e4a7c599
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
Adds a script to create a go module from the generated protobufs as part of #8151.
The module appears to build with the following module declaration:
module github.com/envoyproxy/data-plane-api/api
go 1.12
require (
github.com/census-instrumentation/opencensus-proto v0.2.1
github.com/envoyproxy/protoc-gen-validate v0.1.0
github.com/gogo/protobuf v1.3.0
github.com/golang/protobuf v1.3.2
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
google.golang.org/grpc v1.23.0
)
Add CI automation to trigger the script after the merge to master in envoyproxy.
Risk Level: low
Testing: local build
Docs Changes: none
Release Notes: none
Fixes#8151
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 32e6018f10f9d6de58fd6c3d6844a68a5be7723b
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This is a one-time movement of all UDPA content from envoyproxy/envoy to
cncf/udpa. The permanent home of UDPA will be
https://github.com/cncf/udpa.
Risk level: Low
Testing: Added UDPA service entry to build_test.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e67923fc0b60d0cdf3935fcb30af61d131301fad
This PR provides the TDS API config. The implementation and docs will be
added in a followup, the idea is to unblock some projects that depend on
the details here.
Risk level: Low
Testing: @envoy_api build
Part of #6708
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad2cacc4215477526dc61f838db8a7ddf55f4cc6
As a first step towards TDS, add support for proto/JSON representations of the runtime virtual file
system and plumb this into bootstrap.
"numerator" and "denominator" are now reserved keywords and can't be used in runtime keys; this
simplifies parsing significantly. It is not expected that these are widely used, due to the
difficulty of ensuring atomicity if representing fractions, but this is technically a runtime API
breaking change.
Risk level: Medium (due to the reserved word changes).
Testing: Unit and server tests added.
Part of #6708
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ eb699ab2d53aadf2c5a26ed02fcc1e7de1df7d8a
Imports should be sorted by Golang coding convention
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
Mirrored from https://github.com/envoyproxy/envoy @ eb11b28ceb174d491727195e121ee75159074245
This is a follow up to #4726. In #4726, the access log path became optional, but the admin field
was not itself marked optional. This then led to server_fuzz_test trivially passing due to an early
PGV validation exception, and ~20 bugs being closed out by oss-fuzz. This PR completes the admin
optionality changes.
Risk Level: Low
Testing: Unit tests updated.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 86790c2367558160282d8b0afa1c5e4698e2fed3
This should unblock Windows, fix the long CLI issue and also fuzz weak link problems.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 92971047ca0b8a97c1004e3b5196916203f9185b
adds the required visibility rules and delegates the rest to the generic
api_proto_library. I tested the change by doing the following without
getting errors.
./ci/run_envoy_docker.sh './ci/do_ci.sh docs'
I changed the BUILD files using the following commands.
/envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/api_proto_library(/api_proto_library_internal(/g'
envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/"api_proto_library"/"api_proto_library_internal"/g'
Signed-off-by: mickey <mickeyju@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b871c0ab9350882271a490adcee44e613ed9807
Add support for extracting dynamic metadata from requests. This can then
be used as static metadata would be used (e.g.: for subset load balancer
metadata matches, logging, etc).
Risk Level: Low
Testing: unit-test
Docs Changes: Basic docs.
Release Notes: N/A
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ 827c0a548ab38d55debe00587ee27253786befad
Add api_go_grpc rules for metrics_service, als, and trace_service.
To support those changes, also added the necessary go_proto_library
rules in the repositories.bzl definitions for prometheus_metrics_model,
and io_opencensus_trace, and augmented the go_build_test.go to
verify these changes were correct.
Signed-off-by: William Chang <mr.williamchang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ fb7797fd7df696cd239cc1d3792719361b62c684
Added protos to support Role Based Access Control in Envoy.
Also removed existing auth.proto because the new RBAC proto is a replacement of it.
Ealier discussions at
envoyproxy/data-plane-api#586.
Signed-off-by: Limin Wang <liminwang@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 13de384ab34428af99c53201f6b3c95991b7ae10
Updates the L7 IP tagging filter proto, v1 configuration documentation and overview documentation.
Signed-off-by: Constance Caramanolis <ccaramanolis@lyft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
Created new definitions in api_build_system.bzl that wrap
go_proto_library, go_grpc_library, and go_test. Changed rules in api/BUILD and
test/build/BUILD to use these new definitions. In the future these
definitions could be expanded upon for auto generation in api_proto_library.
Signed-off-by: Kyle Myerson <kmyerson@google.com>
* Added PGV C++ generation support. This (hopefully temporarily)
abandons using native proto_library in favor of pgv_cc_proto_library.
We maintain build support for proto_library for the glorious future in
which we write a Bazel aspect to run PGV against the native
proto_library shadow graph.
* Replace min_len with min_bytes on strings, until PGV gets not-empty or
min_len support for C++.
* Various fixups for places where the PGV plugin objected to
annotations.
Signed-off-by: Harvey Tuch <htuch@google.com>
Turns out that files with just service methods don't get loaded into the
descriptor pool automatically in C++. So, needed to have some messages
in ads.proto. Turns out this was a good opportunity to move some of the
messages that were related to discovery out of base.proto.
This shouldn't break the API, since everything is in the envoy.api.v2
packge space.
This is a design-level update to bootstrap.proto, that plumbs in the
remaining top-level config from v1. It will probably have some small
changes made beyond this as we implement.
Notable differences to v1 are:
* Static/dynamic resources are clearly delineated at top-level, clusters no longer belong to the ClusterManager object.
* Stats sinks are a repeated list of opaque configs, similar to filter.
* Some simplifications to object types, e.g. RLS no longer specifies type (do we want to preserve the v1 generality here?).
Also renamed RLDS back to RLS, I'll admit that it didn't make sense to
cram it into the xDS namespace, it's really a very distinct service on
the data plane and shouldn't be bundled with the control plane services.
We can dynamically discover service/method descriptors in Envoy, so we
don't need to generate any C++ stubs for this. This simplifies the
Google import and removes an unnecessary build output.
This is useful in Travis CI (and also internally at Google in our CI) to
validate basic build/link of the protos.
This exposed a small issue with API compatibility. Since we don't have
additional package namespaces for each individual API, there was a
conflict between the RateLimit mesage in RLDS and RDS. The quick fix was
to move the RLDS message inside the response object (this is fine as
nobody is using the v2 RLDS yet, open to alternatives including per-xDS
API namespaces).
data-plane-api(CircleCI)
Dhi Aurrahman
Constance Caramanolis
Kuat
Piotr Sikora
Gabriel Sagula
kmyerson
William Chang
Yuval Kohavi
htuch
alyssawilk
Matt Klein
ramaraochavali