This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Generate or format next free field annotation via protoxform.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Fixes#8429
Signed-off-by: Yi Tang <ssnailtang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 986173ed516dcc1c3dea7db90659ed993d0aad75
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
Clarifies comments in discovery.proto about when response_nonce is(n't) set for (Delta)DiscoveryRequest. Also left a TODO in grpc_mux_impl_test.cc where spec-deviating behavior is expected.
Risk Level: none
Testing: comment-only change
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d6964e5aa985663f50759ad4cfc7f3d78d944b39
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Due to a seg fault issue with the gogo protobuf library
[https://github.com/gogo/protobuf/issues/568], non nullable repeated
fields in a proto will cause proto.Merge(dst, src) to panic.
The nullable field setting was first added by @kyessenov when he was
re-organizing the protos. Unfortunately, people have been copy pasting it
across several areas in the Envoy proto. To keep the impact radius to a minimum,
I have updated only the fields that are currently causing the segfault
(in go-control-plane) for us.
Its also partly against proto principles. You should be able to determine if
a field is set or not. This non-nullable setting in gogo will insist on initializing
the field to default values.
Risk Level: to go control plane users
Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b22d2b5cf09f779962cfedaaab24969f384cbc48
In order to better support clients such as gRPC-LB that want to access
only a single listener/cluster, provide the scope in the xDS
specification to specify explicit resource hints.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7ef20d7609fb6f570a058fcf4b4e000922d7eeba
Risk Level: none, adding unused proto field
#4991
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 154cdcb25832b01794430bc473e6675d365dd082
Moved all of DeltaSubscriptionImpl's existing tests to be DeltaSubscriptionState tests, since that's what they really were. Added tests that test actual DeltaSubscriptionImpl logic. Followup of #6729.
Risk Level: none, just adding tests
Testing: DeltaSubscriptionState, DeltaSubscriptionImpl
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ae3106b3ee792f697b2098781a0e65d8d73a32ca
Added an optional alias field to the resource message for use in the implementation of on-demand xDS.
Risk Level: Low
Testing: Validated Yaml
Fixes#6128
Signed-off-by: Brian Avery <bavery@redhat.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4d18f6e893b59a323ed0119016f1c1cbdb4fbcf9
Using proto.MarshalAny results in unstable output due to non-deterministic map ordering. This in turn causes Envoy's diff to reload a config since the hash of the structure changes.
Enable stable marshaler for gogoproto to avoid this problem. See #6252
Risk Level: low
Testing: n/a
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 15a19b9cb1cc8bd5a5ec71d125177b3f6c9a3cf5
Description: Code for Envoy to speak delta CDS with a management server. DELTA_GRPC added to config_source.proto's ApiTypes, to allow bootstrap configs to ask for incremental xDS.
Part of #4991. Was #5466; giving up on broken DCO craziness.
Risk Level: medium
Testing: new integration test
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8116b6ddb53409c3cea6f55fb00367aa43d7e845
This PR adds API support required for #5770 . Once #5844 lands, I will output it using the text stat.
Risk Level: Low
Testing: N/A
Signed-off-by: Rama Chavali <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28fbbaa76026f52412c2550ff78664ca79c35487
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
Fixes issue #396. Once this is approved, will make Envoy side changes.
Signed-off-by: Rama rama.rao@salesforce.com
Signed-off-by: Rama <rama.rao@salesforce.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
1) Do RDS other than rate limit. This is large enough will do that next.
2) Manually order top level in v2 with better titles
3) Random other fixes
Note also that I'm going to manually reorder the protos in some cases in a
follow up so the docs make a bit more sense for a casual reader.
Signed-off-by: Matt Klein <mklein@lyft.com>
This patch adds an overview page introduced the v2 API concepts via a
worked example. Brought in the entire transitive dep set of protos from
bootstrap.proto, none of these have been cleaned up beyond the minimum
required to have them build under Sphinx.
Also added the ability to link to the underlying proto in messages/enums
from protodoc.py generated RST.
Signed-off-by: Harvey Tuch <htuch@google.com>
This is a fix for a potential race that exists in the gRPC
implementation today. Namely:
1. Envoy sends a DiscoveryRequest at X with resources {A, B}
2. Management server responds with DiscoveryResponse at X+1.
3. Envoy decides (for whatever reason) to change resources before
beceiving the X+1 response. It sends a DiscoveryRequest at X
with {A, B, C}.
4. Management server treats the X DiscoveryRequest update as a NACK
for the X+1 DiscoveryResponse.
Turns out that files with just service methods don't get loaded into the
descriptor pool automatically in C++. So, needed to have some messages
in ads.proto. Turns out this was a good opportunity to move some of the
messages that were related to discovery out of base.proto.
This shouldn't break the API, since everything is in the envoy.api.v2
packge space.
data-plane-api(CircleCI)
Rama Chavali
Kuat
Matt Klein
alyssawilk
htuch