This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Due to a seg fault issue with the gogo protobuf library
[https://github.com/gogo/protobuf/issues/568], non nullable repeated
fields in a proto will cause proto.Merge(dst, src) to panic.
The nullable field setting was first added by @kyessenov when he was
re-organizing the protos. Unfortunately, people have been copy pasting it
across several areas in the Envoy proto. To keep the impact radius to a minimum,
I have updated only the fields that are currently causing the segfault
(in go-control-plane) for us.
Its also partly against proto principles. You should be able to determine if
a field is set or not. This non-nullable setting in gogo will insist on initializing
the field to default values.
Risk Level: to go control plane users
Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b22d2b5cf09f779962cfedaaab24969f384cbc48
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
To encourage users to use v2 configuration. Related to #2100.
Risk Level: N/A, documentation change.
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Taiki Ono <taiki-ono@cookpad.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1d46c75024ebe3c5449647f8bbb9d5dcc532f836
This allows users to specify literal socket options that may not be compatible with upstream kernels or available in precompiled Envoy binaries.
Risk Level: Low
Testing:
Existing tests pass plus new test to cover new functionality.
Partial fix for #3661.
Signed-off-by: Trevor Schroeder <trevors@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 20c0454545761c254180f77948304083406757be
Adds TCP Keepalive support for upstream connections. This can be configured on the cluster manager level, and overridden on the cluster level.
Risk Level: Medium
Testing:
Unit tests have been added. It appears to run and work.
Docs Changes:
envoyproxy/data-plane-api#614Fixesenvoyproxy/envoy#3028
API Changes:
envoyproxy/data-plane-api#614
Signed-off-by: Jonathan Oddy <jonathan.oddy@transferwise.com>
Mirrored from https://github.com/envoyproxy/envoy @ dd953f99945bb7c6b3251f71bffe252a5f6e9e62
This was found via proto fuzzing the server config.
Risk Level: Low
Testing: New utility_test.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ a87d265d1c1819926f2b2e8e7e866244fd40f574
This follows up from #558 which made IP_FREEBIND a BoolValue for LDS but
not for upstream. I think it makes sense to have it in both places given
the new socket options setup introduce in
https://github.com/envoyproxy/envoy/pull/2734.
Some bonus docs fixups thrown in.
Signed-off-by: Harvey Tuch <htuch@google.com>
Add an option "freebind" in LDS for listeners and the BindConfig which
is used for outgoing upstream connections. If true, set IP_FREEBIND
socket option. Defaults to false.
PR description will be updated with Envoy implementation PR# once it is
created.
Signed-off-by: Dan Noé <dpn@google.com>
I went through and reorganized things to make the v2 docs more
human browsable. I also did a few misc cleanups. There is a lot
more to do here which I'm hoping to find a contractor to pay to
work on, but this is a step in the right direction.
Signed-off-by: Matt Klein <mklein@lyft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
These were lost in my backlog. Required a PGV fix for cross-package enum
validation to deal with the TODOs, see
https://github.com/lyft/protoc-gen-validate/issues/42.
Signed-off-by: Harvey Tuch <htuch@google.com>
To resolve https://github.com/envoyproxy/envoy/issues/2155, it seems
better to fix via docs than implementation. This is because we have a
choice of binding 0.0.0.0 or ::, and the current Envoy idiom is to make
the user be explicit rather than probe.
It's possible to use :: for both IPv4/v6, for example, in
certain environments where /proc/sys/net/ipv6/bindv6only is set to 0. We
could add support to Envoy and the API for IPV6_V6ONLY to override this,
but this is orthogonal to the above issue.
Signed-off-by: Harvey Tuch <htuch@google.com>
* Added PGV C++ generation support. This (hopefully temporarily)
abandons using native proto_library in favor of pgv_cc_proto_library.
We maintain build support for proto_library for the glorious future in
which we write a Bazel aspect to run PGV against the native
proto_library shadow graph.
* Replace min_len with min_bytes on strings, until PGV gets not-empty or
min_len support for C++.
* Various fixups for places where the PGV plugin objected to
annotations.
Signed-off-by: Harvey Tuch <htuch@google.com>
* [#not-implemented-hide:] will hide a message/enum/field from docs.
* [#not-implemented-warn:] will add a "Not implemented yet" warning to a
message/enum/field.
* [#v2-api-diff:<text>] will add a note indicating v2 API difference to
a message/enum/field.
* Switched title annotation to [#protodoc-title:<text>] for consistency.
Signed-off-by: Harvey Tuch <htuch@google.com>
* Import @com_lyft_protoc_gen_validate for validate.proto annotations.
* Example annotation in address.proto (BindConfig).
* Process optional/required annotations in protodoc.
Signed-off-by: Harvey Tuch <htuch@google.com>
Instead of having TOC elements like api/base.proto, we could add a title to protos with prefix
protodoc-title: <title>. These will be converted into page titles and stripped from the docs before rendering.
Signed-off-by: Shriram Rajagopalan <shriram@us.ibm.com>
This is a design-level update to bootstrap.proto, that plumbs in the
remaining top-level config from v1. It will probably have some small
changes made beyond this as we implement.
Notable differences to v1 are:
* Static/dynamic resources are clearly delineated at top-level, clusters no longer belong to the ClusterManager object.
* Stats sinks are a repeated list of opaque configs, similar to filter.
* Some simplifications to object types, e.g. RLS no longer specifies type (do we want to preserve the v1 generality here?).
Also renamed RLDS back to RLS, I'll admit that it didn't make sense to
cram it into the xDS namespace, it's really a very distinct service on
the data plane and shouldn't be bundled with the control plane services.
Since we may want to use a resolver plugin in most places, it doesn't
make sense to have this strong separation between the two in the API.
Also bonus renumbering cleanup in LDS.
This will be useful for Google and other site specific extension of the
resolving of both names and ports. For listeners, ports and bound
addresses can be expressed in a site-specific format. For upstreams,
host names can be resolved using the plugin.
data-plane-api(CircleCI)
htuch
Matt Klein
Dan Noé
Sylvain Mellak
Kuat
Shriram Rajagopalan
Jose Ulises Nino Rivera
Piotr Sikora
alyssawilk