This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Generate or format next free field annotation via protoxform.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Fixes#8429
Signed-off-by: Yi Tang <ssnailtang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 986173ed516dcc1c3dea7db90659ed993d0aad75
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Promote tracing operation field to listener level. This expands the scope of the field to support two use cases:
Tracing TCP connections: istio can send connection events to create a service communication graph. Network filters can benefit from the common knowledge about the intent of the listener/filter chain (client-side vs server-side).
Using ingress/egress designation for other telemetry. The direction of the traffic is a useful label on metrics, and it is not explicit at the moment, unless depending on tracing configuration in HTTP connection manager or naming convention. Both workarounds are not ideal.
Risk Level: low
Testing: all unit tests continue to pass
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ca2af9723598fab4f511b59407396cc5cff9ed94
Description:
this commit bumps protoc-gen-validate to the latest version. this
should unblock `wrowe` in slack working on windows support.
after this I believe we can also take use of the new address validation
type to fix some unique error messages, but the first step is bumping it
as a side note:
- SocketState was using `.message.required` however it was not
a message type. as far as I can tell this was a bug that PGV fixed.
Risk Level: Low
Testing: Ensure that envoy successfully builds.
Docs Changes: None
Release Notes: None
Signed-off-by: Cynthia Coan <ccoan@instructure.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9c00735e68148b9100473eecce2ee536c3072c6b
Description: Add an async data source which supports fetching data from local and remote data source. The async data provider guarantees that data is access before `init manager` is ready.
Risk Level: Low
Testing: Unit test
Docs Changes: N/A
Release Notes: Added
Fixes#7311
Signed-off-by: crazyxy <yxyan@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 42706efea87eda276cd650db99bc318319176a98
Using proto.MarshalAny results in unstable output due to non-deterministic map ordering. This in turn causes Envoy's diff to reload a config since the hash of the structure changes.
Enable stable marshaler for gogoproto to avoid this problem. See #6252
Risk Level: low
Testing: n/a
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 15a19b9cb1cc8bd5a5ec71d125177b3f6c9a3cf5
Users can now choose between buffered tapping (simpler) and
streaming tapping (more flexible but harder to work with).
Streaming tapping for the transport socket will be added in a
follow-up.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ e2e4baaa85a98b14f2bee6ea5aa16dd79cb832d4
This PR adds API support required for #5770 . Once #5844 lands, I will output it using the text stat.
Risk Level: Low
Testing: N/A
Signed-off-by: Rama Chavali <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28fbbaa76026f52412c2550ff78664ca79c35487
Add support of Any as opaque config for extensions. Deprecates Struct configs. Fixes#4475.
Risk Level: Low
Testing: CI
Docs Changes: Added.
Release Notes: Added.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 851f591f4ed84594e5e5041e7ada4167a4f3a273
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
API for #4475.
Risk Level: Low (not implemented)
Testing: CI
Docs Changes: Added but hided
Release Notes: N/A, will add when adding impl.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 45a460fabf34698a875060482de96f7f618bdc9f
limiting user defined header alterations to 25 alterations (add/remove request/response headers) and 1024 bytes per alteration.
Risk Level: medium (if someone needs higher limits we can roll back and roll forward with new values)
Testing: new unit tests of one of the 17 fields affected
Release Notes: noted the new limitations
Fixes#4268
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ b9bee1e649399f838b23cb1cedfba888e1925605
This patch reintroduces PR #4217.
Signed-off-by: Tony Allen <tallen@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ bcc7a96d9e66bc0f70e3ed52161ec3471f835bdb
This reverts commit 75e54d072fa190ab78b9795f9ed7a98e1d2d33a3.
This was breaking Mac CI due to known PGV CLI limits.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8ba41eea553232bdfab90d9aa32ab4b72ca6fb31
To encourage users to use v2 configuration. Related to #2100.
Risk Level: N/A, documentation change.
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Taiki Ono <taiki-ono@cookpad.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1d46c75024ebe3c5449647f8bbb9d5dcc532f836
We have dedicated alternative mechanisms for this in RouteAction, it can
confuse other actions (e.g. prefix_rewrite).
Fixes oss-fuzz issue https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9995.
Risk level: Low
Testing: Unit tests and corpus entry added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b9dc5d9a04411fe72ff516cbaef527dd8c016ae9
Some sites where we consume HeaderValue, e.g. in Google gRPC client library metadata, require
non-empty keys as a precondition. This seems a general property; there shouldn't be any use case for
a header key that is empty.
Found with server_fuzz_test under oss-fuzz (issue 9373). As a bonus, also fixed another proto
descriptor crash that occurs with this corpus addition due to missing proto descriptor pool entries
in server_fuzz_test.
Risk level: Low
Testing: New server_fuzz corpus entry.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8ed7c157c4b672763ae74e526e69ef43dda52549
This allows users to specify literal socket options that may not be compatible with upstream kernels or available in precompiled Envoy binaries.
Risk Level: Low
Testing:
Existing tests pass plus new test to cover new functionality.
Partial fix for #3661.
Signed-off-by: Trevor Schroeder <trevors@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 20c0454545761c254180f77948304083406757be
1) Add ability to runtime filter to configure default, divisor, and
whether independent randomness is used.
2) Also add LE to the comparison filter.
Signed-off-by: Matt Klein <mklein@lyft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
In support of https://github.com/envoyproxy/envoy/issues/2200 and some
Google internal needs, we are planning on adding support to Envoy to
allow a configuration (or possibly build) driven decision on whether to
using the existing Envoy in-built Grpc::AsyncClient or
the Google C++ gRPC client library (https://grpc.io/grpc/cpp/index.html).
To move in this direction, the idea is we have the xDS ApiConfigSources,
rate limit service config and other filter configurations point at a
GrpcService object. This can be configured to use an Envoy cluster,
where Grpc::AsyncClient will orchestrate communication, or to contain
the config needed to establish a channel in Google C++ gRPC client
library.
Signed-off-by: Harvey Tuch <htuch@google.com>
envoyproxy/envoy#2256 adds restrictions to the backing sources for xDS resources. This change documents those restrictions.
Signed-off-by: Jose Nino <jnino@lyft.com>
This patch clarifies the relationship between the CLI flags for local
service configuration and the bootstrap node identifier, where these
concepts are also expressed.
Signed-off-by: Harvey Tuch <htuch@google.com>
These were lost in my backlog. Required a PGV fix for cross-package enum
validation to deal with the TODOs, see
https://github.com/lyft/protoc-gen-validate/issues/42.
Signed-off-by: Harvey Tuch <htuch@google.com>
data-plane-api(CircleCI)
Matt Klein
Kuat
Piotr Sikora
Brian Pane
Jose Ulises Nino Rivera
htuch
William Chang
alyssawilk