In which we convert every v3alpha reference to v3. In future revs of the
stable API versioning policy, we will develop better tooling to support
> 2 alpha and stable versions. For v3, it seems reasonable to just mv
v3alpha to v3, since there should be no external consumers yet.
Risk level: Low
Testing: bazel test //test/..., CI.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5248a4fb7d4c2a3d1fa151f944d3a63f6b7a06cf
Changing from relative name to absolute name, and fixing the fatal-by-defaults that were broken by the v3 switch.
The old way to allow fatal-by-defaults was
envoy.deprecated_features:proto_file.proto:field_name
the new way is
envoy.deprecated_features:full.namespace.field_name
When we switched to v3, all the hard-coded v2 names stopped working. This reinstates them via hopefully more permanent proto annotation.
The only remaining ugly bit is that unfortunately the full namespace and field name are the v3 versions even if the original config was v2. Between @htuch and I we should fix that before merging.
Risk Level: Medium
Testing: added new unit tests
Docs Changes: updated
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 156d7c90083c196a206c07fc03b2de6be8260bd3
To better support config dump, deprecated field detection and other debug, it's helpful to leave a type name breadcrumb and be able to synthesize a Protobuf::Message that corresponds to what was delivered on the wire.
While working on this PR, it became apparent that config dump is broken post v3alpha, since a single config dump might have both v2 and v3 Listeners, etc. The only way to resolve this generically is to make the inner resources in config dump Any. This is a breaking API change, but these are v2alpha/v3alpha at this point, so allowed.
Risk level: Low
Testing: new version converter unit test, config dump tests now verify that the correct versioned inner resource is returned.
Fixes#9612
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 233838a39fb8310ea145e7d758d428d5a37b0306
This allows for a clean separation of config/service in v3. This is a
continuation of #9548.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3bddaee1912fcd1fedc4786aee830b2e4a7c599
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
* api: link to previous message type package in API BUILD files.
We need to include the descriptors from the previous message version in
the build. We opt to do this transitively; when you include v3 of a
package, you get the v2 via a transitive dep. This should work based on
alwayslink semantics for cc_library.
The computation of the deps is based on the previous_message_type
annotation, which will allow cross package migrations.
Part of #8082.
Risk level: Low
Testing: Disabled ip_tagging v2 descriptor hack, observed
version_integration_test. After the BUILD changes, this now passes
again.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7f8fb9509d3189819dd253e25ec76e939ae106e7
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
This provides canonical BUILD formatting and puts protoxform in charge
of being able to determine import paths, without having to worry about
Bazel implications.
Part of #8082.
Risk level: Low
Testing: tools/proto_sync.py, visual inspection of diffs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e53f40f0e5ccc84fca5cd350416fe0f2accf8229
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
The current /listeners admin endpoint only returns a JSON-encoded list of addresses/ports. If a listener config binds to port 0, the endpoint will return the port that was assigned, but if there are multiple listeners binding to port 0, there's no way to tell which listener received which port.
This PR adds a proto that contains the listener's name and address. The /listeners endpoint returns this data in both text and JSON format, similar to the /clusters admin endpoint.
Risk Level: Low (according to #6959 the /listeners admin endpoint is only used for testing)
Testing:
I updated integration_admin_test.cc to test the new output formats for the /listeners endpoint.
I also updated the tools/socket_passing.py script that's used for integration/hotrestart_test.
Docs Changes:
new inline docs in listeners.proto
updated the admin menu description of /listeners
Release Notes:
admin: /listener endpoint now returns :ref:listener.proto<api/envoy/admin/v2alpha/listeners.proto>
Fixes#6959
Signed-off-by: Kaisen Chen <kaisen@yelp.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4461f431112876932e0328f6af31de7cf34fadfc
Implement the scoped RDS (SRDS) API config subscription and provider based on the config protos introduced in #6675 and the ConfigProvider framework introduced in #5243 and #6781.
NOTES:
See parent PR #5839 for full context into these changes. PRs 2a (#6675) and 2b (#6781) have already been merged.
The API is not yet fully implemented. This PR introduces static and dynamic (xDS config subscription) handling of scoped routing configuration, but the new L7 multi tenant routing logic (see #4704) has not yet been introduced.
The API is not yet plumbed into the HttpConnectionManager, that will be done in the next PR.
This PR includes unit tests only; integration tests will follow in the next PR.
Risk Level: Low (this DS API is not yet integrated into the HCM and can not be enabled via config).
Testing: Unit tests added.
Docs Changes: N/A.
Signed-off-by: Andres Guedez <aguedez@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 831d0cbb389f241fd4d1a8b224ffa43e4257a680
This is a MVP for the HTTP tap filter. It includes minimal
infrastructure for the following:
1. Generic tap configuration which in the future will be used for
static config, XDS config, etc. In this MVP the tap can be
configured via a /tap admin endpoint.
2. Generic output configuration which in the future will be used for
different output sinks such as files, gRPC API, etc. In this MVP
the tap results are streamed back out the /tap admin endpoint.
3. Matching infrastructure. In this MVP only matching on request and
response headers are implemented. Both logical AND and logical OR
matches are possible.
4. In this MVP request/response body is not considered at all.
5. All docs are included and with all the caveats the filter is ready
to use for the limited cases it supports (which are likely still to
be useful).
There is a lot of follow on work which I will do in subsequent PRs.
This includes:
1. Merging the existing capture transport socket into this framework.
2. Implementing body support, both for matching on body contents as
well as outputting body data.
3. Tap rate limiting so too many streams do not get tapped.
4. gRPC matching. Using reflection and loaded proto definitions, it will
be possible to match on gRPC fields.
5. JSON matching. If the body parses as JSON, we can allow matching on
JSON fields.
Part of https://github.com/envoyproxy/envoy/issues/1413.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ cf80045587240d494e54e9772949bc9af5eda61f
Converts the existing /server_info admin endpoint to be represented by a protobuf. This will make it easier to extend with new values in the future.
Risk Level: Low
Testing: Updated the existing unit test
Docs Changes: n/a
Release Notes: n/a
Part of #4405
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ 71bd095297ba64712bfad30d0aee1f019fbd32d8
*Description*: Adds support for displaying Subject Alternate Names as a comma separated list of string in `/certs` admin end point
*Risk Level*: Low
*Testing*: Automated
*Docs Changes*: Updated
*Release Notes*: Updated
Signed-off-by: Rama <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7403314b0b255a89ab284b7ed0eead5a5f4aecfd
Adds a /memory endpoint to the admin panel for fast inspection of Envoy::Memory's heap statistics, without needing to query stats.
Risk Level: Low
Testing: Added a test to admin_test.cc.
Docs Changes: Added a description of the option to admin.rst.
Release Notes: N/A
Signed-off-by: James Buckland <jbuckland@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b5ba7f89a4433d283549c61a659c4e61ed564d16
adds the required visibility rules and delegates the rest to the generic
api_proto_library. I tested the change by doing the following without
getting errors.
./ci/run_envoy_docker.sh './ci/do_ci.sh docs'
I changed the BUILD files using the following commands.
/envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/api_proto_library(/api_proto_library_internal(/g'
envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/"api_proto_library"/"api_proto_library_internal"/g'
Signed-off-by: mickey <mickeyju@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b871c0ab9350882271a490adcee44e613ed9807
Added the /clusters?format=json admin endpoint along with a proto representation of /clusters.
Risk Level: Low
Testing: Added a unit test for the new format.
Docs Changes: Added a brief description on the admin docs and linked to the more detailed proto definition.
Release Notes: Added release notes.
Fixes#2020
Signed-off-by: Matt Rice <mattrice@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 64605338ef040a949c5ea205bdd472a8fe42306c
This change does several things:
1) Clarifies how we handle xDS version_info in responses and sets us up
for both top-level/transactional versions as well as per-resource
versions in the future.
2) Moves the config_dump admin endpoint to the v2alpha namespace so that
we can iterate on it in the future.
3) Fills out the config dump proto for the remaining resource types.
These are not implemented but are here to force a discussion about
how we want to handle versions moving forward.
4) Fixes RDS static config dump to actually work and add better tests.
5) Wire up version for the RDS config dump on a per-resource basis.
Once we agree on the general version semantics I will be following up
with dump capability of the remaining resource types.
Part of https://github.com/envoyproxy/envoy/issues/2421
Part of https://github.com/envoyproxy/envoy/issues/2172
Fixes https://github.com/envoyproxy/envoy/issues/3141
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ ada758739907628b50079b9adfccf5481ec9fc5f
See envoyproxy/envoy#2771 for context
I will add documentation once the interfaces and placement of these things is settled. Current location (new admin package) was agreed upon as a good start with @htuch but i don't feel strongly about that or naming.
Signed-off-by: James Sedgwick <jsedgwick@lyft.com>
1) Add ability to runtime filter to configure default, divisor, and
whether independent randomness is used.
2) Also add LE to the comparison filter.
Signed-off-by: Matt Klein <mklein@lyft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
data-plane-api(CircleCI)
James Sedgwick
Matt Klein
Kuat
Piotr Sikora