Currently, the HDS specifier proto gives endpoint information organized by their locality and grouped by cluster. This information is retained when doing health checks, however when responding with health statuses from each endpoint locality and cluster information is dropped. This puts all endpoint information into a flat list when returning, making it the responsibility of the receiver to re-structure data by cluster and locality.
This change follows a similar format to the specifier coming into HDS, organizing it in the same way. The proposed changed adds consistency to how messages are flowing in and out of HDS.
Note that his PR only contains the protos changes, and does not include implementation. This is done to get approval from api-shepherds for the changes before continuing.
Risk Level: Low
Testing: No new tests. New tests will be added upon implementation, but for now the fields remain empty.
Docs Changes:
Inline comments in hds.proto.
Release Notes:
N/A - will append to release notes upon implementation completion.
Deprecated:
The endpoints_health field in EndpointHealthResponse should be deprecated in favor of the new cluster_endpoints_health field.
Signed-off-by: Drew S. Ortega <drewortega@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4f1ee95e36ad233aac0d37a8b1d7c4536d4972c7
This patch enables ext_authz backed with gRPC service to emit metadata.
The authorization can set the dynamic metadata (an opaque
google.protobuf.Struct) as part of the `CheckResponse` when it is
successful (i.e. when `http_response` is `OkHttpResponse`).
Signed-off-by: Dhi Aurrahman <dio@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ ff44d731ecc8d744b971b31aec816ad0b56bad42
These are currently set as [#not-implemented-hide:]. There should be enough plumbing here to start
implementing the first roadmap steps in #11264, i.e. some simple delta discovery flows with core
LDS/CDS/RDS/EDS/SDS.
I've punted on VHDS and SRDS until later, as these will require special case mapping of their
resource keys into context parameters.
Risk level: Low (Unused API changes).
Testing: Built docs and proto_format.sh.
Part of #11264.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 99471fd8f4ef7406f50cf41cf1cfa22bbdfeacc7
Provides a way to specify dynamic rate limit override in the rate limit descriptor from static value or from dynamic metadata. New type, RateLimitUnit was created to share across config protocol and rate limit service protocol. A PR for the reference implementation of the rate limit service will follow after the API changes are discussed and accepted.
Signed-off-by: Petr Pchelko <ppchelko@wikimedia.org>
Mirrored from https://github.com/envoyproxy/envoy @ 7ea1f24bd53522408d9bc55316dff9ed1701bc73
Define filter config discovery. Add FDS for HTTP filters (HTTP extensions is where the pain is felt the most). Modelled after RDS with a twist of config override for re-use.
Risk Level: low (not implemented)
Testing:
Docs Changes:
Release Notes:
Issue: #7867
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ bda15c0a5a0390daed991ed4c96a4873ab3ba7d8
EDS config has been added to config_dump in #11425 and implemented in #11577 .
Risk Level: Low
Testing: N/A
Signed-off-by: Yutong Li <yutongli@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1bb927ed5ca4e292f5bec61cc045bf16de2e020e
Deprecates GoogleRE2.max_program_size field so that client does not need to check this.
Risk Level: Low
Testing: N/A
Docs Changes: Included in PR
Release Notes: Included in PR
Signed-off-by: Mark D. Roth <roth@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 10125161be0d0a759c3ffb02ddcdf8abc0bc6060
Add a send_all_clusters field to LRS response, triggered by a new client capability. This avoids the need for the server to enumerate the full list of clusters if it always wants data for all clusters.
Risk Level: Low
Testing: N/A
Docs Changes: Included in PR.
Release Notes: N/A
Signed-off-by: Mark D. Roth <roth@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b0f45ed98a7a0e6a6c02b28d54bb29e96fe6e6fc
This patch performs a major version freeze and bump by modifying
package_version_status, using the tooling developed in #10636.
Specifically:
v2 APIs are frozen (except for where they are the latest in their
package history and still active)
v3 APIs are transitioned to be active
Candidate v4alpha APIs are generated (not used by Envoy yet)
Fixes#10355
Risk level: medium (entire API's files are modified, visually verified
to ensure things look sane, all tests pass)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 549164c42cae84b59154ca4c36009e408aa10b52
This part of the global rate limiter api is not currently used by envoy, but for anyone implementing the envoy global rate limiter api being able to describe what specific limit you've hit in a human-readable and/or machine-readable way will be useful for debugging, alerting, etc.
Risk Level: Low
Testing: n/a
Docs Changes: n/a
Release Notes: n/a
Fixes#10556
Signed-off-by: David Weitzman <dweitzman@pinterest.com>
Mirrored from https://github.com/envoyproxy/envoy @ 99706022206acb99bd8093c654fc84f6cd5a877e
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
Add Client Status Discovery Service (CSDS) API definition. This can be used by debug tools to obtain config information for specific clients from control plane.
Risk Level: Low
Testing: N/A
Signed-off-by: Fuqiang Gao <fuqianggao@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 10f756efa17e56c8d4d1033be7b4286410db4e01
This PR introduces a parallel v3 API reference documentation tree to the
existing v2 one.
The docs/root/api-v3/ tree was copied from docs/root/api-v2 and the
necessary package path fixups were made manually. As a result, the tree
largely resembles the v2 docs. Long term this is likely to evolve to
reflect the shape of the new extensions tree.
The message type, field and enum anchors are sed'ed to be distinct and
self-consistent inside api-v3/.
There were a number of API proto changes that were made to obtain a
successful Sphinx build:
* References to deprecated fields were replaced by references to the replacement field.
* clang-format line wrapping in protoxform was removed, this breaks RST in some v3 protos.
* Some packages (type/metadata/v2, data/cluster/v2alpha) were force upgraded to v3, to deal with references to types that are distinct in v2/v3. This is OK as these packages probably make sense to bump for v3, in general we're going to have to think about how to do this more
cleanly, supporting dual v2/v3 references alongside each other.
* Some evil hacks for field renaming added to migrate.py for RouteAction.
There's also some additional machinery added to compute distinct v3/v3
build targets to point protodoc at.
Risk level: Low
Testing: Docs build, manual inspection.
Fixes#8087
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ac88316892cd47b6a9b58e3736e20e8863cd0d27
In which we convert every v3alpha reference to v3. In future revs of the
stable API versioning policy, we will develop better tooling to support
> 2 alpha and stable versions. For v3, it seems reasonable to just mv
v3alpha to v3, since there should be no external consumers yet.
Risk level: Low
Testing: bazel test //test/..., CI.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5248a4fb7d4c2a3d1fa151f944d3a63f6b7a06cf
Previously, type_to_endpoint.cc had a lot of hardcoding, which doesn't scale well with multiple API
versions. See https://github.com/envoyproxy/envoy/pull/9526 for an example of the issues
encountered.
This patch switches to using explicit resource type annotations on service descriptors, which is
great for documentation (previously this was sometimes given in comments, sometimes not), and allows
for a reflection driven reverse map from resource type URL to endpoints to be built at runtime.
Risk level: Low
Testing: New unit tests for type_to_endpoint.cc and golden protoxform tests for the new annotations.
Fixes#9454.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ cceab393664429a3063d787cf28cade3c8ab01c7
This allows for a clean separation of config/service in v3. This is a
continuation of #9548.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3bddaee1912fcd1fedc4786aee830b2e4a7c599
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
In order to get file level move annotation, import has to be before options.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 062c895f499382ae61dead16db2a7e78b9146525
Instead of formatting options heuristically, which will erase new annotations without changing protoxform, use proto descriptor to format options, and enforce its order as well.
Risk Level: Low
Testing: CI
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ dfe687d49574ef7eb1bf84867bf571e805a2bf97
* api: link to previous message type package in API BUILD files.
We need to include the descriptors from the previous message version in
the build. We opt to do this transitively; when you include v3 of a
package, you get the v2 via a transitive dep. This should work based on
alwayslink semantics for cc_library.
The computation of the deps is based on the previous_message_type
annotation, which will allow cross package migrations.
Part of #8082.
Risk level: Low
Testing: Disabled ip_tagging v2 descriptor hack, observed
version_integration_test. After the BUILD changes, this now passes
again.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7f8fb9509d3189819dd253e25ec76e939ae106e7
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Set the downstream client X.509 certificate in the source Peer AttributeContext
Risk Level: low
Testing: Tests updated and extended.
Docs Changes: New API additions are documented.
Release Notes: Added.
Fixes#8326
Signed-off-by: Steve Larkin <steve.larkin@volvocars.com>
Mirrored from https://github.com/envoyproxy/envoy @ 766f3fb8dbdafce402631c43c16fda46ed003462
Modifies the pedantic spell checker to better handle camel-case words.
Each part the word is now treated as a separate error in both the check
and fix modes.
Disables run-together mode for aspell, which allowed typos such as
"mananger" (man + anger). Fixes the resulting spelling errors.
Miscellaneous other fixes:
* Provides an option to replace a word without adding the word to the
dictionary (in case aspell's suggestions do not contain the correct
replacement).
* Fixes a bug in the script when no suggestions are returned by aspell.
* Checks the dictionary and added words for invalid characters that cause
aspell errors at dictionary load time.
* Sets the mark flag when in CI runs so that misspelled words are indicated.
* Culls words from the dictionary that are no longer in the codebase, or are
otherwise not needed.
Risk Level: low (comments only)
Testing: n/a
Doc Changes: n/a
Release Notes: n/a
Fixes: #8481
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ de70fe54cdba6b022b3971379afa535f402f2ffe
This reverts commit 596cd4894c8ecd536c1da1dddecaae3531f269ea.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8c4afa6fee5e104c7b195657a3c22e23f062eb11
Modifies the pedantic spell checker to better handle camel-case words.
Each part the word is now treated as a separate error in both the check
and fix modes.
Disables run-together mode for aspell, which allowed typos such as
"mananger" (man + anger). Fixes the resulting spelling errors.
Miscellaneous other fixes:
* Provides an option to replace a word without adding the word to the
dictionary (in case aspell's suggestions do not contain the correct
replacement).
* Fixes a bug in the script when no suggestions are returned by aspell.
* Checks the dictionary and added words for invalid characters that cause
aspell errors at dictionary load time.
* Sets the mark flag when in CI runs so that misspelled words are indicated.
* Culls words from the dictionary that are no longer in the codebase, or are
otherwise not needed.
Risk Level: low (comments only)
Testing: n/a
Doc Changes: n/a
Release Notes: n/a
Fixes: #8481
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 596cd4894c8ecd536c1da1dddecaae3531f269ea
Generate or format next free field annotation via protoxform.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Fixes#8429
Signed-off-by: Yi Tang <ssnailtang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 986173ed516dcc1c3dea7db90659ed993d0aad75
*Ability to add custom upstream headers from ratelimit service/filter.
*For LimitStatus::OK, custom upstream headers are added if RLS service sends upstream headers.
Risk Level: Low
Testing:
Unit and integration tests added.
Verified with modified github.com/lyft/ratelimit service.
Passes "bazel test //test/..." in Linux
Docs Changes: protobuf documentation updated
Release Notes: ratelimit: support for adding custom headers to upstream server
from ratelimit service
** Issues: #6141
Signed-off-by: Nandu Vinodan <nandu.vinodan@freshworks.com>
Mirrored from https://github.com/envoyproxy/envoy @ c2a396c84cc97711eef0231f04d81ce680652abb
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
This provides canonical BUILD formatting and puts protoxform in charge
of being able to determine import paths, without having to worry about
Bazel implications.
Part of #8082.
Risk level: Low
Testing: tools/proto_sync.py, visual inspection of diffs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e53f40f0e5ccc84fca5cd350416fe0f2accf8229
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
* [#not-implemented-warn:] was barely used and its purposes are better
served by [#not-implemented-hide:].
* [#proto-status:] was there for an earlier style of versioning, where
APIs were "frozen" or "draft", etc. Now we have semantic versioning
and a regular API clock as per #6271.
Part of #8371.
Risk level: Low (docs only).
Testing: Docs rebuild.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 2c4b6f2b3d614a15f312e34f4664ebeb96d07d12
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
This patch introduces a new tool, protoxform, that will be the basis of
the v2 -> v3 migration tooling. It operates as a Python protoc plugin,
within the same framework as protodoc, and provides the ability to
operate on protoc AST input and generate proto output.
As a first step, the tool is applied reflexively on v2, and functions as
a formatting tool. In later patches, this will be added to
check_format/fix_format scripts and CI.
Part of #8082.
Risk level: medium (it's possible that some inadvertent wire changes
occur, if they do, this patch should be rolled back).
Testing: manual inspection of diff, bazel test //test/..., some
grep/diff scripts to ensure we haven't lost any comments.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 08b123a8321d359ea66cbbc0e2926545798dabd3
* Move trailing detached comments that protoc ignores to some anchored place. In the near future,
this will be enforced by check_format.
* Remove comments on reserved fields; we won't be using reserved in v3 and it's a bit of a pain
preserving them.
Risk level: Low
Testing: Docs build, diff inspection, comparison against output of proto transform tool.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 22712d26c427091a47324b5f05fd0b493ee4be01
This change defines C++ grpc bindings for the external AuthZ interface. In the istio Security WG we're planning on using this interface to provide transparent authentication of requests. As this interface lies on the data plane we wish to implement it in C++ to maintain consistent and predictable performance
The changes allow a C++ implementation built using bazel to import Envoy as a workspace dependency and generate the C++ gRPC bindings directly without the need for complex import and generation scripts.
Signed-off-by: Nick A. Smith <nickrmc83@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 41932e9e6f3c932f37f77ae0a5191d65bb7ec8eb
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
These were missed in #8125.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 1b3b4ae1180b67bee6395fab5c075896fb1964ec
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Adds support for DNS SAN in ext authz peer validation
Risk Level: Low
Testing: Added
Docs Changes: Added
Release Notes: N/A
Signed-off-by: Rama Chavali <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ d4dc0a5b74acd42aafaaef8bb9c4ebed832674bd
data-plane-api(CircleCI)