This is necessary to provide TLS transport socket docs and to be able to have the TLS transport
socket added to the threat model docs (via its security_posture tag).
I did both v2/v3, since this is not technically a change to v2, justa file re-org, and the shadowing
machinery prefers file consistency across versions.
Risk level: Low (refactoring)
Testing: Docs generation and manual inspection.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ab32f5fd01ca8b23ee16dcffb55b1276e55bf1fa
This patch performs a major version freeze and bump by modifying
package_version_status, using the tooling developed in #10636.
Specifically:
v2 APIs are frozen (except for where they are the latest in their
package history and still active)
v3 APIs are transitioned to be active
Candidate v4alpha APIs are generated (not used by Envoy yet)
Fixes#10355
Risk level: medium (entire API's files are modified, visually verified
to ensure things look sane, all tests pass)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 549164c42cae84b59154ca4c36009e408aa10b52
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
This PR introduces a parallel v3 API reference documentation tree to the
existing v2 one.
The docs/root/api-v3/ tree was copied from docs/root/api-v2 and the
necessary package path fixups were made manually. As a result, the tree
largely resembles the v2 docs. Long term this is likely to evolve to
reflect the shape of the new extensions tree.
The message type, field and enum anchors are sed'ed to be distinct and
self-consistent inside api-v3/.
There were a number of API proto changes that were made to obtain a
successful Sphinx build:
* References to deprecated fields were replaced by references to the replacement field.
* clang-format line wrapping in protoxform was removed, this breaks RST in some v3 protos.
* Some packages (type/metadata/v2, data/cluster/v2alpha) were force upgraded to v3, to deal with references to types that are distinct in v2/v3. This is OK as these packages probably make sense to bump for v3, in general we're going to have to think about how to do this more
cleanly, supporting dual v2/v3 references alongside each other.
* Some evil hacks for field renaming added to migrate.py for RouteAction.
There's also some additional machinery added to compute distinct v3/v3
build targets to point protodoc at.
Risk level: Low
Testing: Docs build, manual inspection.
Fixes#8087
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ac88316892cd47b6a9b58e3736e20e8863cd0d27
In which we convert every v3alpha reference to v3. In future revs of the
stable API versioning policy, we will develop better tooling to support
> 2 alpha and stable versions. For v3, it seems reasonable to just mv
v3alpha to v3, since there should be no external consumers yet.
Risk level: Low
Testing: bazel test //test/..., CI.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5248a4fb7d4c2a3d1fa151f944d3a63f6b7a06cf
Changing from relative name to absolute name, and fixing the fatal-by-defaults that were broken by the v3 switch.
The old way to allow fatal-by-defaults was
envoy.deprecated_features:proto_file.proto:field_name
the new way is
envoy.deprecated_features:full.namespace.field_name
When we switched to v3, all the hard-coded v2 names stopped working. This reinstates them via hopefully more permanent proto annotation.
The only remaining ugly bit is that unfortunately the full namespace and field name are the v3 versions even if the original config was v2. Between @htuch and I we should fix that before merging.
Risk Level: Medium
Testing: added new unit tests
Docs Changes: updated
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ 156d7c90083c196a206c07fc03b2de6be8260bd3
This allows for a clean separation of config/service in v3. This is a
continuation of #9548.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3bddaee1912fcd1fedc4786aee830b2e4a7c599
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
In order to get file level move annotation, import has to be before options.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 062c895f499382ae61dead16db2a7e78b9146525
Instead of formatting options heuristically, which will erase new annotations without changing protoxform, use proto descriptor to format options, and enforce its order as well.
Risk Level: Low
Testing: CI
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ dfe687d49574ef7eb1bf84867bf571e805a2bf97
Adding property use_tcp_for_dns_lookups to bootstrap and cds config, which will instruct dns resolvers to use tcp for DNS queries.
Risk Level: Medium/Low
Fixes#7965
Signed-off-by: Kateryna Nezdolii <nezdolik@spotify.com>
Mirrored from https://github.com/envoyproxy/envoy @ b78fc4e0edc696c2395b7eafbca8cbc62cb0f325
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Generate or format next free field annotation via protoxform.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Fixes#8429
Signed-off-by: Yi Tang <ssnailtang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 986173ed516dcc1c3dea7db90659ed993d0aad75
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
We should start tagging messages with "option deprecated = true;" if we
want them to be auto-deprecated by protoxform going forward.
Risk level: Low (v3alpha is not used yet)
Testing: bazel test @envoy_api//..., manual diff inspection.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e3247539bcc39f3111287b479a85e51655a6f1db
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Add PGV bounds to the stats flush interval (greater than 1ms and less than 5000ms) to prevent Envoy from hanging from too small of a flush time.
Risk Level: Low
Testing: Corpus Entry added
Fixes OSS-Fuzz issue
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16300
Signed-off-by: Asra Ali <asraa@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0a3fc6a091a768ba868ff4dd94cf526af4ffc0ff
Due to a seg fault issue with the gogo protobuf library
[https://github.com/gogo/protobuf/issues/568], non nullable repeated
fields in a proto will cause proto.Merge(dst, src) to panic.
The nullable field setting was first added by @kyessenov when he was
re-organizing the protos. Unfortunately, people have been copy pasting it
across several areas in the Envoy proto. To keep the impact radius to a minimum,
I have updated only the fields that are currently causing the segfault
(in go-control-plane) for us.
Its also partly against proto principles. You should be able to determine if
a field is set or not. This non-nullable setting in gogo will insist on initializing
the field to default values.
Risk Level: to go control plane users
Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b22d2b5cf09f779962cfedaaab24969f384cbc48
Without a distinction between the root and subdir, we can't watch for
symlink swaps that cover multiple layers.
Risk level: Low
Testing: additional test expects added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 49b5a915324e44b9445e3fc5434e0deffaf3f94f
This PR provides the TDS API config. The implementation and docs will be
added in a followup, the idea is to unblock some projects that depend on
the details here.
Risk level: Low
Testing: @envoy_api build
Part of #6708
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad2cacc4215477526dc61f838db8a7ddf55f4cc6
To support TDS and to provide operator flexibility in how it relates to
existing runtime layering, this PR introduces support for explicit
control over layering. This eliminates the distinguished role of
subdirectory and override_subdirectory, for example, in favor of
arbitrary disk overlay layers.
This PR deprecates the existing runtime configuration in favor of the
new layered variant.
As a bonus, the PR fixes some of the runtime stats and related docs.
Risk level: Low
Testing: various unit tests (including server_test) additions.
Part of #6708.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4dafba65baaf9769723f895761268eed31af629b
As a first step towards TDS, add support for proto/JSON representations of the runtime virtual file
system and plumb this into bootstrap.
"numerator" and "denominator" are now reserved keywords and can't be used in runtime keys; this
simplifies parsing significantly. It is not expected that these are widely used, due to the
difficulty of ensuring atomicity if representing fractions, but this is technically a runtime API
breaking change.
Risk level: Medium (due to the reserved word changes).
Testing: Unit and server tests added.
Part of #6708
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ eb699ab2d53aadf2c5a26ed02fcc1e7de1df7d8a
Reintroduce dispatcher stats previously reverted in #6649. Dispatcher stats are now gated on a new bootstrap config parameter, enable_dispatcher_stats, and disabled by default.
Risk Level: Low
Testing: Manually verified that config is respected (see #6582, this still requires integration tests).
Docs Changes: Included a note about how statsd doesn't play well with these high-volume histograms.
Signed-off-by: Dan Rosen <mergeconflict@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 481a127f5ade32e5590b4e8b8271576e44643082
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
This is a follow up to #4726. In #4726, the access log path became optional, but the admin field
was not itself marked optional. This then led to server_fuzz_test trivially passing due to an early
PGV validation exception, and ~20 bugs being closed out by oss-fuzz. This PR completes the admin
optionality changes.
Risk Level: Low
Testing: Unit tests updated.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 86790c2367558160282d8b0afa1c5e4698e2fed3
Now that Envoy can function without an admin listener (#4486), there's no need to specify path or
even an empty admin message, so relax this requirement.
Risk Level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 570a403b8882ea34309bbe73209fee41417fa10e
Accept bootstrap configs that don't specify an admin port, and simply
don't run the admin server in that case.
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0fb5efc50ef2b03ab0073f947ebfcf886c4a32b1
data-plane-api(CircleCI)