This patch performs a major version freeze and bump by modifying
package_version_status, using the tooling developed in #10636.
Specifically:
v2 APIs are frozen (except for where they are the latest in their
package history and still active)
v3 APIs are transitioned to be active
Candidate v4alpha APIs are generated (not used by Envoy yet)
Fixes#10355
Risk level: medium (entire API's files are modified, visually verified
to ensure things look sane, all tests pass)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 549164c42cae84b59154ca4c36009e408aa10b52
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
In which we convert every v3alpha reference to v3. In future revs of the
stable API versioning policy, we will develop better tooling to support
> 2 alpha and stable versions. For v3, it seems reasonable to just mv
v3alpha to v3, since there should be no external consumers yet.
Risk level: Low
Testing: bazel test //test/..., CI.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5248a4fb7d4c2a3d1fa151f944d3a63f6b7a06cf
Previously, type_to_endpoint.cc had a lot of hardcoding, which doesn't scale well with multiple API
versions. See https://github.com/envoyproxy/envoy/pull/9526 for an example of the issues
encountered.
This patch switches to using explicit resource type annotations on service descriptors, which is
great for documentation (previously this was sometimes given in comments, sometimes not), and allows
for a reflection driven reverse map from resource type URL to endpoints to be built at runtime.
Risk level: Low
Testing: New unit tests for type_to_endpoint.cc and golden protoxform tests for the new annotations.
Fixes#9454.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ cceab393664429a3063d787cf28cade3c8ab01c7
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
In order to get file level move annotation, import has to be before options.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 062c895f499382ae61dead16db2a7e78b9146525
Instead of formatting options heuristically, which will erase new annotations without changing protoxform, use proto descriptor to format options, and enforce its order as well.
Risk Level: Low
Testing: CI
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ dfe687d49574ef7eb1bf84867bf571e805a2bf97
Make TCP listeners supporting SO_REUSEPORT, then each worker thread will create
and listen on socket using same address and port.
Signed-off-by: lhuang8 <lhuang8@ebay.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8534ac8f810de72961b9e4399e14cf24fade7a60
Generate or format next free field annotation via protoxform.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Fixes#8429
Signed-off-by: Yi Tang <ssnailtang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 986173ed516dcc1c3dea7db90659ed993d0aad75
This commit introduces optional connection rebalancing
for TCP listeners, targeted as cases where there are a
small number of long lived connections such as service
mesh HTTP2/gRPC egress.
Part of this change involved tracking connection counts
at the per-listener level, which made it clear that we
have quite a bit of tech debt in some of our interfaces
in this area. I did various cleanups in service of this
change which leave the connection handler / accept path
in a cleaner state.
Fixes https://github.com/envoyproxy/envoy/issues/4602
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 587e07974e6badb061ee3c9413660ab47f42750f
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
This patch introduces a new tool, protoxform, that will be the basis of
the v2 -> v3 migration tooling. It operates as a Python protoc plugin,
within the same framework as protodoc, and provides the ability to
operate on protoc AST input and generate proto output.
As a first step, the tool is applied reflexively on v2, and functions as
a formatting tool. In later patches, this will be added to
check_format/fix_format scripts and CI.
Part of #8082.
Risk level: medium (it's possible that some inadvertent wire changes
occur, if they do, this patch should be rolled back).
Testing: manual inspection of diff, bazel test //test/..., some
grep/diff scripts to ensure we haven't lost any comments.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 08b123a8321d359ea66cbbc0e2926545798dabd3
This existed for the v1 -> v2 migration and is no longer relevant.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 358793bce7a0a3eb41f4d9961da5c61b374ceb7d
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Promote tracing operation field to listener level. This expands the scope of the field to support two use cases:
Tracing TCP connections: istio can send connection events to create a service communication graph. Network filters can benefit from the common knowledge about the intent of the listener/filter chain (client-side vs server-side).
Using ingress/egress designation for other telemetry. The direction of the traffic is a useful label on metrics, and it is not explicit at the moment, unless depending on tracing configuration in HTTP connection manager or naming convention. Both workarounds are not ideal.
Risk Level: low
Testing: all unit tests continue to pass
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ca2af9723598fab4f511b59407396cc5cff9ed94
Due to a seg fault issue with the gogo protobuf library
[https://github.com/gogo/protobuf/issues/568], non nullable repeated
fields in a proto will cause proto.Merge(dst, src) to panic.
The nullable field setting was first added by @kyessenov when he was
re-organizing the protos. Unfortunately, people have been copy pasting it
across several areas in the Envoy proto. To keep the impact radius to a minimum,
I have updated only the fields that are currently causing the segfault
(in go-control-plane) for us.
Its also partly against proto principles. You should be able to determine if
a field is set or not. This non-nullable setting in gogo will insist on initializing
the field to default values.
Risk Level: to go control plane users
Signed-off-by: Shriram Rajagopalan <rshriram@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b22d2b5cf09f779962cfedaaab24969f384cbc48
Delta services other than DeltaClusters were missing. Also added those services to proto_descriptors.cc. Also added some other things that proto_descriptors.cc was missing. #4991
Risk Level: low
Signed-off-by: Fred Douglas <fredlas@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 40d8b7f68c64591f11d4e62ac429af5e074ed3a7
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
This reverts commit 36db7761cc963a7790eb31eb7bccd7cfb248146c.
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 553c21b796f9dd7b3d1ae01cdfc4bebdd3b28338
Move the hard-coded 15s timeout in TLS inspector into
the connection handler such that it covers all listener
filters. Also make it configurable as well as add useful
stats to see how many connections are currently undergoing
listener filter processing.
Fixes https://github.com/envoyproxy/envoy/issues/5217
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ e96425aabcce10ab2030de66f69cbb20a867322f
Add a field in listener proto to be able to reverse the order of TCP write filters. The field is set false by default, indicating write filters have the same order as configured in the filter chain. If true, their order will be reversed.
Risk Level: Low
Testing: bazel test //test/...
Part of #4599
Signed-off-by: Qi (Anna) Wang <qiwang@qiwang-macbookpro.roam.corp.google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5da782c8503aa2664ceac1995628d161bbaa6441
This allows users to specify literal socket options that may not be compatible with upstream kernels or available in precompiled Envoy binaries.
Risk Level: Low
Testing:
Existing tests pass plus new test to cover new functionality.
Partial fix for #3661.
Signed-off-by: Trevor Schroeder <trevors@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 20c0454545761c254180f77948304083406757be
This follows up from #558 which made IP_FREEBIND a BoolValue for LDS but
not for upstream. I think it makes sense to have it in both places given
the new socket options setup introduce in
https://github.com/envoyproxy/envoy/pull/2734.
Some bonus docs fixups thrown in.
Signed-off-by: Harvey Tuch <htuch@google.com>
Document the behavior when each option is set to true, set to false,
or unset.
Document that the transparent option should be used in conjunction
the original_dst_filter to restore the original destination address.
Signed-off-by: Romain Lenglet <romain@covalent.io>
Add an option "freebind" in LDS for listeners and the BindConfig which
is used for outgoing upstream connections. If true, set IP_FREEBIND
socket option. Defaults to false.
PR description will be updated with Envoy implementation PR# once it is
created.
Signed-off-by: Dan Noé <dpn@google.com>
Add a "transparent" option to Listener to set the SOL_IP/IP_TRANSPARENT option on listen sockets, which allows using Envoy with the iptables TPROXY target.
Unlike the iptables REDIRECT target, TPROXY allows preserving both the source and destination IP addresses and ports of accepted connections.
API changes for: envoyproxy/envoy#2719
Signed-off-by: Romain Lenglet <romain@covalent.io>
This enables generating generic service stubs for all the data-plane-api
proto services when generating Java classes with protoc.
This is generally not needed when implementing a gRPC server but in our case we're implementing
it behind our legacy protobuf RPC framework which rely on these stubs. As far as I know the only negative
with enabling these is generating some potentially unnecessary Java classes.
Signed-off-by: Snow Pettersen <snowp@squareup.com>
I went through and reorganized things to make the v2 docs more
human browsable. I also did a few misc cleanups. There is a lot
more to do here which I'm hoping to find a contractor to pay to
work on, but this is a step in the right direction.
Signed-off-by: Matt Klein <mklein@lyft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
data-plane-api(CircleCI)
Bjoern Metzdorf
Romain Lenglet
htuch
Matt Klein
Dan Noé
Snow Pettersen
Kuat