Follow-up to #17375 where it was agreed that protolock is not actively maintained enough to depend on. This PR "migrates" the tests from that PR to use buf instead, and also cleans some of the code per a few of the review comments. Still a few outstanding points:
- buf build on the envoy/api folder requires several protobuf dependencies such as udpa to be available to buf to consume. Suggested solution by buf is to point buf's config to necessary BSR modules that the buf team is hosting.
- These lines are commented out in this PR as I had some trouble automating it for the tests, and it is not necessary for the tests to pass
- May introduce issues if buf is not pointing to the same version of modules that bazel builds for envoy. May need to introduce some way to couple them, or (ideally) find a way to run the breaking change detector without building the dependencies
- Currently bazel is using a binary release of buf (for linux). Goal is to move to building it from source in the near future
- It may be in our interest to expand the list of API-breaking-change rules (buf provides an extensive list of rules we could adopt)
Risk Level: Low
Testing: Tests that evaluate buf against "allowed" and "breaking" protobuf API changes. Currently 4 tests are skipped - 3 of them are PGV-related (we need to communicate our desired PGV rules to the buf team so they may add them in the near future). The 4th is a test I had originally written to evaluate protolock but may not apply to buf ("forcing" a breaking change) - refer to comments
Docs Changes:
Release Notes:
Platform Specific Features: buf binary imported by bazel is linux-only. Hopefully the ["manual"] tags attribute prevents any issues for non-linux users
Signed-off-by: Yaseen Alkhafaji <yaseena@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c74cebb3cc7dfb26488f7cac8eb9258cf9a7588e
Risk Level: Low; the protobuf types are not changing at all.
Testing: proto_format and tests
Signed-off-by: Tyler Dixon <tdixon@buf.build>
Co-authored-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ c1bc5e78fa68b86236c8d6237e2db15ce1743459
Fixes a few issues with the generated Go protobuf code:
a) the OT protos are not copied over, which means that their importpath
needs to reference a path where it can pull the protos down from. The
"logs" importpath was nonsensical and couldn't be resolved
b) Bunching several proto packages into one target did not work well
with the go package system. This splits the common protos out into its
own target so that it can reference a different importpath than the
other logs protos
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8037370ce45d8c10654114b2b46e27377525c156
This PR adds an OTLP logger to Envoy, which allows users to flexibly configure their log structure (according to OpenTelemetry protos).
The API users the common gRPC configuration, used in ALS, for gRPC settings. The flexible OT fields (body, attributes) are completely configurable and are filled with request/response information based on Envoy access log formatting.
There are still a couple of TODOs -
The 'filter_state_objects_to_log' from the common gRPC config isn't added to the log
Resource attributes are not configurable, and only Envoy node info is added.
Note: this extension was added as v3alpha API.
Risk Level: Low - a new extension, so unlikely to break any existing setups (marked as v3alpha API).
Testing: Unit tests.
Docs Changes:
Release Notes: access log: added a new :ref:OpenTelemetry access logger <envoy_v3_api_msg_extensions.access_loggers.open_telemetry.v3alpha.OpenTelemetryAccessLogConfig> extension, allowing a flexible log structure with native Envoy access log formatting.
Signed-off-by: Itamar Kaminski <itamark@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4f52f9ef554e1522e763c0eaf6ef6833aab283e5
Moved the db generation into a bazel job - which will allow other tasks that also require this data to make use of it more easily/bazely
this also adds buildozer into the bazel env, so we might want to make use of that elsewhere and potentially remove buildozer from the build-image env
Touch #13229
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ a03209d5caf77b36ca4c40671ade8e328cb93fad
Fix#13799. This means integration with external Apache SkyWalking SDK for C++ and destroyed redundant parts of implementations. Hired library is https://github.com/skyAPM/cpp2sky. But it is not released and some of missing parts of implementations which are necessary to achieve clean integration. Currently working private fork of that SDK here. https://github.com/Shikugawa/cpp2sky-envoy.
Additional Description:
cpp2sky dependency policy check list
### MUST
- [x] Cloud Native Computing Foundation (CNCF) approved license
- [x] Dependencies must not substantially increase the binary size unless they are optional (i.e. confined to specific extensions)
- [x] No duplication of existing dependencies
Maybe, the dependencies are described [here](https://github.com/SkyAPM/cpp2sky/blob/main/bazel/repositories.bzl)
- [x] Hosted on a git repository and the archive fetch must directly reference this repository. We will NOT support intermediate artifacts built by-hand located on GCS, S3, etc.
- [x] CVE history appears reasonable, no pathological CVE arcs
- [x] Code review (ideally PRs) before merge
- [x] Security vulnerability process exists, with contact details and reporting/disclosure process
- [x] Tests run in CI
### SHOULD
- [ ] > 1 contributor responsible for a non-trivial number of commits
- [ ] High test coverage (also static/dynamic analysis, fuzzing)
Parts which is used in Envoy have higher test coverage. But gRPC async segment reporter(which is not used in Envoy) has less test.
- [ ] Envoy can obtain advanced notification of vulnerabilities or of security releases
- [ ] Do other significant projects have shared fate by using this dependency?
- [x] Releases (with release notes)
- [x] Commits/releases in last 90 days
Risk Level: Mid
Testing: Unit/Integration
Docs Changes:
Release Notes:
Platform Specific Features:
Signed-off-by: Shikugawa <rei@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 032a8e75928304e011833babea2ca231f32866dd
This patch adds a new tracer to support the SkyWalking tracing mechanism and format.
Risk Level: Low, a new extension.
Testing: Unit
Docs Changes: Added
Release Notes: Added
Signed-off-by: wbpcode <comems@msn.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7d0f89b1011503ecd22f28e347cf7f76cba73057
- Refactor code responsible for processing repository location specs, i.e. checking for the presence of fields like last_updated and interpolation of version. The same code is now usable by both API repository_locations.bzl and bazel/repository_locations.bzl.
- Cleanup reference to repo locations in repository_locations.bzl, now using a consistent set of macros.
- Add API dependencies to dependency dashboard.
Risk level: Low
Testing: Docs build.
Part of #12673
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9181790e3ab21b53ec268470202986b9517c3723
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a60f6853a2c2ebbbfed79dfff0b5b644fd735980
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
Adds a script to create a go module from the generated protobufs as part of #8151.
The module appears to build with the following module declaration:
module github.com/envoyproxy/data-plane-api/api
go 1.12
require (
github.com/census-instrumentation/opencensus-proto v0.2.1
github.com/envoyproxy/protoc-gen-validate v0.1.0
github.com/gogo/protobuf v1.3.0
github.com/golang/protobuf v1.3.2
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
google.golang.org/grpc v1.23.0
)
Add CI automation to trigger the script after the merge to master in envoyproxy.
Risk Level: low
Testing: local build
Docs Changes: none
Release Notes: none
Fixes#8151
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 32e6018f10f9d6de58fd6c3d6844a68a5be7723b
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This is a one-time movement of all UDPA content from envoyproxy/envoy to
cncf/udpa. The permanent home of UDPA will be
https://github.com/cncf/udpa.
Risk level: Low
Testing: Added UDPA service entry to build_test.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e67923fc0b60d0cdf3935fcb30af61d131301fad
Description:
Un-pin opencensus and googleapis to use master versions
Use SetName span method to set route operation names (aligning with other tracers).
Risk Level: low
Testing: Unit tests
Docs Changes: None
Release Notes: None
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ef054f08695b8c883c94674904ad27210aa9ba38
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ d63aa4d05c0968eb335a891c1b1218d2675beac7
They include fixes for future Bazel changes. In particular, the
repository can now build with --incompatible_depset_is_not_iterable,
which will be included in Bazel 0.27.
Description:
Risk Level: low
Testing: bazel build --nobuild ... --incompatible_depset_is_not_iterable
#6995
Signed-off-by: Laurent Le Brun <laurentlb@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0739cd6c9e33d77df2c9ff30c6f43fbb0676e023
Removes golang/protobuf dependency. This is an old @sesmith177 TODO and now the version of rules_go we import has the requisite well known types (WKT). Bumping rules_go to the latest version 0.18.4 (release notes) aligns with PGV which is on 0.18.3.
Risk Level: low (I think but there are WKT changes)
Testing: bazel test //test/...
Signed-off-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ 7bf53149d137dcb2a7b4eddb4f8c23eda23d66e5
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ ecf1085a5b279d1244a02433f204ebba5306d40f
Dynamic link tests in ASAN to save disk space. Partially addresses #1407.
Risk Level: Low (ASAN only)
Testing: CI
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 8cca5c2992693ae143834ad0726aa630058f5910
The python script will print a machine readable output of Envoy's external dependencies
./tools/print_dependencies.py
I opt'd not to include the bazel rule this time around.
Risk Level: Low
Testing: I tested the changes by invoking the script `ci/build_container/docker_build.sh`
Signed-off-by: Dave Protasowski <dprotaso@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 6046405e68c8772c77f0fa74e4705f7e3c49c059
This updates the lyft/protoc-gen-validate dependency to v0.0.13 in order
to fix a bug in validating repeated enums as described in
lyft/protoc-gen-validate#139.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5aeae5969f945ba4a22279004c016431dbcf908c
Part of #2456
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b475977f649c3614371940a819611f7d17e3a4e
To use --distdir option of Bazel (which allows to use previously
fetched tarballs instead of downloading dependencies during
build), all dependencies should use http instead of git and need
to have sha256 sums specified.
Risk Level: low
Testing: n/a
Docs Changes: updated bazel/EXTERNAL_DEPS.md
Release Notes: n/a
Signed-off-by: Michal Rostecki <mrostecki@suse.de>
Mirrored from https://github.com/envoyproxy/envoy @ dc0d35b9ed93b4f5d2ad83f6a7e78832504aeeea
This should unblock Windows, fix the long CLI issue and also fuzz weak link problems.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 92971047ca0b8a97c1004e3b5196916203f9185b
This patch prepares HTTP response when an upstream gRPC service method
uses google.api.HttpBody proto as its message output type.
Risk Level: Low
Testing: Unit
Docs Changes:
Added notes on sending arbitrary content as HTTP response by using google.api.HttpBody.
Release Notes:
Added support for encoding google.api.HttpBody when building HTTP response.
Fixes#3205
Signed-off-by: Dhi Aurrahman <dio@rockybars.com>
Mirrored from https://github.com/envoyproxy/envoy @ 0cf947f44247e83dad4274a39067e738a17b2994
* Bump protobuf dependency post 3.6.0 to 6a4fec616ec4b20f54d5fb530808b855cb664390. This
brings in the following fixes:
- https://github.com/google/protobuf/pull/4813 (fix bug when Any is directly embedded in a map)
- https://github.com/google/protobuf/pull/4812 (preserve snake/camel-case across Any)
- https://github.com/google/protobuf/pull/4811 (provide meaningful errors messages for missing fields)
* Bump PGV dependency to 345b6b478ef955ad31382955d21fb504e95f38c7. This bumps the Protobuf Go
dependency to 1.0, necessary for the above protobuf dependency bump.
Risk level: Low
Testing: Added proto conversion and validation to integration_admin_test.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9bb7df6bffbf9863939a3798299892a74c16c438
Add api_go_grpc rules for metrics_service, als, and trace_service.
To support those changes, also added the necessary go_proto_library
rules in the repositories.bzl definitions for prometheus_metrics_model,
and io_opencensus_trace, and augmented the go_build_test.go to
verify these changes were correct.
Signed-off-by: William Chang <mr.williamchang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ fb7797fd7df696cd239cc1d3792719361b62c684
Signed-off-by: Daniel Hochman <danielhochman@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8e95e7a7b9a737c6a9b58ef7778367f1416ba33b
Found via proto fuzzing of the server config, unbounded ring sizes can lead to resource exhaustion.
Also bumped PGV version, since even with the bound added to cds.proto, the constraint validation was
skipped due to the bug fixed in https://github.com/lyft/protoc-gen-validate/pull/73.
Risk Level: Medium (PGV bump might result in some configs that passed before failing).
Testing: server_fuzz_test regression.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 176e565eaec82d79ebf28d3f2bd0493f68a95180
This brings in https://github.com/lyft/protoc-gen-validate/pull/57 which
should allow builds that depend on data-plane-api (e.g. envoy) to work when git is
configured with a proxy.
Signed-off-by: Snow Pettersen <snowp@squareup.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
data-plane-api(Azure Pipelines)
jmillikin-stripe
Snow Pettersen
Kuat
Bogdan Drutu
William Chang