This PR is a continuation to #17515 - it adds a script that uses buf to check for breaking changes on proto files in the api folder. It does so by comparing the current api folder against the api folder at the git commit computed by tools/git/last_github_commit.sh - that should hopefully represent the most recent commit on main (if there is a better method to obtain the base branch commit, let me know!).
Adding the script also required re-organizing some of the breaking change detector logic from the previous pr: some levels of abstraction were added, and the detector now expects a git repository and ref as the input for initial state (rather than a proto file).
The script is invoked in do_ci.sh if bazel.api_compat is specified as the CI_TARGET.
This PR also bumps the buf bazel dependency to 0.53.0. If this is preferred to be in a separate PR, let me know and I would be happy to do so
Risk Level: low (hopefully) - the CI script will be invoked in a separate CI pipeline job that can be set to be optional on github. The azure pipeline has been added but needs to be set to optional by a CI maintainer
Testing: New scripts and logic were tested manually; also ran tests from the previous PR and they still pass. hoping to observe more output of this tool through reading CI logs of other PRs once this is merged (this PR should not affect the existing PR workflow - refer to Risk Level)
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: CI script uses a linux binary for buf so it cannot be run outside of docker on non-linux systems
Fixes#3368
Signed-off-by: Yaseen Alkhafaji <yaseena@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ f30c289e456df11d549d7236c56f22da00500e9e
Follow-up to #17375 where it was agreed that protolock is not actively maintained enough to depend on. This PR "migrates" the tests from that PR to use buf instead, and also cleans some of the code per a few of the review comments. Still a few outstanding points:
- buf build on the envoy/api folder requires several protobuf dependencies such as udpa to be available to buf to consume. Suggested solution by buf is to point buf's config to necessary BSR modules that the buf team is hosting.
- These lines are commented out in this PR as I had some trouble automating it for the tests, and it is not necessary for the tests to pass
- May introduce issues if buf is not pointing to the same version of modules that bazel builds for envoy. May need to introduce some way to couple them, or (ideally) find a way to run the breaking change detector without building the dependencies
- Currently bazel is using a binary release of buf (for linux). Goal is to move to building it from source in the near future
- It may be in our interest to expand the list of API-breaking-change rules (buf provides an extensive list of rules we could adopt)
Risk Level: Low
Testing: Tests that evaluate buf against "allowed" and "breaking" protobuf API changes. Currently 4 tests are skipped - 3 of them are PGV-related (we need to communicate our desired PGV rules to the buf team so they may add them in the near future). The 4th is a test I had originally written to evaluate protolock but may not apply to buf ("forcing" a breaking change) - refer to comments
Docs Changes:
Release Notes:
Platform Specific Features: buf binary imported by bazel is linux-only. Hopefully the ["manual"] tags attribute prevents any issues for non-linux users
Signed-off-by: Yaseen Alkhafaji <yaseena@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c74cebb3cc7dfb26488f7cac8eb9258cf9a7588e
In order to unblock a dependency between route_components.proto and the matcher tree, move the alpha Matcher to
its own package. To provide an upgrade path for users using this, we'll keep the old copy around for a single deprecation
cycle.
Risk Level: Low
Testing: n/a
Docs Changes: n/a
Release Notes: Deprecation note
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 40ed33327c23a9e4e88aec448694eb1d03098efd
Risk Level: Low; the protobuf types are not changing at all.
Testing: proto_format and tests
Signed-off-by: Tyler Dixon <tdixon@buf.build>
Co-authored-by: Snow Pettersen <snowp@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ c1bc5e78fa68b86236c8d6237e2db15ce1743459
- update to PGV 0.6.1 (release notes)
- update API versions to corresponding PGV and gRPC versions
- Bazel build tools to 4.0.1
- Zipkin API to 1.0.0
- OpenTelemetry Proto to 0.9.0
Risk Level: Low
Testing:
bazel --nohome_rc test //test/...
bazel --nohome_rc test @envoy_api_canonical//test/... @envoy_api_canonical//tools/...
bazel --nohome_rc build @envoy_api_canonical//envoy/...
tools/dependency/release_dates.py api/bazel/repository_locations.bzl
Signed-off-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ d5ec0351f2a5dc28fc98c1d34deb90e48cf30083
this includes latest xds version which will reduce the warning noise when building protos
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ 25d50667f96ee98d7f6cad8e190a3798ea5cd66f
Moved the db generation into a bazel job - which will allow other tasks that also require this data to make use of it more easily/bazely
this also adds buildozer into the bazel env, so we might want to make use of that elsewhere and potentially remove buildozer from the build-image env
Touch #13229
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ a03209d5caf77b36ca4c40671ade8e328cb93fad
Fix#13799. This means integration with external Apache SkyWalking SDK for C++ and destroyed redundant parts of implementations. Hired library is https://github.com/skyAPM/cpp2sky. But it is not released and some of missing parts of implementations which are necessary to achieve clean integration. Currently working private fork of that SDK here. https://github.com/Shikugawa/cpp2sky-envoy.
Additional Description:
cpp2sky dependency policy check list
### MUST
- [x] Cloud Native Computing Foundation (CNCF) approved license
- [x] Dependencies must not substantially increase the binary size unless they are optional (i.e. confined to specific extensions)
- [x] No duplication of existing dependencies
Maybe, the dependencies are described [here](https://github.com/SkyAPM/cpp2sky/blob/main/bazel/repositories.bzl)
- [x] Hosted on a git repository and the archive fetch must directly reference this repository. We will NOT support intermediate artifacts built by-hand located on GCS, S3, etc.
- [x] CVE history appears reasonable, no pathological CVE arcs
- [x] Code review (ideally PRs) before merge
- [x] Security vulnerability process exists, with contact details and reporting/disclosure process
- [x] Tests run in CI
### SHOULD
- [ ] > 1 contributor responsible for a non-trivial number of commits
- [ ] High test coverage (also static/dynamic analysis, fuzzing)
Parts which is used in Envoy have higher test coverage. But gRPC async segment reporter(which is not used in Envoy) has less test.
- [ ] Envoy can obtain advanced notification of vulnerabilities or of security releases
- [ ] Do other significant projects have shared fate by using this dependency?
- [x] Releases (with release notes)
- [x] Commits/releases in last 90 days
Risk Level: Mid
Testing: Unit/Integration
Docs Changes:
Release Notes:
Platform Specific Features:
Signed-off-by: Shikugawa <rei@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 032a8e75928304e011833babea2ca231f32866dd
Specifically interested in
872b28c457
which reduces the startup cost on golang
Signed-off-by: John Howard <howardjohn@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4cb14ea2da6edb9acd4ad96da387d16d859ca4aa
There is an unfortunate change in 3.14 that changed go_package for WKT, which necessitates updating several go dependencies as well:
protoc-gen-validate to 2020-11-30
transitive dependencies from protoc-gen-validate
gazelle to 0.22.2
rules_go to 0.25.0
go to 1.15.5
Risk Level: low
Signed-off-by: Kuat Yessenov <kuat@google.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b3bb0f9847497bc97951172d248fad3c88096829
As per the decision to move the cncf/udpa repository to cncf/xds branding.
Also updated cncf/udpa hash and updated identifier handling (moved from repeated to a flat string).
Risk level: Low (the only breaking API changes affect not-implemented-hide fields).
Testing: New unit tests for path components.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8c4a3c77a7de016a118aacc4cea933951b85e589
This patch adds a new tracer to support the SkyWalking tracing mechanism and format.
Risk Level: Low, a new extension.
Testing: Unit
Docs Changes: Added
Release Notes: Added
Signed-off-by: wbpcode <comems@msn.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7d0f89b1011503ecd22f28e347cf7f76cba73057
The use of last_updated was ambiguous (is it when an Envoy contributor
creates a PR, merges a commit, or when the dependency is released?).
We really are after the release date as a measure of how stale the
dependency is.
This patch introduces a tool, tools/dependency/release_dates.py, that
uses the GitHub API to compute release date. If a mismatch is detected, an
error is raised.
This patch also introduces a dependency validation CI job that gathers existing
scripts and the release_dates.py script into a single job.
Signed-off-by: Harvey Tuch <htuch@google.com>
Co-authored-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ 91f2bb75a34e1068dcc91de1cafca9dad92feecb
- Refactor code responsible for processing repository location specs, i.e. checking for the presence of fields like last_updated and interpolation of version. The same code is now usable by both API repository_locations.bzl and bazel/repository_locations.bzl.
- Cleanup reference to repo locations in repository_locations.bzl, now using a consistent set of macros.
- Add API dependencies to dependency dashboard.
Risk level: Low
Testing: Docs build.
Part of #12673
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9181790e3ab21b53ec268470202986b9517c3723
This reverts commit ca7dc5344765f3e7fc3c1a47927e8c1e3e6cefd3.
This PGV upgrade is breaking my local build.
Risk Level: Low
Testing: N/a
Signed-off-by: James Buckland <jbuckland@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7feaadb004dc3d577fb1c0f4f383094a0732ee36
This patch introduces support for encoding/decoding
udpa::core::v1::ResourceLocator, in addition to the existing support for
udpa::core::v1::ResourceName. Some refactoring and attention to the
URI/URN/URL distinction (as per RFC3986) is introduced.
Part of #11264
Risk level: Low (not used)
Testing: Additional unit tests for URLs added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 76525d8d9e3fb8b0b979f8ada5e152a3ddf14019
These map between the structured udpa::core::v1::ResourceName message and flat udpa:// URI
representations of resource names.
Risk level: Low
Testing: Unit tests added.
Part of #11264.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ a707b460b12920e79161e0a4ed5033164b683b3e
This PR replaces #11058, taking a slightly different approach. We
utilize field options to annotate fields that should be set for
untrusted environments with [configure_for_untrusted_downstream,
configure_for_untrusted_downstream]. Defaults are provided out-of-band,
in a manifest files in docs/edge_defaults_manifest.yaml.
Protodoc glues the manifest and options together when generating field
documentation, providing an additional notice for sensitive fields.
This PR depends on #11108 first merging to provide the pip3 build
infrastructure.
Risk level: Low (docs only).
Testing: Inspection of generated docs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 31128e7dc22355876020188bc8feb99304663041
This patch introduces the tools/docs for a new flow for protoxform: freeze.
This performs a major version freeze and bump by modifying
package_version_status. Tooling now needs to deal with 3 possible versions (v2,
v3, v4alpha). The following contributions are made in the patch:
* Tooling added to support v2 APIs freeze, transitioning v3 to active and generating v4alpha.
* A new tool, merge_active_shadow.py, is introduced to combine hand-edited active v3 API protos with the v2 implied shadows for deprecated fields.
* The type database now spans v2 and v3 (and implied v4alpha files)
* api_proto_plugins can now take extra args on the Bazel CLI; this is used to plumb in the freeze arg to protoxform.
Docs are updated to reflect the v3 flow that developers should now
follow.
Part of #10355
This is the tools/docs component of #10601. See #10601 for the resulting
API protos generated by a freeze/fix cycle.
Risk level: Low (tooling only, no API changes yet)
Testing: new golden protos tests for protoxform, unit tests for merge_active_shadow.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ de222c3acf1fbf124ca44f50495fc4b8e9aba525
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
This enables "non-strict" header validations, that match the ones in place with Envoy's ASSERT(valid()) code. The default strict: true checks checked for RFC-compliance, which may break previously valid configs.
Part of #10318
Signed-off-by: Asra Ali <asraa@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 97a76d991766d96df2e84a1f2e33b69cae844471
This is the new style for indicating a file is WiP and subject to
breaking changes. Rather than rely on alpha major versions, which are
coarse grained and introduce migration difficulties for operators, we
use a file-level annotation.
Risk level: Low
Testing: API/docs build, manual inspection of docs.
Fixes#9769.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Co-authored-by: htuch <htuch@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 423fe76d5572bb4f1505391ccaaacf39b2bf2c85
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a60f6853a2c2ebbbfed79dfff0b5b644fd735980
Inspired (but no longer required) by #9500. This is a relatively simple
API migration feature to support, so worth adding for future flexibility
Risk level: Low
Testing: New protoxform golden tests added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4efbf45c10e2d2afd84fee2466e476cb8fc0a804
Signed-off-by: William A Rowe Jr <wrowe@pivotal.io>
Signed-off-by: Yechiel Kalmenson <ykalmenson@pivotal.io>
Signed-off-by: Sunjay Bhatia <sbhatia@pivotal.io>
Mirrored from https://github.com/envoyproxy/envoy @ daeb9850dbcd78da21a2e8ff9854564830cefb19
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Updates protoc-gen-validate to a18376249eb51cdd517f67fe8703897322812e6d and
adds tests to the RBAC common filter code to prove that nested validations
work as expected.
Risk Level: low
Testing: unit test
Doc Changes: n/a
Release Notes: n/a
Fixes: #8715, #5324
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28ce96326e154dba0879b4fa330eb33f29581634
Handle udpa.type.v1.TypedStruct embedded into google.protobuf.Any
Risk Level: Low
Testing: Unit and Integration Tests
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b4eaa177332a131fb41fd086fcb4bec63f9130d9
This picks up fixes for the Windows build and a C preprocessor defect
Signed-off-by: Yechiel Kalmenson <ykalmenson@pivotal.io>
Signed-off-by: William Rowe <wrowe@pivotal.io>
Mirrored from https://github.com/envoyproxy/envoy @ bd58c11acbd4c364258aa0e2b5f1a4f130690cb6
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Signed-off-by: Emil Mikulic <g-easy@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 816d6f14fd65329a8511351e74b385fafa3160a8
This is a one-time movement of all UDPA content from envoyproxy/envoy to
cncf/udpa. The permanent home of UDPA will be
https://github.com/cncf/udpa.
Risk level: Low
Testing: Added UDPA service entry to build_test.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e67923fc0b60d0cdf3935fcb30af61d131301fad
Description:
Un-pin opencensus and googleapis to use master versions
Use SetName span method to set route operation names (aligning with other tracers).
Risk Level: low
Testing: Unit tests
Docs Changes: None
Release Notes: None
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ef054f08695b8c883c94674904ad27210aa9ba38
data-plane-api(Azure Pipelines)