Moved the db generation into a bazel job - which will allow other tasks that also require this data to make use of it more easily/bazely
this also adds buildozer into the bazel env, so we might want to make use of that elsewhere and potentially remove buildozer from the build-image env
Touch #13229
Signed-off-by: Ryan Northey <ryan@synca.io>
Mirrored from https://github.com/envoyproxy/envoy @ a03209d5caf77b36ca4c40671ade8e328cb93fad
Fix#13799. This means integration with external Apache SkyWalking SDK for C++ and destroyed redundant parts of implementations. Hired library is https://github.com/skyAPM/cpp2sky. But it is not released and some of missing parts of implementations which are necessary to achieve clean integration. Currently working private fork of that SDK here. https://github.com/Shikugawa/cpp2sky-envoy.
Additional Description:
cpp2sky dependency policy check list
### MUST
- [x] Cloud Native Computing Foundation (CNCF) approved license
- [x] Dependencies must not substantially increase the binary size unless they are optional (i.e. confined to specific extensions)
- [x] No duplication of existing dependencies
Maybe, the dependencies are described [here](https://github.com/SkyAPM/cpp2sky/blob/main/bazel/repositories.bzl)
- [x] Hosted on a git repository and the archive fetch must directly reference this repository. We will NOT support intermediate artifacts built by-hand located on GCS, S3, etc.
- [x] CVE history appears reasonable, no pathological CVE arcs
- [x] Code review (ideally PRs) before merge
- [x] Security vulnerability process exists, with contact details and reporting/disclosure process
- [x] Tests run in CI
### SHOULD
- [ ] > 1 contributor responsible for a non-trivial number of commits
- [ ] High test coverage (also static/dynamic analysis, fuzzing)
Parts which is used in Envoy have higher test coverage. But gRPC async segment reporter(which is not used in Envoy) has less test.
- [ ] Envoy can obtain advanced notification of vulnerabilities or of security releases
- [ ] Do other significant projects have shared fate by using this dependency?
- [x] Releases (with release notes)
- [x] Commits/releases in last 90 days
Risk Level: Mid
Testing: Unit/Integration
Docs Changes:
Release Notes:
Platform Specific Features:
Signed-off-by: Shikugawa <rei@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 032a8e75928304e011833babea2ca231f32866dd
Specifically interested in
872b28c457
which reduces the startup cost on golang
Signed-off-by: John Howard <howardjohn@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4cb14ea2da6edb9acd4ad96da387d16d859ca4aa
There is an unfortunate change in 3.14 that changed go_package for WKT, which necessitates updating several go dependencies as well:
protoc-gen-validate to 2020-11-30
transitive dependencies from protoc-gen-validate
gazelle to 0.22.2
rules_go to 0.25.0
go to 1.15.5
Risk Level: low
Signed-off-by: Kuat Yessenov <kuat@google.com>
Co-authored-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ b3bb0f9847497bc97951172d248fad3c88096829
As per the decision to move the cncf/udpa repository to cncf/xds branding.
Also updated cncf/udpa hash and updated identifier handling (moved from repeated to a flat string).
Risk level: Low (the only breaking API changes affect not-implemented-hide fields).
Testing: New unit tests for path components.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 8c4a3c77a7de016a118aacc4cea933951b85e589
This patch adds a new tracer to support the SkyWalking tracing mechanism and format.
Risk Level: Low, a new extension.
Testing: Unit
Docs Changes: Added
Release Notes: Added
Signed-off-by: wbpcode <comems@msn.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7d0f89b1011503ecd22f28e347cf7f76cba73057
The use of last_updated was ambiguous (is it when an Envoy contributor
creates a PR, merges a commit, or when the dependency is released?).
We really are after the release date as a measure of how stale the
dependency is.
This patch introduces a tool, tools/dependency/release_dates.py, that
uses the GitHub API to compute release date. If a mismatch is detected, an
error is raised.
This patch also introduces a dependency validation CI job that gathers existing
scripts and the release_dates.py script into a single job.
Signed-off-by: Harvey Tuch <htuch@google.com>
Co-authored-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ 91f2bb75a34e1068dcc91de1cafca9dad92feecb
A few changes that wrapup #12673.
* Python/Go dependencies that aren't part of the Envoy binary build
don't make sense to track in repository_locations.bzl, since they
have their own language specific metadata (e.g. requirements.txt)
or are in many cases transitively implied.
* Ensure that the full set of dependencies visible to bazel query
is now validated. This requires that we explicitly call out
transitive dependencies that are implied by direct dependencies
in repository_locations.bzl. A new annotation `implied_untracked_deps`
is used.
Fixes#12673
Risk level: Low
Testing: validate.py.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 499f46a24f59eeb4fc3b28e24f6b9191b009580d
* dependencies: CVE scanner for repository_locations.bzl metadata.
This is a custom CVE scanner that consumes the NIST CVE database and
heuristically matches against the CPEs, versions and last update stamps
in repository_locations.bzl.
Future PRs will create a CI job that runs this on a periodic basis
(every few hours) to provide a CVE early warning system.
Example output:
Based on heuristic matching with the NIST CVE database, Envoy may be vulnerable to:
CVE ID: CVE-2019-19391
CVSS v3 score: 9.1
Severity: CRITICAL
Published date: 2019-11-29
Last modified date: 2019-12-19
Dependencies: com_github_luajit_luajit
Description: ** DISPUTED ** In LuaJIT through 2.0.5, as used in Moonjit before
2.1.2 and other products, debug.getinfo has a type confusion issue
that leads to arbitrary memory write or read operations, because
certain cases involving valid stack levels and > options are
mishandled. NOTE: The LuaJIT project owner states that the debug
libary is unsafe by definition and that this is not a vulnerability.
When LuaJIT was originally developed, the expectation was that the
entire debug library had no security guarantees and thus it made no
sense to assign CVEs. However, not all users of later LuaJIT
derivatives share this perspective.
Affected CPEs:
- cpe:2.3🅰️moonjit_project:moonjit:*
- cpe:2.3🅰️luajit:luajit:*
Risk level: Low
Testing: cve_scan_test.py unit tests, manual.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7d50518ce44aacc702d252eb05eb603c69461834
- Refactor code responsible for processing repository location specs, i.e. checking for the presence of fields like last_updated and interpolation of version. The same code is now usable by both API repository_locations.bzl and bazel/repository_locations.bzl.
- Cleanup reference to repo locations in repository_locations.bzl, now using a consistent set of macros.
- Add API dependencies to dependency dashboard.
Risk level: Low
Testing: Docs build.
Part of #12673
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9181790e3ab21b53ec268470202986b9517c3723
This reverts commit ca7dc5344765f3e7fc3c1a47927e8c1e3e6cefd3.
This PGV upgrade is breaking my local build.
Risk Level: Low
Testing: N/a
Signed-off-by: James Buckland <jbuckland@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7feaadb004dc3d577fb1c0f4f383094a0732ee36
Add CheckedExpr as oneof expression in RBAC proto
Additional Description: CheckedExpr is type check CEL expression. It is future proof and offers better evaluation performance (not for C++ yet, but for other languages). There will be a separate PR to plumb checked expr to RBAC filter.
Risk Level: low
Testing: existing tests
Signed-off-by: jiangtaoli2016 <jiangtao@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 3b8c61b5e5eb4dfddf3c4c0cae9e04eafe1ac61a
This patch introduces support for encoding/decoding
udpa::core::v1::ResourceLocator, in addition to the existing support for
udpa::core::v1::ResourceName. Some refactoring and attention to the
URI/URN/URL distinction (as per RFC3986) is introduced.
Part of #11264
Risk level: Low (not used)
Testing: Additional unit tests for URLs added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 76525d8d9e3fb8b0b979f8ada5e152a3ddf14019
These map between the structured udpa::core::v1::ResourceName message and flat udpa:// URI
representations of resource names.
Risk level: Low
Testing: Unit tests added.
Part of #11264.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ a707b460b12920e79161e0a4ed5033164b683b3e
This PR replaces #11058, taking a slightly different approach. We
utilize field options to annotate fields that should be set for
untrusted environments with [configure_for_untrusted_downstream,
configure_for_untrusted_downstream]. Defaults are provided out-of-band,
in a manifest files in docs/edge_defaults_manifest.yaml.
Protodoc glues the manifest and options together when generating field
documentation, providing an additional notice for sensitive fields.
This PR depends on #11108 first merging to provide the pip3 build
infrastructure.
Risk level: Low (docs only).
Testing: Inspection of generated docs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 31128e7dc22355876020188bc8feb99304663041
This patch introduces the tools/docs for a new flow for protoxform: freeze.
This performs a major version freeze and bump by modifying
package_version_status. Tooling now needs to deal with 3 possible versions (v2,
v3, v4alpha). The following contributions are made in the patch:
* Tooling added to support v2 APIs freeze, transitioning v3 to active and generating v4alpha.
* A new tool, merge_active_shadow.py, is introduced to combine hand-edited active v3 API protos with the v2 implied shadows for deprecated fields.
* The type database now spans v2 and v3 (and implied v4alpha files)
* api_proto_plugins can now take extra args on the Bazel CLI; this is used to plumb in the freeze arg to protoxform.
Docs are updated to reflect the v3 flow that developers should now
follow.
Part of #10355
This is the tools/docs component of #10601. See #10601 for the resulting
API protos generated by a freeze/fix cycle.
Risk level: Low (tooling only, no API changes yet)
Testing: new golden protos tests for protoxform, unit tests for merge_active_shadow.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ de222c3acf1fbf124ca44f50495fc4b8e9aba525
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
This enables "non-strict" header validations, that match the ones in place with Envoy's ASSERT(valid()) code. The default strict: true checks checked for RFC-compliance, which may break previously valid configs.
Part of #10318
Signed-off-by: Asra Ali <asraa@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 97a76d991766d96df2e84a1f2e33b69cae844471
This is the new style for indicating a file is WiP and subject to
breaking changes. Rather than rely on alpha major versions, which are
coarse grained and introduce migration difficulties for operators, we
use a file-level annotation.
Risk level: Low
Testing: API/docs build, manual inspection of docs.
Fixes#9769.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Co-authored-by: htuch <htuch@users.noreply.github.com>
Mirrored from https://github.com/envoyproxy/envoy @ 423fe76d5572bb4f1505391ccaaacf39b2bf2c85
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ a60f6853a2c2ebbbfed79dfff0b5b644fd735980
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
Inspired (but no longer required) by #9500. This is a relatively simple
API migration feature to support, so worth adding for future flexibility
Risk level: Low
Testing: New protoxform golden tests added.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4efbf45c10e2d2afd84fee2466e476cb8fc0a804
This is intended to simplify the internal handling of deprecations during API boosting.
See https://docs.google.com/document/d/1mGO9LtVo7t4Lph7WlmyGCxXye3h6j29z3JZvIBbs_D0/edit
Ultimately the plan is to hide this all as a build artifact in Bazel cache, but due to the technical complexity of the pure Bazel solution (involving changes spanning PGV, protoxform, API build rules), we will use checked-in artifacts for 1.13.0.
Risk level: Low
Testing: Additional API test and protoxform golden test.
Part of #8082
This should be unwound in the future with #9479
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ da5fb5ce6ec7cc6e4693656a5ba2161107f6ac4a
Signed-off-by: William A Rowe Jr <wrowe@pivotal.io>
Signed-off-by: Yechiel Kalmenson <ykalmenson@pivotal.io>
Signed-off-by: Sunjay Bhatia <sbhatia@pivotal.io>
Mirrored from https://github.com/envoyproxy/envoy @ daeb9850dbcd78da21a2e8ff9854564830cefb19
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Updates protoc-gen-validate to a18376249eb51cdd517f67fe8703897322812e6d and
adds tests to the RBAC common filter code to prove that nested validations
work as expected.
Risk Level: low
Testing: unit test
Doc Changes: n/a
Release Notes: n/a
Fixes: #8715, #5324
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 28ce96326e154dba0879b4fa330eb33f29581634
Handle udpa.type.v1.TypedStruct embedded into google.protobuf.Any
Risk Level: Low
Testing: Unit and Integration Tests
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ b4eaa177332a131fb41fd086fcb4bec63f9130d9
This provides canonical BUILD formatting and puts protoxform in charge
of being able to determine import paths, without having to worry about
Bazel implications.
Part of #8082.
Risk level: Low
Testing: tools/proto_sync.py, visual inspection of diffs.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e53f40f0e5ccc84fca5cd350416fe0f2accf8229
As part of #8082, we want to be able to (1) automatically generate BUILD
files and (2) treat packages as atomic from a "upgrade / do not upgrade"
decision perspective. This is simplified by having our BUILD targets at
package granularity, since this is what the protoxform plugin operates
on.
This PR broadens the package-level treatment that was already introduced
for Go in #8003 to Python and C++. This simplifies BUILD files
significantly and opens the way to automated generation.
There is some technical debt introduced, since all visibility controls
have been removed. This is slated for reintroduction in
https://github.com/envoyproxy/envoy/issues/8491.
As a bonus (useful for BUILD file generation), also removed the
inconsistency in BUILD package target naming for packages in envoy.api.*
and envoy.type.*. E.g. //envoy/api/v2:v2 is now //envoy/api/v2:pkg.
Risk level: Low (but this will break internal builds and require BUILD
fixups to consuming projects).
Testing: bazel test //test/... @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4e858f17fe08224c9c089240908ccd0c518e01a7
Adds a script to create a go module from the generated protobufs as part of #8151.
The module appears to build with the following module declaration:
module github.com/envoyproxy/data-plane-api/api
go 1.12
require (
github.com/census-instrumentation/opencensus-proto v0.2.1
github.com/envoyproxy/protoc-gen-validate v0.1.0
github.com/gogo/protobuf v1.3.0
github.com/golang/protobuf v1.3.2
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
google.golang.org/grpc v1.23.0
)
Add CI automation to trigger the script after the merge to master in envoyproxy.
Risk Level: low
Testing: local build
Docs Changes: none
Release Notes: none
Fixes#8151
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 32e6018f10f9d6de58fd6c3d6844a68a5be7723b
Support tags[] arg for more specific build control.
Where the underlying bazel primitives support tags[], envoy_() should support them.
Risk Level: Low
Testing: Local on Windows and Linux CI
Signed-off-by: Yechiel Kalmenson <ykalmenson@pivotal.io>
Mirrored from https://github.com/envoyproxy/envoy @ 0bce3a14ab8ac0dbb9e9c3f291b9fc5e4f25ea8d
This change defines C++ grpc bindings for the external AuthZ interface. In the istio Security WG we're planning on using this interface to provide transparent authentication of requests. As this interface lies on the data plane we wish to implement it in C++ to maintain consistent and predictable performance
The changes allow a C++ implementation built using bazel to import Envoy as a workspace dependency and generate the C++ gRPC bindings directly without the need for complex import and generation scripts.
Signed-off-by: Nick A. Smith <nickrmc83@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 41932e9e6f3c932f37f77ae0a5191d65bb7ec8eb
This picks up fixes for the Windows build and a C preprocessor defect
Signed-off-by: Yechiel Kalmenson <ykalmenson@pivotal.io>
Signed-off-by: William Rowe <wrowe@pivotal.io>
Mirrored from https://github.com/envoyproxy/envoy @ bd58c11acbd4c364258aa0e2b5f1a4f130690cb6
Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.
Risk Level: low
Testing: builds
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ e7f0b7176efdc65f96eb1697b829d1e6187f4502
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
data-plane-api(Azure Pipelines)