Adding a filter for access logs that will decide whether to log based on dynamic metadata.
Signed-off-by: davidraskin <draskin@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d7c7e9a79eed80afa56a0b05cf6adf0516750d14
Changing the HCM to not adjust the date header when serving (side-car) cached responses.
This includes adding a status flag for cached responses which propagates to access logs etc, as well as some test cleanup to support polling for multiple log entries.
Risk Level: Low (only affects pre-alpha cache filter)
Testing: new UT, enhanced IT
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Mirrored from https://github.com/envoyproxy/envoy @ bb74a91ac413d81f4b1874ed6474df93e4ca2782
To resolve#10274, adding max stream duration for upstream connection.
Signed-off-by: shikugawa <rei@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 6151a69f9c0dc4aa7938d987036ec00eedb818d5
This patch performs a major version freeze and bump by modifying
package_version_status, using the tooling developed in #10636.
Specifically:
v2 APIs are frozen (except for where they are the latest in their
package history and still active)
v3 APIs are transitioned to be active
Candidate v4alpha APIs are generated (not used by Envoy yet)
Fixes#10355
Risk level: medium (entire API's files are modified, visually verified
to ensure things look sane, all tests pass)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 549164c42cae84b59154ca4c36009e408aa10b52
Part of https://github.com/envoyproxy/envoy/issues/10355, this patch introduces additional
annotations to the API to support automatic inference (and developer documentation) of where the
active developer editable version of a file is, and which files are frozen or machine generated.
Risk level: Low (API annotations only)
Testing: CI
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4c5f4310bb8a19a38e5377d7a2d2dc6aa4560f47
Modifies the well-known-names of the built-in access loggers to
use the same name as the extension build system.
Risk Level: low, previous name is still accepted
Testing: existing tests
Docs Changes: updated names
Release Notes: updated
Deprecated: old names are logged as deprecated
Signed-off-by: Stephan Zuercher <zuercher@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9cc7a5caf2961947d6c5eea18a1afbbbc13af82b
In which we convert every v3alpha reference to v3. In future revs of the
stable API versioning policy, we will develop better tooling to support
> 2 alpha and stable versions. For v3, it seems reasonable to just mv
v3alpha to v3, since there should be no external consumers yet.
Risk level: Low
Testing: bazel test //test/..., CI.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 5248a4fb7d4c2a3d1fa151f944d3a63f6b7a06cf
This patch introduces a new checker, tools/api/validate_structure.py, that is run as part of the
bazel.api CI job. It ensures that the package layout for the API doesn't violate some constraints,
largely reflecting the heuristics we used for v3alpha migration.
Along the way, I discovered there were some packages that were versionless and not boosted to
v3alpha, and there were some extensions left behind in envoy.config. These are fixed as well to
allow the validation to succeed.
Risk level: Low
Testing: tools/api/validate_structure.py passes, bazel test //test/...
Fixes#9580
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 29b30911dbfb3f9760efeb28238ceac36e1a1a23
This allows for a clean separation of config/service in v3. This is a
continuation of #9548.
Risk level: Low
Testing: bazel test //test/...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c3bddaee1912fcd1fedc4786aee830b2e4a7c599
Description:
Move packages around for #8120 and #8121
Risk Level: Med around messing up build.
Testing: CI
Docs Changes: in API/STYLE.md
Release Notes: N/A (v3alpha is not in use yet)
Fixes#8120
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 1371f2ef46582a72b5b3971147bd87c534011731
In order to get file level move annotation, import has to be before options.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 062c895f499382ae61dead16db2a7e78b9146525
Instead of formatting options heuristically, which will erase new annotations without changing protoxform, use proto descriptor to format options, and enforce its order as well.
Risk Level: Low
Testing: CI
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ dfe687d49574ef7eb1bf84867bf571e805a2bf97
This PR avoids having to include an API type database in the Envoy build
by introducing a message annotation option that allows Envoy to
determine earlier corresponding message types via descriptor inspection.
The ApiTypeDb is now ApiTypeOracle and utilizes these annotations.
Risk level: Low
Testing: Existing API and verison upgrade tests pass.
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 297f7a73b3f93bccf8af73c0a555ae52bce6cecb
Deprecate google.protobuf.Struct config members in the following types:
envoy.api.v2.auth.PrivateKeyProvider
envoy.api.v2.listener.UdpListenerConfig
envoy.config.filter.accesslog.v2.ExtensionFilter
Risk Level: Low
Testing: Unit Test
Docs Changes: N/A
Release Notes: N/A
Fixes: #8403
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d36adbb5c4aad13af21f109b86978b8478f54409
Generate or format next free field annotation via protoxform.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Fixes#8429
Signed-off-by: Yi Tang <ssnailtang@gmail.com>
Mirrored from https://github.com/envoyproxy/envoy @ 986173ed516dcc1c3dea7db90659ed993d0aad75
This patch moves away from the paradigm of sed-style upgrading of every v2
package to v3alpha. Instead, an additional type analysis phase is
performed prior to protoxform by a protoc plugin known as the "type
whisperer".
The type whisperer produces structured type dependency information for
each .proto. The tools/type_whisperer/typedb_gen.py tool then knits
these together to provide an API-wide type dependency graph. This is
then used to determine which types need upgrading (either they have
breaking changes or transitively depend on types with such changes).
Only packages with upgraded types now undergo the v2 -> v3alpha
transition.
The API type database is checked into
source/common/config/api_type_db.pb. This may seem a strange location,
but in the future we will include the type database as a build artifact
for the Envoy binary, as it will be used by the reflection-based version
converter to find the type upgrade path for input proto.
Risk level: Low (the v3alpha protos are not used yet).
Testing: fix_format, manual inspection of diffs, bazel test //test/...,
docs build.
Part of #8082Fixes#8490
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ ad57b58cfbb256af41a467260dce2a8013b7a7fa
Via ./api/migration/v3alpha.sh. This picks up the changes since the last
sync, in particular the major reformat in #8309.
Risk level: Low (not used yet).
Testing: bazel build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ c41cfbf6a33b8115a7e29a2b4a926aad4cd062be
Fixes#7982
Defines a package level proto library and its associated internal go_proto_library.
Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).
I deliberately avoided touching anything under udpa since it's being moved to another repository.
Risk Level: low
Testing: build completes
Signed-off-by: Kuat Yessenov <kuat@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ d504fde0ffd97017d1ddff8caa9a3b46bba9ae48
This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.
The objective is provide a baseline which we can compare the output from
tooling described in #8083 in later PRs, providing smaller visual diffs.
The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.
There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.
Part of #8083.
Risk level: Low
Testing: build @envoy_api//...
Signed-off-by: Harvey Tuch <htuch@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 085d72b490c124a02849812798f5513a8df9ae72
Description:
Before this change, Envoy would silently ignore the `x-envoy-*` header if a
client specifies an invalid value for this header (e.g. `x-envoy-max-retries: 3.0`).
Introduce a `strict_check_headers` config option for `envoy.router` that adds
optional support to reject requests with invalid values for the following headers:
- x-envoy-upstream-rq-timeout-ms
- x-envoy-upstream-rq-per-try-timeout-ms
- x-envoy-max-retries
- x-envoy-retry-on
- x-envoy-retry-grpc-on
On rejection, Envoy responds with HTTP status 400 and sets a new response flag
`IH` to indicate the reason was due to an invalid header.
Risk Level: Low/medium
Testing: unit tests
- unit test: `FilterUtility::StrictHeaderChecker`
- test that router rejects request with HTTP status 400 + setting the `IH` response flag
- test that config validation rejects unsupported values
- manual end-to-end test `client -> envoy -> upstream server` to verify that
Envoy returns a 400 and sets the response flag in the logs
Docs Changes:
- add inline docs to `router.proto` for `strict_check_headers`
- add inline docs to `accesslog.proto` for `IH` response flag
Release Notes: updated for router and accesslog
Fixes#6482
Signed-off-by: Xiao Yu <xyu@stripe.com>
Mirrored from https://github.com/envoyproxy/envoy @ ecd03a4eed07e1cfea9e9844e519b7fffada437a
This adds a new response flag SI that is set when the request is
canceled due to a stream idle timeout.
_Risk Level_: Low
_Testing_: Updated existing access log UTs, added test for setting flag
_Docs Changes_: Updated list of response flags
_Release Notes_: Added release note
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ a22d983cc87170677243d4aeba6457a4972d7980
Add support of Any as opaque config for extensions. Deprecates Struct configs. Fixes#4475.
Risk Level: Low
Testing: CI
Docs Changes: Added.
Release Notes: Added.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 851f591f4ed84594e5e5041e7ada4167a4f3a273
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
This identifies downstream disconnects from within the HTTP Connection
Manager and eventually echoes it out via the access log.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 988f7d81f973e17c76db3f900cf145c371d6e8b3
API for #4475.
Risk Level: Low (not implemented)
Testing: CI
Docs Changes: Added but hided
Release Notes: N/A, will add when adding impl.
Signed-off-by: Lizan Zhou <lizan@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 45a460fabf34698a875060482de96f7f618bdc9f
This reverts commit ac0bd74. But leaves the API changes as 'not implemented' in order to not scramble the proto field.
#4073 had a bug. The cause has been identified, and a fix PR is forthcoming. However, in the meantime, we want to leave master clean.
Mirrored from https://github.com/envoyproxy/envoy @ 9d094e590f814485b53c7e46a4fb1066a71c36d3
* support failure_mode_deny in ratelimit
Signed-off-by: Rama <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ ac0bd74f6f9716e3a44d1412f795317c30ca770a
Fixes https://github.com/envoyproxy/envoy/issues/743
This is a general cleanup of all of the access logging documentation.
I have reorganized a bunch of things and hidden the various gRPC logging
fields that are not implemented yet.
I've also moved the existing tap protos into a new "output" directory. This
is the best name I could come up for cleanly separating output data that might
be stored outside of any service or configuration.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ c15019e79c832d9f0a09468affaadabc4be3e115
define a access log filter to filter requests based on the
value of a specified header.
This is the initial data plane api change for the issue envoyproxy/envoy#2544.
Signed-off-by: Kevin Chan <kchan@evernote.com>
1) Add ability to runtime filter to configure default, divisor, and
whether independent randomness is used.
2) Also add LE to the comparison filter.
Signed-off-by: Matt Klein <mklein@lyft.com>
In the proxy we've added an Unauthorized response flag. This PR adds the same to filter access logs.
Once this PR is merged it will be possible to set the flag in source/common/access_log/grpc_access_log_impl.cc
See also, comment in envoyproxy/envoy#2415
Signed-off-by: Saurabh Mohan <saurabh+github@tigera.io>
I went through and reorganized things to make the v2 docs more
human browsable. I also did a few misc cleanups. There is a lot
more to do here which I'm hoping to find a contractor to pay to
work on, but this is a step in the right direction.
Signed-off-by: Matt Klein <mklein@lyft.com>
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
data-plane-api(CircleCI)
Kchan
Matt Klein
Saurabh Mohan
Daniel Hochman
Kuat