Also optimize the buffering path to reduce copies.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 59d599981fc94d37485ee59b820c929560e25438
This adds a new response flag SI that is set when the request is
canceled due to a stream idle timeout.
_Risk Level_: Low
_Testing_: Updated existing access log UTs, added test for setting flag
_Docs Changes_: Updated list of response flags
_Release Notes_: Added release note
Signed-off-by: Snow Pettersen <snowp@squareup.com>
Mirrored from https://github.com/envoyproxy/envoy @ a22d983cc87170677243d4aeba6457a4972d7980
1) Add request/response body tapping
2) Add buffered body limits (TBI for transport socket)
3) Add the JSON_BODY_AS_BYTES and JSON_BODY_AS_STRING output
formats for convenience when the body is known to be human
readable.
4) Add JSON output for the file per tap sink.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 9a06dc0777d2809195cb1fc414b05ae7c0660193
1) Add request/response trailer matching
2) Output request/response trailers
3) Refactor matchers to reduce boilerplate and make it harder
to make mistakes when adding new update functions.
4) Split out a match configuration for HTTP request and response
into individual things like request headers, request trailers,
etc. This will make it easier and more logical to add various
types of body matching and wire it up using the existing and/or
logic.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ fbcf6bcaac6ac3c2b13f8c51fb1470a25b240abd
This commit refactors the tap transport socket to use the common
tap extension configuration and tap matching infrastructure. More
match conditions will be added in a future PR as well as additional
cleanups that have been marked with TODOs.
One result of this PR is that the HTTP tap filter can now have a static
configuration as well as write to a file per tap sink.
All future tap PRs should be smaller and more targeted after this one.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ f37ebdc14f4c0adf0e90aabddae833355c0cec1b
This is a rename PR only. It renames the capture transport socket
and associated tools to the tap transport socket. It also updates
some documentation. In a subsequent PR I'm going to refactor the
tap transport socket to use the new common tap framework so that
the tap transport socket can be configured via admin, the HTTP
tap filter can write to a file, the tap transport socket can have
matching, etc.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 7a5849f2a8bcc55fa16da3eaee94d9c99a11147c
This is a MVP for the HTTP tap filter. It includes minimal
infrastructure for the following:
1. Generic tap configuration which in the future will be used for
static config, XDS config, etc. In this MVP the tap can be
configured via a /tap admin endpoint.
2. Generic output configuration which in the future will be used for
different output sinks such as files, gRPC API, etc. In this MVP
the tap results are streamed back out the /tap admin endpoint.
3. Matching infrastructure. In this MVP only matching on request and
response headers are implemented. Both logical AND and logical OR
matches are possible.
4. In this MVP request/response body is not considered at all.
5. All docs are included and with all the caveats the filter is ready
to use for the limited cases it supports (which are likely still to
be useful).
There is a lot of follow on work which I will do in subsequent PRs.
This includes:
1. Merging the existing capture transport socket into this framework.
2. Implementing body support, both for matching on body contents as
well as outputting body data.
3. Tap rate limiting so too many streams do not get tapped.
4. gRPC matching. Using reflection and loaded proto definitions, it will
be possible to match on gRPC fields.
5. JSON matching. If the body parses as JSON, we can allow matching on
JSON fields.
Part of https://github.com/envoyproxy/envoy/issues/1413.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ cf80045587240d494e54e9772949bc9af5eda61f
* api: add proto options for java
* add ci for checking proto options
Signed-off-by: Penn (Dapeng) Zhang <zdapeng@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 02659d411332e9f20d229f482931c15304ea17fd
This identifies downstream disconnects from within the HTTP Connection
Manager and eventually echoes it out via the access log.
Signed-off-by: Venil Noronha <veniln@vmware.com>
Mirrored from https://github.com/envoyproxy/envoy @ 988f7d81f973e17c76db3f900cf145c371d6e8b3
Currently health check failure events are only log if the HealthFlag for a host transition from non-FAILED_ACTIVE_HC to FAILED_ACTIVE_HC. However, since hosts are initialized in the FAILED_ACTIVE_HC state, hosts that never became healthy have no events associated with it.
Since the current health check events only log transitions, we'll have to scan the entire log in order to find the hosts in a current failing state. Then we'll still have to filter the hosts permanently removed from the cluster by the discovery service. This makes the events very difficult to use in operations.
Proposed solution
Both of these 2 issues can be solved by emitting a health check failure event if either of these conditions are true:
If the active health check failed and it's the first health check for a host. This ensures we have events for hosts that never became healthy.
If the active health check failed and a AlwaysLogFailures configuration is set to true, by default this flag is set to false. This makes it very easy to find the hosts currently failing by looking at the last few seconds of logs.
Signed-off-by: Henry Yang <hyang@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ 11e196b67ee9124f33c45f5adf542841386e3c39
*Description*: PGV picks up unused imports in `api/envoy/data/core/v2alpha/health_check_event.proto`.
Error message is:
```
INFO: From ProtoGenValidateCcGenerate external/envoy_api/envoy/data/core/v2alpha/health_check_event.pb.validate.h:
envoy/data/core/v2alpha/health_check_event.proto: warning: Import envoy/api/v2/core/base.proto but not used.
envoy/data/core/v2alpha/health_check_event.proto: warning: Import google/protobuf/wrappers.proto but not used.
envoy/data/core/v2alpha/health_check_event.proto: warning: Import google/protobuf/duration.proto but not used.
```
*Risk Level*: Low
*Testing*: `bazel test //test/...` and running on local instances
*Docs Changes*: none required
*Release Notes*: none required
Signed-off-by: Michael Payne <michael@sooper.org>
Mirrored from https://github.com/envoyproxy/envoy @ c951e6088a5e1214c864448b0ccfd104bf2131ee
This reverts commit ac0bd74. But leaves the API changes as 'not implemented' in order to not scramble the proto field.
#4073 had a bug. The cause has been identified, and a fix PR is forthcoming. However, in the meantime, we want to leave master clean.
Mirrored from https://github.com/envoyproxy/envoy @ 9d094e590f814485b53c7e46a4fb1066a71c36d3
* support failure_mode_deny in ratelimit
Signed-off-by: Rama <rama.rao@salesforce.com>
Mirrored from https://github.com/envoyproxy/envoy @ ac0bd74f6f9716e3a44d1412f795317c30ca770a
This PR adds timestamp field to the HealthCheckEvent message to allow
having it rendered inside the JSON serialized log of a health check
event.
Signed-off-by: Dhi Aurrahman <dio@tetrate.io>
Mirrored from https://github.com/envoyproxy/envoy @ 8d0680feb074999c18998930b7f5f261f8f4a7a0
adds the required visibility rules and delegates the rest to the generic
api_proto_library. I tested the change by doing the following without
getting errors.
./ci/run_envoy_docker.sh './ci/do_ci.sh docs'
I changed the BUILD files using the following commands.
/envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/api_proto_library(/api_proto_library_internal(/g'
envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/"api_proto_library"/"api_proto_library_internal"/g'
Signed-off-by: mickey <mickeyju@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b871c0ab9350882271a490adcee44e613ed9807
Fixes https://github.com/envoyproxy/envoy/issues/743
This is a general cleanup of all of the access logging documentation.
I have reorganized a bunch of things and hidden the various gRPC logging
fields that are not implemented yet.
I've also moved the existing tap protos into a new "output" directory. This
is the best name I could come up for cleanly separating output data that might
be stored outside of any service or configuration.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ c15019e79c832d9f0a09468affaadabc4be3e115
data-plane-api(CircleCI)