Add a new config option under access_log called json_format. This is a single level dictionary that contains strings as keys, and envoy access log format specifiers (such as %PROTOCOL%) as values.
The specifiers will be replaced with actual values at logging time. I call this dictionary the "format dictionary" (as opposed to "format string").
You can specify only one of format (format string) or json_format (format dictionary). If neither are there, we fall back to the default string format.
Add the correct plumbing inside the configuration parsing to handle this.
Add a new access log formatter class that is instantiated with the format dictionary. It maintains the mapping of dictionary keys to loggers
Create a new class called FormatterProvider, to distinguish things that actually extract the information from a request. The things that combine together a bunch of FormatterProviders are still called Formatters. This is primarily a semantic/naming difference, but imo these are two conceptually separate things. There is, however no API difference, and if people are truly opposed to this, I could just merge them back into one Formatter class. This also provides a better foundation for adding more log formats in the future.
At present, only one specifier per key in the format dictionary is allowed. This is because the whole point of JSON logging is to make logs easily machine-parseable. If you can include multiple formats in the same field, then you'll be right back to parsing those manually
At present, only top-level keys are allowed in the format dictionary. This is validated at config load time. In the future, we can expand this to have nested dictionaries.
Risk Level: Low. It's an optional feature that has to be explicitly enabled.
Testing: Unit testing for the actual formatter, and config load. Also manually tested using an example config file.
Docs:
Amended Access Log docs to create a notion of "Format Strings" and "Format Dictionaries".
Put things that are common to access logs in general under "Format Rules", and then distinguished how strings and dictionaries are different.
Called out restrictions on format dictionaries
Added protobuf comments for format and json_format
Signed-off-by: Aaltan Ahmad <aa@stripe.com>
Mirrored from https://github.com/envoyproxy/envoy @ de039269f54aa21aa0da21da89a5075aa3db3bb9
adds the required visibility rules and delegates the rest to the generic
api_proto_library. I tested the change by doing the following without
getting errors.
./ci/run_envoy_docker.sh './ci/do_ci.sh docs'
I changed the BUILD files using the following commands.
/envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/api_proto_library(/api_proto_library_internal(/g'
envoy/api$ find . -type f -name BUILD | xargs sed -i -e 's/"api_proto_library"/"api_proto_library_internal"/g'
Signed-off-by: mickey <mickeyju@google.com>
Mirrored from https://github.com/envoyproxy/envoy @ 4b871c0ab9350882271a490adcee44e613ed9807
Fixes https://github.com/envoyproxy/envoy/issues/743
This is a general cleanup of all of the access logging documentation.
I have reorganized a bunch of things and hidden the various gRPC logging
fields that are not implemented yet.
I've also moved the existing tap protos into a new "output" directory. This
is the best name I could come up for cleanly separating output data that might
be stored outside of any service or configuration.
Signed-off-by: Matt Klein <mklein@lyft.com>
Mirrored from https://github.com/envoyproxy/envoy @ c15019e79c832d9f0a09468affaadabc4be3e115
There are several main changes in this PR:
Create envoy.api.v2.core packages to break circular dependencies from xDS on to subpackages on to base protos.
Create individual packages for each filter and add independent versioning to each filter.
Add visibility constraints to prevent formation of dependency cycles.
Add gogoproto annotations to improve go code generation.
After moving xDS service definitions and top-level resource protos back to envoy.core.api.v2, cycles were created, since the second-level definitions depend on base protobuf definitions, and are in turn included from xDS; however xDS and base definitions are in the same package.
The solution is to split the base protos into another package, envoy.api.v2.core. That eliminates dependency cycles (validated using go-control-plane).
Added a few gogoproto annotations to improve golang code generation.
Signed-off-by: Kuat Yessenov <kuat@google.com>
data-plane-api(CircleCI)
Kuat
Piotr Sikora