From d3da61529d6b3b6c02de48c67bebb5cf432d9374 Mon Sep 17 00:00:00 2001 From: "data-plane-api(CircleCI)" Date: Thu, 18 Oct 2018 23:12:36 +0000 Subject: [PATCH] access log: add JSON logging mode (#4693) Add a new config option under access_log called json_format. This is a single level dictionary that contains strings as keys, and envoy access log format specifiers (such as %PROTOCOL%) as values. The specifiers will be replaced with actual values at logging time. I call this dictionary the "format dictionary" (as opposed to "format string"). You can specify only one of format (format string) or json_format (format dictionary). If neither are there, we fall back to the default string format. Add the correct plumbing inside the configuration parsing to handle this. Add a new access log formatter class that is instantiated with the format dictionary. It maintains the mapping of dictionary keys to loggers Create a new class called FormatterProvider, to distinguish things that actually extract the information from a request. The things that combine together a bunch of FormatterProviders are still called Formatters. This is primarily a semantic/naming difference, but imo these are two conceptually separate things. There is, however no API difference, and if people are truly opposed to this, I could just merge them back into one Formatter class. This also provides a better foundation for adding more log formats in the future. At present, only one specifier per key in the format dictionary is allowed. This is because the whole point of JSON logging is to make logs easily machine-parseable. If you can include multiple formats in the same field, then you'll be right back to parsing those manually At present, only top-level keys are allowed in the format dictionary. This is validated at config load time. In the future, we can expand this to have nested dictionaries. Risk Level: Low. It's an optional feature that has to be explicitly enabled. Testing: Unit testing for the actual formatter, and config load. Also manually tested using an example config file. Docs: Amended Access Log docs to create a notion of "Format Strings" and "Format Dictionaries". Put things that are common to access logs in general under "Format Rules", and then distinguished how strings and dictionaries are different. Called out restrictions on format dictionaries Added protobuf comments for format and json_format Signed-off-by: Aaltan Ahmad Mirrored from https://github.com/envoyproxy/envoy @ de039269f54aa21aa0da21da89a5075aa3db3bb9 --- envoy/config/accesslog/v2/file.proto | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/envoy/config/accesslog/v2/file.proto b/envoy/config/accesslog/v2/file.proto index d1ca2d1e..a53c5aab 100644 --- a/envoy/config/accesslog/v2/file.proto +++ b/envoy/config/accesslog/v2/file.proto @@ -4,6 +4,7 @@ package envoy.config.accesslog.v2; option go_package = "v2"; import "validate/validate.proto"; +import "google/protobuf/struct.proto"; // [#protodoc-title: File access log] @@ -17,5 +18,11 @@ message FileAccessLog { // Access log format. Envoy supports :ref:`custom access log formats // ` as well as a :ref:`default format // `. - string format = 2; + oneof access_log_format { + // Access log :ref:`format string` + string format = 2; + + // Access log :ref:`format dictionary` + google.protobuf.Struct json_format = 3; + } }