access log: add JSON logging mode (#4693)

Add a new config option under access_log called json_format. This is a single level dictionary that contains strings as keys, and envoy access log format specifiers (such as %PROTOCOL%) as values.

The specifiers will be replaced with actual values at logging time. I call this dictionary the "format dictionary" (as opposed to "format string").

You can specify only one of format (format string) or json_format (format dictionary). If neither are there, we fall back to the default string format.

Add the correct plumbing inside the configuration parsing to handle this.

Add a new access log formatter class that is instantiated with the format dictionary. It maintains the mapping of dictionary keys to loggers

Create a new class called FormatterProvider, to distinguish things that actually extract the information from a request. The things that combine together a bunch of FormatterProviders are still called Formatters. This is primarily a semantic/naming difference, but imo these are two conceptually separate things. There is, however no API difference, and if people are truly opposed to this, I could just merge them back into one Formatter class. This also provides a better foundation for adding more log formats in the future.

At present, only one specifier per key in the format dictionary is allowed. This is because the whole point of JSON logging is to make logs easily machine-parseable. If you can include multiple formats in the same field, then you'll be right back to parsing those manually

At present, only top-level keys are allowed in the format dictionary. This is validated at config load time. In the future, we can expand this to have nested dictionaries.

Risk Level: Low. It's an optional feature that has to be explicitly enabled.

Testing: Unit testing for the actual formatter, and config load. Also manually tested using an example config file.

Docs:

Amended Access Log docs to create a notion of "Format Strings" and "Format Dictionaries".
Put things that are common to access logs in general under "Format Rules", and then distinguished how strings and dictionaries are different.
Called out restrictions on format dictionaries
Added protobuf comments for format and json_format

Signed-off-by: Aaltan Ahmad <aa@stripe.com>

Mirrored from https://github.com/envoyproxy/envoy @ de039269f54aa21aa0da21da89a5075aa3db3bb9

envoy/config/accesslog/v2/file.proto

@@ -4,6 +4,7 @@ package envoy.config.accesslog.v2;
 option go_package = "v2";
 
 import "validate/validate.proto";
+import "google/protobuf/struct.proto";
 
 // [#protodoc-title: File access log]
 
@@ -17,5 +18,11 @@ message FileAccessLog {
   // Access log format. Envoy supports :ref:`custom access log formats
   // <config_access_log_format>` as well as a :ref:`default format
   // <config_access_log_default_format>`.
-  string format = 2;
+  oneof access_log_format {
+    // Access log :ref:`format string<config_access_log_format_strings>`
+    string format = 2;
+
+    // Access log :ref:`format dictionary<config_access_log_format_dictionaries>`
+    google.protobuf.Struct json_format = 3;
+  }
 }