access_log: metadata formatter extension (#17457)

This PR addresses several issues:
1. MetadataFormatter base class takes a pointer to a function to access specific type of metadata. All other methods operating on internals of metadata structure are moved to base class. This simplified adding new types of metadata.
2. new type of  access_log METADATA tag is introduced to handle all types of metadata. It is implemented as formatter extension. It handles existing tags DYNAMIC_METADATA and CLUSTER_METADATA, so two ways of accessing dynamic and cluster metadata are allowed now. Maybe we should obsolete DYNAMIC_METADATA and CLUSTER_METADATA in the future?
3. Adds handler for ROUTE metadata. See issue #3006.

Follow up to #16121

Risk Level: Low
Testing: Unit tests.
Docs Changes: Yes.
Release Notes: Yes.
Platform Specific Features: No

Signed-off-by: Christoph Pakulski <christoph@tetrate.io>

Mirrored from https://github.com/envoyproxy/envoy @ 43b597e719230c8266998394301c5d0b3c862688

BUILD

@@ -180,6 +180,7 @@ proto_library(
         "//envoy/extensions/filters/network/zookeeper_proxy/v3:pkg",
         "//envoy/extensions/filters/udp/dns_filter/v3alpha:pkg",
         "//envoy/extensions/filters/udp/udp_proxy/v3:pkg",
+        "//envoy/extensions/formatter/metadata/v3:pkg",
         "//envoy/extensions/formatter/req_without_query/v3:pkg",
         "//envoy/extensions/health_checkers/redis/v3:pkg",
         "//envoy/extensions/http/header_formatters/preserve_case/v3:pkg",

envoy/extensions/formatter/metadata/v3/BUILD

@@ -0,0 +1,9 @@
+# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.
+
+load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")
+
+licenses(["notice"])  # Apache 2
+
+api_proto_package(
+    deps = ["@com_github_cncf_udpa//udpa/annotations:pkg"],
+)

envoy/extensions/formatter/metadata/v3/metadata.proto

@@ -0,0 +1,56 @@
+syntax = "proto3";
+
+package envoy.extensions.formatter.metadata.v3;
+
+import "udpa/annotations/status.proto";
+
+option java_package = "io.envoyproxy.envoy.extensions.formatter.metadata.v3";
+option java_outer_classname = "MetadataProto";
+option java_multiple_files = true;
+option (udpa.annotations.file_status).package_version_status = ACTIVE;
+
+// [#protodoc-title: Formatter extension for printing various types of metadata]
+// [#extension: envoy.formatter.metadata]
+
+// Metadata formatter extension implements METADATA command operator that
+// prints all types of metadata. The first parameter taken by METADATA operator defines
+// type of metadata. The following types of metadata are supported (case sensitive):
+//
+// * DYNAMIC
+// * CLUSTER
+// * ROUTE
+//
+// See :ref:`here <config_access_log>` for more information on access log configuration.
+
+// %METADATA(TYPE:NAMESPACE:KEY):Z%
+//    :ref:`Metadata <envoy_v3_api_msg_config.core.v3.Metadata>` info,
+//    where TYPE is type of metadata (see above for supported types),
+//    NAMESPACE is the filter namespace used when setting the metadata, KEY is an optional
+//    lookup up key in the namespace with the option of specifying nested keys separated by ':',
+//    and Z is an optional parameter denoting string truncation up to Z characters long.
+//    The data will be logged as a JSON string. For example, for the following ROUTE metadata:
+//
+//    ``com.test.my_filter: {"test_key": "foo", "test_object": {"inner_key": "bar"}}``
+//
+//    * %METADATA(ROUTE:com.test.my_filter)% will log: ``{"test_key": "foo", "test_object": {"inner_key": "bar"}}``
+//    * %METADATA(ROUTE:com.test.my_filter:test_key)% will log: ``foo``
+//    * %METADATA(ROUTE:com.test.my_filter:test_object)% will log: ``{"inner_key": "bar"}``
+//    * %METADATA(ROUTE:com.test.my_filter:test_object:inner_key)% will log: ``bar``
+//    * %METADATA(ROUTE:com.unknown_filter)% will log: ``-``
+//    * %METADATA(ROUTE:com.test.my_filter:unknown_key)% will log: ``-``
+//    * %METADATA(ROUTE:com.test.my_filter):25% will log (truncation at 25 characters): ``{"test_key": "foo", "test``
+//
+//  .. note::
+//
+//    For typed JSON logs, this operator renders a single value with string, numeric, or boolean type
+//    when the referenced key is a simple value. If the referenced key is a struct or list value, a
+//    JSON struct or list is rendered. Structs and lists may be nested. In any event, the maximum
+//    length is ignored.
+//
+//  .. note::
+//
+//    METADATA(DYNAMIC:NAMESPACE:KEY):Z is equivalent to :ref:`DYNAMIC_METADATA(NAMESPACE:KEY):Z<config_access_log_format_dynamic_metadata>`
+//    METADATA(CLUSTER:NAMESPACE:KEY):Z is equivalent to :ref:`CLUSTER_METADATA(NAMASPACE:KEY):Z<config_access_log_format_cluster_metadata>`
+
+message Metadata {
+}

versioning/BUILD

@@ -130,6 +130,7 @@ proto_library(
         "//envoy/extensions/filters/network/zookeeper_proxy/v3:pkg",
         "//envoy/extensions/filters/udp/dns_filter/v3alpha:pkg",
         "//envoy/extensions/filters/udp/udp_proxy/v3:pkg",
+        "//envoy/extensions/formatter/metadata/v3:pkg",
         "//envoy/extensions/formatter/req_without_query/v3:pkg",
         "//envoy/extensions/health_checkers/redis/v3:pkg",
         "//envoy/extensions/http/header_formatters/preserve_case/v3:pkg",