Hi, I would like to suggest five changes/additions to gRPC access loger and accesslog proto file. These changes mainly benefit using Envoy as a TLS transparent proxy. I am not an experienced c++ developer and any feedback is welcome.
The problem is explained here: [#22407](https://github.com/envoyproxy/envoy/issues/22407)
After digging into the code, the proposal is to make five modifications/additions to the gRPC logger. A detailed description per modification/addition is listed below.
1. Set tls_sni_hostname also when TLS connection is not terminated.
2. Add duration
3. Add upstream_request_attempt_count
4. Add connection_termination_details
5. Add ja3_fingerprint when ja3_fingerprinting is enabled
Details
1. Set tls_sni_hostname also when TLS connection is not terminated.
This enables it to log the requested hostname when the tls listener filter is used, but the connection is simply forwarded to the upstream cluster. This is for example useful to see if envoy forwards to the correct cluster based on the requested hostname.
2. Add duration
The existing duration fields in the acceslog.proto are not present when using the tcp listener. The complete duration, based on stream_info.requestComplete(), however is available. A separate duration field is in line with the file access log.
3. Add upstream_request_attempt_count
Logs the number of times the connection request is attempted upstream. Note that the field is omitted when the connect request was never attempted upstream. This is particularly useful when the max_connect_attempts field is set in the tcp_proxy.
4. Add connection_termination_details
Connection termination details may provide additional information about why the connection was terminated by Envoy for L4 reasons. This is especiall usefull when network rbac envoy.filters.network.rbac . The connection termination details contains the info why a connection was blocked/allowed.
5. Add ja3_fingerprint when ja3_fingerprinting is enabled
The [tls inspector listner filter]( https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/listener/tls_inspector/v3/tls_inspector.proto#envoy-v3-api-msg-extensions-filters-listener-tls-inspector-v3-tlsinspector) has an option to generate ja3 fingerprint (default false). When available this logs the ja3 fingerprint. This field is omitted when the ja3 fingerprint is not available. The field is present when eiter envoy terminates the tls connection or acts as a transparent proxy.
Risk Level:
Low/Medium - four additional fields are added, no fields removed or changed (no behavior changes expected)
The only change is that tls_properties is also added when it is not a terminated tls connection. In this scenario, of the tls_properties only the tls_sni_hostname is available. The other tls_properties fields are not present. As far as I can see, this is consistent with the behavior of the tls listener filter.
This could be a breaking change if the receiving application of the access log message checks for the presence of the tls_properries and when present expect all the fields to be there, or assumes this must be a tls terminated connection.
Testing:
unit tests
Development build using mac os x (m1 if that info is relevant), but with ‘default’ extensions.
All tests pass.
manual testing
Using a sample grpc log service. Displaying accesslog stream using golang protojson.Marshal()
Tested
- Tcp with tls
- Tcp no tls
- Http no tls
Docs Changes:
I have added the documentation in the proto file [accesslog.proto](https://github.com/envoyproxy/envoy/blob/main/api/envoy/data/accesslog/v3/accesslog.proto) for data.accesslog.v3.AccessLogCommon, and data.accesslog.v3.TLSProperties.
Am I right in assuming that the documentation is generated from this file? If not, I have to modify the documentation.
Release Notes:
**access_log:** log `duration`, `upstream_request_attempt_count`, `connection_termination_details` and tls `ja3` field in the grpc access log and also log the tls `sni` and `ja3` field in the grpc access log when envoy is configured as a tls forward proxy.
Platform Specific Features: N/A
Issues:
Fixes: [#22407](https://github.com/envoyproxy/envoy/issues/22407)
[Optional API Considerations:]
ls_sni_hostname:
- See risk level above. The other option would be to add a requested_server_name field to the access_log_common properties. That would be present when either the tls connection is terminated or forwarded.
- The downside of this solution would be that this would create a duplicate field for tls_sni_hostname when envoy terminates the tls connection.
- The proposed solution: Filling tls_properties->tls_sni_hostame either when envoy terminates or forwards the tls connection is consistent with file access logging.
upstream_request_attempt_count
- In file logging this field is 0 when there is no upstream connection attempt made. I believe that omitting this field when there is no value is more in line with the grcp ALS service.
- In most scenarios the value of this field is 1. This field could also be adapted to a retry count. In that solution the field would be present only if the attempt > 1. This is however not consistent with file logging. The proposed solution is.
Signed-off-by: Niek Temme <niek@bubl.cloud>
Mirrored from https://github.com/envoyproxy/envoy @ 06625e63d1e780c5affe7938c9d7debe13c36caf
data-plane-api(Azure Pipelines)