From 44053ea490410b8698564c6547a01684c5543569 Mon Sep 17 00:00:00 2001 From: Tanvi Jagtap <139093547+tanvi-jagtap@users.noreply.github.com> Date: Mon, 24 Jun 2024 20:45:01 -0700 Subject: [PATCH] [Documentation][Gpr_To_Absl_Logging] Fix GRPC_VERBOSITY docs (#37011) [Documentation][Gpr_To_Absl_Logging] Fix GRPC_VERBOSITY docs Closes #37011 COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37011 from tanvi-jagtap:fix_docs 18bf235baf5a53a1700091a1ceb9a4deb4206b7c PiperOrigin-RevId: 646315660 --- TROUBLESHOOTING.md | 57 +++++++++++++++++++++--------------- doc/environment_variables.md | 16 +++++----- 2 files changed, 41 insertions(+), 32 deletions(-) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 6a2cf4a6f09..4f0ae6f4918 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -7,44 +7,47 @@ This guide is for troubleshooting gRPC implementations based on C core library ( Extra logging can be very useful for diagnosing problems. It can be used to increase the amount of information that gets printed to stderr. -## GRPC_VERBOSITY +## Setting Logging Severity and Verbosity - +[gRPC uses absl logging](https://abseil.io/docs/cpp/guides/logging). +Verbosity can be set using absl flags such as +`--minloglevel`, `--v` and `--vmodule`. - -`GRPC_VERBOSITY` is used to set the minimum level of log messages printed by gRPC (supported values are `DEBUG`, `INFO` and `ERROR`). If this environment variable is unset, only `ERROR` logs will be printed. `ERROR` is recomeded for production systems. - +These can also be programmatically set using +[these absl APIs.](https://github.com/abseil/abseil-cpp/blob/master/absl/log/globals.h) -## GRPC_TRACE +Example +``` +# Disable all logs other than FATAL for the entire application +./helloworld_application_using_grpc --v=-1 --minloglevel=3 +``` + +## GRPC_VERBOSITY (DEPRECATED) - +[Environment Variables Overview](doc/environment_variables.md) -`GRPC_TRACE` can be used to enable extra logging for some internal gRPC components. Enabling the right traces can be invaluable -for diagnosing for what is going wrong when things aren't working as intended. Possible values for `GRPC_TRACE` are listed in [Environment Variables Overview](doc/environment_variables.md). +## GRPC_TRACE + +`GRPC_TRACE` can be used to enable extra logging for specific internal gRPC components. Enabling the right traces can be invaluable +for diagnosing for what is going wrong when things aren't working as intended. Possible values for `GRPC_TRACE` are [listed here](doc/trace_flags.md). Multiple traces can be enabled at once (use comma as separator). ``` -# Enable debug logs for an application -GRPC_VERBOSITY=debug ./helloworld_application_using_grpc +# Enable debug logs for the entire application +./helloworld_application_using_grpc --v=2 --minloglevel=0 ``` ``` # Print information about invocations of low-level C core API. -# Note that trace logs of log level DEBUG won't be displayed. -# Also note that most tracers user log level INFO, so without setting -# GPRC_VERBOSITY accordingly, no traces will be printed. -GRPC_VERBOSITY=info GRPC_TRACE=api ./helloworld_application_using_grpc +# Note that trace logs that use `VLOG` won't be displayed. +# Many tracers user log level INFO. +# So unless absl settings are correct, no traces will be printed. +GRPC_TRACE=api ./helloworld_application_using_grpc --v=-1 --minloglevel=0 ``` ``` -# Print info from 3 different tracers, including tracing logs with log level DEBUG -GRPC_VERBOSITY=debug GRPC_TRACE=tcp,http,api ./helloworld_application_using_grpc +# Print info from 3 different tracers, including tracing logs +GRPC_TRACE=tcp,http,api ./helloworld_application_using_grpc --v=2 --minloglevel=0 ``` Known limitations: `GPRC_TRACE=tcp` is currently not implemented for Windows (you won't see any tcp traces). @@ -52,3 +55,11 @@ Known limitations: `GPRC_TRACE=tcp` is currently not implemented for Windows (yo Please note that the `GRPC_TRACE` environment variable has nothing to do with gRPC's "tracing" feature (= tracing RPCs in microservice environment to gain insight about how requests are processed by deployment), it is merely used to enable printing of extra logs. + +## Preventing gRPC Log Noise + +Log noise could consume a lot of resources. We recommend tuning settings for production systems very carefully. +* Avoid using GRPC_VERBOSITY flag. This has been deprecated. If this value of this flag is anything other than "ERROR" or "NONE" it will cause log noise. +* Always avoid setting --v and --vmodule to anything other than -1 for production systems. +* Avoid setting --minloglevel=0 for production systems. Anyting greater than 0 should be fine. +* If setting this does not eliminate your log noise, look for instances of functions `--v`, `--vmodule`, `absl::SetVLogLevel` and `absl::SetMinLogLevel` in your entire codebase and any libraries/components/configs that you may be using. diff --git a/doc/environment_variables.md b/doc/environment_variables.md index bce55266ee7..ef27e91f803 100644 --- a/doc/environment_variables.md +++ b/doc/environment_variables.md @@ -46,14 +46,13 @@ some configuration as environment variables that can be set. Available tracers and their usage can be found in [gRPC Trace Flags](trace_flags.md) -* GRPC_VERBOSITY - +* GRPC_VERBOSITY (DEPRECATED) -`GRPC_VERBOSITY` is used to set the minimum level of log messages printed by gRPC (supported values are `DEBUG`, `INFO` and `ERROR`). If this environment variable is unset, only `ERROR` logs will be printed. + `GRPC_VERBOSITY` is used to set the minimum level of log messages printed. Supported values are `DEBUG`, `INFO`, `ERROR` and `NONE`. + + We only support this flag for legacy reasons. If this environment variable is set, then gRPC will set absl MinLogValue and absl SetVLogLevel. This will alter the log settings of the entire application, not just gRPC code. For that reason, it is not recommended. Our recommendation is to avoid using this flag and [set log verbosity using absl](https://abseil.io/docs/cpp/guides/logging). + gRPC logging verbosity - one of: - DEBUG - log INFO, WARNING, ERROR and FATAL messages. Also sets absl VLOG(2) logs enabled. This is not recommended for production systems. This will be expensive for staging environments too, so it can be used when you want to debug a specific issue. - INFO - log INFO, WARNING, ERROR and FATAL messages. This is not recommended for production systems. This may be slightly expensive for staging environments too. We recommend that you use your discretion for staging environments. @@ -65,9 +64,8 @@ END_GOOGLE_INTERNAL_DOCUMENTATION --> - If nothing is set by the external application also, the default set by absl will be honoured. -* GRPC_STACKTRACE_MINLOGLEVEL - Minimum loglevel to print the stack-trace - one of DEBUG, INFO, ERROR, and NONE. - NONE is a default value. +* GRPC_STACKTRACE_MINLOGLEVEL (DEPRECATED) + This will not work anymore. * GRPC_TRACE_FUZZER if set, the fuzzers will output trace (it is usually suppressed).