xDS interop: custom before_sleep_log only logging primitive returns (#30757)
When we use retryers with `log_level=logging.INFO`, tenacity logs the result value (or an exception) after each unsuccessful retry attempt. We often retry methods that return objects, resulting in unreadable log messages: ``` I0820 03:16:29.027635 140613877811008 before_sleep.py:45] Retrying framework.xds_k8s_testcase.IsolatedXdsKubernetesTestCase.cleanup in 10.0 seconds as it raised RetryError: RetryError[Attempts: 21, Value: {'api_version': 'v1', 'kind': 'Namespace', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2022, 8, 20, 2, 55, 32, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': datetime.datetime(2022, 8, 20, 3, 6, 27, tzinfo=tzlocal()), 'finalizers': None, 'generate_name': None, 'generation': None, 'labels': {'kubernetes.io/metadata.name': 'psm-interop-server-20220820-0253-yrmam', 'name': 'psm-interop-server-20220820-0253-yrmam', 'owner': 'xds-k8s-interop-test'}, 'managed_fields': [{'api_version': 'v1', 'fields_type': 'FieldsV1', 'fields_v1': {'f:metadata': {'f:labels': {'.': {}, 'f:kubernetes.io/metadata.name': {}, ... (82 more lines) ``` This PR introduces custom `before_sleep` logger, that only logs the value if it's a primitive: `int, str, bool`. Otherwise, it logs the type, example: ``` k8s_base_runner.py:311] Waiting for pod psm-grpc-client-5d5648478f-7vsf7 to start retryers.py:192] Retrying framework.infrastructure.k8s.KubernetesNamespace.get_pod in 1.0 seconds as it returned type <class 'kubernetes.client.models.v1_pod.V1Pod'>. retryers.py:192] Retrying framework.infrastructure.k8s.KubernetesNamespace.get_pod in 1.0 seconds as it returned type <class 'kubernetes.client.models.v1_pod.V1Pod'>. ``` Note that this only changes the behavior of the unsuccessful retries, and doesn't affect the new feature that prints formatted k8s status field on if the *final* retry attempt failed.pull/30768/head
parent
1e42fbeb52
commit
1837dae106
1 changed files with 37 additions and 2 deletions
Loading…
Reference in new issue