In some rare occasions on Win machines (0,3-0,4%), the tests are stuck when we execute the loop of 10 DoRpc calls. We receive Deadline Exceeded for such cases. The PR bumps the deadline from 10 to 60s (no flakes for --runs_per_test=10000).
Closes#37844
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37844 from erm-g:seqFix 8644db8194
PiperOrigin-RevId: 681891281
This is the last piece of gRFC A83 (https://github.com/grpc/proposal/pull/438).
Note that although this is the first use-case for this "blackboard" mechanism, we will also use it in the future for the xDS rate-limiting filter on the gRPC server side.
Closes#37646
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37646 from markdroth:gcp_auth_filter_state 72d0d96c79
PiperOrigin-RevId: 679707134
The following files have been moved:
- src/core/lib/avl/*
- src/core/lib/backoff/*
- src/core/lib/debug/event_log*
- src/core/lib/iomgr/gethostname*
- src/core/lib/iomgr/grpc_if_nametoindex*
- src/core/lib/matchers/*
- src/core/lib/uri/* (renamed from uri_parser.* to uri.*)
- src/core/lib/gprpp/* (existing src/core/util/time.cc was renamed to gpr_time.cc to avoid conflict)
Closes#36792
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36792 from markdroth:reorg_util d4e8996f48
PiperOrigin-RevId: 676947640
Looks like our MSAN build is just taking longer at the moment, so increase sharding to reduce per-shard runtime.
Closes#37780
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37780 from ctiller:flake-fightas-13 07c422e977
PiperOrigin-RevId: 676869265
Issue noticed on xds_end2end_test and is made worse worse when reducing `xds_resource_does_not_exist_timeout_ms` to 500 and running it on tsan.
Closes#37678
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37678 from yashykt:XdsClientOnTimerDebugging 1d31e28d2c
PiperOrigin-RevId: 673479242
Fix some flakiness caused by fixing jitter in the backoff library in #37595.
In the xDS retry tests, the additional jitter made it such that an extra attempt snuck in before the call deadline, so I adjusted the knobs to ensure that exactly the expected number of attempts fit into the tests.
In the PF test, I rewrote the test to use a connection injector, so that it can more accurately tell the time between the connection attempts, without seeing skew due to the server startup time.
Closes#37629
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37629 from markdroth:xds_retry_backoff_flake_fix f35ac902a0
PiperOrigin-RevId: 671120341
Fallback interop test is fully deployed. This variable is no longer needed.
Closes#37620
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37620 from eugeneo:no-fallback-var c21509d0a5
PiperOrigin-RevId: 670738146
Final piece of gRFC A83 (https://github.com/grpc/proposal/pull/438): the GCP authentication filter itself.
Infrastructure changes include:
- Added a general-purpose LRU cache library that can be reused elsewhere.
- Fixed the client channel code to use the channel args returned by the resolver for the dynamic filters. This was necessary so that the GCP auth filter could access the `XdsConfig` object, which is passed via a channel arg.
- Unlike the other xDS HTTP filters we support, the GCP auth filter does not support config overrides, and its configuration includes a cache size parameter that we always need at the channel level, not per-call. As a result, I had to change the xDS HTTP filter API to give it the ability to set top-level fields in the service config, not just per-method fields. (We use the service config as a way of passing configuration down into xDS HTTP filters.) Note that for now, this works only on the client side, because we don't have machinery for a top-level service config on the server side.
- The GCP auth filter is also the first case where the filter needs to know its instance name from the xDS config, so I changed the xDS HTTP filter API to plumb that through.
- Fixed a bug in the HTTP client library that prevented the override functions from declining to override a particular request.
Closes#37550
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37550 from markdroth:xds_gcp_auth_filter 19eaefb52f
PiperOrigin-RevId: 669371249
Fixes a bug in the backoff implementation whereby we were incorrectly failing to apply jitter to the initial backoff.
Also change the API to return `Duration` instead of `Timestamp`. The only caller that actually wants to count the backoff from the start of the previous attempt instead of the end of the previous attempt is the subchannel code, and it handles that on its end.
Closes#37595
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37595 from markdroth:backoff_fixes_and_api_improvement 39d083c0f4
PiperOrigin-RevId: 669112557
[Gpr_To_Absl_Logging] Remove gpr logging header include from other headers
Some of the cc files are using gpr_log_verbosity_init() functions.
So I removed the #include <grpc/support/log.h> from all headers and put it selectively only in the cc files that used gpr_log_verbosity_init()
Closes#37513
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37513 from tanvi-jagtap:remove_gpr_headers_from_headers_01 612ca6d0f7
PiperOrigin-RevId: 663811895
[Gpr_To_Absl_Logging] Remove logging header from example and test/cpp folder
Closes#37488
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37488 from tanvi-jagtap:remove_header_examples 1423ee77ff
PiperOrigin-RevId: 663597077
[Gpr_To_Absl_Logging] Remove GRPC_API_TRACE .
This will be replaced by GRPC_TRACE_LOG
Closes#37190
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37190 from tanvi-jagtap:grpc_api_trace_fix 6dae72bc6e
PiperOrigin-RevId: 652397141
As per gRFC A81 (https://github.com/grpc/proposal/pull/435).
A few small improvements along the way:
- Use `DownCast<>` instead of `static_cast<>` in `ServiceConfigCallData::GetCallAttribute()`.
- Add a convenient templated wrapper for `ClientChannelLbCallState::GetCallAttribute()`, similar to the one on `ServiceConfigCallData`.
- Added a `ParseBoolValue()` helper to xds_common_types_parser, and use it in various places in xDS resource parsing.
Closes#37087
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37087 from markdroth:xds_authority_rewriting 3679d2b061
PiperOrigin-RevId: 650293912
As part of the xDS authority rewriting implementation in #37087, I need to write some tests showing the authority being rewritten. However, the fake security connector currently crashes on unexpected authorities. (As a side note, I think the fake security connector is very cumbersome and should be redesigned, but that's a separate project.) As a result, I need a way to use InsecureCreds on a per-test basis.
I thought about just adding an option to `XdsTestType` to trigger use of InsecureCreds, but the logic we use for determining which creds type to use for what is already very cumbersome, and adding another option there would have just made that worse. Instead, I have switched to a simpler approach where the individual tests can decide what creds type to use directly. This both unblocks my other PR and makes the existing code more maintainable.
Closes#37156
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37156 from markdroth:xds_e2e_test_creds_improvement 2e813834d0
PiperOrigin-RevId: 649225248
We don't have a guarantee of timeliness for connectivity issues to clear up with resource quota, and so this wait for ready setting is causing very long delays for the test to complete.
Closes#36967
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36967 from ctiller:rq 4e76af9bda
PiperOrigin-RevId: 644502087
Also made some minor improvements to the `ConfigSelector` API.
Closes#36877
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36877 from markdroth:client_channel_v3_dynamic_filters 6a539fe320
PiperOrigin-RevId: 642755276
- ensure ordering of `OnAccept` and `Shutdown` callbacks from thready event engine (previously these could be reordered and this caused spurious failures)
- enable thready_tsan for one C++ e2e test
- don't filter thready_tsan for local builds (only CI)
Closes#36886
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36886 from ctiller:erm e3b88e7d86
PiperOrigin-RevId: 642702724
The following attributes were completely unused:
- kOffset
- kIndex
- kSize
- kFilename
- kKey
- kValue
The following attributes were added but never programmatically accessed, and I've moved them into the status messages themselves, which is another step toward #22883:
- kErrorNo
- kTsiCode
- kWsaError
- kHttpStatus
- kOsError
- kSyscall
- kTargetAddress
- kRawBytes
- kTsiError
Closes#36523
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36523 from markdroth:grpc_error_attribute_cleanup b289c399fe
PiperOrigin-RevId: 639147583
This fixes a fairly embarrassing bug and lack of testing from #33234. Prior to this fix, attempting to use the "tls" creds type would always cause a crash.
@gtcooke94@matthewstevenson88 Note that the root cause of this bug was that when I wrote this code, I assumed that `grpc_tls_credentials_options` had a reasonable default for the cert verifier. But it turns out that it doesn't do that directly; instead, we are only imposing that default in [`CredentialOptionSanityCheck()`](621aa4e5ce/src/core/lib/security/credentials/tls/tls_credentials.cc (L85)), which is called only when we call [`grpc_tls_credentials_create()`](621aa4e5ce/src/core/lib/security/credentials/tls/tls_credentials.cc (L160)), not when we directly instantiate `TlsCredentials` as my code was doing. As part of the TlsCreds API cleanup you're working on, we should fix this so that callers get the right behavior even if they are internal callers that instantiate the TlsCreds object directly rather than calling the C-core API.
Closes#36726
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36726 from markdroth:xds_bootstrap_mtls_creds_fix dac2789e10
PiperOrigin-RevId: 637993734
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36678
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36678 from tanvi-jagtap:test_cpp_gpr_log 81b8f4179f
PiperOrigin-RevId: 636410944
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36677
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36677 from tanvi-jagtap:regex_test_end2end dfb803ebff
PiperOrigin-RevId: 635807701
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - BUILD
In this CL we are just editing the build and bzl files to add dependencies.
This is done to prevent merge conflict and constantly having to re-make the make files using generate_projects.sh for each set of changes.
Closes#36607
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36607 from tanvi-jagtap:build_test_cpp 3e17d778d0
PiperOrigin-RevId: 633523097
This adds compile-time checking that the type of the value and the size of the labels and optional labels passed when recording a metric must match with the type and the size specified when the metric is registered.
The RegistrationBuilder API idea is credited to @ctiller.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36449
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36449 from yijiem:registration-builder-api a727810136
PiperOrigin-RevId: 632271022
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging - gpr_log
In this CL we are migrating from gRPCs own gpr logging mechanism to absl logging mechanism. The intention is to deprecate gpr_log in the future.
We have the following mapping
1. gpr_log(GPR_INFO,...) -> LOG(INFO)
2. gpr_log(GPR_ERROR,...) -> LOG(ERROR)
3. gpr_log(GPR_DEBUG,...) -> VLOG(2)
Reviewers need to check :
1. If the above mapping is correct.
2. The content of the log is as before.
gpr_log format strings did not use string_view or std::string . absl LOG accepts these. So there will be some elimination of string_view and std::string related conversions. This is expected.
Closes#36550
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36550 from tanvi-jagtap:tjagtap_test_cpp_e2e_xds d242076993
PiperOrigin-RevId: 632024853
erm-g
Mark D. Roth
Craig Tiller
Nathan Baulch
Yash Tibrewal
Eugene Ostroukhov
Esun Kim
Tanvi Jagtap
AJ Heller
Yijie Ma