<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
~5% flake introduced earlier this week, where the pool was not scaling
when draining work @ shutdown. Maintaining a busy thread count at
shutdown allows the lifeguard to notice when all threads are blocked,
and spawn new threads if necessary.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
We were previously trying to adjust the initial guess flow control
window *and* the adjustment methodology, which created some problems.
The latter is more important for us to get in, so don't change the
former for now (we can do that later).
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
recv_initial_metadata_ in `RetryFilter::CallAttempt` is deleted before
it's accessed in
`ClientChannel::FilterBasedLoadBalancedCall::RecvTrailingMetadataReady`
- instead of accessing it, pull out peer string and keep a ref
- switch to json_object_loader for config parsing
- use `absl::string_view` instead of `const char*` for cert provider
names
- change cert provider registry to use a map instead of a vector
- remove unused mesh_ca cert provider factory
Allow usage in production tasks
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
I generated a new client key and cert where a Spiffe ID is added as the
URI SAN. As such, we are able to test the audit log contains the
principal correctly.
Update: I switched to use the test logger to verify the log content and
removed stdout logger here because one the failure of [RBE Windows Debug
C/C++](https://source.cloud.google.com/results/invocations/c3187f41-bb1f-44b3-b2b1-23f38e47386d).
Update again: Refactored the test logger in a util such that the authz
engine test also uses the same logger. Subsequently, xDS e2e test will
also use it.
---------
Co-authored-by: rockspore <rockspore@users.noreply.github.com>
The PR does the following:
* Splits the single experiments.yaml file into two files:
experiments.yaml and rollouts.yaml.
* The experiments.yaml will now only include experiment definitions. The
default values of the experiments must now be specified in rollouts.yaml
* Removes the 'release' default value because it is not used.
* Adds an additional_constraints character string to ExperimentMetadata.
* Introduces a hook in src/core/lib/experiments/config.h to allow
registering arbitrary experiment constraint validation callbacks. These
callbacks would take an ExperimentMetadata object as input and return
the correct value to use for an experiment subject to additional
constraints.
Parties prefer to wakeup inline, however there are some mechanisms that
want an out-of-line wakeup (say due to a previously held mutex that may
be re-taken). To help those cases permit a guaranteed asynchronous
wakeup.
(needed now for resolver wakeups on the client call path)
We defaulted this on 5 months ago, and it seems to be working... let's
remove the experiment bit!
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
- Accept JSON null for any optional field.
- Do *not* accept JSON null for wrapper types (`absl::optional<>`,
`std::unique_ptr<>`, and `RefCountedPtr<>`) that are *not* marked as
optional fields.
As the [issue](https://github.com/grpc/grpc/issues/10136) documents, the
behavior of AsyncNotifyWhenDone is documented as:
"The comment on `AsyncNotifyWhenDone` states "Has to be called before
the rpc starts" but it seems that if the request tag is returned with
ok=false (i.e. because the CQ is shutting down) then the async done tag
is never received. Instead, I expect the async done tag to be received
regardless of whether or not an incoming call request was successfully
received."
The TODO item is marked closed as stale, and it seems unlikely this will
be resolved, without breaking
existing users whose code is written under the assumption that the tag
is not seen if the call never starts, so it may be time to documented
the idiosyncratic corner case and make it the expected behavior.
(This is a re-open PR for https://github.com/grpc/grpc/pull/32999, which
was closed accidentally due to the branch re-base and force-push)
Implement the frame serialization/deserialization method in chaotic-good
transport.
Previous comments from Craig:
- Since messages are not part of the framing system anymore, I think we
should remove ReceiveMessage (and therefore ReceivePadding) from this
type.
(instead we should add some helper functions to get the message lengths)
-- Resolved
- This approach will cause all frame manipulation code to know about
this serialization detail, rather than just the code that's serializing
it - I think it would be better to keep the type, flags separation (even
if we need to change the flags representation)
-- Done, changed back to type, flags separation.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
I have not been able to reproduce the non-empty pool @ shutdown bug in
around 200k runs of various kinds. Now that experiments are marked flaky
by default, any similar failures should not block PR submission, and
this will give me good signal if the bugs reproduce more frequently in
the CI environment.
I have a fix in theory, but I don't think it should be necessary. If the
bug reproduces, I'll try the fix.
This brings the `ThreadPoolTest.ForkStressTest` down from ~4s to < 1s,
by reducing the lower bound on worker thread wait time between fork
checks when no work is available.
Also changed lifetime of lifeguard's `thread_running_`. I don't believe
this fixes the rare TSAN race we've been seeing, but I believe this
range is more correct.
Will be used to evaluate experiment effects on memory usage once they're
toggled on.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Make the change to improve branch prediction.
For the details, please see #33204.
With the change, the QPS increases by ~0.5% in
"cpp_protobuf_sync_streaming_qps_unconstrained_10mps_secure" test.
cpp_protobuf_sync_streaming_qps_unconstrained_10mps_secure: +0.515%
Most of these data structures need to scale a bit like per-cpu, but not
entirely. We can have more than one cpu hit the same instance in most
cases, and probably want to cap out before the hundreds of shards some
platforms have.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This reverts the changes made to ring_hash in #29872. The comment in the
picker code specifically says not to change these variables to an
unsigned type, but that's exactly what that PR did. I don't know if this
has actually been causing any problems, but given that we duplicated
this algorithm (and that comment) from elsewhere without doing a
detailed analysis of it, it seems prudent to stick with the types that
the original code suggested were important.
To avoid causing problems for ObjC, I have changed this such that
ring_hash is not built unless we are building with xDS support, which we
exclude on mobile. Currently, there is no way to use ring_hash without
xDS, although that might change in the future; if it does, we can deal
with any problems that arise at that point.
Upgrade apple platform deployment_target versions to fix the cocoapods
push of BoringSSL-GRPC about the following error:
```
ld: file not found: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_macosx.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
ref: https://developer.apple.com/forums/thread/725300
This also aligns with the versions required by
[protobuf](https://github.com/protocolbuffers/protobuf/pull/10652)
```
ios.deployment_target = '10.0'
osx.deployment_target = '10.12'
tvos.deployment_target = '12.0'
watchos.deployment_target = '6.0'
```
This test mode tries to create threads wherever it legally can to
maximize the chances of TSAN finding errors in our codebase.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Without this fix it can be the case in the filter based stack that a
unary request that sees the response payload from the wire, prior to
being cancelled, can see the response payload even though we report
cancelled - since we unilaterally remove the error status from the batch
op just prior to evaluating whether to scrub the payload or not.
(also tweaks a couple of log lines that were useful in diagnosing this
bug)
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This PR adds lookup cancellation to the client channel resolver fuzzer,
and adds an optional call to `resolver->Orphan()` at a few specific
points.
I also significantly sped up the fuzzer by removing all mutexes, waits,
and sleeps. It's single-threaded, after all.
---------
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
Use an index instead of a string to select tests (and use that index
module total test count to ensure whatever the fuzzer selects we always
run a test).
This will make the fuzzer corpus unstable when the test count changes,
which I think is fine - it'll regenerate.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
ChannelArgs fuzz configuration is expected to be used in other fuzzing
targets as well. This PR extracts the common code from the API fuzzer
and converts to use the C++ types.
Allow for multiple `--grpc_experiments`, `--grpc_trace` command line
arguments to be added, accumulate them, and provide them to gRPC as one
thing.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Reverts grpc/grpc#33088
The internal test is failing because after the change, transitive
dependencies will be included in response too, thus we need add
`empty2_pb2` to response since it's a dependency of
`empty2_extensions_pb2`.
We already did that in OSS test, we need do the same for internal test
case.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Craig Tiller
AJ Heller
Paulo Castello da Costa
Mark D. Roth
Luwei Ge
Vignesh Babu
Mehrdad Afshari
nanahpang
Steven Wang
Esun Kim
Hannah Shi
Wanlin Du
Eugene Ostroukhov
Yash Tibrewal
Xuan Wang