I have not been able to reproduce the non-empty pool @ shutdown bug in
around 200k runs of various kinds. Now that experiments are marked flaky
by default, any similar failures should not block PR submission, and
this will give me good signal if the bugs reproduce more frequently in
the CI environment.
I have a fix in theory, but I don't think it should be necessary. If the
bug reproduces, I'll try the fix.
This brings the `ThreadPoolTest.ForkStressTest` down from ~4s to < 1s,
by reducing the lower bound on worker thread wait time between fork
checks when no work is available.
Also changed lifetime of lifeguard's `thread_running_`. I don't believe
this fixes the rare TSAN race we've been seeing, but I believe this
range is more correct.
Make the change to improve branch prediction.
For the details, please see #33204.
With the change, the QPS increases by ~0.5% in
"cpp_protobuf_sync_streaming_qps_unconstrained_10mps_secure" test.
cpp_protobuf_sync_streaming_qps_unconstrained_10mps_secure: +0.515%
Most of these data structures need to scale a bit like per-cpu, but not
entirely. We can have more than one cpu hit the same instance in most
cases, and probably want to cap out before the hundreds of shards some
platforms have.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This reverts the changes made to ring_hash in #29872. The comment in the
picker code specifically says not to change these variables to an
unsigned type, but that's exactly what that PR did. I don't know if this
has actually been causing any problems, but given that we duplicated
this algorithm (and that comment) from elsewhere without doing a
detailed analysis of it, it seems prudent to stick with the types that
the original code suggested were important.
To avoid causing problems for ObjC, I have changed this such that
ring_hash is not built unless we are building with xDS support, which we
exclude on mobile. Currently, there is no way to use ring_hash without
xDS, although that might change in the future; if it does, we can deal
with any problems that arise at that point.
Upgrade apple platform deployment_target versions to fix the cocoapods
push of BoringSSL-GRPC about the following error:
```
ld: file not found: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_macosx.a
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
ref: https://developer.apple.com/forums/thread/725300
This also aligns with the versions required by
[protobuf](https://github.com/protocolbuffers/protobuf/pull/10652)
```
ios.deployment_target = '10.0'
osx.deployment_target = '10.12'
tvos.deployment_target = '12.0'
watchos.deployment_target = '6.0'
```
This test mode tries to create threads wherever it legally can to
maximize the chances of TSAN finding errors in our codebase.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Without this fix it can be the case in the filter based stack that a
unary request that sees the response payload from the wire, prior to
being cancelled, can see the response payload even though we report
cancelled - since we unilaterally remove the error status from the batch
op just prior to evaluating whether to scrub the payload or not.
(also tweaks a couple of log lines that were useful in diagnosing this
bug)
Allow for multiple `--grpc_experiments`, `--grpc_trace` command line
arguments to be added, accumulate them, and provide them to gRPC as one
thing.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Reverts grpc/grpc#33088
The internal test is failing because after the change, transitive
dependencies will be included in response too, thus we need add
`empty2_pb2` to response since it's a dependency of
`empty2_extensions_pb2`.
We already did that in OSS test, we need do the same for internal test
case.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
When we deprecate services in proto-file:
```proto
syntax = "proto3";
option php_namespace = "Test";
service MyService {
option deprecated = true;
rpc Call(Test) returns (Test) {
option deprecated = true;
};
}
message Test {
}
```
It might be a good idea to propagate this flat in the generated classes
so that the final users will be aware of the changes coming soon.
Example of the generated code:
```php
<?php
// GENERATED CODE -- DO NOT EDIT!
namespace Test;
/**
* @deprecated
*/
class MyServiceClient extends \Grpc\BaseStub {
/**
* @param string $hostname hostname
* @param array $opts channel options
* @param \Grpc\Channel $channel (optional) re-use channel object
*/
public function __construct($hostname, $opts, $channel = null) {
parent::__construct($hostname, $opts, $channel);
}
/**
* @deprecated
* @param \Test\Test $argument input argument
* @param array $metadata metadata
* @param array $options call options
* @return \Grpc\UnaryCall
*/
public function Call(\Test\Test $argument,
$metadata = [], $options = []) {
return $this->_simpleRequest('/MyService/Call',
$argument,
['\Test\Test', 'decode'],
$metadata, $options);
}
}
```
These tags will cause IDE to highlight the class/method to bring the
developer's attention.
This is consistent with how "protoc" handles the "deprecated" option for
PHP
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This reverts commit 1624542ea4, relanding
https://github.com/grpc/grpc/pull/32956
Because of some proto dependency and build problems internally, I've
removed the ServiceConfig proto fuzzing component. These build issues
can hopefully be resolved soon, and then we can re-add the deleted
implementation from commit
[b078c9c](b078c9c015)
in this PR.
`tools/run_tests/sanity/check_absl_mutex.sh` was broken, a missing paren
crashed the script if run locally. It's unclear yet how our sanity
checks were not complaining about this, `run_tests.py` does not save the
log.
Rare bug: server initial metadata gets stranded in the outbound pipe.
(fix is a little unpleasant, but we'll do better at the five pipes
stage)
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
re2 previously failed to compile if:
1. An old `re2` version is installed with a non-standard system prefix,
such as `/opt/local`.
2. The environment variable is set: `CPPFLAGS=-I/opt/local/include`.
Running `make` would result in function prototype mismatches because the
Makefile would previously attempt to use the headers from
`/opt/local/include/re2` before the `third_party/re2/re2` directory.
https://github.com/grpc/grpc/pull/27660 caused `CPPFLAGS` to inherit
from the environment, but this can cause the Makefile to use external
include files for re2 and other libraries if `-I` flags are defined.
This commit reverts to the original behavior of only using
`RbConfig::CONFIG` values to avoid using the wrong headers.
The logger uses `absl::FPrintF` to write to stdout. After reading a
number of sources online, I got the impression that `std::fwrite` which
is used by `absl::FPrintF` is atomic so there is no locking required
here.
---------
Co-authored-by: rockspore <rockspore@users.noreply.github.com>
This is a mistake made in https://github.com/grpc/grpc/pull/33030.
`sizeof()` would count the null byte terminated the C string and would
cause us to skip a byte if it is used as the index to
`result->substr()`. This would also crash if `result` only contains
`grpc_config=` as @drfloob pointed out.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This metadata doesn't actually encode so passing it through from an app
will force a crash.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Instead just Utf-16 encode the null byte when dumping the value to a
string form.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
#thistimeforsure
a863532c62 adds some debug to help track
which batches get leaked by a transport
3203e75ec5 makes connected_channel respect
the high level intent of cancellation better (and fixes the last reason
we needed to turn these tests off)
aaf5fa036b re-enables testing of c++ e2e
tests with server based promise calls
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Was leading to a nullptr deref, and we just don't need this one anymore.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
With these, it is actually possible to have typed client stubs where the
return type is correctly inferred.
It's only for the non-streaming calls, because there is
`RequestIterableType` for the streaming ones (but it's just Any with
extra steps and would require much more work).
---------
Co-authored-by: Xuan Wang <xuanwn@google.com>
In collaboration with @Vignesh2208 . This supersedes
https://github.com/grpc/grpc/pull/32622. The original description is
below.
----
The current endpoint semantics are as follows:
On endpoint shutdown, the socket is immediately closed regardless and
any pending registered read/write closures are not immediately executed.
The pending read/write closures get executed with aborted error whenever
the next grpc_iocp_work method runs.
However the grpc_iocp_work may run only during grpc_shutdown and
grpc_shutdown may only get scheduled after the pending registered
read/write closures execute.
This PR changes the shutdown semantics to match shutdown semantics used
in posix - i.e On endpoint shutdown, the socket is immediately closed
and any pending registered read/write closures are executed immediately.
Additional care is taken to ensure that the socket is not immediately
deleted because the pending I/O ops still need to be flushed later
during grpc_shutdown.
---------
Co-authored-by: Vignesh Babu <vigneshbabu@google.com>
@sampajano This should fix b/265779666. If the CFEngine ends up taking
some time to land with rollbacks, bugs and whatnot, this can work in the
meantime.
Fixes https://github.com/grpc/grpc/issues/32481.
Please test this with the (excellent) repro case in
https://github.com/grpc/grpc/pull/33000, and consider merging _just_ the
test from that PR.
Per #32481, the issue was bisected to
https://github.com/grpc/grpc/pull/30101. What changed in that PR is that
the epoll1 engine is only checked for availablily once per process at
iomgr initialization (which as a side effect initializes the engine),
but the engine was being shutdown with `grpc_shutdown` anyhow. With
repeated cycles of grpc init & shutdown in the same process, the second
attempt to reinit and use gRPC finds the epoll1 engine in an invalid
state.
Reverts grpc/grpc#32909
It's breaking some internal test
(//net/grpc/python:internal_tests/unit/_default_reflection_test), revert
for now to investigate.
AJ Heller
Steven Wang
Craig Tiller
Mark D. Roth
Hannah Shi
Vignesh Babu
Esun Kim
Luwei Ge
Xuan Wang
Eugene Ostroukhov
Yijie Ma
Mikhail Galanin
Stan Hu
Vojtěch Boček
Alisha Nanda