This change simplifies `EventEngine::DNSResolver`'s API based on the
proposal:
[go/event-engine-dns-resolver-api-changes](http://go/event-engine-dns-resolver-api-changes).
Note that this API change + the implementation described in
[go/event-engine-dns-resolver-implementation](http://go/event-engine-dns-resolver-implementation)
has already been tested against our main test suites and are passing
them.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Fixes#24206 by ensuring that IOCP/socket errors in the iomgr on_read
callback are properly annotated with the gRPC Unavailable status. The
WindowsEventEngine was already doing this correctly (try running the
client with `$env:GRPC_EXPERIMENTS="event_engine_client"`).
This also adds two small cleanups:
* Cleanly prints statuses with their child statuses in a few spots
within the chttp2 transport logging (previously, child messages were
printed with garbled bits)
* Adds friendly names to a subset of WSA errors that we're likely to see
from common operations. The top-level status message will no longer just
say "WSA Error" in many cases.
CC @Hamza-Q
This fixes a dumb bug from #33359, where I used `==` instead of `<` in
the comparator functor.
It also avoids unnecessary down-casting of the `unique_ptr<>`.
This reverts commit e107ff5e99.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Noticed this in a nested forking test in
https://github.com/grpc/grpc/pull/33430 (by nested fork - forked child
process forks again).
Before this change, the EventEngine and non-EventEngine pollers were
competing with each other in their calls to
`Fork::SetResetChildPollingEngineFunc`. In the first child's after-fork
handler, the EE engine would [clear
out](123da4a866/src/core/lib/event_engine/posix_engine/ev_epoll1_linux.cc (L260))
the post-fork handlers of the non-EE poller, leaving the grandchild
without the right post-fork cleanup method.
New versions of Clang (and GCC) warn about this
grpc_authorization_policy_provider.cc:124:17: error: format specifies
type 'int' but the argument has type 'absl::StatusCode'
[-Werror,-Wformat]
124 | "authorization policy reload status. code=%d error_details=%s",
| ~~
125 | status.code(), std::string(status.message()).c_str());
In chttp2: a pending but not yet sent goaway should block incoming
requests just like a sent one (we will sent that data momentarily!)
In the test:
- handle the case of the connection idle timeout happening before the
request arrives at the server
- disable retries, as these cause the request to get stuck (as we don't
have an additional server to retry on)
Fix b/287897932
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Noticed some inconsistencies in our keepalive configuration -
* Earlier, even if keepalive pings were disabled, we would be scheduling
keepalive pings at an interval of INT_MAX ms.
* We were not using `g_default_client_keepalive_permit_without_calls` /
`g_default_server_keepalive_permit_without_calls`. They are both false
by default but they can be overridden in
`grpc_chttp2_config_default_keepalive_args`.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Child classes of polling resolvers might have use cases where they would want to request a resolution explicitly which honor the min time between requests. As RequestReresolutionLocked needs to be called in from work_serializer which is usually owned by polling resolver completely, allow passing the work serializer back to the child class.
In rare cases across gRPC's end2end tests (only in Windows RBE with
debug builds, interestingly enough), the endpoint may be destroyed
before the final IOCP read callback has run. This edge case is only
triggered when all the following are true:
* the previous read operation received data (not an error, not
0-length), so more data is expected in the stream.
* the socket has not yet shut down
* the application destroyed its endpoint before (or during) the IOCP
callback execution.
* the Read operation has not yet called the client's on_read callback.
This is a valid scenario, and it is expected that the engine
implementation should call the application callbacks with error statuses
when this occurs. This PR fixes two associated bugs.
Previously we were running these at call termination, but internally we
have things that want to access finalizer cleaned up data after that
point.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Fix fuzzer found bug b/286716972
Follows up on https://github.com/grpc/grpc/pull/33266 but gets the edge
case right of when there's a read queued before the peer closes - in
that case we weren't waking up the read.
PanCakes to the rescue!
We noticed that our 'sanity' test was going to fail, but we think we can
fix that automatically, so we put together this PR to do just that!
If you'd like to opt-out of these PR's, add yourself to NO_AUTOFIX_USERS
in .github/workflows/pr-auto-fix.yaml
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Otherwise when we fill in peer in `SetCommonEntryFields` it will be
empty/invalid.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
The following bugs are fixed:
* Missing ExecCtx in event engine endpoints and listeners
* Ref counting issue with iomgr endpoint which causes crashes in
overloaded situations
The PR includes a test which triggers these bugs by simulating an
overloaded system.
With some delay, this is a PR for
https://github.com/grpc/grpc/issues/32564 (and previously
https://github.com/grpc/grpc/pull/31791).
I looked into adding a regular `py_test` for this change [as
suggested](https://github.com/grpc/grpc/pull/31791#issuecomment-1423245116)
but I am not aware of any effect that the presence of a .pyi stub file
would have at runtime and where some sort of type-checking in a .py
script would be affected. Stub files are only for use by type checkers &
IDE's. I mean, something like this would work:
```
import helloworld_pb2
py_file = helloworld_pb2.__file__
pyi_file = py_file + 'i’
self.assertTrue(os.path.exists(pyi_file))
```
But that seems really hacky to me. Instead I created a simple rule test
for `py_proto_library` with Bazel Skylib which tests the declared
outputs for an example `py_proto_library` target. Indirectly, this also
tests that the declared output files are actually generated. Please let
me know if this is sufficient.
- Switched from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies
Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.
To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
The function grpc_rb_call_run_batch has many places that could raise
errors, including in child functions. Since a raised error will longjump
out of the function, it will cause memory leaks since the function
cannot perform any clean up. This commit fixes the issue by wrapping the
whole function in an rb_ensure, which will ensure that a cleanup
function is ran before the error is propagated upwards.
The function grpc_rb_server_request_call has many places that could
raise errors, including in child functions. Since a raised error will
longjump out of the function, it will cause memory leaks since the
function cannot perform any clean up. This commit fixes the issue by
wrapping the whole function in an rb_ensure, which will ensure that a
cleanup function is ran before the error is propagated upwards.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Revert "Revert "[core] Add support for vsock transport"
(https://github.com/grpc/grpc/pull/33276)"
This reverts commit
c5ade3011a.
And fix the issue which broke the python build.
@markdroth@drfloob please review this PR. Thank you very much.
---------
Co-authored-by: AJ Heller <hork@google.com>
Mark D. Roth
Yijie Ma
AJ Heller
Yash Tibrewal
Esun Kim
apolcyn
Vignesh Babu
Eugene Ostroukhov
Benjamin Kramer
Craig Tiller
Yuriy Chernyshov
Anirudh Ramachandra
github-actions[bot]
Florian Ferstl
Sergii Tkachenko
Peter Zhu
Yousuk Seung
Yadong