The previous approach of generating strings was not converging well.
Instead, load a bitfield from the protobuf and use the bits to select
experiments. The fuzzers can explore this space swiftly.
Downside is that as experiments rotate in/out the corpus gets a bit
messed up, but I'm reasonably confident we'll recover quickly.
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
… c-ares versions (#34314)"
This reverts commit eb37b91072.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
The "override" is not added on purpose to remain compatible with
Protobuf < 22.x, as already written in the comment on top of these two
functions.
CC @veblush as the author of this code.
Note: I am personally not super enthousiastic about this change. As an
alternative, I can propose to selectively add the `override` keyword,
based on the value of the `PROTOBUF_VERSION` macro (comparing it to
`4022000`). Tell me if you prefer this version instead.
Tested by changing c-ares to `1.14.0` in Bazel dependency and verify
that the `posix_event_engine_test` build. The test would fail though
since c-ares versions < `1.16.0` does not come with address sorting
capability so the relevant test cases will fail. But this is probably
sufficient to make the Cloud C++ CI job pass and unblock the 1.58
release.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Towards https://github.com/grpc/grpc/issues/33032,
Reopen after botched force-push in #33175 that then got "merged" and
cannot be reopened anymore.
More context in that PR.
---------
Co-authored-by: Isuru Fernando <isuruf@gmail.com>
Co-authored-by: David Chamberlin <david.chamberlin@ln.email.gs.com>
Instead of having the per-call code hop into the `WorkSerializer` to
unref the pickers, have the `SubchannelWrapper` itself hop into the
`WorkSerializer` before it is destroyed.
This also reverts the change made to the WRR picker in #34077, since
that is no longer necessary.
@ctiller PTAL for core configuration changes. I converted the type from
std::function to absl::AnyInvocable. Do you think the functor in
RegisteredBuilder should be callable just once or multiple times?
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Splitting off from https://github.com/grpc/grpc/pull/34273
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
In certain situations the current flow control algorithm can result in
sending one flow control update write for every write sent (known
situation: rollout of promise based server calls with qps_test).
Fix things up so that the updates are only sent when truly needed, and
then fix the fallout (turns out our fuzzer had some bugs)
I've placed actual logic changes behind an experiment so that it can be
incrementally & safely rolled out.
Changes -
1) Change local mesh labels to not be reported on 'started' metrics at
all (even those that we know about) to be consistent. (Since xDS labels
atleast on the server side would not be available on started metric.)
2) Add mesh_id as a local label that is populated by reading the xDS
bootstrap. As part of this, also added a minimal xds bootstrap parsing
logic.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
* Remove `fetch_build_eggs` since they're deprecated and those deps will
be installed by `setuptools`.
* Fix indentation on `run_test` so we don't miss `native` test cases.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
### Background
* `distutils` is deprecated with removal planned for Python 3.12
([pep-0632](https://peps.python.org/pep-0632/)), thus we're trying to
replace all distutils usage with setuptools.
* Please note that user still have access to `distutils` if setuptools
is installed and `SETUPTOOLS_USE_DISTUTILS` is set to `local` (The
default in setuptools, more details can be found [in this
discussion](https://github.com/pypa/setuptools/issues/2806#issuecomment-1193336591)).
### How we decide the replacement
* We're following setuptools [Porting from Distutils
guide](https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html#porting-from-distutils)
when deciding the replacement.
#### Replacement not mentioned in the guide
* Replaced `distutils.utils.get_platform()` with
`sysconfig.get_platform()`.
* Based on the [answer
here](https://stackoverflow.com/questions/71664875/what-is-the-replacement-for-distutils-util-get-platform),
and also checked the document that `sysconfig.get_platform()` is good
enough for our use cases.
* Replaced `DistutilsOptionError` with `OptionError`.
* `setuptools.error` is exporting it as `OptionError` [in the
code](https://github.com/pypa/setuptools/blob/v59.6.0/setuptools/errors.py).
* Upgrade `setuptools` in `test_packages.sh` and changed the version
ping to `59.6.0` in `build_artifact_python.bat`.
* `distutils.errors.*` is not fully re-exported until `59.0.0` (See
[this issue](https://github.com/pypa/setuptools/issues/2698) for more
details).
### Changes not included in this PR
* We're patching some compiler related functions provided by distutils
in our code
([example](ee4efc31c1/src/python/grpcio/_spawn_patch.py (L30))),
but since `setuptools` doesn't have similar interface (See [this issue
for more details](https://github.com/pypa/setuptools/issues/2806)), we
don't have a clear path to replace them yet.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
This API was [removed in Python
3.12](https://github.com/python/cpython/issues/98040).
Fixes Python 3.12 support in `grpcio` tests.
This is relevant to https://github.com/grpc/grpc/issues/33063.
See also https://github.com/grpc/grpc/pull/33492.
----
I have actually only tested this in a form backported to grpc 1.48.4,
and I am not able to test the change to `bazel/_gevent_test_main.py`
directly. However, the backported form allows me to build grpc 1.48.4
for Fedora Rawhide with Python 3.12, and I believe the version in this
PR to be correct—especially, if CI passes for Python 3.11, I believe
this part of the test code will continue to work in Python 3.12.
We print the status of all experiments at grpc init time, but that
status is in experiment declaration order - which isn't particularly
useful for being able to find if your experiment is enabled or not
quickly.
Change that to lexically sort experiments.
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
This should address one of the failures we're seeing in #34224.
The test failure is caused by the changes in timing triggering a race
condition. In the code at head, we delay sending out the subscription
for the first CDS watch until we've already seen the other two CDS
watches, because the previous send_message op has not yet completed, and
by the time it does, we've seen all 3 watches, so we can send a
subscription for all 3 at the same time. With the WorkSerializer change,
the send_message op is complete by the time we see the first CDS watch,
so we subscribe to only that resource, and then later add the other two.
The result is that we'll NACK twice with two different messages, the
first one including only the error about the first resource, and the
second one including all three.
I suspect this same race condition would have been triggered eventually
by the EventEngine migration anyway; the current test basically depends
on the single-thread timing of the iomgr approach. So I'm addressing it
by replacing the e2e test with a unit test that covers the same cases
without the timing issue.
This test is not giving us good signals while adding some long term
maintenance cost. The way the test is set up and run was not standard.
There are other sanitizer tests to test for memory leaks.
Without this change, calling `find_package(gRPC REQUIRED)` errors out
with
CMake Error at /opt/homebrew/lib/cmake/grpc/gRPCConfig.cmake:8
(find_dependency):
Unknown CMake command "find_dependency".
The issue is that `find_dependency` is provided by the
`CMakeFindDependencyMacro` module[^1], so we need to `include` it before
use.
[^1]:
https://cmake.org/cmake/help/v3.26/module/CMakeFindDependencyMacro.html
Rolls forward part of the dualstack changes, mostly from #33427 and a
little bit from #32692, both of which were reverted in #33718.
Specifically:
- For petiole policies, unconditionally start health watch on
subchannels, even if client side health checking is not enabled; in this
case, the health watch will report the subchannel's raw connectivity
state.
- Fix edge cases in health check reporting that occur when a watcher is
started before the initial state is reported.
- When client-side health checking fails, add the subchannel's address
to the RPC failure status message.
- Outlier detection now works only via the health checking watch, not
via the raw connectivity state watch.
- Remove now-unnecessary hack to ensure that outlier detection does not
work for pick_first.
This reverts https://github.com/grpc/grpc/pull/34170. We (protobuf) have
decided not to release the plugin feature that would have required this
without an actual use-case asking for it. Since this is *technically* a
breaking change, it's better to roll it back in gRPC until we actually
need it.
- Update to windows RBE docker image, followed by subsequent regenerate
of win RBE toolchain (basically verifying that the README.md for windows
RBE toolchain works well)
- Also move windows docker image from GCR to GAR (which is where our
other testing images already live).
- (cleanup) Move windows RBE docker image to a better location in our
repo.
This PR should be sufficient to demonstrate how we would go about
upgrading the windows RBE build to VS2022 once needed.
Esun Kim
Craig Tiller
nanahpang
Romain Geissler @ Amadeus
yifeizhuang
Yijie Ma
Stanley Cheung
h-vetinari
AJ Heller
Yash Tibrewal
Mark D. Roth
Andre Gaschler
Arvind Bright
Eugene Ostroukhov
Sergii Tkachenko
Xuan Wang
Ben Beasley
Carlo Cabrera
Jan Tattermusch
Mike Kruskal