The client promise code seems to cause a problem with iomgr pollset shutdown which is causing flakiness.
Right now I don't think it's likely that we'll get this code rolled out before the event engine client lands, so I'm making the experiment dependent on event engine polling.
Closes#35678
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35678 from ctiller:flake 4512fa81b0
PiperOrigin-RevId: 601926443
* Generate observability and grpcio-tools dependency file when building project.
* Also removed deps file from black formatter.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35681
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35681 from XuanWang-Amos:fix_build_o11y 84958c2d59
PiperOrigin-RevId: 601873635
rebuilt `tools/dockerfile/distribtest/csharp_ubuntu2204_x64/Dockerfile` (without actually changing anything in the docker instructions) in an attempt to fix `The SSL connection could not be established, see inner exception.` errors happening in `dotnet restore` commands in distrib tests using this image.
Closes#35674
PiperOrigin-RevId: 601856906
This PR:
* adds FD extensions to the public headers
* Adds the query extension interface to EventEngine, Listener, and Endpoint, via a new `Extensible` interface
* Refactors the PosixEventEngine to use the Extensible interface.
Closes#35648
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35648 from drfloob:posix-ee-query-interface 7cae28e0b3
PiperOrigin-RevId: 601794970
When testing CSM Observability, we discovered that the c++ xds interop client is not sending any payload with the `UnaryCall` RPCs so most of the metrics will have a value of 0.
Adding a payload to the xds interop client here.
We need this fix so that we can verify that the metrics are recording the right number of bytes being sent / received. So we need a non-trivial payload to be sent with the `UnaryCall` RPC between the xds interop client and server.
Closes#35545
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35545 from stanley-cheung:xds-client-payload 11be4e6f4f
PiperOrigin-RevId: 601596246
Fix: https://github.com/grpc/grpc/issues/35061
Add note to indicate that iterator and read/write APIs can't be mixed in a single RPC.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35598
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35598 from XuanWang-Amos:aio_mix_apis_doc 4c4cfbf06e
PiperOrigin-RevId: 601580411
This adds the commit info (author, title, link to commit, link to CL) for the commits that are missing the PR number to the release note draft.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35658
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35658 from yijiem:fix-release-note-script 7254e4f7de
PiperOrigin-RevId: 601539726
This reverts commit 6318e9e7e9.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35667
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35667 from ctiller:a 032999b51e
PiperOrigin-RevId: 601495207
Fix: https://github.com/grpc/grpc/issues/30424
Currently the implementation of `maximum_concurrent_rpcs` flag in AIO is different with docstring, in implementation, we're waiting for RPCs to finish and continue execution instead of raising `resource_exhausted` as mentioned in docstring.
This PR changes the implementation of `maximum_concurrent_rpcs` flag in AIO stack to match the docstring.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35376
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35376 from XuanWang-Amos:fix_aio_concurrent_rpc 5b2b59fb9f
PiperOrigin-RevId: 601490418
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Just to be future-proof, I'm amending the `void` return status of `BuildAndRegisterGlobal` in `OpenTelemetryPluginBuilder` to absl::Status.
This will be backported to 1.61 as well.
Closes#35659
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35659 from yashykt:UpdateOtelApiToAddStatus 07d3f41b8a
PiperOrigin-RevId: 601458408
We are no longer sure about this API, so re-experimentalizing it.
This PR will be backported to 1.61 as well.
Closes#35660
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35660 from yashykt:ReexperimentalizeCsmPluginOption 4f114a54d9
PiperOrigin-RevId: 601378856
BoringSSL-GRPC uses sources from https://github.com/google/boringssl.git, which doesn't contain PrivacyInfo.xcprivacy file.
This PR embeds the manifest data into the podspec and extract during pod install, similar to file boringssl_prefix_symbols.h
--
Will need to regenerate the podspec after #35542 merged.
Closes#35616
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35616 from HannahShiSFB:privacy-manifests-boring-ssl d068be244b
PiperOrigin-RevId: 601274520
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35573
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35573 from yijiem:enable-oss-ee-dns-posix-real 017b99312f
PiperOrigin-RevId: 601245249
Passed [gamma.gamma_baseline_test](https://github.com/grpc/psm-interop/blob/main/tests/gamma/affinity_test.py) and [gamma.affinity_test](https://github.com/grpc/psm-interop/blob/main/tests/gamma/affinity_test.py) locally with those changes.
The docker image used for testing was pushed to Container Registry and can be tested locally using this flag:
`--client_image=gcr.io/grpc-testing/xds-interop/python-client:test_ssa_interop_python`
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35596
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35596 from XuanWang-Amos:python_ssa_client_test af026547ef
PiperOrigin-RevId: 601164317
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35633
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35633 from yijiem:labels-injector-patch 6876243943
PiperOrigin-RevId: 600931754
Example:
```
virtualenv Python module not installed. Attempting to install via pip.
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
############################
Your administrator is _insisting_ on managing your packages themself. Try running `sudo apt-get install python3-virtualenv`
############################
```
Closes#35642
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35642 from gnossen:fix_fix_fix_protobuf 047470aaa2
PiperOrigin-RevId: 600909347
Instead, connect a real client, which gives us the ability to exercise the listener portion of the code too.
Closes#35625
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35625 from ctiller:shush-connect b8f78993c6
PiperOrigin-RevId: 600836565
This ensures that if a cluster is used both in the RouteConfig and via a ClusterSpecifierPlugin, and then the entry in the RouteConfig goes away, we won't unsubscribe and then resubscribe to the CDS resource.
Closes#35627
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35627 from markdroth:xds_rls_dynamic_subscription_fix c78afaf6ce
PiperOrigin-RevId: 600801766
`ProtoBitGen` provides a random number generator that returns values directly from fuzzer selected values, which allows us to test-into random selection algorithms deterministically.
Since the list of values provided by the fuzzer is limited, we need a fallback implementation. Previously we'd used something that was very correlated, and some of the distribution algorithms get into a very slow convergence mode when we do that (so we repeatedly return the same value for billions of iterations and cause timeouts in fuzzers).
Instead, when we run out of fuzzer supplied values, seed an mt19937 generator with the fuzzer selected values and use that from there on. Said generator will then produce values deterministically (for a given fuzzer input), but with a better distribution to allow convergence for fiddly algorithms.
Closes#35621
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35621 from ctiller:cg-timeout 6c9ef9cac5
PiperOrigin-RevId: 600607424
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Follow-up to add privacy manifests to #35042
- [x] Update podspec template files
- [x] Align on manifest update:
```
git grep mach_absolute_time
src/core/lib/gpr/posix/time.cc:static uint64_t g_time_start = mach_absolute_time();
src/core/lib/gpr/posix/time.cc: ((double)(mach_absolute_time() - g_time_start)) * g_time_scale;
```
cc: @paulb777Closes#35542
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35542 from ncooke3:nc/privacy-manifests b0e0e57d2e
PiperOrigin-RevId: 600595032
Actually build O11y artifacts.
### Testing
* Manually installed the `.whl` and verified it's working locally (For Python 3.8 + Linux).
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35578
PiperOrigin-RevId: 600566829
As announced in https://discourse.llvm.org/t/clang-16-notice-of-potentially-breaking-changes/65562, clang 16 defaults `-Wincompatible-function-pointer-types` to be on. This causes a number of hard errors due to implicit conversions between `void *` and `void`, such as:
```
../../../../src/ruby/ext/grpc/rb_channel.c:770:47: error: incompatible function pointer types passing 'VALUE (VALUE)' (aka 'unsigned long (unsigned long)') to parameter of type 'VALUE (*)(void *)' (aka 'unsigned long (*)(void *)') [-Wincompatible-function-pointer-types]
g_channel_polling_thread = rb_thread_create(run_poll_channels_loop, NULL);
^~~~~~~~~~~~~~~~~~~~~~
/root/.rbenv/versions/3.1.4/include/ruby-3.1.0/ruby/internal/intern/thread.h:190:32: note: passing argument to parameter 'f' here
VALUE rb_thread_create(VALUE (*f)(void *g), void *g);
^
../../../../src/ruby/ext/grpc/rb_channel.c:780:41: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
void grpc_rb_channel_polling_thread_stop() {
^
void
../../../../src/ruby/ext/grpc/rb_channel.c:786:30: error: incompatible function pointer types passing 'void (void *)' to parameter of type 'void *(*)(void *)' [-Wincompatible-function-pointer-types]
rb_thread_call_without_gvl(run_poll_channels_loop_unblocking_func, NULL, NULL,
```
This commit fixes these pointer types using wrapper functions where necessary.
This issue was also raised on the FreeBSD port of the grpc gem: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271540
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34481
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34481 from stanhu:sh-ruby-fix-clang-16 63c7424ad0
PiperOrigin-RevId: 600548467
Using `AF_UNSPEC` for both IPv4 and IPv6 queries does not work in all cases. Specifically, for `localhost:<>`, c-ares only returns the IPv6 record i.e. `::1`.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35530
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35530 from yijiem:enable-oss-ee-dns-posix 452b5a2d81
PiperOrigin-RevId: 599989537
Craig Tiller
Xuan Wang
AJ Heller
apolcyn
Yash Tibrewal
Esun Kim
Stanley Cheung
Yijie Ma
Hannah Shi
Vignesh Babu
Eugene Ostroukhov
Richard Belleville
Mark D. Roth
Nick Cooke
Stan Hu
Alisha Nanda