As documented in [0], there are two certificate verification callbacks in the OpenSSL/BoringSSL TLS API. The one taken as a parameter to SSL_CTX_set_verify is the "verify callback". It is called multiple times during a single certificate verification is used to suppress errors and otherwise be notified about various events during verification.
Such a callback is not appropriate for accepting all certificates (you waste time processing things that will be thrown away), nor for post-verification inspection of the result (it will run multiple times). This is, however, what gRPC does with it.
Rather, gRPC should have used SSL_CTX_set_cert_verify_callback, which swaps out the verification process entirely. That is called exactly once per handshake and allows you to skip the verification, or verify and then inspect the results afterwards. Fix gRPC to heed the documentation.
In addition, this PR fixes a lifetime bug in gRPC's handling of the root certificate. RootCertExtractCallback stashes the root certificate without retaining it anywhere, but the X509_STORE_CTX will shortly be destroyed. There is no immediate guarantee the X509 object lasts as long as the SSL object. It most likely does because the object is often cached in the X509_STORE, which lives on the SSL_CTX, but this is at best, non-obvious. Instead, gRPC should have made
g_ssl_ex_verified_root_cert_index own a refcount to the X509 object by registering a free function and calling X509_up_ref when saving the value.
[0] https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_CTX_set_verify
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35369
It looks like this ended up getting deleted in https://github.com/grpc/grpc/pull/34350 probably when merging.
Also, the `Init` method in the otel test library is getting unwieldy. I'm going to send out a follow-up PR to convert this into a builder instead.
Closes#35532
We're having some issues internally, rolling this change back for now and wait for the affected users to figure out a solution.
This reverts: https://github.com/grpc/grpc/pull/35002, https://github.com/grpc/grpc/pull/35482 and 6872a7a473
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35522
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35210
New source of truth: https://github.com/grpc/psm-interop.
This PR removes PSM Interop framework source code from `tools/run_tests/xds_k8s_test_driver`, and all references to it.
Closes#35466
Recently two more openssl tests were added to the portability test suite. At-head tests are using the same set, having an unintended big surge in the test time, causing timeout. So I've changed at-head tests not to run openssl tests.
Closes#35520
We probably want to reconsider this types role in the system, but for now removing it would be a large job. Move it to somewhere isolated for now.
Closes#35504
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.5 to 1.26.18.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/urllib3/urllib3/releases">urllib3's releases</a>.</em></p>
<blockquote>
<h2>1.26.18</h2>
<ul>
<li>Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other" redirect responses. (GHSA-g4mx-q9vg-27p4)</li>
</ul>
<h2>1.26.17</h2>
<ul>
<li>Added the <code>Cookie</code> header to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set via <code>Retry.remove_headers_on_redirect</code>. (GHSA-v845-jxx5-vc9f)</li>
</ul>
<h2>1.26.16</h2>
<ul>
<li>Fixed thread-safety issue where accessing a <code>PoolManager</code> with many distinct origins would cause connection pools to be closed while requests are in progress (<a href="https://redirect.github.com/urllib3/urllib3/issues/2954">#2954</a>)</li>
</ul>
<h2>1.26.15</h2>
<ul>
<li>Fix socket timeout value when HTTPConnection is reused (<a href="https://redirect.github.com/urllib3/urllib3/issues/2645">urllib3/urllib3#2645</a>)</li>
<li>Remove "!" character from the unreserved characters in IPv6 Zone ID parsing (<a href="https://redirect.github.com/urllib3/urllib3/issues/2899">urllib3/urllib3#2899</a>)</li>
<li>Fix IDNA handling of 'x80' byte (<a href="https://redirect.github.com/urllib3/urllib3/issues/2901">urllib3/urllib3#2901</a>)</li>
</ul>
<h2>1.26.14</h2>
<ul>
<li>Fixed parsing of port 0 (zero) returning None, instead of 0 (<a href="https://redirect.github.com/urllib3/urllib3/issues/2850">#2850</a>)</li>
<li>Removed deprecated <code>HTTPResponse.getheaders()</code> calls in <code>urllib3.contrib</code> module.</li>
</ul>
<h2>1.26.13</h2>
<ul>
<li>Deprecated the <code>HTTPResponse.getheaders()</code> and <code>HTTPResponse.getheader()</code> methods.</li>
<li>Fixed an issue where parsing a URL with leading zeroes in the port would be rejected even when the port number after removing the zeroes was valid.</li>
<li>Fixed a deprecation warning when using cryptography v39.0.0.</li>
<li>Removed the <code><4</code> in the <code>Requires-Python</code> packaging metadata field.</li>
</ul>
<h2>1.26.12</h2>
<ul>
<li>Deprecated the <code>urllib3[secure]</code> extra and the <code>urllib3.contrib.pyopenssl</code> module. Both will be removed in v2.x. See this <a href="https://redirect.github.com/urllib3/urllib3/issues/2680">GitHub issue</a> for justification and info on how to migrate.</li>
</ul>
<h2>1.26.11</h2>
<p><strong>If you or your organization rely on urllib3 consider supporting us via <a href="https://github.com/sponsors/urllib3">GitHub Sponsors</a>.</strong></p>
<p>⚠️ <strong>urllib3 v2.0 will drop support for Python 2</strong>: <a href="https://urllib3.readthedocs.io/en/latest/v2-roadmap.html">Read more in the v2.0 Roadmap</a></p>
<ul>
<li>Fixed an issue where reading more than 2 GiB in a call to HTTPResponse.read would raise an OverflowError on Python 3.9 and earlier.</li>
</ul>
<h2>1.26.10</h2>
<p><strong>If you or your organization rely on urllib3 consider supporting us via <a href="https://github.com/sponsors/urllib3">GitHub Sponsors</a>.</strong></p>
<p>⚠️ <strong>urllib3 v2.0 will drop support for Python 2</strong>: <a href="https://urllib3.readthedocs.io/en/latest/v2-roadmap.html">Read more in the v2.0 Roadmap</a></p>
<p>🔐 <strong>This is the first release to be signed with Sigstore!</strong> You can verify the distributables using the <code>.sig</code> and <code>.crt</code> files included on this release.</p>
<ul>
<li>Removed support for Python 3.5</li>
<li>Fixed an issue where a <code>ProxyError</code> recommending configuring the proxy as HTTP instead of HTTPS could appear even when an HTTPS proxy wasn't configured.</li>
</ul>
<h2>1.26.9</h2>
<p><strong>If you or your organization rely on urllib3 consider supporting us via <a href="https://github.com/sponsors/urllib3">GitHub Sponsors</a>.</strong></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's changelog</a>.</em></p>
<blockquote>
<h1>1.26.18 (2023-10-17)</h1>
<ul>
<li>Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other" redirect responses.</li>
</ul>
<h1>1.26.17 (2023-10-02)</h1>
<ul>
<li>Added the <code>Cookie</code> header to the list of headers to strip from requests when redirecting to a different host. As before, different headers can be set via <code>Retry.remove_headers_on_redirect</code>. (<code>[#3139
](https://github.com/urllib3/urllib3/issues/2865) <https://github.com/urllib3/urllib3/issues/2865></code>__)</li>
</ul>
<h1>1.26.13 (2022-11-23)</h1>
<ul>
<li>Deprecated the <code>HTTPResponse.getheaders()</code> and <code>HTTPResponse.getheader()</code> methods.</li>
<li>Fixed an issue where parsing a URL with leading zeroes in the port would be rejected
even when the port number after removing the zeroes was valid.</li>
<li>Fixed a deprecation warning when using cryptography v39.0.0.</li>
<li>Removed the <code><4</code> in the <code>Requires-Python</code> packaging metadata field.</li>
</ul>
<h1>1.26.12 (2022-08-22)</h1>
<ul>
<li>Deprecated the <code>urllib3[secure]</code> extra and the <code>urllib3.contrib.pyopenssl</code> module.
Both will be removed in v2.x. See this <code>GitHub issue <https://github.com/urllib3/urllib3/issues/2680></code>_
for justification and info on how to migrate.</li>
</ul>
<h1>1.26.11 (2022-07-25)</h1>
<ul>
<li>Fixed an issue where reading more than 2 GiB in a call to <code>HTTPResponse.read</code> would</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="9c2c2307dd"><code>9c2c230</code></a> Release 1.26.18 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3159">#3159</a>)</li>
<li><a href="b594c5ceac"><code>b594c5c</code></a> Merge pull request from GHSA-g4mx-q9vg-27p4</li>
<li><a href="944f0eb134"><code>944f0eb</code></a> [1.26] Use vendored six in urllib3.contrib.securetransport</li>
<li><a href="c9016bf464"><code>c9016bf</code></a> Release 1.26.17</li>
<li><a href="01220354d3"><code>0122035</code></a> Backport GHSA-v845-jxx5-vc9f (<a href="https://redirect.github.com/urllib3/urllib3/issues/3139">#3139</a>)</li>
<li><a href="e63989f97d"><code>e63989f</code></a> Fix installing <code>brotli</code> extra on Python 2.7</li>
<li><a href="2e7a24d087"><code>2e7a24d</code></a> [1.26] Configure OS for RTD to fix building docs</li>
<li><a href="57181d6ea9"><code>57181d6</code></a> [1.26] Improve error message when calling urllib3.request() (<a href="https://redirect.github.com/urllib3/urllib3/issues/3058">#3058</a>)</li>
<li><a href="3c0148048a"><code>3c01480</code></a> [1.26] Run coverage even with failed jobs</li>
<li><a href="d94029b7e2"><code>d94029b</code></a> Release 1.26.16</li>
<li>Additional commits viewable in <a href="https://github.com/urllib3/urllib3/compare/1.26.5...1.26.18">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/grpc/grpc/network/alerts).
</details>
> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.
Closes#34714
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35494
(Required recursively splitting some other targets too)
Gets us closer to eliminating `grpc_base`, and (more importantly) sets up some pieces of transport to be split out so I can rebuild them in coming months.
Closes#35496
Add token based authentication example
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
`distribtest.cpp_linux_x64_debian10_aarch64_cross_cmake_aarch64_cross` has been timed out recently about 50% hitting 45 min deadline so let's bump this to 60 mins. (The timeout for windows is bumped as well for consistency)
Closes#35479
Make sure there is no unnecessary delays when there are multiple reports in the queue.
This change also adds a test for the custom LB policy.
Closes#35467
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35482
Fixed
```
/var/local/git/grpc/src/core/ext/transport/chaotic_good/frame_header.cc:69:15: error: no member named 'StrCat' in namespace 'absl'
absl::StrCat("Invalid header length: ", header.header_length));
~~~~~~^
```
Closes#35478
Based on [OpenTelemetry Metrics gRFC](https://github.com/grpc/proposal/blob/master/A66-otel-stats.md#opentelemetry-metrics), we should recored unregistered RPC method name as `other`, this PR adds the ability to pass register method information when creating a call.
We'll consider calls created using generated stubs as registered, note that this won't prevent user from setting `registered_method=True` when creating calls manually.
This is also enabled for simple stub flow but **NOT enabled for AsyncIO**, we'll add that later when start working on AsyncIO Observability.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35002
Adapts work from https://github.com/grpc/grpc/pull/34728 and previous changes from @nanahpang, implements new v3 filter/transport interface, and brings up the core of the chaotic good transport.
The next change will bring a more complete test suite (for this and inproc).
Closes#35400
I appreciate the elegance of using `\` and `/` to create ASCII art, but trailing backslashes in a `//` comment generates a lot of warnings with some compilers:
```
INFO: From Compiling src/core/lib/channel/promise_based_filter.cc:
In file included from external/com_github_grpc_grpc/src/core/lib/surface/server.h:48,
from external/com_github_grpc_grpc/src/core/lib/surface/call.h:53,
from external/com_github_grpc_grpc/src/core/lib/channel/promise_based_filter.h:65,
from external/com_github_grpc_grpc/src/core/lib/channel/promise_based_filter.cc:17:
external/com_github_grpc_grpc/src/core/lib/channel/call_tracer.h:47:1: warning: multi-line comment [-Wcomment]
47 | // / \
| ^
external/com_github_grpc_grpc/src/core/lib/channel/call_tracer.h:49:1: warning: multi-line comment [-Wcomment]
49 | // / \
| ^
```
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35464
Eventually for call-v3 we're going to want to have registration of filters generate the appropriate glue into the channel runtime to execute a call.
Begin that process now and gradually by introducing the new syntax and allowing a piecemeal migration to it - by the time we're done converting filters to the v3 APIs we'll also have the registration piece done.
PiperOrigin-RevId: 596013927
David Benjamin
Craig Tiller
Yash Tibrewal
Mark D. Roth
Xuan Wang
Gregory Cooke
Yijie Ma
Sergii Tkachenko
Esun Kim
dependabot[bot]
AJ Heller
Eugene Ostroukhov
Vignesh Babu
Carlos O'Ryan