We're having issues with github runners right now where they're queueing for long periods, and this has become disruptive. We've gotten the flake rate under 2% total (<0.5% per build), and can ramp this up again later if we want to reduce it further.
PiperOrigin-RevId: 575227138
This should help us catch problems that come up when libprotobuf and libprotoc
are built as shared libraries. The motivating example was that we recently had
build failures due to `PROTOBUF_EXPORT` being missing from a symbol that needed
it, but none of our existing tests caught this. (Technically this test wouldn't
catch it either since that particular issue affected C++17 only, but at least
this should help with similar problems.)
PiperOrigin-RevId: 575218922
Bumps [shivammathur/setup-php](https://github.com/shivammathur/setup-php) from 2.24.0 to 2.26.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/shivammathur/setup-php/releases">shivammathur/setup-php's releases</a>.</em></p>
<blockquote>
<h2>2.26.0</h2>
<h2>Changelog</h2>
<h3>Microsoft SQL Server extensions. (<a href="https://redirect.github.com/shivammathur/setup-php/issues/758">#758</a>, <a href="https://redirect.github.com/shivammathur/setup-php/issues/766">#766</a>)</h3>
<p>The latest supported version of <code>sqlsrv</code> and <code>pdo_sqlsrv</code> for the PHP version will be installed.
Also, on Windows, these extensions will be installed from <a href="https://github.com/microsoft/msphpsql"><code>microsoft/msphpsql</code></a> GitHub releases and will fallback to <code>pecl</code>.</p>
<pre lang="yaml"><code>- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.2'
extensions: sqlsrv, pdo_sqlsrv
</code></pre>
<h3>PHP 8.4 Support (<a href="https://redirect.github.com/shivammathur/setup-php/issues/762">#762</a>)</h3>
<p>Added support for PHP <code>8.4.0-dev</code> from the master branch of <a href="https://github.com/php/php-src"><code>php/php-src</code></a> for all supported OS.
<strong>Note</strong>: PHP <code>8.3.0-dev</code> is now built from the new <a href="https://github.com/php/php-src/tree/PHP-8.3"><code>PHP-8.3</code></a> branch.</p>
<pre lang="yaml"><code>- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.4'
</code></pre>
<h2>Fixes</h2>
<h3>Blackfire Player</h3>
<p>Resolved issues affecting the <code>blackfire-player</code> on Linux and macOS. It now installs the <code>uuid</code> extension that is required for the tool.
Dropped support for it on Windows as <code>uuid</code> extension is not available for the OS.</p>
<pre lang="yaml"><code>- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
tools: blackfire-player
</code></pre>
<h2>Updates</h2>
<h3>Update actions/checkout</h3>
<p>Updated the use of <code>actions/checkout</code> to v4 in the documentation and workflows.</p>
<pre lang="yaml"><code>- name: Checkout
uses: actions/checkout@v4
</code></pre>
<h3>Update Node.js Version</h3>
<p><strong>Note</strong>: Node.js 16 reached End-of-Life on 2023-09-11 (<a href="https://nodejs.dev/en/about/releases/">Ref</a>).</p>
<p>Updated <code>action.yml</code> to use the <code>node20</code> binary. If you use <code>setup-php</code> on a self-hosted runner, please make sure it is <a href="https://github.com/actions/runner/releases/tag/v2.308.0">v2.308.0</a> or newer to use this release or the major version tag <code>v2</code>.</p>
<h3>Node.js Dependencies</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="7fdd3ece87"><code>7fdd3ec</code></a> Fix blackfire-player on Linux and macOS</li>
<li><a href="0de5aa9140"><code>0de5aa9</code></a> Bump version to 2.26.0</li>
<li><a href="a6e0b1404f"><code>a6e0b14</code></a> Bump Node.js dependencies</li>
<li><a href="5d259c66b1"><code>5d259c6</code></a> Bump node version to 20 in action.yml</li>
<li><a href="6207829361"><code>6207829</code></a> Update use of actions/checkout to v4</li>
<li><a href="319feb83c2"><code>319feb8</code></a> Add support to install sqlsrv and pdo_sqlsrv from GitHub releases</li>
<li><a href="661ad4bf75"><code>661ad4b</code></a> Fix enabling latest pecl extensions</li>
<li><a href="04c15e2314"><code>04c15e2</code></a> Remove hardcoded latest version for sqlsrv and pdo_sqlsrv</li>
<li><a href="5aa416d77b"><code>5aa416d</code></a> Merge pull request <a href="https://redirect.github.com/shivammathur/setup-php/issues/766">#766</a> from GrahamCampbell/patch-1</li>
<li><a href="a9a661ccd4"><code>a9a661c</code></a> Use sqlsvr 5.11.1 on PHP 8</li>
<li>Additional commits viewable in <a href="d30ad8b184...7fdd3ece87">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Closes#14115
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14115 from protocolbuffers:dependabot/github_actions/shivammathur/setup-php-2.26.0 f4423b44f0
PiperOrigin-RevId: 573315535
These tests were right near the threshold for disk space on default runners, and recently got pushed over (~18G). The cheapest large runner has 150G of SSD storage, and won't likely hit this issue again.
PiperOrigin-RevId: 572596499
This makes the file layout a bit more consistent with the `protos ->
protos_generator` pattern. I also replaced the `upbc` namespace with
`upb::generator`.
PiperOrigin-RevId: 569264372
This change moves almost everything in the `upb/` directory up one level, so
that for example `upb/upb/generated_code_support.h` becomes just
`upb/generated_code_support.h`. The only exceptions I made to this were that I
left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids
conflict with other files and the current locations seem reasonable for now.
The `python/` directory is a little bit of a challenge because we had to merge
the existing directory there with `upb/python/`. I made `upb/python/BUILD` into
the BUILD file for the merged directory, and it effectively loads the contents
of the other BUILD file via `python/build_targets.bzl`, but I plan to clean
this up soon.
PiperOrigin-RevId: 568651768
Hi, I'd like to suggest the adoption of the OpenSSF Scorecard Action.
The scorecard action runs the OpenSSF Scorecard checks on the repository often to provide feedbacks on how to improve (on the security dashboard) and transparency for the users about the current security posture (shown in the badge).
It is also a good way to keep track of new security practices and to provide users easy information about it.
See more about scorecard at [OpenSSF Scorecard](https://github.com/ossf/scorecard) and the [Show off your security score](https://openssf.org/blog/2022/09/08/show-off-your-security-score-announcing-scorecards-badges/)
PiperOrigin-RevId: 568644059
This will retry up to 3 times if we hit networks flakes updating our submodules. It will also allow us to easily inject other stability fixes to this step in the future.
PiperOrigin-RevId: 568306356
This restores the Python wheel CI runs from the old upb repo with only minor
changes. I had to update a path in one of the `py_wheel` rules and also make a
slight tweak to ensure that the `descriptor.upb_minitable.{h,c}` files make it
into the source wheels. The change in text_format_test.py is not strictly
necessary but is a small simplification I made while I was trying to debug an
issue with CRLF newlines.
I had to update test_util.py to use `importlib` to access the golden files from
the installed `protobuftests` package. I suspect the previous incarnation of
thse test runs was somehow reading the goldens from the repo checkout, but I
think the intention is to read them from `protobuftests` instead. This was a
bit tricky to get working because Python versions before 3.9 do not support
`importlib.resources.files()`. I set up the code to fall back on
`importlib.resources.open_binary()` in that case, but that function does not
support subdirectories, so this required putting an `__init__.py` file inside
the `testdata` directory to make sure it is treated as a Python package.
PiperOrigin-RevId: 567366695
I am getting ready to move almost everything under the upb/ directory up one
level to integrate upb better into its new location in the protobuf repo. This
change makes a few tweaks to prepare for that:
- Delete upb's LICENSE and CONTRIBUTING.md files since we already have similar
files at the top level.
- Rename `//python:python_version` so that it won't conflict later with
`//upb/python:python_version`.
- Move the contents of python/BUILD.bazel out to a Bazel macro to facilitate
merging that BUILD.bazel file with upb/python/BUILD.
PiperOrigin-RevId: 567119840
This restores a CI run that used to exist in the upb repo. I still need to add
test runs to exercise the wheels, but this change adds the functionality for
building and uploading them.
PiperOrigin-RevId: 564896637
A couple weeks ago we moved upb into the protobuf Git repo, and this change
continues the merger of the two repos by making them into a single Bazel repo.
This was mostly a matter of deleting upb's WORKSPACE file and fixing up a bunch
of references to reflect the new structure.
Most of the changes are pretty mechanical, but one thing that needed more
invasive changes was the Python script for generating CMakeLists.txt,
make_cmakelists.py. The WORKSPACE file it relied on no longer exists with this
change, so I updated it to hardcode the information it needed from that file.
PiperOrigin-RevId: 564810016
It seems that the janitor.yml workflow only needs pull-requests: write permission -- to close the PRs. I've also granted the contents: read just in case.
PiperOrigin-RevId: 561444486
This removes the need for us to manually rebuild these amalgamation files. I
added the "manual" tag to the corresponding staleness tests so that we won't
get presubmit errors if the amalgamation files are out of date. I also had to
tweak the Ruby BUILD.bazel file to make sure it uses Bazel-generated
amalgamations instead of relying on the checked-in files.
I added deliberate errors to the Ruby and PHP amalgamation files to test that
everything is working. We should not see any CI errors, and the files should
quickly be auto-updated after the change lands on main.
PiperOrigin-RevId: 561187352
## Description
I would like to suggest a security practice recommended by the [OpenSSF Scorecard][scorecard-repo] which is to hash pin dependencies to prevent typosquatting and tag renaming attacks.
The change would only be applied to GitHub workflows.
This means hash pinning GitHub Workflow actions.
Along with hash-pinning dependencies, I also recommend adopting dependabot (or other dependency update tool) to help keep the dependencies up to date. Most tools can update hashes and associated semantic version comments.
Any questions or concerns just let me know.
Thanks!
## Additional Context
A tag renaming attack is a type of attack whereby an attacker:
- Hijack an action.
- Upload a malicious version.
- Replace existing tags with malicious versions.
A [typosquatting attack][typosquatting] is a type of attack whereby an attacker:
- Create a malicious package
- Publish it with a similar name of a known package (example: numpi instead of numpy)
For more informations about the dependency-update tools:
- [Dependabot][dependabot]
[scorecard-repo]: https://github.com/ossf/scorecard
[deps-confusion]: https://www.websecuritylens.org/how-dependency-confusion-attack-works-and-how-to-prevent-it/
[typosquatting]: https://snyk.io/blog/typosquatting-attacks/
[dependabot]: https://github.blog/2020-06-01-keep-all-your-packages-up-to-date-with-dependabot/
[renovatebot]:https://www.mend.io/renovate/
PiperOrigin-RevId: 561019142
Currently all our upb CI coverage is running from within the protobuf workspace
and referring to upb as the external repo `@upb`. This change makes the GCC
test run cd into upb and use its workspace instead, just to make sure that
builds in the upb workspace have some coverage.
PiperOrigin-RevId: 560197956
Mike Kruskal
Eric Salo
dependabot[bot]
Adam Cozzette
Matt Kulukundis
Matt Kulukundis
Protobuf Team Bot
Sandy Zhang