These tests were right near the threshold for disk space on default runners, and recently got pushed over (~18G). The cheapest large runner has 150G of SSD storage, and won't likely hit this issue again.
PiperOrigin-RevId: 572596499
This makes the file layout a bit more consistent with the `protos ->
protos_generator` pattern. I also replaced the `upbc` namespace with
`upb::generator`.
PiperOrigin-RevId: 569264372
This change moves almost everything in the `upb/` directory up one level, so
that for example `upb/upb/generated_code_support.h` becomes just
`upb/generated_code_support.h`. The only exceptions I made to this were that I
left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids
conflict with other files and the current locations seem reasonable for now.
The `python/` directory is a little bit of a challenge because we had to merge
the existing directory there with `upb/python/`. I made `upb/python/BUILD` into
the BUILD file for the merged directory, and it effectively loads the contents
of the other BUILD file via `python/build_targets.bzl`, but I plan to clean
this up soon.
PiperOrigin-RevId: 568651768
Hi, I'd like to suggest the adoption of the OpenSSF Scorecard Action.
The scorecard action runs the OpenSSF Scorecard checks on the repository often to provide feedbacks on how to improve (on the security dashboard) and transparency for the users about the current security posture (shown in the badge).
It is also a good way to keep track of new security practices and to provide users easy information about it.
See more about scorecard at [OpenSSF Scorecard](https://github.com/ossf/scorecard) and the [Show off your security score](https://openssf.org/blog/2022/09/08/show-off-your-security-score-announcing-scorecards-badges/)
PiperOrigin-RevId: 568644059
This will retry up to 3 times if we hit networks flakes updating our submodules. It will also allow us to easily inject other stability fixes to this step in the future.
PiperOrigin-RevId: 568306356
This restores the Python wheel CI runs from the old upb repo with only minor
changes. I had to update a path in one of the `py_wheel` rules and also make a
slight tweak to ensure that the `descriptor.upb_minitable.{h,c}` files make it
into the source wheels. The change in text_format_test.py is not strictly
necessary but is a small simplification I made while I was trying to debug an
issue with CRLF newlines.
I had to update test_util.py to use `importlib` to access the golden files from
the installed `protobuftests` package. I suspect the previous incarnation of
thse test runs was somehow reading the goldens from the repo checkout, but I
think the intention is to read them from `protobuftests` instead. This was a
bit tricky to get working because Python versions before 3.9 do not support
`importlib.resources.files()`. I set up the code to fall back on
`importlib.resources.open_binary()` in that case, but that function does not
support subdirectories, so this required putting an `__init__.py` file inside
the `testdata` directory to make sure it is treated as a Python package.
PiperOrigin-RevId: 567366695
I am getting ready to move almost everything under the upb/ directory up one
level to integrate upb better into its new location in the protobuf repo. This
change makes a few tweaks to prepare for that:
- Delete upb's LICENSE and CONTRIBUTING.md files since we already have similar
files at the top level.
- Rename `//python:python_version` so that it won't conflict later with
`//upb/python:python_version`.
- Move the contents of python/BUILD.bazel out to a Bazel macro to facilitate
merging that BUILD.bazel file with upb/python/BUILD.
PiperOrigin-RevId: 567119840
This restores a CI run that used to exist in the upb repo. I still need to add
test runs to exercise the wheels, but this change adds the functionality for
building and uploading them.
PiperOrigin-RevId: 564896637
A couple weeks ago we moved upb into the protobuf Git repo, and this change
continues the merger of the two repos by making them into a single Bazel repo.
This was mostly a matter of deleting upb's WORKSPACE file and fixing up a bunch
of references to reflect the new structure.
Most of the changes are pretty mechanical, but one thing that needed more
invasive changes was the Python script for generating CMakeLists.txt,
make_cmakelists.py. The WORKSPACE file it relied on no longer exists with this
change, so I updated it to hardcode the information it needed from that file.
PiperOrigin-RevId: 564810016
It seems that the janitor.yml workflow only needs pull-requests: write permission -- to close the PRs. I've also granted the contents: read just in case.
PiperOrigin-RevId: 561444486
This removes the need for us to manually rebuild these amalgamation files. I
added the "manual" tag to the corresponding staleness tests so that we won't
get presubmit errors if the amalgamation files are out of date. I also had to
tweak the Ruby BUILD.bazel file to make sure it uses Bazel-generated
amalgamations instead of relying on the checked-in files.
I added deliberate errors to the Ruby and PHP amalgamation files to test that
everything is working. We should not see any CI errors, and the files should
quickly be auto-updated after the change lands on main.
PiperOrigin-RevId: 561187352
## Description
I would like to suggest a security practice recommended by the [OpenSSF Scorecard][scorecard-repo] which is to hash pin dependencies to prevent typosquatting and tag renaming attacks.
The change would only be applied to GitHub workflows.
This means hash pinning GitHub Workflow actions.
Along with hash-pinning dependencies, I also recommend adopting dependabot (or other dependency update tool) to help keep the dependencies up to date. Most tools can update hashes and associated semantic version comments.
Any questions or concerns just let me know.
Thanks!
## Additional Context
A tag renaming attack is a type of attack whereby an attacker:
- Hijack an action.
- Upload a malicious version.
- Replace existing tags with malicious versions.
A [typosquatting attack][typosquatting] is a type of attack whereby an attacker:
- Create a malicious package
- Publish it with a similar name of a known package (example: numpi instead of numpy)
For more informations about the dependency-update tools:
- [Dependabot][dependabot]
[scorecard-repo]: https://github.com/ossf/scorecard
[deps-confusion]: https://www.websecuritylens.org/how-dependency-confusion-attack-works-and-how-to-prevent-it/
[typosquatting]: https://snyk.io/blog/typosquatting-attacks/
[dependabot]: https://github.blog/2020-06-01-keep-all-your-packages-up-to-date-with-dependabot/
[renovatebot]:https://www.mend.io/renovate/
PiperOrigin-RevId: 561019142
Currently all our upb CI coverage is running from within the protobuf workspace
and referring to upb as the external repo `@upb`. This change makes the GCC
test run cd into upb and use its workspace instead, just to make sure that
builds in the upb workspace have some coverage.
PiperOrigin-RevId: 560197956
This test validates that upb Python targets can be built successfully even if
Python is not installed locally.
I also updated our pinned upb version to pull in some recent fixes needed for
this test run.
PiperOrigin-RevId: 559504790
This change adds test runs for GCC, Windows, and Mac. The 32-bit ASAN test
turned out to be somewhat difficult to migrate, so I left a TODO for that one.
I made sure to give each build environment its own cache prefix, so that we
don't end up with artifacts built by different toolchains conflicting with each
other in the same cache.
PiperOrigin-RevId: 559197655
There are still a bunch more test runs to migrate, but this change adds a few
more that were unblocked by the `exclude-targets` flag.
PiperOrigin-RevId: 558206606
My editor reformatted this file to add a missing newline, and this ended up
breaking the test. The added newline caused there to be a newline in the middle
of the Bazel command. So this change adds the missing newline and fixes up the
Bazel command by using the `>-` syntax to treat this as a multiline string
without adding any newlines.
PiperOrigin-RevId: 558149295
Mike Kruskal
Adam Cozzette
Protobuf Team Bot
Sandy Zhang
dependabot[bot]
Joshua Haberman
Eric Salo