This flips the default behavior to "fetch", downloading local copies of required dependencies. This can be disabled by setting `-Dprotobuf_FETCH_DEPENDENCIES=OFF`, in which case we will look for a local installation using find_package. Setting `-Dprotobuf_ABSL_PROVIDER=package` will continue to have the same behavior as before.
See https://protobuf.dev/news/2024-10-02/#replace-cmake-submods for more details.
#test-continuous
PiperOrigin-RevId: 686649864
This ports upb's WORKSPACE scraping logic to protobuf, and allows us to dynamically fetch our dependencies at the exact same pinned version as in Bazel via protobuf_FETCH_DEPENDENCIES=ON. This is mostly for development purposes, and is preferable to git submodules. In a later cl we will flip the default behavior to "package"
#test-continuous
PiperOrigin-RevId: 686265348
This ensures the names of these jobs will properly be prefixed with "[SKIPPED] (Continuous)" and will show as passed instead of skipped.
PiperOrigin-RevId: 666918220
If there's an unexpected failure message or an unexpected succeeding test from a wildcard expansion users will be made to remove the wildcarded equivalent. Once removed, they must rerun the conformance test to add the failures contained within the removed wildcarded equivalent.
PiperOrigin-RevId: 663040062
Before this PR, we stored a list internally of tests that must pass on presubmit and tried to keep it up to date.
This PR moves that information keeping into GitHub by adding a 'continuous-only' variable to most testing matrices to allow authors to specify which of their tests should be skipped on presubmit. During presubmit, tests that were specified to not run on presubmit will not be run and their names will be prefixed with "[SKIPPED]". All continuous only tests will be suffixed with "(Continuous)".
At the end of running all the tests, we have a single "All Blocking Tests" signal that will tell us whether all of the necessary tests have passed (either for presubmit or continuous based on how the test was triggered).
I've tested this from a different branch [here](https://github.com/protocolbuffers/protobuf/actions/runs/9602443750?pr=17151) and from a different fork [here](https://github.com/protocolbuffers/protobuf/actions/runs/9602554500?pr=17192). These should be the same and are as far as I can tell.
I also have a continuous test run [here](https://github.com/protocolbuffers/protobuf/actions/runs/9603824200) which runs the entire test suite.
Closes#17198
PiperOrigin-RevId: 662940724
Create WORKSPACE.bzlmod. Before building with Bzlmod resulted in use of full WORKSPACE.
Some repos are still there, but the file should eventually be empty.
Add dep to rules_kotlin 1.9.0. This was the first version available on BCR. It pushed upgrade of
rules_jvm_external to 6.0 and rules_java to 6.5.2 (keep 6.0.0 on Bazel 6.3.0).
Add missing maven and other deps to MODULE.bazel
CI changes:
Disable Bazel 6.4.0 with bzlmod. rules_jvm_external 6.0 use use_repo_rule, which is not supported by Bazel 6.
Add C++ build "Bazel7 with Bzlmod" enabled.
Add Java builds with "Bazel 7 with/without Bzlmod".
Fixes: https://github.com/protocolbuffers/protobuf/issues/17176
PiperOrigin-RevId: 652773197
This will allow them to reuse our bazelrc and remote caching setup. This also silences the non-bzlmod windows test that's hitting the windows path length.
PiperOrigin-RevId: 626390416
We're seeing a high flake rate due to remote cache-misses only on Bazel 7 builds. The key change here seems to be adding the remote_download_output flag, but this also upgrades us to 7.1.1 and adds some retry behavior to caching issues.
PiperOrigin-RevId: 625892332
This installs a upb's static library, headers, and three protoc plugin binaries (upb, upbdefs, upb_minitable). These will now be enabled by default, but can be disabled by setting protobuf_BUILD_LIBUPB=OFF.
To qualify this, we hook into our existing install test infrastructure which attempts to build and run our tests without any of the installation artifacts available from the source tree. Public headers are deleted, and builds of exported libraries/binaries are disabled.
PiperOrigin-RevId: 617376961
The latest github runner image of ubuntu-22 includes a kernel update that breaks the sanitizers we use in our docker images. Long-term, we'll likely need to upgrade the images to new sanitizers and clang.
PiperOrigin-RevId: 616319042
This makes third_party/utf8_range no longer a Git subtree, but instead the
permanent location and source of truth for utf8_range. It is also now
incorporated into the @com_google_protobuf Bazel repo. Utf8_range still has its
own separate CMake build for now, though.
PiperOrigin-RevId: 580682733
We will soon be moving utf8_range into the protobuf repo, not as a subtree
anymore but as the real source of truth. This change adds CI coverage in
advance so that there will not be a lapse in coverage.
I also upgraded our pinned versions of rules_fuzzing and rules_python, to fix
some errors that came up with Bazel 6 and Python 3.12. I had to patch
rules_fuzzing but I am working on upstreaming the fixes.
PiperOrigin-RevId: 579987379
setup-gcloud doesn't honor the environment variables set by setup-python, so we need to manually set CLOUDSDK_PYTHON before running it.
PiperOrigin-RevId: 578674246
This should help us catch problems that come up when libprotobuf and libprotoc
are built as shared libraries. The motivating example was that we recently had
build failures due to `PROTOBUF_EXPORT` being missing from a symbol that needed
it, but none of our existing tests caught this. (Technically this test wouldn't
catch it either since that particular issue affected C++17 only, but at least
this should help with similar problems.)
PiperOrigin-RevId: 575218922
These tests were right near the threshold for disk space on default runners, and recently got pushed over (~18G). The cheapest large runner has 150G of SSD storage, and won't likely hit this issue again.
PiperOrigin-RevId: 572596499
This will retry up to 3 times if we hit networks flakes updating our submodules. It will also allow us to easily inject other stability fixes to this step in the future.
PiperOrigin-RevId: 568306356
## Description
I would like to suggest a security practice recommended by the [OpenSSF Scorecard][scorecard-repo] which is to hash pin dependencies to prevent typosquatting and tag renaming attacks.
The change would only be applied to GitHub workflows.
This means hash pinning GitHub Workflow actions.
Along with hash-pinning dependencies, I also recommend adopting dependabot (or other dependency update tool) to help keep the dependencies up to date. Most tools can update hashes and associated semantic version comments.
Any questions or concerns just let me know.
Thanks!
## Additional Context
A tag renaming attack is a type of attack whereby an attacker:
- Hijack an action.
- Upload a malicious version.
- Replace existing tags with malicious versions.
A [typosquatting attack][typosquatting] is a type of attack whereby an attacker:
- Create a malicious package
- Publish it with a similar name of a known package (example: numpi instead of numpy)
For more informations about the dependency-update tools:
- [Dependabot][dependabot]
[scorecard-repo]: https://github.com/ossf/scorecard
[deps-confusion]: https://www.websecuritylens.org/how-dependency-confusion-attack-works-and-how-to-prevent-it/
[typosquatting]: https://snyk.io/blog/typosquatting-attacks/
[dependabot]: https://github.blog/2020-06-01-keep-all-your-packages-up-to-date-with-dependabot/
[renovatebot]:https://www.mend.io/renovate/
PiperOrigin-RevId: 561019142
Evan Brown
Tony Liao
Mike Kruskal
Protobuf Team Bot
Deanna Garcia
Yamil Morales
Joshua Haberman
Adam Cozzette
Sandy Zhang