These will still use MSVC as the compiler, but will no longer generate Visual Studio projects for the builds. Visual Studio is particularly bad at parallelizing builds, and is hostile to ccache.
This change also tweaks the ccache setup to prevent unbounded the growth observed in our github caches. Windows builds have had debug symbols stripped to reduce ccache size by a factor of 2x, and ccache maximum have been tweaked so that we persist fewer older builds.
Before this change, each CMake build took 12 minutes on every run (plus some constant overhead from staleness/gcloud), even with caching or on large multi-core runners. No amount of caching or parallelization made any noticeable difference above noise. With this change, we see the following improvements:
- 12 minutes to build from scratch on normal runners (unchanged)
- 4 minutes on 32-core runners from scratch
- 1 minute with optimal caches available on normal runners.
- 30 seconds on 32-core runners with optimal caches
PiperOrigin-RevId: 508799909
Use the bash GHA to ensure WKTs are current, and then just directly invoke
Xcode. This means the script full_mac_build.sh can go back to just being
a simpler helper for folks doing local builds and we can remove all the extra
support that was needed for CI.
PiperOrigin-RevId: 508762464
Changes to these files *can't* be tested in forked PRs, so we should explicitly block them with an error message to explain why.
PiperOrigin-RevId: 508701535
Both 'schedule' and 'workflow_dispatch' are valid alternatives today. What we really want here is anything *except* pull request events.
PiperOrigin-RevId: 509123777
These will still use MSVC as the compiler, but will no longer generate Visual Studio projects for the builds. Visual Studio is particularly bad at parallelizing builds, and is hostile to ccache.
This change also tweaks the ccache setup to prevent unbounded the growth observed in our github caches. Windows builds have had debug symbols stripped to reduce ccache size by a factor of 2x, and ccache maximum have been tweaked so that we persist fewer older builds.
Before this change, each CMake build took 12 minutes on every run (plus some constant overhead from staleness/gcloud), even with caching or on large multi-core runners. No amount of caching or parallelization made any noticeable difference above noise. With this change, we see the following improvements:
- 12 minutes to build from scratch on normal runners (unchanged)
- 4 minutes on 32-core runners from scratch
- 1 minute with optimal caches available on normal runners.
- 30 seconds on 32-core runners with optimal caches
PiperOrigin-RevId: 508799909
Use the bash GHA to ensure WKTs are current, and then just directly invoke
Xcode. This means the script full_mac_build.sh can go back to just being
a simpler helper for folks doing local builds and we can remove all the extra
support that was needed for CI.
PiperOrigin-RevId: 508762464
Changes to these files *can't* be tested in forked PRs, so we should explicitly block them with an error message to explain why.
PiperOrigin-RevId: 508701535
- General naming up to "Objective-C" that some linters want, so just
use it every where for consistence.
- Add a bazel macOS config
- Includes the conformance test, so drop it from the Xcode flows.
- Tweak the naming on the Xcode flows so they are a little more clear.
- CocoaPods flows:
- Give them (hopefully) clear naming
- Use the same `image` as the Xcode tests
- Pin the Xcode version to match the Xcode tests
PiperOrigin-RevId: 508156130
This should substantially reduce network flakes, since most builds will no longer need to download external dependencies. Because these caches actually get pretty expensive (~300 MB just from 1 openjdk dependency), we've taken some precautions:
1) The caches only get updated on push events (i.e. post-submit jobs)
2) Only a single job will push a new cache for any os/branch/commit combination. This will be whichever job finishes its bazel run first, and *also* has updates for the cache. Over time, the cache should expand to the full set of transitive dependencies as the quicker jobs stop pushing updates.
3) A clear_caches workflow has been introduced to reset the caches in case they grow too large. This will run monthly for now, but can also be run manually.
PiperOrigin-RevId: 508133044
This has the consequence of removing the path filter, but allows us to actually use these test results as presubmits. Since these are relatively cheap tests, this seems like a better situation. A future enhancement may restore path filters to these and other tests.
PiperOrigin-RevId: 507872208
The `BuildInParallel` flag only controls parallelism at the *project* level, while `/MP` enables it at the *compilation unit* level.
PiperOrigin-RevId: 507837073
This uses ccache + github caching to substantially decrease the time it takes to run XCode builds. Due to Bazel caching, these are some of our slowest tests, causing one of the biggest presubmit bottlenecks
PiperOrigin-RevId: 507794241
This uses ccache + github caching to substantially decrease the time it takes to run CMake builds. Due to Bazel caching, these are some of our slowest tests, causing one of the biggest presubmit bottlenecks
PiperOrigin-RevId: 507667813
Pull the first index with jq instead of looking for "main"
Git add all artifacts since "git commit -a" doesn't add new ones
PiperOrigin-RevId: 506933714
Pull the first index with jq instead of looking for "main"
Git add all artifacts since "git commit -a" doesn't add new ones
PiperOrigin-RevId: 506933714
This no longer served its original purpose and was actually testing an *older* version of MacOS than the rest of our test suite. This did lock down CMake builds on Mac though, so we'll keep that coverage.
PiperOrigin-RevId: 506776948
Our pkg_tar rule was missing a few template files for python. Using setup.py will guarantee conformance with python package requirements.
Adds MANIFEST.in to include c header/inc files in the source distribution.
PiperOrigin-RevId: 506638326
This migrates from reusable workflows to composite GHA actions. This has the following advantages:
1) We can split them into smaller, easier to reason about chunks
2) We can reuse them more freely between each other and workflows
3) They don't complicate the job names
4) In theory they'll be easier to test in isolation. While composite actions can't be unit-tested, we can easily break them up into nodejs or docker actions that can be.
As a proof of concept, some of our non-Bazel tests are migrated to GHA as well (CMake + Composer)
Closes#11718
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11718 from protocolbuffers:gha-actions 5403307bc00616e94816aa460813939d8f37a1bd
PiperOrigin-RevId: 506375417
Our repo settings have already been updated to only allow merges from the BTR team and protobuf managers. Except for some very specific circumstances, PRs should *never* be merged and should instead be imported via Copybara.
PiperOrigin-RevId: 506371362
Mike Kruskal
Protobuf Team Bot
Deanna Garcia
Sandy Zhang