This installs a upb's static library, headers, and three protoc plugin binaries (upb, upbdefs, upb_minitable). These will now be enabled by default, but can be disabled by setting protobuf_BUILD_LIBUPB=OFF.
To qualify this, we hook into our existing install test infrastructure which attempts to build and run our tests without any of the installation artifacts available from the source tree. Public headers are deleted, and builds of exported libraries/binaries are disabled.
PiperOrigin-RevId: 617376961
My org, as part of its reproducible-build hygiene, builds projects with the source trees in read-only mode. I approached the protobuf build in the same way, but encountered this error (among others):
```
[ 30%] Built target statusor
[ 34%] Built target libprotobuf-lite
[ 45%] Built target libprotobuf
Consolidate compiler generated dependencies of target libprotoc
[ 62%] Built target libprotoc
Consolidate compiler generated dependencies of target protoc
[ 62%] Built target protoc
Consolidate compiler generated dependencies of target scoped_mock_log
[ 62%] Built target scoped_mock_log
[ 62%] Generating /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.h, /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.cc
/home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.cc: Read-only file system
make[2]: *** [CMakeFiles/libtest_common.dir/build.make:387: /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.h] Error 1
make[1]: *** [CMakeFiles/Makefile2:1194: CMakeFiles/libtest_common.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
```
(`/home/src` is a read-only mount)
Generated source files are effectively build artifacts, and should be written to the build tree, not the source tree. This PR accordingly enables a successful build and test run (lite, full, and conformance) with read-only sources.
(Beyond this, I would add that CMake usually does not need source file paths to be qualified with `CMAKE_SOURCE_DIR` or `CMAKE_BINARY_DIR`; in most cases it knows to look in both locations, favoring the latter if a file is in both. The CMake code could be simplified by relying on this behavior.)
Closes#14455
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14455 from iskunk:feature/ro-source da7510c24a
PiperOrigin-RevId: 607031010
In CMake >= 3.0 it is more idiomatic to set per-target compiler options than global compiler settings. I have kept these options and defines as `PRIVATE` so they won't be exported with the target.
Closes#12916
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/12916 from coryan:cleanup-cmake-avoid-global-compile-settings 3d586dc0e8
PiperOrigin-RevId: 536517165
Policy CMP0141 broke our hack to remove /Zi, and there doesn't seem to be a way to strip debug symbols with this policy enabled. We can disable the policy when it exists to get ccache working again and speed up our tests.
PiperOrigin-RevId: 534958215
CMake has fairly strict backwards compatibility behavior. Saying `cmake_minimum_required(VERSION 3.10)` has two effects:
- It requires CMake to be version 3.10 or higher
- It configures CMake to behave exactly as it did in version 3.10
The code used to override some policies to get newer behavior if it was available. The alternative syntax in this PR says "require at least 3.10, and use the behavior (when available) up to 3.26". We can omit policy overrides by doing so. It does not seem like any other policies are important, so we can use the simpler syntax.
Closes#12712
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/12712 from coryan:cleanup-cmake-automatically-use-newer-policies 36db0fa17e
PiperOrigin-RevId: 530376446
Allows the use of an external `jsoncpp` library to be used. Replicates the model used by `abseil-cpp` as a "package" or "module" to the `protobuf_JSONCPP_PROVIDER` option.
Resolves: #11827Closes#12577
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/12577 from omenos:omenos/cmake-jsoncpp 073edd69b4
PiperOrigin-RevId: 528492610
For now, this only covers linux on the two architectures we have testing support for. However, it serves as a good sanity check and can be expanded in the future.
PiperOrigin-RevId: 514449399
For now, this only covers linux on the two architectures we have testing support for. However, it serves as a good sanity check and can be expanded in the future.
PiperOrigin-RevId: 514449399
This change also removes policy settings that are now already set to
`NEW` by default. There are still two remaining policies that were
introduced in CMake > 3.10, so I left those, and documented the CMake
version they were introduced in so we know when we can later remove them
(when our min version advances beyond that).
The min CMake version supported by Protobuf is documented here:
https://github.com/google/oss-policies-info/blob/main/foundational-cxx-support-matrix.md
PiperOrigin-RevId: 511897379
These will still use MSVC as the compiler, but will no longer generate Visual Studio projects for the builds. Visual Studio is particularly bad at parallelizing builds, and is hostile to ccache.
This change also tweaks the ccache setup to prevent unbounded the growth observed in our github caches. Windows builds have had debug symbols stripped to reduce ccache size by a factor of 2x, and ccache maximum have been tweaked so that we persist fewer older builds.
Before this change, each CMake build took 12 minutes on every run (plus some constant overhead from staleness/gcloud), even with caching or on large multi-core runners. No amount of caching or parallelization made any noticeable difference above noise. With this change, we see the following improvements:
- 12 minutes to build from scratch on normal runners (unchanged)
- 4 minutes on 32-core runners from scratch
- 1 minute with optimal caches available on normal runners.
- 30 seconds on 32-core runners with optimal caches
PiperOrigin-RevId: 508799909
These will still use MSVC as the compiler, but will no longer generate Visual Studio projects for the builds. Visual Studio is particularly bad at parallelizing builds, and is hostile to ccache.
This change also tweaks the ccache setup to prevent unbounded the growth observed in our github caches. Windows builds have had debug symbols stripped to reduce ccache size by a factor of 2x, and ccache maximum have been tweaked so that we persist fewer older builds.
Before this change, each CMake build took 12 minutes on every run (plus some constant overhead from staleness/gcloud), even with caching or on large multi-core runners. No amount of caching or parallelization made any noticeable difference above noise. With this change, we see the following improvements:
- 12 minutes to build from scratch on normal runners (unchanged)
- 4 minutes on 32-core runners from scratch
- 1 minute with optimal caches available on normal runners.
- 30 seconds on 32-core runners with optimal caches
PiperOrigin-RevId: 508799909
This uses ccache + github caching to substantially decrease the time it takes to run CMake builds. Due to Bazel caching, these are some of our slowest tests, causing one of the biggest presubmit bottlenecks
PiperOrigin-RevId: 507667813
Protobuf Team Bot
Mike Kruskal
Daniel Richard G
Dennis Shao
Kevin King
Jie Luo
Protobuf Team Bot
ericsalo
Carlos O'Ryan
deannagarcia
Mike Rochefort
zhangskz
Greg Miller