My org, as part of its reproducible-build hygiene, builds projects with the source trees in read-only mode. I approached the protobuf build in the same way, but encountered this error (among others):
```
[ 30%] Built target statusor
[ 34%] Built target libprotobuf-lite
[ 45%] Built target libprotobuf
Consolidate compiler generated dependencies of target libprotoc
[ 62%] Built target libprotoc
Consolidate compiler generated dependencies of target protoc
[ 62%] Built target protoc
Consolidate compiler generated dependencies of target scoped_mock_log
[ 62%] Built target scoped_mock_log
[ 62%] Generating /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.h, /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.cc
/home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.cc: Read-only file system
make[2]: *** [CMakeFiles/libtest_common.dir/build.make:387: /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.h] Error 1
make[1]: *** [CMakeFiles/Makefile2:1194: CMakeFiles/libtest_common.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
```
(`/home/src` is a read-only mount)
Generated source files are effectively build artifacts, and should be written to the build tree, not the source tree. This PR accordingly enables a successful build and test run (lite, full, and conformance) with read-only sources.
(Beyond this, I would add that CMake usually does not need source file paths to be qualified with `CMAKE_SOURCE_DIR` or `CMAKE_BINARY_DIR`; in most cases it knows to look in both locations, favoring the latter if a file is in both. The CMake code could be simplified by relying on this behavior.)
Closes#14455
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14455 from iskunk:feature/ro-source da7510c24a
PiperOrigin-RevId: 607031010
These utilities provide a way to embed a FeatureSetDefaults message into generators or runtimes that need to implement feature resolution. They use protoc to handle the tricky reflection-based algorithm over feature protos, leaving only simple merges to be implemented in other languages. See docs/design/editions/editions-life-of-a-featureset.md for more information.
PiperOrigin-RevId: 574554333
This is likely the cause of our macos cmake flakes. Both the lite and full tests trigger codegen of the lite protos, but there's no dependency between them. This creates a race condition, where it's possible for the files to be invalid by the time one of their builds is triggered. This is consistent with the error message we see:
liblibtest_common_lite.a(unittest_lite.pb.cc.o) has no symbols
PiperOrigin-RevId: 573381442
This will reduce the number of times our test protos are built, and may fix the flakes we're seeing in aarch64. At the very least, it should reduce them by a factor of 3 and marginally speed up our builds.
PiperOrigin-RevId: 568963559
This works for both built-in generators and plugins written in C++. Any feature extensions specified by this virtual method will end up fully resolved in the descriptors.
PiperOrigin-RevId: 561446773
This represents the future direction of protobuf, replacing proto2/proto3 syntax with editions. These will enable more incremental evolution of protobuf APIs through features, which are individual behaviors (such as whether field presence is explicit or implicit). For more details see https://protobuf.dev/editions/overview/.
This PR contains a working implementation of editions for the protoc frontend and C++ code generation, along with various infrastructure improvements to support it. It gives early access for anyone who wants to a preview of editions, but has no effect on proto2/proto3 syntax. It is flag-guarded behind the `--experimental_editions` flag, and is an experimental feature with no guarantees.
PiperOrigin-RevId: 544805690
This deletes our old branches of internal Abseil code in favor of their newly open-sourced library. Notably, this removes the ability to turn CHECK crashes into exceptions.
PiperOrigin-RevId: 504460562
This cl hit an issue during the shared library cmake build from ODR violations, leading to mismatched absl hash seeds. The problem was pre-existing but didn't manifest until now, and can be traced to the fact that in shared library builds we linked Abseil statically. All of the cmake changes here remove the underlying ODR violation.
PiperOrigin-RevId: 485787671
* Using glob to remove headers instead of cyclic file_lists
* Simplify CMake config and include missing files
* Don't remove generated proto headers
* Fix broken CMake proto dependencies instead of opting out pb.h files.
* Fixing cyclic dependency
* Fixing typos
* Revert new files that were deleted by sync script
* Fix CMake breakages
* bump upb version
* Sync from Piper @468772608
PROTOBUF_SYNC_PIPER
* Adding abseil to include path for python C++ extension
* Adding abseil linkage for python C++ extension
* Fixing linkage order
* Initial implementation of cmake tests for linux
* Reverting accidental distcheck changes
* Deleting extract_includes.bat now that it's been replaced with the generated file_lists.cmake
* Removing unnecessary endif conditions
This commit changes update_file_lists.sh to use Bazel as the source of truth. Currently, only the cmake build is actually affected, but Automake should be fairly straightforward to convert.
The file lists were originally factored out in PR#10027. This commit actually switches over to the Bazel-generated lists.
This is in preparation for generating file lists automatically.
I am putting the list of files under src/, not under cmake/, so that it will be next to the file list for automake (see #10029). I can certainly put the list back under cmake/, but in general, I think we probably want to move the cmake targets e.g. to src/CMakeLists.txt anyhow.
* Revert "Added cmake abseil include guard"
This reverts commit b6ee841d7c.
* Revert "Update CMake configuration to add a dependency on Abseil (#9793)"
This reverts commit e9246cd789.
* Proof of concept for CMake Abseil dependency
* Hooked up Abseil linking
* Adding test binaries
* Reverting absl::string view use added for testing. This will still be broken for automake and bazel
* Adding new cmake config to dist list
* Whitespace fixes and an attempt at fix for Windows Python Release (CMake version < 3.13)
* Hook up LTS to windows cmake
* Fix 'git cd' typo
* Adding clarifying comment for fix
* Adding abseil dependency to protobuf-lite too
* Update submodules instead of cloning abseil-cpp
* Adding explicit pointer to abseil root directory
Co-authored-by: Jason Lunn <jason.lunn@gmail.com>
Co-authored-by: Jorg Brown <jorg.brown@gmail.com>
The new option will allow us to generate test XML output.
CTest can discover tests from googletest, but it generates one file
per case. We have several thousand cases, so one file each would be
far from optimal. The approach in this change will generate one file
per test (executable).
This change adds separate libraries for common test utilities and test proto definitions, which are shared by different test targets.
Without this change, cmake fails when generating Xcode files with the error:
```
CMake Error in CMakeLists.txt:
The custom command generating
.../protobuf/src/google/protobuf/unittest_lite.pb.cc
is attached to multiple targets:
tests
lite-test
lite-arena-test
but none of these is a common dependency of the other(s). This is not
allowed by the Xcode "new build system".
```
If `-Dprotobuf_USE_EXTERNAL_GTEST=ON` is passed to CMake, it will use an external Google Test copy (i.e. by calling `find_package(GTest REQUIRED)`) rather than the one provided as a submodule.
This makes sense for larger projects that might already include Google Test and want to use a more standard CMake approach.
Also updated build instructions with this information, and more idiomatic usage.
Co-authored-by: Adam Cozzette <acozzette@google.com>
Convenience feature: enable users to test via the familiar `ctest` command rather than making the `check` target.
They would be able to use the familiar CMake pattern:
```
cmake -S source/protobuf -B build/protobuf ...
cmake --build build/protobuf
ctest --test-dir build/protobuf
cmake --install build/protobuf
```
This is a follow-up to 9f447fc9d3da93da29b8301f1a8ca57b1ea812d7
Daniel Richard G
Mike Kruskal
Sandy Zhang
Adam Cozzette
Mike Kruskal
Elliotte Rusty Harold
David L. Jones
Arfrever Frehtes Taifersar Arahesis
Florin Crișan
Stephen Kennedy
Joshua Haberman
Bo Yang
listonjesse
Yannic Bonenberger