My org, as part of its reproducible-build hygiene, builds projects with the source trees in read-only mode. I approached the protobuf build in the same way, but encountered this error (among others):
```
[ 30%] Built target statusor
[ 34%] Built target libprotobuf-lite
[ 45%] Built target libprotobuf
Consolidate compiler generated dependencies of target libprotoc
[ 62%] Built target libprotoc
Consolidate compiler generated dependencies of target protoc
[ 62%] Built target protoc
Consolidate compiler generated dependencies of target scoped_mock_log
[ 62%] Built target scoped_mock_log
[ 62%] Generating /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.h, /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.cc
/home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.cc: Read-only file system
make[2]: *** [CMakeFiles/libtest_common.dir/build.make:387: /home/src/protobuf/src/google/protobuf/util/message_differencer_unittest_proto3.pb.h] Error 1
make[1]: *** [CMakeFiles/Makefile2:1194: CMakeFiles/libtest_common.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
```
(`/home/src` is a read-only mount)
Generated source files are effectively build artifacts, and should be written to the build tree, not the source tree. This PR accordingly enables a successful build and test run (lite, full, and conformance) with read-only sources.
(Beyond this, I would add that CMake usually does not need source file paths to be qualified with `CMAKE_SOURCE_DIR` or `CMAKE_BINARY_DIR`; in most cases it knows to look in both locations, favoring the latter if a file is in both. The CMake code could be simplified by relying on this behavior.)
Closes#14455
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14455 from iskunk:feature/ro-source da7510c24a
PiperOrigin-RevId: 607031010
This caused an issue on some platforms where `protobuf/java/core/src/main/java/com/google/protobuf/java_features.proto` was getting turned into the invalid path `java/core/src/main/java/com/google/java_features.proto`.
Fixes#14862
PiperOrigin-RevId: 586527713
Unlike Arena::CreateMessage, Arena::Create creates only the top level object
from arena even if it is arena constructalble; e.g. messages, RepeatedPtrField,
etc. This renders arenas less effective.
Instead of asking users to be aware of such nuances to use the right API for
the right type, this CL makes Arena::Create recognizes and fully supports arena
constructable types.
While extremly rare, some users try to emulate Arena::CreateMessage with
Arena::Create by passing arena parameter twice. For example,
```
auto foo = Arena::Create<Foo>(&arena, &arena); // bad
```
This pattern is not supported and will break after this change. The following
is recommended instead.
```
auto foo = Arena::CreateMessage<Foo>(&arena); // recommended
auto foo = Arena::Create<Foo>(&arena); // after this change
```
PiperOrigin-RevId: 585709990
We transformed the proto2/proto3 protos to editions, and then run the same set of tests over both. This will verify that migrating to editions preserves the same proto2/proto3 behavior. These will not be enabled by default, and require a flag `--maximum_edition=2023`.
Future changes will:
- add more targeted editions-specific tests
- clean up our conformance test framework to allow for more targeted tests
- add wildcards to failure lists in limited cases to reduce noise
- add feature resolution conformance tests
PiperOrigin-RevId: 574570607
These utilities provide a way to embed a FeatureSetDefaults message into generators or runtimes that need to implement feature resolution. They use protoc to handle the tricky reflection-based algorithm over feature protos, leaving only simple merges to be implemented in other languages. See docs/design/editions/editions-life-of-a-featureset.md for more information.
PiperOrigin-RevId: 574554333
This is likely the cause of our macos cmake flakes. Both the lite and full tests trigger codegen of the lite protos, but there's no dependency between them. This creates a race condition, where it's possible for the files to be invalid by the time one of their builds is triggered. This is consistent with the error message we see:
liblibtest_common_lite.a(unittest_lite.pb.cc.o) has no symbols
PiperOrigin-RevId: 573381442
The current logic of finding a suitable include path relies on CMake to always return a meaningful relative path. On Windows, however, if the two paths reside on different drives (e.g. `c:/Users/Me` and `d:/project/`) the absolute path gets returned unaltered. This yields the invalid final path `c:/Users/Me/d:/project/`.
This case is now detected and the corresponding include path is deemed unsuitable.
(fixes#9290)
Closes#14025
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14025 from cgudrian:issue-9290-wrong-relative-path-in-protobuf-generate 3c0fd37324
PiperOrigin-RevId: 573315544
This will reduce the number of times our test protos are built, and may fix the flakes we're seeing in aarch64. At the very least, it should reduce them by a factor of 3 and marginally speed up our builds.
PiperOrigin-RevId: 568963559
This works for both built-in generators and plugins written in C++. Any feature extensions specified by this virtual method will end up fully resolved in the descriptors.
PiperOrigin-RevId: 561446773
This will allow us to run the CMake test from within the protobuf repo's Bazel
workspace. The directory structure is a little bit different depending on which
workspace the test is invoked in.
PiperOrigin-RevId: 557275295
I found that this test fails when I try to run it from our protobuf repo CI
setup, probably because rsync is not available in the Docker image there. Let's
just use `cp` instead so that the test works even if rsync is unavailable.
PiperOrigin-RevId: 557225733
The next in a series of CLs to split upb/BUILD into subdirs.
Create mem/internal/
Delete the deprecated upb/arena.h and upb/alloc.h stub headers
PiperOrigin-RevId: 552864952
pkg_files() has some odd quirks, like breaking if a filename exists in multiple directories. filegroup() does everything we need, without these quirks.
PiperOrigin-RevId: 547523447
This represents the future direction of protobuf, replacing proto2/proto3 syntax with editions. These will enable more incremental evolution of protobuf APIs through features, which are individual behaviors (such as whether field presence is explicit or implicit). For more details see https://protobuf.dev/editions/overview/.
This PR contains a working implementation of editions for the protoc frontend and C++ code generation, along with various infrastructure improvements to support it. It gives early access for anyone who wants to a preview of editions, but has no effect on proto2/proto3 syntax. It is flag-guarded behind the `--experimental_editions` flag, and is an experimental feature with no guarantees.
PiperOrigin-RevId: 544805690
After this change, `mini_table` only has MiniTable definitions themselves. Everything having to do with the MiniDescriptor wire format is in `mini_descriptor`.
Also rearranged some of the files in mini_table to have better structure for `internal/`.
This CL contains no functional change.
PiperOrigin-RevId: 543529112
h-vetinari
Daniel Richard G
Sandy Zhang
Protobuf Team Bot
Mike Kruskal
Christian Gudrian
Joshua Haberman
Adam Cozzette
Protobuf Team Bot
Eric Salo
Euclid