This is a cleanup that is intended to reduce the noise in current and planned protocol compiler code, and improve readability of the code.
PiperOrigin-RevId: 550631662
Currently this corner case (discovered by fuzzing) is not considered. The code throws `ArrayIndexOutOfBoundsException` which can escape `protobuf.toString()` method.
PiperOrigin-RevId: 550514062
When `DescriptorBuilder::BuildFileImpl` is called, `alloc.AllocateArray` appears to be allocating an uninitialized object. This means every field inside FileDescriptor should be initialized, and edition_ may be ignored if not present. This can then lead to a SEGFAULT later in `FileDescriptor::edition()`, with a check of `edition_ == nullptr ? anti_hyrum_string : *edition_;` potentially dereferencing `edition_` when it's uninitialized.
PiperOrigin-RevId: 549712519
Design details:
- We store the split repeated fields inline in the Split struct by pointer. We use pointers so that we can save memory when the fields are absent and so that the split struct will still be trivially copyable.
- If we had instead stored the repeated fields by value, then we would need to add virtual functions to each Message that would call the constructors of the repeated fields which we would call during parsing after allocating non-default split structs.
PiperOrigin-RevId: 549673919
If a proto_library has more than one srcs, we designate the first one as the primary (that file will be passed to rustc as the crate root). All other files will represent (internal) submodules of the crate.
In general, Rust users won't see which .proto file a message came from, they will only see a crate corresponding to the entire proto_library, and in it public submodules for all `package` statements in all .proto files in the proto_library sources. Therefore in this CL we reexport all messages from non primary sources into their corresponding public modules (= packages declared in their owning .proto files).
Besides the common case this CL also handles .proto files without package statement, and a subset of behaviors needed for public import functionality.
PiperOrigin-RevId: 549543321
This optimizes non const and copy initialization of arena string values. String hardening is automatically performed on as needed by the ArenaStringPtr class, removing the need for extra clutter or set operations added by the protocol compiler in generated code.
PiperOrigin-RevId: 549480061
Only handles proto2 syntax atm, but I couldn't find any other implementation of something that seemed rather simple and sometimes useful, to integrate with third-party tools meant to run directly on .proto files.
Example usage: `protoc Example.proto --descriptor_set_in file_descriptor_set.pb --proto_out .`
Closes#13338
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13338 from mycroftjr:patch-1 b25408d05f
PiperOrigin-RevId: 549351811
This adds `#![deny(unsafe_op_in_unsafe_fn)]` which removes the
implicit `unsafe` block that `unsafe fn` does.
It also adds many more `SAFETY` docs, corrects some incomplete
ones, and catches a null pointer returned by `upb_Arena_New`.
PiperOrigin-RevId: 549067106
This is just cruft used for selectively rolling out the editions changes. Since it's not tested, it would be dangerous to keep it in place.
PiperOrigin-RevId: 547979600
Editions are still flag-guarded by the `--experimental_editions` flag for now, but once that's removed in a later release generators will need to explicitly specify that their support. This will avoid cases where generators may happen to work for editions but produce incorrect code.
PiperOrigin-RevId: 547959326
There are 4 different feature sets for every descriptor:
* Runtime/resolved - Used to make runtime decisions
* Source/resolved - Used to make codegen decisions
* Runtime/unresolved - Used to reason about the original proto file during runtime
* Source/unresolved - Used to validate features during codegen
Plugins will receive all 4 of these. Unresolved features will show up in the `raw_features` fields nested inside `features`. The runtime features will be in the `proto_file` field of the request, while the source features will be in the unstripped `source_file_descriptors` field.
PiperOrigin-RevId: 547925275
There are 4 different feature sets for every descriptor:
* Runtime/resolved - Used to make runtime decisions
* Source/resolved - Used to make codegen decisions
* Runtime/raw - Used to reason about the original proto file during runtime
* Source/raw - Used to validate features during codegen
All of these will need to be shipped to code generators. While C++ generators can use the C++ runtime to reconstruct these, they won't end up in the FileDescriptorProtos we send to runtimes. Similarly, non-C++ generators wouldn't be able to get them without duplicating our feature resolution logic. This change adds a helper that allows us to bundle all of these into the protos we send to generators.
PiperOrigin-RevId: 547849243
The `unittest_proto3_bad_macros.proto` test is failing on FreeBSD due to the GID_MAX/UID_MAX macros. This is because `port_def.inc` is missing these entries (as well as TRUE/FALSE) for FreeBSD. This PR adds those missing macros.
See compile errors below:
```
[ 87%] Building CXX object CMakeFiles/tests.dir/src/google/protobuf/unittest_preserve_unknown_enum2.pb.cc.o
[ 87%] Building CXX object CMakeFiles/tests.dir/src/google/protobuf/unittest_proto3.pb.cc.o
[ 87%] Building CXX object CMakeFiles/tests.dir/src/google/protobuf/unittest_proto3_arena.pb.cc.o
[ 87%] Building CXX object CMakeFiles/tests.dir/src/google/protobuf/unittest_proto3_arena_lite.pb.cc.o
[ 88%] Building CXX object CMakeFiles/tests.dir/src/google/protobuf/unittest_proto3_bad_macros.pb.cc.o
In file included from /usr/home/jpb/protobuf/src/google/protobuf/unittest_proto3_bad_macros.pb.cc:4:
/usr/home/jpb/protobuf/src/google/protobuf/unittest_proto3_bad_macros.pb.h:71:15: error: expected unqualified-id
constexpr GID GID_MAX = static_cast<GID>(0);
^
/usr/include/sys/limits.h:85:18: note: expanded from macro 'GID_MAX'
#define GID_MAX UINT_MAX /* max value for a gid_t */
^
/usr/include/sys/limits.h:59:18: note: expanded from macro 'UINT_MAX'
#define UINT_MAX __UINT_MAX /* max value for an unsigned int */
^
/usr/include/x86/_limits.h:58:20: note: expanded from macro '__UINT_MAX'
#define __UINT_MAX 0xffffffff /* max value for an unsigned int */
^
In file included from /usr/home/jpb/protobuf/src/google/protobuf/unittest_proto3_bad_macros.pb.cc:4:
/usr/home/jpb/protobuf/src/google/protobuf/unittest_proto3_bad_macros.pb.h:102:15: error: expected unqualified-id
constexpr UID UID_MAX = static_cast<UID>(0);
^
/usr/include/sys/limits.h:86:18: note: expanded from macro 'UID_MAX'
#define UID_MAX UINT_MAX /* max value for a uid_t */
^
/usr/include/sys/limits.h:59:18: note: expanded from macro 'UINT_MAX'
#define UINT_MAX __UINT_MAX /* max value for an unsigned int */
^
/usr/include/x86/_limits.h:58:20: note: expanded from macro '__UINT_MAX'
#define __UINT_MAX 0xffffffff /* max value for an unsigned int */
^
2 errors generated.
gmake[2]: *** [CMakeFiles/tests.dir/build.make:2038: CMakeFiles/tests.dir/src/google/protobuf/unittest_proto3_bad_macros.pb.cc.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1178: CMakeFiles/tests.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
```
Closes#13303
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13303 from jbloggz:freebsd_macro_fix 12d6baf35a
PiperOrigin-RevId: 547820047
Jason Lunn
Martijn Vels
Protobuf Team Bot
Jie Luo
Protobuf Team Bot
Marcel Hlopko
Hong Shin
MycroftJr
Chris Kennelly
kang jinci
Thomas Van Lenten
Nevena Kotlaja
Mike Kruskal
jbloggz