Instead of using the message's corresponding field to denote the end of the stream, just abort it on *any* end group tag. If this is an invalid tag, the caller will handle the failure.
PiperOrigin-RevId: 622859234
Trying to build `protobuf/3.21.12` my tests is failing on Android due to the GID_MAX/UID_MAX macros. This is because port_def.inc is missing these entries for Android. This PR adds those missing macros. I am not sure if I should apply the same port for the `TRUE`/`FALSE` macros as well, any opinion here would be appreciated.
See compile errors below:
```
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=aarch64-none-linux-android29 --gcc-toolchain=C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchai
ns/llvm/prebuilt/windows-x86_64 --sysroot=C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot -IC:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9
-armv8-17-debug -isystem C:/Users/edukaj/.conan2/p/b/proto97c417c206dce/p/include -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat
-Werror=format-security -O0 -fno-limit-debug-info -std=c++17 -fPIE -MD -MT CMakeFiles/test_package.dir/addressbook.pb.cc.o -MF CMakeFiles\test_package.dir\addressbook.pb.cc.o.d -o CMakeFiles/test_package.dir/addressbook.p
b.cc.o -c C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.cc
In file included from C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.cc:4:
C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.h:834:24: error: expected member name or ';' after declaration specifiers
static constexpr GID GID_MAX =
~~~~~~~~~~~~~~~~~~~~ ^
C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include\limits.h:118:18: note: expanded from macro 'GID_MAX'
# define GID_MAX UINT_MAX /* max value for a gid_t */
^
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\include\limits.h:56:20: note: expanded from macro 'UINT_MAX'
#define UINT_MAX (__INT_MAX__ *2U +1U)
^
<built-in>:92:21: note: expanded from here
#define __INT_MAX__ 2147483647
^
In file included from C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.cc:4:
C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.h:834:24: error: expected ')'
C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include\limits.h:118:18: note: expanded from macro 'GID_MAX'
# define GID_MAX UINT_MAX /* max value for a gid_t */
^
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\include\limits.h:56:20: note: expanded from macro 'UINT_MAX'
#define UINT_MAX (__INT_MAX__ *2U +1U)
^
<built-in>:92:21: note: expanded from here
#define __INT_MAX__ 2147483647
^
C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.h:834:24: note: to match this '('
C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include\limits.h:118:18: note: expanded from macro 'GID_MAX'
# define GID_MAX UINT_MAX /* max value for a gid_t */
^
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\include\limits.h:56:19: note: expanded from macro 'UINT_MAX'
#define UINT_MAX (__INT_MAX__ *2U +1U)
^
In file included from C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.cc:4:
C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.h:862:24: error: expected member name or ';' after declaration specifiers
static constexpr UID UID_MAX =
~~~~~~~~~~~~~~~~~~~~ ^
C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include\limits.h:117:18: note: expanded from macro 'UID_MAX'
# define UID_MAX UINT_MAX /* max value for a uid_t */
^
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\include\limits.h:56:20: note: expanded from macro 'UINT_MAX'
#define UINT_MAX (__INT_MAX__ *2U +1U)
^
<built-in>:92:21: note: expanded from here
#define __INT_MAX__ 2147483647
^
In file included from C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.cc:4:
C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.h:862:24: error: expected ')'
C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include\limits.h:117:18: note: expanded from macro 'UID_MAX'
# define UID_MAX UINT_MAX /* max value for a uid_t */
^
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\include\limits.h:56:20: note: expanded from macro 'UINT_MAX'
#define UINT_MAX (__INT_MAX__ *2U +1U)
^
<built-in>:92:21: note: expanded from here
#define __INT_MAX__ 2147483647
^
C:/qxr/repo/conan-center-index/recipes/protobuf/all/test_package/build/clang-9-armv8-17-debug/addressbook.pb.h:862:24: note: to match this '('
C:/Users/edukaj/.conan2/p/b/andro6a387be0e6a99/p/bin/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include\limits.h:117:18: note: expanded from macro 'UID_MAX'
# define UID_MAX UINT_MAX /* max value for a uid_t */
^
C:\Users\edukaj\.conan2\p\b\andro6a387be0e6a99\p\bin\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\9.0.9\include\limits.h:56:19: note: expanded from macro 'UINT_MAX'
#define UINT_MAX (__INT_MAX__ *2U +1U)
^
4 errors generated.
ninja: build stopped: subcommand failed.
```
Closes#16292
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/16292 from elvisdukaj:fix/macros_conflict_with_android_ndk 4c3801cf17
PiperOrigin-RevId: 622422810
These are already supported, but this will lock down that there's no issues like the ones we hit with text-format.
This also fixes an unrelated bug in our BinaryToJsonString algorithm related to unknown group handling. We still can't actually test binary->JSON in conformance tests for extensions due to TypeResolver's lack of support though.
PiperOrigin-RevId: 622360970
This updates all our text parsers and serializers to better handle tag-delimited fields under editions. Under proto2, groups were the only tag-delimited fields possible, and the group name (i.e. the message type) was guaranteed to be unique. Text-format and various generators used this instead of the synthetic field name (lower-cased group name) to represent these fields.
Under editions, we've removed group syntax and allowed any message field to be tag-delimited. This breaks those cases when adding new tag-delimited fields where the message type might not be unique or correspond to the field name. Code generators have already been fixed to treat "group-like" fields using the old behavior, and treat new fields like any other sub-message.
This change addresses the text-format issue. Text parsers will accept *either* the type or field name for "group-like" fields, and only the field name for every other message field. Text serializers will continue to emit the message name for "group-like" fields, but also use the field name for everything else.
This creates some awkward capitalization behavior for fields that happen to *look* like proto2 groups, but it won't lead to any conflicts or invalid encodings. A feature will likely be added to edition 2024 to allow for migration off this legacy behavior.
PiperOrigin-RevId: 622260327
A few of our generators previously used proto2 group's message name to generate code. In order to keep these backwards compatible but still functional under editions, we define a new IsGroupLike helper function. Message fields consistent with proto2 groups will continue using the old generated APIs, but all new ones possible under editions will correctly use the field name.
Edition 2024 will likely include a feature to allow for incremental migrations off this pattern, and follow-up changes will address related text-format issues.
PiperOrigin-RevId: 622204864
They are not needed after the rules are move into protobuf repo.
Except for the reference to toolchain type, which is currently in rules_proto and can be moved after the implementation is moved into protobuf repo.
PiperOrigin-RevId: 622176865
This will allow users to export any metadata attached to this descriptor without also constructing every nested descriptor. This is consistent with FileDescriptor::CopyHeadingTo, which serves a similar purpose at a different scope.
PiperOrigin-RevId: 621995153
function call with precalculated values.
We have to query every field anyway, so better to do it once at the start and
keep the results.
PiperOrigin-RevId: 621650969
All parsing now happens via a TcParser table, accessible from `ClassData` either directly or through the `get_tc_table` callback.
All codegen types insert their generated table in their ClassData. Dynamic types use the callback to generate it on demand.
PiperOrigin-RevId: 621632525
DynamicMessage now eagerly creates the tc_table doing proper crosslinking of TDP tables.
Remove all kTvDefault cases from TcParser.
PiperOrigin-RevId: 620943275
Also, we remove the ArenaDtor and register the Cord for destruction on demand.
This avoids registering the ArenaDtor on every message with a lazy field.
PiperOrigin-RevId: 620857434
* The alignment may be incorrect.
* Using delete[] otherwise mismatches, since it was not allocated with
overaligned operator new.
Additionally, we add a test to ensure we do not value initialize in this case.
PiperOrigin-RevId: 620241337
If it doesn't properly do constant initialization on globals we should
not try to call virtual functions on these objects during dynamic
initialization.
Turn off the optimization in these cases.
PiperOrigin-RevId: 619251810
Mike Kruskal
Elvis Dukaj
Protobuf Team Bot
Protobuf Team Bot
Dave MacLachlan
Marcel Hlopko
Chris Kennelly
Adam Cozzette
Alex Eagle
Anton Grbin