and enable Wnullability-completeness in proto headers via port_def.inc to help check for backsliding.
A few nonnulls are added for nested pointers.
A follow-on will add the rest of the Nonnull annotations.
We'll also add annotations for the runtime later.
Not yet active in open source (more of a comment).
PiperOrigin-RevId: 712901449
There's already path compression which guarantees amortized fast times (halving the cost of subsequent lookups, alas not the inverse ackermann), but there's still no need to redo work and acquire/release atomics the whole way along the path. This also takes advantage of the fast-path relaxed-only read for querying the root of a root node.
PiperOrigin-RevId: 712770023
* Add acquire/release where necessary for all atomic ops
* Add sentinel member to ensure safe publication when tsan is active; tsan will not catch the previous errors without this member.
* For all operations using relaxed memory order, comment why relaxed order is safe
* Add a test that exercises racy fuses and space allocated checks without mutexes or other memory barriers from the test harness. This test proved the existence of several races not caught by the existing tests, including one with a confident comment about why relaxed memory order was safe.
* Add a test that exercises racing allocation and destruction among fused arenas, which doesn't use locks and substitutes a custom allocator that verifies its memory blocks.
Test coverage and assert/tsan instrumentation is now sufficient to cause test failures if any call site is further relaxed.
PiperOrigin-RevId: 712751905
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
Temporarily, we will need to hardcode the migrated test name in test_upb.yml
because not all tests under google.protobuf.internal support UPB yet.
(UPB testing for selected tests are added in 21e9aa6cac).
PiperOrigin-RevId: 712672890
implemented on top of `UntypedMapBase` visitation.
Reduces code duplication in large binaries.
More to come in future changes.
PiperOrigin-RevId: 712658107
The unnecessary [< >] around the ident in that specific macro expansion makes it the name as a literal which would then later be reparsed back into an ident, which makes it lose the r# in the process and break.
PiperOrigin-RevId: 712562736
This was disabled because it couldn't pass with the upb C generated accessors. Now that we only use the upb minitables, this can be safely enabled.
PiperOrigin-RevId: 712560415
This fixes:
* MSVC with `/std:c11 /experimental:c11atomics` on recent versions now emits atomics
* Clang with `-std=c11 -fgnuc-version=0` now emits atomics
* Clang and GCC 14 when built with `-std=c99 -pedantic-errors` will now compile, and not emit atomics
PiperOrigin-RevId: 712538312
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
Temporarily, we will need to hardcode the migrated test name in test_upb.yml
because not all tests under google.protobuf.internal support UPB yet.
(UPB testing for selected tests are added in 21e9aa6cac).
PiperOrigin-RevId: 711837521
When there is an #if or #ifdef in raw strings, clang-format wants to put it at the start of the line, which can leave a big gap compared to the rest of the code, and throws off indent calculation.
PiperOrigin-RevId: 711828681
Add protobuf local macro similar to the very recently added absl version cd9dd4266c The macro form allows Wnullability-completeness to check (vs absl's template form of annotation)
The macro will be empty unless you define PROTOBUF_TEMPORARY_NULL_ANNOTATIONS. We will switch that on once all annotations are in. The annotations will also be off in open source until we are ready.
PiperOrigin-RevId: 711805580
I just tried removing a bunch of if api_implementation.Type() guards in
message_test.py, and a bunch of them just pass without specialization! The bugs
that used to cause trouble seem to be now fixed.
PiperOrigin-RevId: 711764185
implemented on top of `UntypedMapBase` visitation.
Reduces code duplication in large binaries.
More to come in future changes.
PiperOrigin-RevId: 710756618
We no longer need to traverse the linked list of blocks to check allocated space, which means we also no longer need atomics in the linked list or even its head. This is especially beneficial as the previous implementation contained a race where we could dereference uninitialized memory; because the setting of the `next` pointers did not use release semantics and the reading of them in `SpaceAllocated` reads with relaxed order, there's no guarantee that `size` has actually been initialized - but worse, *there is also no guarantee that `next` has been!*. Simplified:
```
AddBlock:
1 ptr = malloc();
2 ptr->size = 123;
3 ptr->next = ai->blocks;
4 ai->blocks = ptr (release order);
```
```
SpaceAllocated:
5 block = ai->blocks (relaxed order)
6 block->size (acquire, but probably by accident)
7 block = block->next (relaxed order)
```
So I think a second thread calling SpaceAllocated could see the order 1, 4, 5, 6, 7, 2, 3 and read uninitialized memory - there is no data-dependency relationship or happens-before edge that this order violates, and so it would be valid for a compiler+hardware to produce.
In reality, operation 4 will produce an `stlr` on arm (forcing an order of 1, 2, 3 before 4), and `block->next` has a data dependency on `ai->blocks` which would force an ordering in the hardware between 5->6 and 5->7 even for regular `ldr` instructions.
Delete arena contains, it's private and the only user is its own test.
PiperOrigin-RevId: 709918443
This fixes that we accidentally imported absl ascii.h within the protobuf namespace, and avoids unused imports warnings for imports that are only used on windows.
PiperOrigin-RevId: 709846366
While in general Proto enum numbers are more of arbitary wire tag numbers, sometimes the ordering is used for semantic meaning, and this allows for the enums to be used in things that require ordering eg BTreeMap.
Rust defaults to language enums _not_ being Ord with the derive(Ord) opt in, but it is largely for reasons which don't apply to protobuf enums.
PiperOrigin-RevId: 709831238
This avoids installing every header required by protoc, and only installs the ones we've explicitly marked as public in bazel (and their transitive dependencies).
PiperOrigin-RevId: 709815861
Hong Shin
Protobuf Team Bot
Protobuf Team Bot
なつき
Tony Liao
Sandy Zhang
Chris Kennelly
Jie Luo
y-yagi
Mike Kruskal