On the read side avoid writing to memory via volatile. The volatile access will generate its own tsan checks because it is touching memory.
On the write side also avoid the volatile access by unconditionally writing to the memory. The address used for the tsan validation is dedicated for it so it doesn't matter if we write arbitrary values to it.
PiperOrigin-RevId: 559841351
This allows the compiler to statically detect use-after-free bugs.
This change touches a subset of field types. More changes to follow.
PiperOrigin-RevId: 559832961
- `generate_minimal_imports` is now `yes`, this reduces the imports generated
into the generated source.
- `headers_use_forward_declarations` is now `no`; this causes the generated
.pbobj.h files to directly import all the headers needed to define types used by
this types defined in the file. This provides better compatibility for Swift
interop especially when the ObjC protos are used in multiple libraries as
forward declarations can cause issues for importing into Swift.
If you need to override either options, please open an issue on github to report
the reason. A future release of the library will remove these options locking in
the new behaviors.
PiperOrigin-RevId: 559737716
This places all of the tricky reflection logic of feature resolution into a single location that outputs a portable protobuf message. With this message, feature resolution becomes drastically simpler and easy to reimplement in other languages.
Follow-up changes will provide:
- An API to specify generator features from the CodeGenerator class for C++ plugins
- A CLI for building these default mappings
- Bazel/CMake rules to help embed these mappings in target languages (i.e. for runtimes and non-C++ plugins)
PiperOrigin-RevId: 559615720
This test validates that upb Python targets can be built successfully even if
Python is not installed locally.
I also updated our pinned upb version to pull in some recent fixes needed for
this test run.
PiperOrigin-RevId: 559504790
Disable the alignment check in 32-bit msvc.
This toolchain has a difference between "natural" and "required" alignment of
types and it can have the alignment of members of type `T` to be smaller than
`alignof(T)`.
Also, disable AnyTest.TestPackFromSerializationExceedsSizeLimit there because it can't allocate that much memory.
PiperOrigin-RevId: 559495544
This makes a few changes:
- It changes generated messages to reference message innards as a type in `__runtime` instead of branching on what fields should be there. That results in much less bifurcation in gencode and lets runtime-agnostic code reference raw message innards.
- It adds a generic mechanism for creating vtable-based mutators. These vtables point to thunks generated for interacting with C++ or upb fields. Right now, the design results in 2-word (msg+vtable) mutators for C++ and 3-word mutators (msg+arena+vtable) for UPB. See upb.rs for an explanation of the design options. I chose the `RawMessage+&Arena` design for mutator data as opposed to a `&MessageInner` design because it did not result in extra-indirection layout changes for message mutators. We could revisit this in the future with performance data, since this results in all field mutators being 3 words large instead of the register-friendly 2 words.
- And lastly, as a nearby change that touches on many of the same topics, it adds some extra SAFETY comments for Send/Sync in message gencode.
PiperOrigin-RevId: 559483437
This fixes Python/C++ and upb, and pushes the buggy behavior to pure python. There, it's very difficult to handle options on the bootstrapped proto with the current architecture. Future changes will attempt to address this more isolated issue.
PiperOrigin-RevId: 559450900
This will allow us to run all the upb tests from the protobuf repo, which
already uses this flag. I just had to remove a couple glob patterns that did
not match any files.
PiperOrigin-RevId: 559434154
If there is no local Python installation then the contents of this glob will be
empty. We want the build to succeed in that scenario, so this change will allow
that to happen.
PiperOrigin-RevId: 559407373
This will be used to seed feature resolution, which becomes just an edition lookup followed by proto merges. Runtime and generators that need to resolve their own features will be able to leverage this to avoid reimplementing all of the reflective logic that goes into the default calculation.
PiperOrigin-RevId: 559271634
This change adds test runs for GCC, Windows, and Mac. The 32-bit ASAN test
turned out to be somewhat difficult to migrate, so I left a TODO for that one.
I made sure to give each build environment its own cache prefix, so that we
don't end up with artifacts built by different toolchains conflicting with each
other in the same cache.
PiperOrigin-RevId: 559197655
1. getter function to access the upb_Map corresponding to a map proto field
2. conversion function to get a Go map from a upb_Map. The conversion function uses generics, so it must be a standalone function independent of `MessageAccessor`.
Accessing map fields is not as efficient as accessing other proto field types, because of two reasons:
- we need to make a copy of map values in Go, since the memory representations are different in UPB C
- we need to call C functions from Go, because the upb_Map struct fields are not exported and we can't access the bits directly, causing performance overhead
Luckily the map fields can be quite rare, so the performance overhead should not be too significant overall.
PiperOrigin-RevId: 559191359
The issue is that if another component (MediaPipe in this case) calls PackFrom, there'll be linker errors because kTypeGoogleApisComPrefix wasn't found.
PiperOrigin-RevId: 559132962
The Python comparison protocol requires that if an object doesn't know how to
compare itself to an object of a different type, it returns NotImplemented
rather than False. The interpreter will then try performing the comparison using
the other operand. This translates, for protos, to:
If a proto message doesn't know how to compare itself to an object of
non-message type, it returns NotImplemented. This way, the interpreter will then
try performing the comparison using the comparison methods of the other object,
which may know how to compare itself to a message. If not, then Python will
return the combined result (e.g., if both objects don't know how to perform
__eq__, then the equality operator `==` return false).
This change allows one to compare a proto with custom matchers such as mock.ANY
that the message doesn't know how to compare to, regardless of whether
mock.ANY is on the right-hand side or left-hand side of the equality (prior to
this change, it only worked with mock.ANY on the left-hand side).
See https://github.com/protocolbuffers/protobuf/issues/9173
PiperOrigin-RevId: 559056804
Remove message/message.h as a hdr from :message_internal
Lock down the visibility of :message_types to upb-only
message/types.h is not an internal header because the definitions there are part of the public surface. But the fact that it needs to exist is an implementation detail that should remain opaque, so make it a private dep.
PiperOrigin-RevId: 558960776
Alyssa Haroldsen
Adam Cozzette
Protobuf Team Bot
Protobuf Team Bot
Thomas Van Lenten
Richard Levasseur
Mike Kruskal
Eric Salo
Joshua Haberman
martijneken