I also added a blanket implementation of `IntoProxied<T> for T` so that we
don't have to duplicate this no-op implementation for all our types.
PiperOrigin-RevId: 656465755
- We introduce two new view types ProtoStringCow and ProtoBytesCow.
- In UPB, for cord field accessors we always return a Cow::Borrowed.
- In C++, for coed field accessors we check if the underlying absl::Cord is flat (contigous) and if so return a Cow::Borrowed. If it's not flat we copy the data to a ProtoString and return a Cow::Owned.
- We expect the absl::Cord to be flat almost all the time. We have experimentally verified that for small strings (<4 KiB) and less than 6 appends the cord is in fact flat [1].
- This change lifts the requirement of all ViewProxy types to be Copy. Our Cow types cannot be Copy because the owned types aren't copy.
[1] https://source.corp.google.com/piper///depot/google3/experimental/users/buchgr/cords/cords.cc
PiperOrigin-RevId: 655485943
New serialization test will verify the amount of bytes it takes to serialize a message with a field of type INT32 set to different values.
PiperOrigin-RevId: 641044189
The intent is to avoid codegen issues for simple cases where a message of the shape:
message M {
int32 x = 3
string set_x = 8;
}
Which would otherwise break due to the first field having a setter whose name collides with the second field's getter.
By seeing that 'set_x' matches another field with a common accessor prefixed, it will generate all accessors for that field as though it was named `set_x_8`.
This does not avoid all possible collisions, but should mitigate the vast majority of situations of an accidental collision.
PiperOrigin-RevId: 627776907
The shared tests which access `protobuf_upb` or `protobuf_cpp`
have access to more items than the `protobuf` library itself.
This is because the former don't go through the same re-exporting based
on kernel.
I fix this by creating two test-only libraries that perform the same re-exporting
as the `protobuf` library, but with the kernel explicitly set, and changing the shared
tests to reference that instead of the inner runtime library.
This is needed to reliably test macros, where item paths are relative to the invocation,
not eagerly checked at the macro source.
PiperOrigin-RevId: 624328817
Add convenience _opt for singular message fields.
Users can detect if the message is set or unset, and unwrap via into_inner.
PiperOrigin-RevId: 602749065
Before this change if a field type was defined in a imported .proto file then our codegen would not generate the field. After this change such fields are correctly generated (see tests). This change is rather trivial as all the supporting infra has been implemented as part of the .proto -> crate mapping CLs.
PiperOrigin-RevId: 601443383
To satisfy the layering check, we need to depend on :gtest for the headers, in
addition to :gtest_main which provides the main() function.
There are a bunch of formatting changes as a side effect of this, but they
should be harmless.
PiperOrigin-RevId: 594318263
accessors_test.rs is getting a bit unwieldy, approaching 1K LOC. This CL breaks out the repeated tests into their own file (accessors_repeated_test.rs), as we expect the tests here to grow. This follows the precedent of `accessors_map_test.rs`.
PiperOrigin-RevId: 588554476
This CL implements Maps for scalar types for the C++ runtime. It's orthogonal to cl/580453646. This CL is constrained by having to force template instantiation of proto2::Map<K, V>. Put differently, a Rust protobuf::Map<K, V> implementation needs to call 'extern "C"' functions with both key and value type in the function name (e.g. __pb_rust_Map_i32_f64_get()). We use macros to generate a Map implementation for every (K,V)-pair. An alternative would have been to use vtables.
Luckily a key in a protobuf map can only be integer types, bool and string. So the number of key types is bounded by the specification, while the number of value types is not i.e. any protobuf message can be a value in a map. Given these constraints we introduce one 'MapKeyOps' trait per key type e.g. MapKeyBOOLOps or MapKeyI32Ops. These traits need to be implemented for every value type e.g. 'impl MapKeyBOOLOps for i32' will implement 'Map::<bool, i32>'. In particular the MapKeyOps traits can also be implemented for generated messages without violating the orphan rule.
This CL also contains significant changes to the UPB runtime so that both upb.rs and cpp.rs export a similar interface to simplify the implementation in map.rs and the generated code.
This CL does not yet implement the Proxied trait.
PiperOrigin-RevId: 582951914
We've had access to views for submessages for a while:
If you hit some_message.submsg().some_int(), you'll get a view for that int.
Until now, there hasn't been a way to get some_message.submsg_mut(), so we introduce the mutational pathway here.
We haven't added fully-functioning mutation, but this is a step towards that goal.
subview was inaccurate, so I've refactored and renamed: { accessor_fns_for_views, accessor_fns_for_muts }.
PiperOrigin-RevId: 581984371
We continue our foray into googletest-rust [1], this time with homegrown matchers.
Our unit tests will now be able to check is_unset() and is_set() for all types that implement ProxiedWithPresence. In practice, this boils down to [u8] and ProtoStr.
Note that we've broken out matchers_upb and matchers_cpp, similar to what was done with aliasing here [2].
[1] https://github.com/google/googletest-rust
[2] 9a0bc392b3 (diff-08e5182ff36ad340a3bfb628995524a2a36a89b59a514ba027b0f25e048dd5c3R90)
PiperOrigin-RevId: 573895179
We had previously commented out the upb portion of simple_nested_test.
This is because nonmutable getters have submessages being NULL by default.
This means that trying to fetch anything, like a simple scalar from that nested message would segfault.
This CL makes the externC return an Option<RawMessage> since we've discovered that upb can return NULL. This way, we can check for `None` and handle the NULL case appropriately.
We know that the NULL pathway can only come from terra upb, since
cpp automagically constructs submsgs if they don't exist.
We've augmented upb.rs to contain a scratch space that allocates a zeroed-out contiguous chunk of memory @64KB. Since a block of zeroed-out memory is a legit message from upb's point of view, we can provide $pbr$::ScratchSpace::zeroed_block() to upb in order to get the default submessage behavior we want from upb.
This block is lazily allocated upon first request. This means that a consumer of the cpp kernel will not incur an additional cost.
PiperOrigin-RevId: 573840755
This makes a few changes:
- It changes generated messages to reference message innards as a type in `__runtime` instead of branching on what fields should be there. That results in much less bifurcation in gencode and lets runtime-agnostic code reference raw message innards.
- It adds a generic mechanism for creating vtable-based mutators. These vtables point to thunks generated for interacting with C++ or upb fields. Right now, the design results in 2-word (msg+vtable) mutators for C++ and 3-word mutators (msg+arena+vtable) for UPB. See upb.rs for an explanation of the design options. I chose the `RawMessage+&Arena` design for mutator data as opposed to a `&MessageInner` design because it did not result in extra-indirection layout changes for message mutators. We could revisit this in the future with performance data, since this results in all field mutators being 3 words large instead of the register-friendly 2 words.
- And lastly, as a nearby change that touches on many of the same topics, it adds some extra SAFETY comments for Send/Sync in message gencode.
PiperOrigin-RevId: 559483437
Only emit has_field() if the field support presence. Only emit field_opt() getter if the field is both optional and supports presence.
PiperOrigin-RevId: 555133374
If a proto_library has more than one srcs, we designate the first one as the primary (that file will be passed to rustc as the crate root). All other files will represent (internal) submodules of the crate.
In general, Rust users won't see which .proto file a message came from, they will only see a crate corresponding to the entire proto_library, and in it public submodules for all `package` statements in all .proto files in the proto_library sources. Therefore in this CL we reexport all messages from non primary sources into their corresponding public modules (= packages declared in their owning .proto files).
Besides the common case this CL also handles .proto files without package statement, and a subset of behaviors needed for public import functionality.
PiperOrigin-RevId: 549543321
Before this CL all messages were generated in the top-level crate module. With
this change we generate messages under the module specified by the package
declaration in the .proto file.
Dots are interpreted as submodule separator in consistency with how C++
namespaces are handled.
Note that name of the proto_library target still remains to be used as the crate name. This CL only adds crate submodules dependeing on the specified package.
PiperOrigin-RevId: 524235162
This turns out to be quite of a yak shave to be able to perfectly test both kernels without having to pass extra Blaze flags.
PiperOrigin-RevId: 521850709
Adam Cozzette
Protobuf Team Bot
Jakob Buchgraber
Dmitri Gribenko
Derek Benson
Marcel Hlopko
Alyssa Haroldsen
Hong Shin
Sandy Zhang