When you have a `T: SomeTrait + ?Sized` and `trait SomeTrait:Sized`, the ?Sized has no effect (its not able to "remove" the requirement).
PiperOrigin-RevId: 654836513
Rather than two traits (MutProxy subtrait of ViewProxy), instead have three traits (MutProxy subtrait of Proxy, and ViewProxy subtrait of Proxy).
This makes things more consistent, including that (MutProxied subtraits Proxied) is now more parallel to (MutProxy subtraits Proxy).
ViewProxy is largely a marker trait here but leaves a spot for methods that should be on ViewProxies but not MutProxies if those do show up later.
PiperOrigin-RevId: 653661953
Our ASAN test runs have not had the heap checker enabled, so this has allowed a
few memory leaks to slip in. This CL fixes all of them so that we can turn on
the heap checker.
The first one takes place whenever we add an entry into a string-valued map
using the C++ kernel. The problem is that `InnerProtoString::into_raw()` gives
up ownership of the raw `std::string` pointer it holds, but then we never
delete that pointer. This CL fixes the problem by deleting the pointer in C++
right after we perform the map insertion. To simplify things, I created a
`MakeCleanup()` helper function that we always call in our map insertion
thunks, but it's a no-op in the cases where we don't need to free anything.
There were a couple similar memory leaks related to repeated field accessors in
the C++ kernel, and those were simple to fix just by adding the necessary
`delete` call.
Finally, there were two benign memory leaks in the upb kernel involving global
variables used for empty repeated fields and maps. It turned out that we did
not need to use `Box` at all here, so removing that simplified things and fixed
the leaks.
PiperOrigin-RevId: 652947042
Calling into_proxied() already does a copy and before this change we were doing a second one.
I am not using set_allocated_<field(std::string* s) because the method is not generated when [features.(pb.cpp).string_type = VIEW] is specified.
PiperOrigin-RevId: 650612909
* The public Repeated::{push, set} and Map::insert methods now accept any value that implements IntoProxied<T>, allowing us to move owned values instead of copying them.
* This change also updates the FFI layer for strings/bytes in the repeated and maps thunks to accept a std::string* that can be moved rather than a PtrAndLen type that needs to be copied.
* Tests are updated to no longer .as_view() when setting a message / string on a repeated / map field. The IntoProxied trait makes calling .as_view() obsolete.
PiperOrigin-RevId: 650580788
The test I am removing fails on 32-bit ARM, as pointers are 4-byte aligned. `upb_MessageValue` is a union. The alignment of a union is the max. alignment of any of its fields. Among the fields are 8-byte types like u64 and thus `upb_MessageValue` is 8-byte aligned even on 32-bit ARM. 4 != 8 and it fails.
I am split on the usefulness of the test, as I find it somewhat unlikely to catch any divergence between the FFI and C types in the future. I can't imagine a practical scenario where upb_MessageValue would change in C code and this test would fail. IMO our actual unit & integration tests plus the IFTT lints are much better guards.
We could fix the test by testing the alignment of u64 instead of *const c_void, which is always 8 bytes on the hw platforms we care about.
PiperOrigin-RevId: 638976482
We modify set_<repeated_field> to accept the IntoProxied type as the value and move the value (avoid copying) whenever possible.
For UPB:
- We fuse the arena of Repeated<T> with the parent message arena.
- We use upb_Message_SetBaseField to set the upb_Array contained in the Repeated<T>.
For C++:
- We generate an additional setter thunk that moves the value.
- The move assignment operator of RepeatedField/RepeatedPtrField is specialized. In order to adhere to the layering check we need to add '#include' statements for all .proto imports to the generated thunks.pb.cc.
PiperOrigin-RevId: 631010333
The last callside that used PrimitiveMut was in our enums code. This change makes it so that enums nolonger implement MutProxied and thus no longer need the PrimitiveMut type.
PiperOrigin-RevId: 629017282
This change removes the only remaining instance of SettableValue in a generated accessor. The principled fix is to implement IntoProxied for ProtoStr/[u8], but this will have to wait until we have agreed on & implemented the 1.0 string types. So we'll use AsRef in the meantime in order to not break any user code while allowing us to make progress on IntoProxied and Proxied changes.
PiperOrigin-RevId: 627735404
The intent of this directory would be for a layer of Rust bindings that directly map to upb semantics; Rust Protobuf runtime would be layer on top of that Rust, instead of directly on the upb C api.
PiperOrigin-RevId: 624282429
The actual construction of the zeroed block is now entirely safe.
This will be accessed in nearly every program using protobuf.
Using a static in .bss has much less overhead than an atomically-constructed
dynamic allocation and is far more predictable for space-constrained systems.
In the future, if dynamic allocation is kept, it should use std::sync::OnceLock
instead of the much less safe Once combined with `static mut`.
PiperOrigin-RevId: 609635555
This memory management should be handled by Rust.
I've confirmed this works by running the new included tests with msan.
The sanitizer is necessary to detect an incorrect copy_from impl
that uses-after-free from the upb arena.
PiperOrigin-RevId: 604689154
A public "raw" field in a safe wrapper is guaranteed unsound!
This makes repeated and map inner access consistent and
avoids exposing raw internals.
It also provides the accessors necessary for implementing map
access for external types.
PiperOrigin-RevId: 604405543
Part 3 of 4 (added a stage).
The getter and mut_getter bifurcate based on the kernel.
upb returns Option<RawMessage>, while cpp returns the RawMessage.
This means that we can't have a unified MessageVTable in vtable.rs, so we've split these out in {upb.rs and cpp.rs}.
$field$_entry is now prepped and populated for the $field$_mut swappage in the following CL.
PiperOrigin-RevId: 601230880
We want to return $pb$::FieldEntry<'_, $msg_type$> for msg_mut accessors as opposed to the current state (returning $Msg$Mut directly).
In this CL, we pave the way to implementing field entry returns.
We introduce { MessagePresentMutData, MessageAbsentMutData } and impl { ProxiedWithRawVTable, ProxiedWithRawOptionalVTable }. I initially tried a blanket impl approach, but it collided with the already existing PrimitiveVTable constructs; perhaps worth revisiting post 0.6.
In a followup, we'll flesh out the bodies. Lastly, we'll perform the swapover by
replacing $field$_mut with $field$_entry, updating all related tests.
PiperOrigin-RevId: 599282850
We now support fields with bytes as map values e.g. map<i32, bytes>. The implementation for the C++ runtime was straightforward. The majority of the changes in this CL are about the UPB runtime. In UPB, when we insert Rust bytes/string into the map we need to first copy the bytes onto the maps arena. To support this I have rewritten the macro that implements the ProxiedInMapValue types. I refactored the functionality to convert between UPB and Rust types into the 'UpbTypeConversions' trait. This trait has a function 'to_message_value_if_required' which does the copying for bytes and strings.
PiperOrigin-RevId: 599118416
This adds private methods of:
-- .raw_msg() to Msg+MsgMut+MsgView
-- .raw_arena() to Msg+MsgMut [upb kernel only]
And updates the accessors to use the self.raw_msg() / self.raw_arena().
A couple more things will need to be changed before the accessors can be verbatim reused in Msg/MsgView/MsgMut which will be mailed separately.
PiperOrigin-RevId: 598869392
- ProxiedInMapValue is defined in maps.rs, and no longer in the runtime files {upb, cpp}.rs.
- ProxiedInMapValue's methods accept and return Proxied types.
- InnerMapMut no longer has any generic type parameters.
- Through this refactoring the Map type is no longer a ZST. Creating a new map is now as simple as `Map::new()`.
PiperOrigin-RevId: 597765165
This change is a pure refactoring and simplification of the code. We replace all MapsWith<TYPE>KeyOps traits through a single generic ProxiedInMapValue<K> trait. Through connecting the runtime maps implementation with Proxied the code gets a lot simpler e.g. we can use View<T> instead of hardcoding the concrete type behind it.
I also expect this change to be beneficial for the gencode. In a subsequent CL we'll implement message values for maps. After this change we'll only have to implement a single trait, while before we had to implement num(key types) many traits.
PiperOrigin-RevId: 596562909
For the cpp runtime, call the `Message::CopyFrom` method.
For the upb runtime, expose the message `MiniTable` and call `upb_Message_DeepCopy`.
PiperOrigin-RevId: 595166276
- Rename most usage of `'a` to `'msg`
- Remove a no-op unused lifetime param for `remove` in maps
- Elide lifetimes recommended by clippy
PiperOrigin-RevId: 589878364
This change implements maps with keys and values of type string e.g. Map<ProtoStr, i32> and Map<ProtoStr, ProtoStr>.
Implementing the Map type for ProtoStr has been different from scalar types because ProtoStr is an unsized type i.e. its size is not known at compile time. The existing Map implementation assumed sized types in many places. To make unsized types fit into the existing code architecture I have added an associated type 'Value' to the MapWith*KeyOps traits. The associated type needs to be sized and is the type returned by the Map::get(self, key) method e.g. for aProtoStr, the `type Value = &ProtoStr`.
PiperOrigin-RevId: 588783751
This change implements the Proxied trait for the Map type.
It leaves a TODO to implement SettableValue. I haven't implemented SettableValue yet because I have not yet been able to verify that we get the right copy_from semantics for both kernels. I'll implement set_on in a follow up.
PiperOrigin-RevId: 584285061
Protobuf Team Bot
Adam Cozzette
Derek Benson
Jakob Buchgraber
Marcel Hlopko
Alyssa Haroldsen
Hong Shin
Kevin King