The test I am removing fails on 32-bit ARM, as pointers are 4-byte aligned. `upb_MessageValue` is a union. The alignment of a union is the max. alignment of any of its fields. Among the fields are 8-byte types like u64 and thus `upb_MessageValue` is 8-byte aligned even on 32-bit ARM. 4 != 8 and it fails.
I am split on the usefulness of the test, as I find it somewhat unlikely to catch any divergence between the FFI and C types in the future. I can't imagine a practical scenario where upb_MessageValue would change in C code and this test would fail. IMO our actual unit & integration tests plus the IFTT lints are much better guards.
We could fix the test by testing the alignment of u64 instead of *const c_void, which is always 8 bytes on the hw platforms we care about.
PiperOrigin-RevId: 638976482
We modify set_<repeated_field> to accept the IntoProxied type as the value and move the value (avoid copying) whenever possible.
For UPB:
- We fuse the arena of Repeated<T> with the parent message arena.
- We use upb_Message_SetBaseField to set the upb_Array contained in the Repeated<T>.
For C++:
- We generate an additional setter thunk that moves the value.
- The move assignment operator of RepeatedField/RepeatedPtrField is specialized. In order to adhere to the layering check we need to add '#include' statements for all .proto imports to the generated thunks.pb.cc.
PiperOrigin-RevId: 631010333
The last callside that used PrimitiveMut was in our enums code. This change makes it so that enums nolonger implement MutProxied and thus no longer need the PrimitiveMut type.
PiperOrigin-RevId: 629017282
This change removes the only remaining instance of SettableValue in a generated accessor. The principled fix is to implement IntoProxied for ProtoStr/[u8], but this will have to wait until we have agreed on & implemented the 1.0 string types. So we'll use AsRef in the meantime in order to not break any user code while allowing us to make progress on IntoProxied and Proxied changes.
PiperOrigin-RevId: 627735404
The intent of this directory would be for a layer of Rust bindings that directly map to upb semantics; Rust Protobuf runtime would be layer on top of that Rust, instead of directly on the upb C api.
PiperOrigin-RevId: 624282429
The actual construction of the zeroed block is now entirely safe.
This will be accessed in nearly every program using protobuf.
Using a static in .bss has much less overhead than an atomically-constructed
dynamic allocation and is far more predictable for space-constrained systems.
In the future, if dynamic allocation is kept, it should use std::sync::OnceLock
instead of the much less safe Once combined with `static mut`.
PiperOrigin-RevId: 609635555
This memory management should be handled by Rust.
I've confirmed this works by running the new included tests with msan.
The sanitizer is necessary to detect an incorrect copy_from impl
that uses-after-free from the upb arena.
PiperOrigin-RevId: 604689154
A public "raw" field in a safe wrapper is guaranteed unsound!
This makes repeated and map inner access consistent and
avoids exposing raw internals.
It also provides the accessors necessary for implementing map
access for external types.
PiperOrigin-RevId: 604405543
Part 3 of 4 (added a stage).
The getter and mut_getter bifurcate based on the kernel.
upb returns Option<RawMessage>, while cpp returns the RawMessage.
This means that we can't have a unified MessageVTable in vtable.rs, so we've split these out in {upb.rs and cpp.rs}.
$field$_entry is now prepped and populated for the $field$_mut swappage in the following CL.
PiperOrigin-RevId: 601230880
We want to return $pb$::FieldEntry<'_, $msg_type$> for msg_mut accessors as opposed to the current state (returning $Msg$Mut directly).
In this CL, we pave the way to implementing field entry returns.
We introduce { MessagePresentMutData, MessageAbsentMutData } and impl { ProxiedWithRawVTable, ProxiedWithRawOptionalVTable }. I initially tried a blanket impl approach, but it collided with the already existing PrimitiveVTable constructs; perhaps worth revisiting post 0.6.
In a followup, we'll flesh out the bodies. Lastly, we'll perform the swapover by
replacing $field$_mut with $field$_entry, updating all related tests.
PiperOrigin-RevId: 599282850
We now support fields with bytes as map values e.g. map<i32, bytes>. The implementation for the C++ runtime was straightforward. The majority of the changes in this CL are about the UPB runtime. In UPB, when we insert Rust bytes/string into the map we need to first copy the bytes onto the maps arena. To support this I have rewritten the macro that implements the ProxiedInMapValue types. I refactored the functionality to convert between UPB and Rust types into the 'UpbTypeConversions' trait. This trait has a function 'to_message_value_if_required' which does the copying for bytes and strings.
PiperOrigin-RevId: 599118416
This adds private methods of:
-- .raw_msg() to Msg+MsgMut+MsgView
-- .raw_arena() to Msg+MsgMut [upb kernel only]
And updates the accessors to use the self.raw_msg() / self.raw_arena().
A couple more things will need to be changed before the accessors can be verbatim reused in Msg/MsgView/MsgMut which will be mailed separately.
PiperOrigin-RevId: 598869392
- ProxiedInMapValue is defined in maps.rs, and no longer in the runtime files {upb, cpp}.rs.
- ProxiedInMapValue's methods accept and return Proxied types.
- InnerMapMut no longer has any generic type parameters.
- Through this refactoring the Map type is no longer a ZST. Creating a new map is now as simple as `Map::new()`.
PiperOrigin-RevId: 597765165
This change is a pure refactoring and simplification of the code. We replace all MapsWith<TYPE>KeyOps traits through a single generic ProxiedInMapValue<K> trait. Through connecting the runtime maps implementation with Proxied the code gets a lot simpler e.g. we can use View<T> instead of hardcoding the concrete type behind it.
I also expect this change to be beneficial for the gencode. In a subsequent CL we'll implement message values for maps. After this change we'll only have to implement a single trait, while before we had to implement num(key types) many traits.
PiperOrigin-RevId: 596562909
For the cpp runtime, call the `Message::CopyFrom` method.
For the upb runtime, expose the message `MiniTable` and call `upb_Message_DeepCopy`.
PiperOrigin-RevId: 595166276
- Rename most usage of `'a` to `'msg`
- Remove a no-op unused lifetime param for `remove` in maps
- Elide lifetimes recommended by clippy
PiperOrigin-RevId: 589878364
This change implements maps with keys and values of type string e.g. Map<ProtoStr, i32> and Map<ProtoStr, ProtoStr>.
Implementing the Map type for ProtoStr has been different from scalar types because ProtoStr is an unsized type i.e. its size is not known at compile time. The existing Map implementation assumed sized types in many places. To make unsized types fit into the existing code architecture I have added an associated type 'Value' to the MapWith*KeyOps traits. The associated type needs to be sized and is the type returned by the Map::get(self, key) method e.g. for aProtoStr, the `type Value = &ProtoStr`.
PiperOrigin-RevId: 588783751
This change implements the Proxied trait for the Map type.
It leaves a TODO to implement SettableValue. I haven't implemented SettableValue yet because I have not yet been able to verify that we get the right copy_from semantics for both kernels. I'll implement set_on in a follow up.
PiperOrigin-RevId: 584285061
Before this change the runtimes export a Map and MapInner. This change merges the Map and MapInner types into a single MapInner type, removing the Map type from the runtime (upb.rs, cpp.rs).
The motivation for this change is twofold:
1) A separate Map type is not strictly needed by the runtime. I hope this reduces some complexity.
2) After this change we can introduce a runtime-agnostic protobuf::Map type that implements Proxied.
PiperOrigin-RevId: 582978008
This CL implements Maps for scalar types for the C++ runtime. It's orthogonal to cl/580453646. This CL is constrained by having to force template instantiation of proto2::Map<K, V>. Put differently, a Rust protobuf::Map<K, V> implementation needs to call 'extern "C"' functions with both key and value type in the function name (e.g. __pb_rust_Map_i32_f64_get()). We use macros to generate a Map implementation for every (K,V)-pair. An alternative would have been to use vtables.
Luckily a key in a protobuf map can only be integer types, bool and string. So the number of key types is bounded by the specification, while the number of value types is not i.e. any protobuf message can be a value in a map. Given these constraints we introduce one 'MapKeyOps' trait per key type e.g. MapKeyBOOLOps or MapKeyI32Ops. These traits need to be implemented for every value type e.g. 'impl MapKeyBOOLOps for i32' will implement 'Map::<bool, i32>'. In particular the MapKeyOps traits can also be implemented for generated messages without violating the orphan rule.
This CL also contains significant changes to the UPB runtime so that both upb.rs and cpp.rs export a similar interface to simplify the implementation in map.rs and the generated code.
This CL does not yet implement the Proxied trait.
PiperOrigin-RevId: 582951914
We've had access to views for submessages for a while:
If you hit some_message.submsg().some_int(), you'll get a view for that int.
Until now, there hasn't been a way to get some_message.submsg_mut(), so we introduce the mutational pathway here.
We haven't added fully-functioning mutation, but this is a step towards that goal.
subview was inaccurate, so I've refactored and renamed: { accessor_fns_for_views, accessor_fns_for_muts }.
PiperOrigin-RevId: 581984371
Marcel Hlopko
Jakob Buchgraber
Derek Benson
Protobuf Team Bot
Alyssa Haroldsen
Hong Shin
Kevin King