`Vec<u8>` is a more idiomatic Rust type to return for serialization.
For the C++ kernel, we are able to return this type with no extra copying. We
still use `SerializedData` type for FFI, but convert the result into a
`Vec<u8>` using a new `into_vec` method.
The upb kernel serializes onto an arena, so for upb we do need to copy the data
to get it into a `Vec<u8>`.
PiperOrigin-RevId: 644444571
This change adds a cfg attribute 'cpp_lite' to the C++ kernel of Protobuf Rust. If C++ lite is selected on the command line, the cfg attribute 'cpp_lite' is set. The root cause of the test failure was that the Debug implementation for full C++ protos uses text proto which is not available in C++ lite. The fix uses the 'cpp_lite' cfg attribute to select a different Debug implementation that doesn't rely on text proto
PiperOrigin-RevId: 640552701
We modify set_<repeated_field> to accept the IntoProxied type as the value and move the value (avoid copying) whenever possible.
For UPB:
- We fuse the arena of Repeated<T> with the parent message arena.
- We use upb_Message_SetBaseField to set the upb_Array contained in the Repeated<T>.
For C++:
- We generate an additional setter thunk that moves the value.
- The move assignment operator of RepeatedField/RepeatedPtrField is specialized. In order to adhere to the layering check we need to add '#include' statements for all .proto imports to the generated thunks.pb.cc.
PiperOrigin-RevId: 631010333
The last callside that used PrimitiveMut was in our enums code. This change makes it so that enums nolonger implement MutProxied and thus no longer need the PrimitiveMut type.
PiperOrigin-RevId: 629017282
This change removes the only remaining instance of SettableValue in a generated accessor. The principled fix is to implement IntoProxied for ProtoStr/[u8], but this will have to wait until we have agreed on & implemented the 1.0 string types. So we'll use AsRef in the meantime in order to not break any user code while allowing us to make progress on IntoProxied and Proxied changes.
PiperOrigin-RevId: 627735404
The intent of this directory would be for a layer of Rust bindings that directly map to upb semantics; Rust Protobuf runtime would be layer on top of that Rust, instead of directly on the upb C api.
PiperOrigin-RevId: 624282429
This change implements a custom Debug for messages, views and muts in the C++ kernel. Debug defers to proto2::Utf8Format.
It implements this only for the C++ kernel. We will need to pull in additional dependencies beyond minitables to implement it for UPB as well. This will be done at a later point.
PiperOrigin-RevId: 613191236
It now uses the same prefix as other thunks needed for the proxied type,
so the RawMapThunk helper can be used for enums.
Calling it a "iter next" thunk is misleading.
It does not increment the iterator as "next" implies,
it only gets the current key/value the iterator points to.
PiperOrigin-RevId: 609527442
A public "raw" field in a safe wrapper is guaranteed unsound!
This makes repeated and map inner access consistent and
avoids exposing raw internals.
It also provides the accessors necessary for implementing map
access for external types.
PiperOrigin-RevId: 604405543
Part 3 of 4 (added a stage).
The getter and mut_getter bifurcate based on the kernel.
upb returns Option<RawMessage>, while cpp returns the RawMessage.
This means that we can't have a unified MessageVTable in vtable.rs, so we've split these out in {upb.rs and cpp.rs}.
$field$_entry is now prepped and populated for the $field$_mut swappage in the following CL.
PiperOrigin-RevId: 601230880
We want to return $pb$::FieldEntry<'_, $msg_type$> for msg_mut accessors as opposed to the current state (returning $Msg$Mut directly).
In this CL, we pave the way to implementing field entry returns.
We introduce { MessagePresentMutData, MessageAbsentMutData } and impl { ProxiedWithRawVTable, ProxiedWithRawOptionalVTable }. I initially tried a blanket impl approach, but it collided with the already existing PrimitiveVTable constructs; perhaps worth revisiting post 0.6.
In a followup, we'll flesh out the bodies. Lastly, we'll perform the swapover by
replacing $field$_mut with $field$_entry, updating all related tests.
PiperOrigin-RevId: 599282850
We now support fields with bytes as map values e.g. map<i32, bytes>. The implementation for the C++ runtime was straightforward. The majority of the changes in this CL are about the UPB runtime. In UPB, when we insert Rust bytes/string into the map we need to first copy the bytes onto the maps arena. To support this I have rewritten the macro that implements the ProxiedInMapValue types. I refactored the functionality to convert between UPB and Rust types into the 'UpbTypeConversions' trait. This trait has a function 'to_message_value_if_required' which does the copying for bytes and strings.
PiperOrigin-RevId: 599118416
- ProxiedInMapValue is defined in maps.rs, and no longer in the runtime files {upb, cpp}.rs.
- ProxiedInMapValue's methods accept and return Proxied types.
- InnerMapMut no longer has any generic type parameters.
- Through this refactoring the Map type is no longer a ZST. Creating a new map is now as simple as `Map::new()`.
PiperOrigin-RevId: 597765165
This change is a pure refactoring and simplification of the code. We replace all MapsWith<TYPE>KeyOps traits through a single generic ProxiedInMapValue<K> trait. Through connecting the runtime maps implementation with Proxied the code gets a lot simpler e.g. we can use View<T> instead of hardcoding the concrete type behind it.
I also expect this change to be beneficial for the gencode. In a subsequent CL we'll implement message values for maps. After this change we'll only have to implement a single trait, while before we had to implement num(key types) many traits.
PiperOrigin-RevId: 596562909
- Rename most usage of `'a` to `'msg`
- Remove a no-op unused lifetime param for `remove` in maps
- Elide lifetimes recommended by clippy
PiperOrigin-RevId: 589878364
This change implements maps with keys and values of type string e.g. Map<ProtoStr, i32> and Map<ProtoStr, ProtoStr>.
Implementing the Map type for ProtoStr has been different from scalar types because ProtoStr is an unsized type i.e. its size is not known at compile time. The existing Map implementation assumed sized types in many places. To make unsized types fit into the existing code architecture I have added an associated type 'Value' to the MapWith*KeyOps traits. The associated type needs to be sized and is the type returned by the Map::get(self, key) method e.g. for aProtoStr, the `type Value = &ProtoStr`.
PiperOrigin-RevId: 588783751
This change implements the Proxied trait for the Map type.
It leaves a TODO to implement SettableValue. I haven't implemented SettableValue yet because I have not yet been able to verify that we get the right copy_from semantics for both kernels. I'll implement set_on in a follow up.
PiperOrigin-RevId: 584285061
Before this change the runtimes export a Map and MapInner. This change merges the Map and MapInner types into a single MapInner type, removing the Map type from the runtime (upb.rs, cpp.rs).
The motivation for this change is twofold:
1) A separate Map type is not strictly needed by the runtime. I hope this reduces some complexity.
2) After this change we can introduce a runtime-agnostic protobuf::Map type that implements Proxied.
PiperOrigin-RevId: 582978008
This CL implements Maps for scalar types for the C++ runtime. It's orthogonal to cl/580453646. This CL is constrained by having to force template instantiation of proto2::Map<K, V>. Put differently, a Rust protobuf::Map<K, V> implementation needs to call 'extern "C"' functions with both key and value type in the function name (e.g. __pb_rust_Map_i32_f64_get()). We use macros to generate a Map implementation for every (K,V)-pair. An alternative would have been to use vtables.
Luckily a key in a protobuf map can only be integer types, bool and string. So the number of key types is bounded by the specification, while the number of value types is not i.e. any protobuf message can be a value in a map. Given these constraints we introduce one 'MapKeyOps' trait per key type e.g. MapKeyBOOLOps or MapKeyI32Ops. These traits need to be implemented for every value type e.g. 'impl MapKeyBOOLOps for i32' will implement 'Map::<bool, i32>'. In particular the MapKeyOps traits can also be implemented for generated messages without violating the orphan rule.
This CL also contains significant changes to the UPB runtime so that both upb.rs and cpp.rs export a similar interface to simplify the implementation in map.rs and the generated code.
This CL does not yet implement the Proxied trait.
PiperOrigin-RevId: 582951914
We've had access to views for submessages for a while:
If you hit some_message.submsg().some_int(), you'll get a view for that int.
Until now, there hasn't been a way to get some_message.submsg_mut(), so we introduce the mutational pathway here.
We haven't added fully-functioning mutation, but this is a step towards that goal.
subview was inaccurate, so I've refactored and renamed: { accessor_fns_for_views, accessor_fns_for_muts }.
PiperOrigin-RevId: 581984371
This makes a few changes:
- It changes generated messages to reference message innards as a type in `__runtime` instead of branching on what fields should be there. That results in much less bifurcation in gencode and lets runtime-agnostic code reference raw message innards.
- It adds a generic mechanism for creating vtable-based mutators. These vtables point to thunks generated for interacting with C++ or upb fields. Right now, the design results in 2-word (msg+vtable) mutators for C++ and 3-word mutators (msg+arena+vtable) for UPB. See upb.rs for an explanation of the design options. I chose the `RawMessage+&Arena` design for mutator data as opposed to a `&MessageInner` design because it did not result in extra-indirection layout changes for message mutators. We could revisit this in the future with performance data, since this results in all field mutators being 3 words large instead of the register-friendly 2 words.
- And lastly, as a nearby change that touches on many of the same topics, it adds some extra SAFETY comments for Send/Sync in message gencode.
PiperOrigin-RevId: 559483437
Marcel Hlopko
Adam Cozzette
Jakob Buchgraber
Protobuf Team Bot
Derek Benson
Alyssa Haroldsen
Hong Shin
Kevin King
Joshua Haberman
Adrian Sadłocha