With the C++ kernel for Rust, we currently need to generate quite a few C++
thunks for operations on map fields. For each message we generate, we generate
these thunks for all possible map types that could have that message as a
value. These operations are for things such as insertion, removal, clearing,
iterating, etc.
The reason we do this is that templated types don't play well with FFI, so we
effectively need separate FFI endpoints for every possible combination of key
and value types used (or even potentially used) as a map field.
This CL fixes the problem by replacing the generated thunks with functions in
the runtime that can operate on `proto2::MessageLite*` without needing to care
about the specific message type.
The way it works is that we implement the operations using either
`UntypedMapBase` (the base class of all map types, which knows nothing about
the key and value types) or `KeyMapBase`, which knows the key type but not the
value type. I roughly followed the example of the table-driven parser, which
has a similar problem of needing to operate generically on maps without having
access to the concrete types.
I removed 54 thunks per message (that's 6 key types times 9 operations per
key), but had to add two new thunks per message:
- The `size_info` thunk looks up the `MapNodeSizeInfoT`, which is stored in a
small constant table. The important thing here is an offset indicating where
to look for the value in each map entry. This offset can be different for
every pair of key and value types, but we can safely assume that the result
does not depend on the signedness of the key. As a result we only need to
store four entries per message: one each for i32, i64, bool, and string.
- The `placement_new` thunk move-constructs a message in place. We need this
to be able to efficiently implement map insertion.
There are two big things that this CL does not address yet but which I plan to
follow up on:
- Enums still generate many map-related C++ thunks that could be replaced with
a common implementation. This should actually be much easier to handle than
messages, because every enum has the same representation as an i32.
- We still generate six `ProxiedInMapValue` implementations for every message,
but it should be possible to replace these with a blanket implementation that
works for all message types.
PiperOrigin-RevId: 657681421
I also added a blanket implementation of `IntoProxied<T> for T` so that we
don't have to duplicate this no-op implementation for all our types.
PiperOrigin-RevId: 656465755
* The public Repeated::{push, set} and Map::insert methods now accept any value that implements IntoProxied<T>, allowing us to move owned values instead of copying them.
* This change also updates the FFI layer for strings/bytes in the repeated and maps thunks to accept a std::string* that can be moved rather than a PtrAndLen type that needs to be copied.
* Tests are updated to no longer .as_view() when setting a message / string on a repeated / map field. The IntoProxied trait makes calling .as_view() obsolete.
PiperOrigin-RevId: 650580788
We now support fields with bytes as map values e.g. map<i32, bytes>. The implementation for the C++ runtime was straightforward. The majority of the changes in this CL are about the UPB runtime. In UPB, when we insert Rust bytes/string into the map we need to first copy the bytes onto the maps arena. To support this I have rewritten the macro that implements the ProxiedInMapValue types. I refactored the functionality to convert between UPB and Rust types into the 'UpbTypeConversions' trait. This trait has a function 'to_message_value_if_required' which does the copying for bytes and strings.
PiperOrigin-RevId: 599118416
This change is a pure refactoring and simplification of the code. We replace all MapsWith<TYPE>KeyOps traits through a single generic ProxiedInMapValue<K> trait. Through connecting the runtime maps implementation with Proxied the code gets a lot simpler e.g. we can use View<T> instead of hardcoding the concrete type behind it.
I also expect this change to be beneficial for the gencode. In a subsequent CL we'll implement message values for maps. After this change we'll only have to implement a single trait, while before we had to implement num(key types) many traits.
PiperOrigin-RevId: 596562909
This change implements maps with keys and values of type string e.g. Map<ProtoStr, i32> and Map<ProtoStr, ProtoStr>.
Implementing the Map type for ProtoStr has been different from scalar types because ProtoStr is an unsized type i.e. its size is not known at compile time. The existing Map implementation assumed sized types in many places. To make unsized types fit into the existing code architecture I have added an associated type 'Value' to the MapWith*KeyOps traits. The associated type needs to be sized and is the type returned by the Map::get(self, key) method e.g. for aProtoStr, the `type Value = &ProtoStr`.
PiperOrigin-RevId: 588783751
Dmitri Gribenko
Adam Cozzette
Jakob Buchgraber
Marcel Hlopko
Bastien Jacot-Guillarmod
Protobuf Team Bot
Alyssa Haroldsen