protobuf

Commit Graph

Author	SHA1	Message	Date
Protobuf Team Bot	a5920a728d	Reimplement DynamicMapField to use UntypedMapBase directly. It uses the same nodes that Map<> would use for the respectice types. This makes the type compatible with visitation and removes some custom types supporting DynamicMapField. Remove most of the vtable, since this can now be done directly in MapFieldBase without derived type support. PiperOrigin-RevId: 717984824	1 month ago
Protobuf Team Bot	828716eb57	Use generic DeleteNode to reduce code size of `erase` in `Map` and to simplify the parsing logic in `MpMap`. PiperOrigin-RevId: 704832360	3 months ago
Protobuf Team Bot	7fbb3d2cc7	Make conversion functions for into c++ string types from PtrAndLen. The invariant is that PtrAndLen may hold ptr+len values which are legal for _either_ C++ string_view or Rust slices: constructing one from either Rust or C++ permits the laxest constraints, and care must be taken when converting a PtrAndLen into either type. For "into Rust slice" case to handle is that len=0 ptr must be non-null, so len=0 ptr=null gets turned into an arbitrary non-null pointer as provided by std::ptr::NonNull::dangling() (any preexisting non-null ptr is kept in that direction). For the "into C++ slice" the risk is more obscure that a Rust non-null pointer could potentially be an illegal pointer (such that ptr+0 is not a legal operation in C++), so when going into C++ we map any len=0 cases into ptr=null to avoid this as a possible risk. PiperOrigin-RevId: 704776210	3 months ago
Protobuf Team Bot	7588e512b6	Use visitation to implement allocation/initialization/deallocation of nodes in generic code. This simplifies Rust bindings, and the table-driven parser. PiperOrigin-RevId: 700702876	3 months ago
Protobuf Team Bot	aa9868e137	Reimplement ClearTable using the stored TypeInfo. Clean up the rust bindings to use this better interface. PiperOrigin-RevId: 699199230	3 months ago
Protobuf Team Bot	d6b90bfd66	Add TypeInfo type UntypedMapBase for generic operation support. The data is not currently used. This change is what we need to hook it up from Rust. In a future change we will start using the data to simplify the interface. No expected performance change. The data inserted via static typing is constant evaluated, and fits in the existing padding (in 64-bit builds). Also, fix Rust bindings to take float/double into account now that the enum lists them. PiperOrigin-RevId: 698780360	3 months ago
Adam Cozzette	cbb3edd86d	Rust C++: get all map fields onto a common implementation of ProxiedInMapValue This CL migrates messages, enums, and primitive types all onto the same blanket implementation of the `ProxiedInMapValue` trait. This gets us to the point where messages and enums no longer need to generate any significant amount of extra code just in case they might be used as a map value. There are a few big pieces to this: - I generalized the message-specific FFI endpoints in `rust/cpp_kernel/map.cc` to be able to additionally handle enums and primitive types as values. This mostly consisted of replacing `MessageLite*` parameters with a new `MapValue` tagged union. - On the Rust side, I added a new blanket implementation of `ProxiedInMapValue` in rust/cpp.rs. It relies on its value type to implement a new `CppMapTypeConversions` trait so that it can convert to and from the `MapValue` tagged union used for FFI. - In the Rust generated code, I deleted the generated `ProxiedInMapValue` implementations for messages and enums and replaced them with implementations of the `CppMapTypeConversions` trait. PiperOrigin-RevId: 687355817	4 months ago
Adam Cozzette	d900d6114c	Rust: remove use of `MapNodeSizeInfoT` from generated code We generate these constants to enable map operations, but this is no longer necessary now that we can get the relevant size and alignment information for each message through its vtable. PiperOrigin-RevId: 680712939	5 months ago
Adam Cozzette	5c3d1e8c30	Rust protobuf: remove the need for a generated `placement_new` thunk We have been relying on a per-message generated `placement_new` function for implementing map insertion, but this CL simplifies things by removing that. Instead, we do a reflective swap if possible, or else fall back on a copy. This will probably make insertions a bit slower, but I think it may be worth it because it should make it much simpler to have a blanket implementation for ProxedInMapValue that works for all map types. It looks like it should be possible to make this faster in the future by implementing a bitwise move that will work for any message. PiperOrigin-RevId: 676495920	5 months ago
Mike Kruskal	5695a882bd	Move -Werror to our test/dev bazelrc files. Putting it into BUILD files unintentionally forces it on all our downstream users. Instead, we just want to enable this during testing and let them choose for themselves in their builds. Note, that this expands the scope of -Werror to our entire repo for CI, so a bunch of fixes and opt-outs had to be applied to get this change passing. Closed #14714 PiperOrigin-RevId: 666903224	6 months ago
Adam Cozzette	6ab302d3a3	Rust: cut down on the amount of generated C++ code needed for maps With the C++ kernel for Rust, we currently need to generate quite a few C++ thunks for operations on map fields. For each message we generate, we generate these thunks for all possible map types that could have that message as a value. These operations are for things such as insertion, removal, clearing, iterating, etc. The reason we do this is that templated types don't play well with FFI, so we effectively need separate FFI endpoints for every possible combination of key and value types used (or even potentially used) as a map field. This CL fixes the problem by replacing the generated thunks with functions in the runtime that can operate on `proto2::MessageLite*` without needing to care about the specific message type. The way it works is that we implement the operations using either `UntypedMapBase` (the base class of all map types, which knows nothing about the key and value types) or `KeyMapBase`, which knows the key type but not the value type. I roughly followed the example of the table-driven parser, which has a similar problem of needing to operate generically on maps without having access to the concrete types. I removed 54 thunks per message (that's 6 key types times 9 operations per key), but had to add two new thunks per message: - The `size_info` thunk looks up the `MapNodeSizeInfoT`, which is stored in a small constant table. The important thing here is an offset indicating where to look for the value in each map entry. This offset can be different for every pair of key and value types, but we can safely assume that the result does not depend on the signedness of the key. As a result we only need to store four entries per message: one each for i32, i64, bool, and string. - The `placement_new` thunk move-constructs a message in place. We need this to be able to efficiently implement map insertion. There are two big things that this CL does not address yet but which I plan to follow up on: - Enums still generate many map-related C++ thunks that could be replaced with a common implementation. This should actually be much easier to handle than messages, because every enum has the same representation as an i32. - We still generate six `ProxiedInMapValue` implementations for every message, but it should be possible to replace these with a blanket implementation that works for all message types. PiperOrigin-RevId: 657681421	7 months ago
Jakob Buchgraber	0d6e9794d1	Migrate Repeated::{push, set} and Map::insert to use the IntoProxied trait. * The public Repeated::{push, set} and Map::insert methods now accept any value that implements IntoProxied<T>, allowing us to move owned values instead of copying them. * This change also updates the FFI layer for strings/bytes in the repeated and maps thunks to accept a std::string* that can be moved rather than a PtrAndLen type that needs to be copied. * Tests are updated to no longer .as_view() when setting a message / string on a repeated / map field. The IntoProxied trait makes calling .as_view() obsolete. PiperOrigin-RevId: 650580788	8 months ago
Protobuf Team Bot	c07de7c9df	Change to proto2_rust C prefix and proto2::rust C++ namespace PiperOrigin-RevId: 648791688	8 months ago
Protobuf Team Bot	a9bc366522	Stop using double underscores for our C function names and standardize on the 'rust_proto_' prefix. Besides unnecessary inconsistency on our C symbols, double underscores anywhere in the name are reserved for stdlib use. In practice its unlikely these symbols would ever hit a collision problem (maybe the prior name 'utf8_debug_string' with no prefix as having some risk), but safer to just standardize on this and have no concerns going forward. PiperOrigin-RevId: 648709299	8 months ago
Protobuf Team Bot	419760f873	Split up cpp_api.h/.cc into smaller units. PiperOrigin-RevId: 647663342	8 months ago

15 Commits (117e7bbe74ac7c7faa9b6f44c1b22de366302854)