This change implements maps with keys and values of type string e.g. Map<ProtoStr, i32> and Map<ProtoStr, ProtoStr>.
Implementing the Map type for ProtoStr has been different from scalar types because ProtoStr is an unsized type i.e. its size is not known at compile time. The existing Map implementation assumed sized types in many places. To make unsized types fit into the existing code architecture I have added an associated type 'Value' to the MapWith*KeyOps traits. The associated type needs to be sized and is the type returned by the Map::get(self, key) method e.g. for aProtoStr, the `type Value = &ProtoStr`.
PiperOrigin-RevId: 588783751
This change implements the Proxied trait for the Map type.
It leaves a TODO to implement SettableValue. I haven't implemented SettableValue yet because I have not yet been able to verify that we get the right copy_from semantics for both kernels. I'll implement set_on in a follow up.
PiperOrigin-RevId: 584285061
Before this change the runtimes export a Map and MapInner. This change merges the Map and MapInner types into a single MapInner type, removing the Map type from the runtime (upb.rs, cpp.rs).
The motivation for this change is twofold:
1) A separate Map type is not strictly needed by the runtime. I hope this reduces some complexity.
2) After this change we can introduce a runtime-agnostic protobuf::Map type that implements Proxied.
PiperOrigin-RevId: 582978008
This CL implements Maps for scalar types for the C++ runtime. It's orthogonal to cl/580453646. This CL is constrained by having to force template instantiation of proto2::Map<K, V>. Put differently, a Rust protobuf::Map<K, V> implementation needs to call 'extern "C"' functions with both key and value type in the function name (e.g. __pb_rust_Map_i32_f64_get()). We use macros to generate a Map implementation for every (K,V)-pair. An alternative would have been to use vtables.
Luckily a key in a protobuf map can only be integer types, bool and string. So the number of key types is bounded by the specification, while the number of value types is not i.e. any protobuf message can be a value in a map. Given these constraints we introduce one 'MapKeyOps' trait per key type e.g. MapKeyBOOLOps or MapKeyI32Ops. These traits need to be implemented for every value type e.g. 'impl MapKeyBOOLOps for i32' will implement 'Map::<bool, i32>'. In particular the MapKeyOps traits can also be implemented for generated messages without violating the orphan rule.
This CL also contains significant changes to the UPB runtime so that both upb.rs and cpp.rs export a similar interface to simplify the implementation in map.rs and the generated code.
This CL does not yet implement the Proxied trait.
PiperOrigin-RevId: 582951914
We've had access to views for submessages for a while:
If you hit some_message.submsg().some_int(), you'll get a view for that int.
Until now, there hasn't been a way to get some_message.submsg_mut(), so we introduce the mutational pathway here.
We haven't added fully-functioning mutation, but this is a step towards that goal.
subview was inaccurate, so I've refactored and renamed: { accessor_fns_for_views, accessor_fns_for_muts }.
PiperOrigin-RevId: 581984371
We shouldn't give easy access to the zeroed block, so let's pass in pbi::private to deter misuse.
Also saw that arena wasn't used mutably in a repeatedfield test, so let mut -> let;
PiperOrigin-RevId: 574896460
We had previously commented out the upb portion of simple_nested_test.
This is because nonmutable getters have submessages being NULL by default.
This means that trying to fetch anything, like a simple scalar from that nested message would segfault.
This CL makes the externC return an Option<RawMessage> since we've discovered that upb can return NULL. This way, we can check for `None` and handle the NULL case appropriately.
We know that the NULL pathway can only come from terra upb, since
cpp automagically constructs submsgs if they don't exist.
We've augmented upb.rs to contain a scratch space that allocates a zeroed-out contiguous chunk of memory @64KB. Since a block of zeroed-out memory is a legit message from upb's point of view, we can provide $pbr$::ScratchSpace::zeroed_block() to upb in order to get the default submessage behavior we want from upb.
This block is lazily allocated upon first request. This means that a consumer of the cpp kernel will not incur an additional cost.
PiperOrigin-RevId: 573840755
This makes a few changes:
- It changes generated messages to reference message innards as a type in `__runtime` instead of branching on what fields should be there. That results in much less bifurcation in gencode and lets runtime-agnostic code reference raw message innards.
- It adds a generic mechanism for creating vtable-based mutators. These vtables point to thunks generated for interacting with C++ or upb fields. Right now, the design results in 2-word (msg+vtable) mutators for C++ and 3-word mutators (msg+arena+vtable) for UPB. See upb.rs for an explanation of the design options. I chose the `RawMessage+&Arena` design for mutator data as opposed to a `&MessageInner` design because it did not result in extra-indirection layout changes for message mutators. We could revisit this in the future with performance data, since this results in all field mutators being 3 words large instead of the register-friendly 2 words.
- And lastly, as a nearby change that touches on many of the same topics, it adds some extra SAFETY comments for Send/Sync in message gencode.
PiperOrigin-RevId: 559483437
These are more type safe, and more clearly distinguish between a
raw message and serialized data.
This also defines a macro to create new opaque pointer types, and
switches `RawArena` to using it.
PiperOrigin-RevId: 552957136
This CL sets up the basic plumbing end-to-end for singular message fields.
We add skeletonized support for `Proxied` messages. This is done
by creating structs for $Msg$View and $Msg$Mut, and providing
stubbed impls.
PiperOrigin-RevId: 552609955
This adds `#![deny(unsafe_op_in_unsafe_fn)]` which removes the
implicit `unsafe` block that `unsafe fn` does.
It also adds many more `SAFETY` docs, corrects some incomplete
ones, and catches a null pointer returned by `upb_Arena_New`.
PiperOrigin-RevId: 549067106
No need to say "represents" when describing a type (all types represent
something in real world, they are not the real thing), and "ABI-compatible"
needs a dash.
PiperOrigin-RevId: 546813197
In this CL I'd like to call existing C++ Protobuf API from the V0 Rust API. Since parts of the C++ API are defined inline and using (obviously) C++ name mangling, we need to create a "thunks.cc" file that:
1) Generates code for C++ API function we use from Rust
2) Exposes these functions without any name mangling (meaning using `extern "C"`)
In this CL we add Bazel logic to generate "thunks" file, compile it, and propagate its object to linking. We also add logic to protoc to generate this "thunks" file.
The protoc logic is rather rudimentary still. I hope to focus on protoc code quality in my followup work on V0 Rust API using C++ kernel.
PiperOrigin-RevId: 523479839
In this CL we're adding the barebones infrastructure to generate Rust proto messages using UPB as a backend. The API is what we call a V0, not yet production-quality, not yet rigorously designed, just something to enable parallel work.
The interesting part of switching backend between UPB and C++ will come in a followup.
PiperOrigin-RevId: 517089760
* Follows proper autoloading standards
- Splits PHP classes in descriptor.php into separate files
- Splits MapFieldIter and RepeatedFieldIter into separate files
- Moves descriptor.php to Internal/functions.php
- Moves all namespaced functions into Iternal/functions.php
* fixes Makefile.am for added php files
* [PHP] moves all functions to GPBUtil
* removes description.php from the makefile
General
* License changed from Apache 2.0 to New BSD.
* It is now possible to define custom "options", which are basically
annotations which may be placed on definitions in a .proto file.
For example, you might define a field option called "foo" like so:
import "google/protobuf/descriptor.proto"
extend google.protobuf.FieldOptions {
optional string foo = 12345;
}
Then you annotate a field using the "foo" option:
message MyMessage {
optional int32 some_field = 1 [(foo) = "bar"]
}
The value of this option is then visible via the message's
Descriptor:
const FieldDescriptor* field =
MyMessage::descriptor()->FindFieldByName("some_field");
assert(field->options().GetExtension(foo) == "bar");
This feature has been implemented and tested in C++ and Java.
Other languages may or may not need to do extra work to support
custom options, depending on how they construct descriptors.
C++
* Fixed some GCC warnings that only occur when using -pedantic.
* Improved static initialization code, making ordering more
predictable among other things.
* TextFormat will no longer accept messages which contain multiple
instances of a singular field. Previously, the latter instance
would overwrite the former.
* Now works on systems that don't have hash_map.
Python
* Strings now use the "unicode" type rather than the "str" type.
String fields may still be assigned ASCII "str" values; they will
automatically be converted.
* Adding a property to an object representing a repeated field now
raises an exception. For example:
# No longer works (and never should have).
message.some_repeated_field.foo = 1
Jakob Buchgraber
Protobuf Team Bot
Hong Shin
Kevin King
Sandy Zhang
Joshua Haberman
Adrian Sadłocha
Alyssa Haroldsen
Dmitri Gribenko
Marcel Hlopko
Miguel Young de la Sota
listonjesse
Paul Yang
Brent Shaffer
Adam Cozzette
Feng Xiao
xiaofeng@google.com
kenton@google.com
Jon Skeet
temporal