This updates all our text parsers and serializers to better handle tag-delimited fields under editions. Under proto2, groups were the only tag-delimited fields possible, and the group name (i.e. the message type) was guaranteed to be unique. Text-format and various generators used this instead of the synthetic field name (lower-cased group name) to represent these fields.
Under editions, we've removed group syntax and allowed any message field to be tag-delimited. This breaks those cases when adding new tag-delimited fields where the message type might not be unique or correspond to the field name. Code generators have already been fixed to treat "group-like" fields using the old behavior, and treat new fields like any other sub-message.
This change addresses the text-format issue. Text parsers will accept *either* the type or field name for "group-like" fields, and only the field name for every other message field. Text serializers will continue to emit the message name for "group-like" fields, but also use the field name for everything else.
This creates some awkward capitalization behavior for fields that happen to *look* like proto2 groups, but it won't lead to any conflicts or invalid encodings. A feature will likely be added to edition 2024 to allow for migration off this legacy behavior.
PiperOrigin-RevId: 622260327
For now, these are limited to tests of text-form for delimited fields that locks down our problematic behavior in editions. Follow up changes will adjust the behavior to behave better under editions.
PiperOrigin-RevId: 622211473
A few of our generators previously used proto2 group's message name to generate code. In order to keep these backwards compatible but still functional under editions, we define a new IsGroupLike helper function. Message fields consistent with proto2 groups will continue using the old generated APIs, but all new ones possible under editions will correctly use the field name.
Edition 2024 will likely include a feature to allow for incremental migrations off this pattern, and follow-up changes will address related text-format issues.
PiperOrigin-RevId: 622204864
They are not needed after the rules are move into protobuf repo.
Except for the reference to toolchain type, which is currently in rules_proto and can be moved after the implementation is moved into protobuf repo.
PiperOrigin-RevId: 622176865
In the _RegularMessageToJsonObject method, there is no str type conversion for non-bytes type values in the repeated content. This causes an exception in the MessageToJson method, as the jsons data does not allow for the occurrence of byte array type property values.
BUG EG (contains proto and python code)：
------------common.proto------------
message SafetyInfo{//
repeated LoginDevice deviceList = 1;
}
message LoginDevice {
optional string uuid = 1 [default = ""];
optional string deviceName = 2 [default = ""];
optional string deviceType = 3 [default = ""];
required uint32 lastTime = 4;
}
------------test code(python)------------
from google.protobuf.json_format import MessageToJson
import common_pb2 # generate by common.proto
pb = common_pb2.SafetyInfo()
pb_hex = "0a4e0a2039323833663530356533363332396338356638623866343832613561323061651211636d4a38426c307a4d20446576696365731a0f6950686f6e6520694f532031332e3720dc85a9b00628010a410a206233356161326632366236343966313536393466663761336263303434323163120950432dcdacd0cbb4ef1a0a57696e646f777320313020d5d2c6e705280010001800200028003001" pb.ParseFromString(bytes.fromhex(pb_hex))
print(pb)
print(MessageToJson(pb))
Closes#16382
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/16382 from zhangzibao:main 3af2569265
PiperOrigin-RevId: 622156823
This will allow users to export any metadata attached to this descriptor without also constructing every nested descriptor. This is consistent with FileDescriptor::CopyHeadingTo, which serves a similar purpose at a different scope.
PiperOrigin-RevId: 621995153
This is needed after python/dist:dist's limited_api_wheels was updated from 3.7->3.8 and was causing us to mislabel the python_tag for our 3.8 limited api wheels as the system python version of 3.7.
Also:
* Add LINT to prevent next time
* Clean up some documentation
PiperOrigin-RevId: 621915013
function call with precalculated values.
We have to query every field anyway, so better to do it once at the start and
keep the results.
PiperOrigin-RevId: 621650969
All parsing now happens via a TcParser table, accessible from `ClassData` either directly or through the `get_tc_table` callback.
All codegen types insert their generated table in their ClassData. Dynamic types use the callback to generate it on demand.
PiperOrigin-RevId: 621632525
I am introducing custom field options, and subsequently message options as extensions to a new options.proto file. I would like to reserve extension number for the same.
Closes#16378
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/16378 from shashankram:add-ext 803510526b
PiperOrigin-RevId: 621548777
This is needed to make protobuf/bazel package minimal for other proto rules.
Keep 4 public bzl files in upb/bazel. They end up under protobuf/bazel, and they are legitimately used by other repositories.
Move upb_proto_library_internal/* under bazel/private. Those are utilities used in the rules. Moving them one level deeper makes protobuf/bazel package clean for other rules.
Move build_defs.bzl and amalgamation under /upb/bazel. Those are utilities used in the build.
Move lua.BUILD and python* uner /python/dist. Those are used in the WORKSPACE dependency setup.
PiperOrigin-RevId: 621442236
The “in” operator will be consistent with HasField but a little different with Proto Plus.
The detail behavior of “in” operator in Nextgen for Struct (to be consist with old Struct behavior):
-Raise TypeError if not pass a string
-Check if the key is in the struct.fields
The detail behavior of “in” operator in Nextgen(for other message):
-Raise ValueError if not pass a string
-Raise ValueError if the string is not a field
-For Oneof: Check any field under the oneof is set
-For has-presence field: check if set
-For non-has-presence field (include repeated fields): raise ValueError
PiperOrigin-RevId: 621240977
Mike Kruskal
Protobuf Team Bot
Protobuf Team Bot
Dave MacLachlan
zhangzibao
Sandy Zhang
Marcel Hlopko
Shashank Ram
Jason Lunn
Jie Luo
Hong Shin