Features need to be validated within the pool being built, since the generated pool only contains extensions linked into the binary (e.g. protoc or a runtime building dynamic protos). The generated pool may be missing extensions used in this proto or it may have version skew. Moving to the build pool requires reflective parsing, which in general can't be done from inside the pool's database lock. This required some refactoring to add a post-build validation phase outside of the lock.
For now, the feature support spec is optional and the checks only are only applied when it's present. Follow-up changes will add these specs to our existing features and then require them for all FeatureSet extensions.
PiperOrigin-RevId: 623630219
This updates all our text parsers and serializers to better handle tag-delimited fields under editions. Under proto2, groups were the only tag-delimited fields possible, and the group name (i.e. the message type) was guaranteed to be unique. Text-format and various generators used this instead of the synthetic field name (lower-cased group name) to represent these fields.
Under editions, we've removed group syntax and allowed any message field to be tag-delimited. This breaks those cases when adding new tag-delimited fields where the message type might not be unique or correspond to the field name. Code generators have already been fixed to treat "group-like" fields using the old behavior, and treat new fields like any other sub-message.
This change addresses the text-format issue. Text parsers will accept *either* the type or field name for "group-like" fields, and only the field name for every other message field. Text serializers will continue to emit the message name for "group-like" fields, but also use the field name for everything else.
This creates some awkward capitalization behavior for fields that happen to *look* like proto2 groups, but it won't lead to any conflicts or invalid encodings. A feature will likely be added to edition 2024 to allow for migration off this legacy behavior.
PiperOrigin-RevId: 622260327
In the _RegularMessageToJsonObject method, there is no str type conversion for non-bytes type values in the repeated content. This causes an exception in the MessageToJson method, as the jsons data does not allow for the occurrence of byte array type property values.
BUG EG (contains proto and python code)：
------------common.proto------------
message SafetyInfo{//
repeated LoginDevice deviceList = 1;
}
message LoginDevice {
optional string uuid = 1 [default = ""];
optional string deviceName = 2 [default = ""];
optional string deviceType = 3 [default = ""];
required uint32 lastTime = 4;
}
------------test code(python)------------
from google.protobuf.json_format import MessageToJson
import common_pb2 # generate by common.proto
pb = common_pb2.SafetyInfo()
pb_hex = "0a4e0a2039323833663530356533363332396338356638623866343832613561323061651211636d4a38426c307a4d20446576696365731a0f6950686f6e6520694f532031332e3720dc85a9b00628010a410a206233356161326632366236343966313536393466663761336263303434323163120950432dcdacd0cbb4ef1a0a57696e646f777320313020d5d2c6e705280010001800200028003001" pb.ParseFromString(bytes.fromhex(pb_hex))
print(pb)
print(MessageToJson(pb))
Closes#16382
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/16382 from zhangzibao:main 3af2569265
PiperOrigin-RevId: 622156823
The “in” operator will be consistent with HasField but a little different with Proto Plus.
The detail behavior of “in” operator in Nextgen for Struct (to be consist with old Struct behavior):
-Raise TypeError if not pass a string
-Check if the key is in the struct.fields
The detail behavior of “in” operator in Nextgen(for other message):
-Raise ValueError if not pass a string
-Raise ValueError if the string is not a field
-For Oneof: Check any field under the oneof is set
-For has-presence field: check if set
-For non-has-presence field (include repeated fields): raise ValueError
PiperOrigin-RevId: 621240977
Features are designed as temporary migration tools, and any unbounded type leaves the system open to unexpected use. Features should have a fixed set of values, with well defined behaviors.
PiperOrigin-RevId: 617933544
This prevents these from using abc.ABCMeta metaclass to avoid deprecation warning:
```
DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.
```
Fixes#15077Fixes#12186
PiperOrigin-RevId: 613029479
The replacement always_print_without_presence_fields should be used instead, which is very similar but has consistent handling of optional fields by not affecting them.
PiperOrigin-RevId: 604381178
This flag has consistent behavior between proto2 and proto3 optionals (by not including either one), unlike including_default_value_fields which does include proto2 optional but excludes proto3 optionals.
including_default_value_fields is now deprecated and will be removed in an upcoming release.
PiperOrigin-RevId: 603156447
Also hardened the text format printer against invalid UTF-8 in string fields. The output string will always be valid UTF-8, even if string fields contain invalid UTF-8.
PiperOrigin-RevId: 600990001
Include AddFileDescriptor, AddDescriptor, AddEnumDescriptor,
AddExtensionDescriptor, AddServiceDescriptor.
Those Deprecated APIs may add unlinked descriptors to descriptor_pool which is
is wrong. Should use Add() or AddSerializedFile() instead. Those APIs were
raising deprecated warnings since 2019
PiperOrigin-RevId: 595831718
Users should use the add-on unknown_fields.py support.
Old usage example:
unknown_field_set = msg.UnknownFields()
New usage should be:
from google.protobuf import unknown_fields
unknown_field_set = unknown_fields.UnknownFieldSet(msg)
PiperOrigin-RevId: 589969095
This has been replaced by edition and feature getters. Any code depending on syntax will likely be broken by editions files, and should be migrated to the finer-grained feature helpers.
PiperOrigin-RevId: 589866745
As a prerequisite of enabling go/protobuf-weak-speed-messages downcasts using built-in cast operations need to be changed to cast functions recommended by protobuf team (go/protobuf-downcast-recommendation)
**More information**: go/protobuf-stripping-lsc
#codehealth
PiperOrigin-RevId: 582977861
This also fixes a few minor bugs in the editions implementation that were caught in python/conformance tests, and adds a new SetFeatureSetDefaults API to the def pool for consistency with C++ and other python implementations.
PiperOrigin-RevId: 581384108
This was already fully implemented in C++, but we need to expose features methods to the Python runtime to fully enable it. This also enables conformance and unit-testing for C++.
PiperOrigin-RevId: 581364355
This change only covers pure python, and follow-up changes will handle C++/upb variants and actually enable editions support. The C++ one works (as evident from the conformance tests), but needs some APIs added to allow for testing.
PiperOrigin-RevId: 580304039
Mike Kruskal
zhangzibao
Jie Luo
Anton Grbin
Eric Salo
Protobuf Team Bot
Sandy Zhang
Dennis Shao
Joshua Haberman
Dmitri Gribenko
Jie Luo
Protobuf Team Bot