This is just part of an effort at locking down (and understand well) how map
entries are serialized into textproto when they have empty fields.
Right now the C++ behaviour for DynamicMessages is that for implicit-presence
(i.e. "proto3") fields, the empty MapEntry fields are not serialized even if
explicitly set. For example, `value: ""` would not show up in textproto
serialization for proto3 MapEntry messages. This contrasts with C++ map
reflection behaviour, because C++ MapEntry messages are always generated with
hasbits (even if they are declared in a proto3 file and have implicit presence
according to their descriptors).
For this Python test, the DynamicMessage fallback path was triggered because
the implementation of the C++ proto was not found. When python is lacking in
implementation, it calls DynamicMessage which respects field presence, even for
map entries. If the corresponding `map_unittest_cc_proto` is linked, this test
actually fails, because the C++ MapEntry implementation, which is always
generated with hasbits even for implicit presence, is used.
PiperOrigin-RevId: 681944020
This should rename the self parameter name to "self_" if any field is named "self", not rename the corresponding keyword parameter name to "self_" if the first field is named "self".
PiperOrigin-RevId: 681872250
This is a very narrow edge case where touching a packed extension via generated APIs first, and then doing so reflectively will trigger a DCHECK. Otherwise, reflective APIs will work but not use packed encoding for the extension. This was likely a pre-existing bug dating back to proto3, where it would only be visible on custom options (the only extensions allowed in proto3).
To help qualify this and uncover similar issues, unittest.proto was migrated to editions. This turned up some other minor issues in DebugString and python.
PiperOrigin-RevId: 675785611
In the worst case, the source object is a constant (eg a global default instance) and modifying them has undefined behavior.
PiperOrigin-RevId: 673402981
This enables enforcement of lifetime specifications on individual enum values for features. It will allow us to add new values to existing features, as well as deprecate/and remove existing values. By default, each value will be scoped to the lifetime spec of its corresponding feature field. However, individual lifetime boundaries can be overridden at the value-level for finer grained control.
In the near-term, this will allow us to deprecate/remove required field presence, and add a stricter utf8 validation feature.
PiperOrigin-RevId: 672710484
This CL is mostly a no-op, except that now google3-only code is actually stripped from OSS, instead of being preserved in `# begin:google_only` blocks.
This follows the conventions of the greater Copybara ecosystem.
PiperOrigin-RevId: 669513564
This simplifies upb by removing differences between google3 and OSS.
This also points upb at the protobuf license, instead of keeping a separate copy around for upb.
PiperOrigin-RevId: 669447145
These are very old unit-tests that appear to have been written to provide some conformance and fuzzing capabilities before we had more comprehensive solutions. Today, they should be already covered by our conformance tests and fuzz tests and create unnecessary noise.
The goal of this change is to remove the checked-in binary golden data that we no longer know how to regenerate. There is a lot of nearby code that could likely also be cleaned up in a follow-up
PiperOrigin-RevId: 667604310
Our bootstrapping setup compiles multiple versions of the generated code for `descriptor.proto` and `plugin.proto`, one for each stage of the bootstrap. For source files (`.c`), we can always select the correct version of the file in the BUILD rules, but for header files we need to make sure the correct stage's file is always selected via `#include`.
Previously we used `cc_library(includes=[])` to make it appear as though our bootstrapped headers had the same names as the "real" headers. This allowed a lot of the code to be agnostic to whether a bootstrap header was being used, which simplified things because we did not have to change the code performing the `#include`.
Unfortunately, due to build system limitations, this sometimes led to the incorrect header getting included. This should not have been possible, because we had a clean BUILD graph that should have removed all ambiguity about which header should be available. But in non-sandboxed builds, the compiler was able to find headers that were not actually in `deps=[]`, and worse it preferred those headers over the headers that actually were in `deps=[]`. This led to unintended results and errors about layering check violations.
This CL fixes the problem by removing all use of `includes=[]`. We now spell a full pathname to all bootstrap headers, so this class of errors is no longer possible. Unfortunately this adds some complexity, as we have to hard-code these full paths in several places.
A nice improvement in this CL is that `bootstrap_upb_proto_library()` can now only be used for bootstrapping; it only exposes the `descriptor_bootstrap.h` / `plugin_bootstrap.h` files. Anyone wanting to use the normal `net/proto2/proto/descriptor.upb.h` file should depend on `//net/proto2/proto:descriptor_upb_c_proto` target instead.
PiperOrigin-RevId: 664953196
Pure python and upb do not support it and filtered out the test. This API does
not exists in any other language(except protobuf c++).
GetDebugString() for cpp extension will be removed in Jan 2025
PiperOrigin-RevId: 662640110
-Raise warnings for deprecated google/protobuf/service.py APIs.
service.py APIs are marked as deprecated since 2010. These APIs will be
removed in Jan 2025
PiperOrigin-RevId: 653280370
Python dict is now able to be assigned (by create and copy, not reference) and compared with the Protobuf Struct field.
Python list is now able to be assigned (by create and copy, not reference) and compared with the Protobuf ListValue field.
example usage:
dictionary = {'key1': 5.0, 'key2': {'subkey': 11.0, 'k': False},}
list_value = [6, 'seven', True, False, None, dictionary]
msg = more_messages_pb2.WKTMessage(
optional_struct=dictionary, optional_list_value=list_value
)
self.assertEqual(msg.optional_struct, dictionary)
self.assertEqual(msg.optional_list_value, list_value)
PiperOrigin-RevId: 646099987
This was previously fixed in C++ (https://github.com/protocolbuffers/protobuf/issues/16549), but not ported to other languages. Delimited field encoding can be inherited by fields where it's invalid, such as non-messages and maps. In these cases, the encoding should be ignored and length-prefixed should be used.
PiperOrigin-RevId: 642792988
There is a special case where message factories can be confused: if a module
written in C++ with pybind11 links against a self-recursive message, and that
message is part of another message loaded from Python, then the confusion
will happen.
Example:
# This one is also linked into the C++ module.
message SelfRecursive {
optional SelfRecursive self_recursive = 1;
}
# This one is used only in Python and not linked.
message OnlyUsedInPython {
optional SelfRecursive self_recursive = 2;
}
The caching through message_factory::RegisterMessageClass then happens on one
instance of the factory, but traversal with the lookup in another.
This occurs in the pure Python and upb implementations that have their own
default descriptor pools (and thus message factory).
Fix this by using the already passed message factory to registering the
message class to cache.
A test accounts for this case to avoid regressions.
PiperOrigin-RevId: 642551744
Timestamp and Duration are now have more support with datetime and timedelta:
- Allows assign python datetime to protobuf DateTime field in addition to current FromDatetime/ToDatetime (Note: will throw exceptions for the differences in supported ranges)
- Allows assign python timedelta to protobuf Duration field in addition to current FromTimedelta/ToTimedelta
- Calculation between Timestamp, Duration, datetime and timedelta will also be supported.
example usage:
from datetime import datetime, timedelta
from event_pb2 import Event
e = Event(start_time=datetime(year=2112, month=2, day=3),
duration=timedelta(hours=10))
duration = timedelta(hours=10))
end_time = e.start_time + timedelta(hours=4)
e.duration = end_time - e.start_time
PiperOrigin-RevId: 640639168
Tony Liao
Samuel Freilich
Mike Kruskal
Nathan Baulch
Protobuf Team Bot
Mike Kruskal
Hood Chatham
Joshua Haberman
Jie Luo
zhangskz
jensbjorgensen