When adding UPB as a backend to to reflection_test, I found that the test had
lots of if-conditions that skip certain sections of the test based on an
assumption that anything where api_implementation.Type() != 'python' is C++.
With UPB in the mix, that's no longer true. So it seemed like a good
opportunity to go in and lock down the nonconformance a bit tighter.
As it turns out, some of the nonconformance has been fixed already over the
past 10 years. :) So we can simply remove some of the if checks and enjoy a
better conformance test coverage across all python backends!
PiperOrigin-RevId: 712945117
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
Since this is the last remaining test file in the pb_unit_tests directory, I
have removed the entire directory. Future UPB tests will be in the same
directory as C++ and pure python implementations, namely:
//python/google/protobuf/internal.
Temporarily, we will need to hardcode the migrated test name in test_upb.yml
because not all tests under google.protobuf.internal support UPB yet.
(UPB testing for selected tests are added in 21e9aa6cac).
The reflection_test is a slightly more complicated test. In this commit I've
removed exemption of testParsingNestedClass in reflection_test. A follow-up
commit will ensure further conformance within the reflection_test. There are
quite a few if-conditions that were added to skip certain checks due to
nonconformant behavior -- they seem to no longer apply.
PiperOrigin-RevId: 712943652
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
Temporarily, we will need to hardcode the migrated test name in test_upb.yml
because not all tests under google.protobuf.internal support UPB yet.
(UPB testing for selected tests are added in 21e9aa6cac).
PiperOrigin-RevId: 712672890
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
Temporarily, we will need to hardcode the migrated test name in test_upb.yml
because not all tests under google.protobuf.internal support UPB yet.
(UPB testing for selected tests are added in 21e9aa6cac).
PiperOrigin-RevId: 711837521
I just tried removing a bunch of if api_implementation.Type() guards in
message_test.py, and a bunch of them just pass without specialization! The bugs
that used to cause trouble seem to be now fixed.
PiperOrigin-RevId: 711764185
Update Message.__dir__ method to ensure all proto fields of a message
are included in the list of attributes. This solves the inconsistency
between the Fast C++ and UPB proto implementation. The Fast C++
implementation was always returning fields while UPB never. Improve
a unit test to detect that.
PiperOrigin-RevId: 708262631
Checking whether the object is a numpy array checking is pretty expensive, so we only want to do it when the object is not trivially a float.
This makes appending to a repeated field of float ~3x faster in a common case:
```
_bench_append, 1000000, 284.1
_bench_extend, 1000000, 209.0
_bench_assign, 1000000, 175.8
_bench_pybind11, 1000000, 3.7
```
```
_bench_append, 1000000, 128.4
_bench_extend, 1000000, 67.1
_bench_assign, 1000000, 57.2
_bench_pybind11, 1000000, 3.5
```
PiperOrigin-RevId: 707811151
I've been gradually moving the location of our UPB tests to make them "more
default" (see an example here:
a02ec0f353)
It turns out that reduced some of our open-source coverage around UPB python
unit tests. In this commit, I temporarily hard-code the tests I've migrated,
and eventually I'll change it into a wildcard expansion to be more robust. We
can't do wildcards right now because not all tests in the
google.protobuf.internal namespace support UPB by default yet.
#test-continuous
PiperOrigin-RevId: 707661811
The previous treatment was a conformance violation, where implicit present float fields with a non-default value of -0.0 could get dropped.
PiperOrigin-RevId: 705728806
This is only used in some narrow edge cases, and is a less-safe version of our unknown field decoders. Switching to those reduces some duplication and improves error handling.
PiperOrigin-RevId: 704899254
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
PiperOrigin-RevId: 702723759
Enable Bazel 7 macOS test coverage which otherwise fails with
```
Undefined symbols for architecture arm64:
"_PyModule_AddIntConstant", referenced from:
_PyInit__api_implementation in api_implementation.o
"_PyModule_Create2", referenced from:
_PyInit__api_implementation in api_implementation.o
"__Py_Dealloc", referenced from:
_PyInit__api_implementation in api_implementation.o
ld: symbol(s) not found for architecture arm64
```
Fixes https://github.com/protocolbuffers/protobuf/issues/19454
#test-continuous
PiperOrigin-RevId: 702375059
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
PiperOrigin-RevId: 700829382
Add a new implementation of Message.__dir__ method that filters out
attributes which are not accessible. Such an attribute is currently
only `Extensions`, which is only accessible if any extension exists.
Add a unit test that verifies all remaining attributes can be accessed.
PiperOrigin-RevId: 700778715
Before, the upb_ExtensionRegistry_AddArray API would just return a boolean
indicating whether the operation succeeded or failed. This is not descriptive
enough in some cases and made the error output confusing.
Specifically, when trying to register an extension with a duplicate extension
number, AddArray first performs a map lookup before inserting the extension
entry into the registry. The code handled lookup failure (due to duplicates)
the same way as insertion failure (due to OOM), and printed an error message
that showed OOM when there is a duplicate array entry.
This was acknolwedged in a TODO in the AddArray code comment -- which is now
fixed. :)
PiperOrigin-RevId: 700764584
`PyObject_HEAD` already expands to a field with a semicolon; the extra semicolon is unnecessary, and makes some compilers unhappy.
Closes#16123
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/16123 from goffrie:pyobject_head 947438678f
PiperOrigin-RevId: 700735003
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
PiperOrigin-RevId: 700127942
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
PiperOrigin-RevId: 699321395
Struct has a default bool function, and empty Struct returns False for bool().
Should compare with None instead of bool() when check if Struct is able to be
created.
PiperOrigin-RevId: 698205106
-setdefault will be similar with dict for ScalarMap. But both key and value must be set.
-setdefault will be rejected for MessageMap.
PiperOrigin-RevId: 695768629
The old behavior requires a prefix of '.' for empty packages like '.Foo'. To not break old behaviors, both 'Foo' and '.Foo' will be supported. The support for leading '.' will be removed around 2026 Jan.
For example:
db.FindFileContainingSymbol('Foo')
db.FindFileContainingSymbol('.Foo') # with a warning
will have same result
PiperOrigin-RevId: 695493356
Considering that protobuf depends on absl already, we don't need protobuf's
version of PREDICT_TRUE|FALSE. This CL shrinks port_def.inc.
PiperOrigin-RevId: 694015588
cpp extension added the API in https://github.com/protocolbuffers/protobuf/pull/7498
Pure python and upb do not support it and filtered out the test
This API does not exists in any other language except C++.
PiperOrigin-RevId: 691870623
Jie Luo
Tony Liao
Matej Aleksandrov
Protobuf Team Bot
Mike Kruskal
Sandy Zhang
Geoffry Song
Zoey Greer
gongsu832
Nevena Kotlaja