Notably, @jhump identified the following holes:
- Non-messages can have message_encoding set
- Extensions can have implicit presence set
- Repeated fields can have explicit presence set
PiperOrigin-RevId: 557571375
This change moves the upb Fastbuild, Optimized, and FastTable test runs over to
the protobuf repo CI in preparation for moving the upb codebase itself. There
are a bunch more test runs to move, but this initial change handles the easy
ones first.
I also updated our pinned upb version to the current head to pick up some
recent fixes.
PiperOrigin-RevId: 557486174
This will allow us to run the CMake test from within the protobuf repo's Bazel
workspace. The directory structure is a little bit different depending on which
workspace the test is invoked in.
PiperOrigin-RevId: 557275295
Every language has very different handling of utf8 validation. Any with proto2/proto3 differences will receive language-specific features for edition zero to better model these subtle differences.
PiperOrigin-RevId: 557258923
This helps make the API more complete, since the FeatureSet object will always be fully resolved on any accessible features. This specifically targets C++ plugins though, which will now have their features filled in by default. Before, any proto files that didn't include the language-specific features would result in unresolved extensions in the generators.
PiperOrigin-RevId: 557232654
I found that this test fails when I try to run it from our protobuf repo CI
setup, probably because rsync is not available in the Docker image there. Let's
just use `cp` instead so that the test works even if rsync is unavailable.
PiperOrigin-RevId: 557225733
https://github.com/protocolbuffers/protobuf/pull/13204 refactored the Ruby object cache to use a key of `LL2NUM(key_val)` instead of `LL2NUM(key_val >> 2)`. On 32-bit systems, `LL2NUM(key_val)` returns inconsistent results because a large value has to be stored as a Bignum on the heap. This causes cache lookups to fail.
This commit restores the previous behavior of using `ObjectCache_GetKey`, which discards the lower 2 bits, which are zero. This enables a key to be stored as a Fixnum on both 32 and 64-bit platforms.
As https://patshaughnessy.net/2014/1/9/how-big-is-a-bignum describes, a Fixnum uses:
* 1 bit for the `FIXNUM_FLAG`.
* 1 bit for the sign flag.
Therefore the largest possible Fixnum value on a 64-bit value is 4611686018427387903 (2^62 - 1). On a 32-bit system, the largest value is 1073741823 (2^30 - 1).
For example, a possible VALUE pointer address on a 32-bit system:
0xff5b4af8 => 4284173048
Dropping the lower 2 bits makes up for the loss of range to these flags. In the example above, we see that shifting by 2 turns the value into a 30-bit number, which can be represented as a Fixnum:
(0xff5b4af8 >> 2) => 1071043262
This bug can also manifest on a 64-bit system if the upper bits are 0xff.
Closes#13481Closes#13494
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13494 from stanhu:sh-fix-ruby-protobuf-32bit d63122a6fc
PiperOrigin-RevId: 557211768
This addresses #13441. Second try, now with more internal fixes.
This preserves the similar check at the _point of use_ of invalid messages in `DescriptorBuilder` (and there's an existing test that verifies that check still works).
But it adds another check in the parser, to catch this error at the _point of definition_ of an invalid message. And the corresponding test is updated: we no longer need a usage of the message to catch the error, and the reported position is the definition of the option, not the usage site of the message.
The way this works feels kinda gross, but I wasn't sure of a better way to do it. The only place we know for certain that it was an explicit option (vs. auto-added by the parser when synthesizing a map entry message) is when after processing the message body, we can look at the uninterpreted options. So that's what this does. If you have ideas on better/cleaner approaches, I'd be happy to revise.
Closes#13479
PiperOrigin-RevId: 557199190
This avoids a call to mutable(), and results in smaller code, branching on the field being not default, which should be a well predicted, rarely taken branch as the probability of a split being instantiated with more than 1 field is very low.
PiperOrigin-RevId: 557132505
This will replace the frontend string_field_validation field, which we've discovered can't accurately model all of the unique behaviors in each language.
PiperOrigin-RevId: 556996541
This addresses #13442. This also backfills the tests to add tests for the two checks that were already implemented as well as one for the newly added check.
Finally, this fixes the location information so that positions for reserved ranges are correctly reported. (The previous check that already existed, for enums, failed to show line and column info.)
Closes#13474
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13474 from jhump:jh/validate-msg-reserved-range-order 661adc745a
PiperOrigin-RevId: 556979032
There is a new boolean ObjC Generator Option called
`strip_custom_options` that defaults to `True` that causes all
extensions for the descriptor custom options to not generated.
See https://protobuf.dev/programming-guides/proto2/#customoptions
for more information on these. Since the ObjC runtime doesn't
capture this data, we're not aware of any reason they would be need.
If you do enable this option, please open an issue and let us know
as we'd rather remove this option and have this be the only
behavior.
PiperOrigin-RevId: 556805883
Scrub all the objc related sources for references to "syntax"
and update them to no longer tie things accordingly.
The ObjC Protos `GPBFileDescriptor` concept still exposes the
`syntax`, but it was already marked as deprecated. This will be
removed in the future as nothing should need to look at the value.
Validate that all generation calls are on the helpers to get the
editions support from the C++ code.
PiperOrigin-RevId: 555971288
This proto is only used in C++ tests, and doesn't need to be built in python. Python doesn't support editions yet, so we should just remove it for now. This also fixes some other issues with setup.py where tests fail.
Closes#13501
PiperOrigin-RevId: 555681295
Mike Kruskal
Eric Salo
Protobuf Team Bot
Adam Cozzette
Jonas Kvinge
Stan Hu
Sandy Zhang
Matt Kulukundis
Martijn Vels
Joshua Humphries
Protobuf Team Bot
Thomas Van Lenten
zhangskz