https://github.com/protocolbuffers/protobuf/pull/13204 refactored the Ruby object cache to use a key of `LL2NUM(key_val)` instead of `LL2NUM(key_val >> 2)`. On 32-bit systems, `LL2NUM(key_val)` returns inconsistent results because a large value has to be stored as a Bignum on the heap. This causes cache lookups to fail.
This commit restores the previous behavior of using `ObjectCache_GetKey`, which discards the lower 2 bits, which are zero. This enables a key to be stored as a Fixnum on both 32 and 64-bit platforms.
As https://patshaughnessy.net/2014/1/9/how-big-is-a-bignum describes, a Fixnum uses:
* 1 bit for the `FIXNUM_FLAG`.
* 1 bit for the sign flag.
Therefore the largest possible Fixnum value on a 64-bit value is 4611686018427387903 (2^62 - 1). On a 32-bit system, the largest value is 1073741823 (2^30 - 1).
For example, a possible VALUE pointer address on a 32-bit system:
0xff5b4af8 => 4284173048
Dropping the lower 2 bits makes up for the loss of range to these flags. In the example above, we see that shifting by 2 turns the value into a 30-bit number, which can be represented as a Fixnum:
(0xff5b4af8 >> 2) => 1071043262
This bug can also manifest on a 64-bit system if the upper bits are 0xff.
Closes#13481Closes#13494
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13494 from stanhu:sh-fix-ruby-protobuf-32bit d63122a6fc
PiperOrigin-RevId: 557211768
This addresses #13441. Second try, now with more internal fixes.
This preserves the similar check at the _point of use_ of invalid messages in `DescriptorBuilder` (and there's an existing test that verifies that check still works).
But it adds another check in the parser, to catch this error at the _point of definition_ of an invalid message. And the corresponding test is updated: we no longer need a usage of the message to catch the error, and the reported position is the definition of the option, not the usage site of the message.
The way this works feels kinda gross, but I wasn't sure of a better way to do it. The only place we know for certain that it was an explicit option (vs. auto-added by the parser when synthesizing a map entry message) is when after processing the message body, we can look at the uninterpreted options. So that's what this does. If you have ideas on better/cleaner approaches, I'd be happy to revise.
Closes#13479
PiperOrigin-RevId: 557199190
This avoids a call to mutable(), and results in smaller code, branching on the field being not default, which should be a well predicted, rarely taken branch as the probability of a split being instantiated with more than 1 field is very low.
PiperOrigin-RevId: 557132505
This will replace the frontend string_field_validation field, which we've discovered can't accurately model all of the unique behaviors in each language.
PiperOrigin-RevId: 556996541
This addresses #13442. This also backfills the tests to add tests for the two checks that were already implemented as well as one for the newly added check.
Finally, this fixes the location information so that positions for reserved ranges are correctly reported. (The previous check that already existed, for enums, failed to show line and column info.)
Closes#13474
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13474 from jhump:jh/validate-msg-reserved-range-order 661adc745a
PiperOrigin-RevId: 556979032
There is a new boolean ObjC Generator Option called
`strip_custom_options` that defaults to `True` that causes all
extensions for the descriptor custom options to not generated.
See https://protobuf.dev/programming-guides/proto2/#customoptions
for more information on these. Since the ObjC runtime doesn't
capture this data, we're not aware of any reason they would be need.
If you do enable this option, please open an issue and let us know
as we'd rather remove this option and have this be the only
behavior.
PiperOrigin-RevId: 556805883
Scrub all the objc related sources for references to "syntax"
and update them to no longer tie things accordingly.
The ObjC Protos `GPBFileDescriptor` concept still exposes the
`syntax`, but it was already marked as deprecated. This will be
removed in the future as nothing should need to look at the value.
Validate that all generation calls are on the helpers to get the
editions support from the C++ code.
PiperOrigin-RevId: 555971288
This proto is only used in C++ tests, and doesn't need to be built in python. Python doesn't support editions yet, so we should just remove it for now. This also fixes some other issues with setup.py where tests fail.
Closes#13501
PiperOrigin-RevId: 555681295
ctype can not be used for none string/bytes fields. ctye=CORD can not be used for extensions. A new macro PROTOBUF_FUTURE_REMOVE_WRONG_CTYPE is added that will be flipped for our next breaking release.
PiperOrigin-RevId: 555615362
This addresses #13441.
This preserves the similar check at the _point of use_ of invalid messages in `DescriptorBuilder` (and there's an existing test that verifies that check still works).
But it adds another check in the parser, to catch this error at the _point of definition_ of an invalid message. And the corresponding test is updated: we no longer need a usage of the message to catch the error, and the reported position is the definition of the option, not the usage site of the message.
The way this works feels kinda gross, but I wasn't sure of a better way to do it. The only place we know for certain that it was an explicit option (vs. auto-added by the parser when synthesizing a map entry message) is when after processing the message body, we can look at the uninterpreted options. So that's what this does. If you have ideas on better/cleaner approaches, I'd be happy to revise.
Closes#13479
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13479 from jhump:jh/map-option-not-allowed 4d95e9b839
PiperOrigin-RevId: 555577734
This is in preparation for moving upb into the protobuf repo. Upb requires C++
17. Protobuf supports C++ 14, and our non-upb CI tests will keep using C++ 14
to ensure that it still works. Our Bazel CI runs use the bazelrc files in the
ci/ directory, so these runs should be unaffected by this change.
PiperOrigin-RevId: 555223181
Only emit has_field() if the field support presence. Only emit field_opt() getter if the field is both optional and supports presence.
PiperOrigin-RevId: 555133374
Jonas Kvinge
Stan Hu
Sandy Zhang
Matt Kulukundis
Protobuf Team Bot
Martijn Vels
Mike Kruskal
Joshua Humphries
Protobuf Team Bot
Thomas Van Lenten
Adam Cozzette
zhangskz
Jie Luo
ericsalo
Mike Kruskal
Eric Salo