The new naming will use our new vocabulary, and restrict feature access:
Unresolved source features - source-retention features that generators need to validate their own features. These should not be sent to the runtime or used for any decision-making.
Resolved source features - fully resolved source-retention features. These should be used to make codegen decisions, but shouldn't be sent to runtimes.
PiperOrigin-RevId: 558287302
This is a no-op change. Since Copybara will automatically format PRs with ClangFormat, this can lead to spurious changes that make CLs harder to review, as in cl/549058046. To avoid this, we format everything with clang-format in a single no-op change.
This CL also forces all pointer alignment to left pointers, for consistency (clang-format would normally deduce based on existing style in each file).
Also added `// clang-format off` comments in some places where PHP conventions lead to badly formatted code.
PiperOrigin-RevId: 558260317
Remove array.h and map.h as hdrs from :collections_internal
Remove alloc.h and arena.h as hdrs from :mem_internal (and add them to :mem)
Remove common.h and decode.h and encode.h as hdrs from :wire_internal
Lock down the visibility of :wire_internal to upb-only
Merge :mini_descriptor_encode_internal into :mini_descriptor_internal
PiperOrigin-RevId: 558235138
There are still a bunch more test runs to migrate, but this change adds a few
more that were unblocked by the `exclude-targets` flag.
PiperOrigin-RevId: 558206606
This allows the compiler to statically detect use-after-free bugs.
This change touches a subset of field types. More changes to follow.
PiperOrigin-RevId: 558180721
My editor reformatted this file to add a missing newline, and this ended up
breaking the test. The added newline caused there to be a newline in the middle
of the Bazel command. So this change adds the missing newline and fixes up the
Bazel command by using the `>-` syntax to treat this as a multiline string
without adding any newlines.
PiperOrigin-RevId: 558149295
This version is a light-weight, inlined version implemented in terms of `MergeFrom` that extends the protocol buffer API with arena enabled copy construction, but without any explicit (optimized) implementation.
Going forward the protocol compiler will provide an optimized implementation that is expected to save a non trivial amount of compute.
PiperOrigin-RevId: 558124029
Since protoc can't know the full set of features that matter to a plugin, sending partially resolved features is unnecessarily confusing. Plugins written in C++ will have automatic access to the fully resolved features via previous enhancements. Plugins written in other languages will need to reimplement feature resolution logic. Future changes will document this algorithm and provide conformance testing.
PiperOrigin-RevId: 557969789
TDP replaced most of the class' use. MapFieldLite does not need it anymore.
We can also simplify MapFieldLite by removing some now unused template parameters.
PiperOrigin-RevId: 557599609
Notably, @jhump identified the following holes:
- Non-messages can have message_encoding set
- Extensions can have implicit presence set
- Repeated fields can have explicit presence set
PiperOrigin-RevId: 557571375
This change moves the upb Fastbuild, Optimized, and FastTable test runs over to
the protobuf repo CI in preparation for moving the upb codebase itself. There
are a bunch more test runs to move, but this initial change handles the easy
ones first.
I also updated our pinned upb version to the current head to pick up some
recent fixes.
PiperOrigin-RevId: 557486174
This will allow us to run the CMake test from within the protobuf repo's Bazel
workspace. The directory structure is a little bit different depending on which
workspace the test is invoked in.
PiperOrigin-RevId: 557275295
Every language has very different handling of utf8 validation. Any with proto2/proto3 differences will receive language-specific features for edition zero to better model these subtle differences.
PiperOrigin-RevId: 557258923
This helps make the API more complete, since the FeatureSet object will always be fully resolved on any accessible features. This specifically targets C++ plugins though, which will now have their features filled in by default. Before, any proto files that didn't include the language-specific features would result in unresolved extensions in the generators.
PiperOrigin-RevId: 557232654
I found that this test fails when I try to run it from our protobuf repo CI
setup, probably because rsync is not available in the Docker image there. Let's
just use `cp` instead so that the test works even if rsync is unavailable.
PiperOrigin-RevId: 557225733
https://github.com/protocolbuffers/protobuf/pull/13204 refactored the Ruby object cache to use a key of `LL2NUM(key_val)` instead of `LL2NUM(key_val >> 2)`. On 32-bit systems, `LL2NUM(key_val)` returns inconsistent results because a large value has to be stored as a Bignum on the heap. This causes cache lookups to fail.
This commit restores the previous behavior of using `ObjectCache_GetKey`, which discards the lower 2 bits, which are zero. This enables a key to be stored as a Fixnum on both 32 and 64-bit platforms.
As https://patshaughnessy.net/2014/1/9/how-big-is-a-bignum describes, a Fixnum uses:
* 1 bit for the `FIXNUM_FLAG`.
* 1 bit for the sign flag.
Therefore the largest possible Fixnum value on a 64-bit value is 4611686018427387903 (2^62 - 1). On a 32-bit system, the largest value is 1073741823 (2^30 - 1).
For example, a possible VALUE pointer address on a 32-bit system:
0xff5b4af8 => 4284173048
Dropping the lower 2 bits makes up for the loss of range to these flags. In the example above, we see that shifting by 2 turns the value into a 30-bit number, which can be represented as a Fixnum:
(0xff5b4af8 >> 2) => 1071043262
This bug can also manifest on a 64-bit system if the upper bits are 0xff.
Closes#13481Closes#13494
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13494 from stanhu:sh-fix-ruby-protobuf-32bit d63122a6fc
PiperOrigin-RevId: 557211768
This addresses #13441. Second try, now with more internal fixes.
This preserves the similar check at the _point of use_ of invalid messages in `DescriptorBuilder` (and there's an existing test that verifies that check still works).
But it adds another check in the parser, to catch this error at the _point of definition_ of an invalid message. And the corresponding test is updated: we no longer need a usage of the message to catch the error, and the reported position is the definition of the option, not the usage site of the message.
The way this works feels kinda gross, but I wasn't sure of a better way to do it. The only place we know for certain that it was an explicit option (vs. auto-added by the parser when synthesizing a map entry message) is when after processing the message body, we can look at the uninterpreted options. So that's what this does. If you have ideas on better/cleaner approaches, I'd be happy to revise.
Closes#13479
PiperOrigin-RevId: 557199190
This avoids a call to mutable(), and results in smaller code, branching on the field being not default, which should be a well predicted, rarely taken branch as the probability of a split being instantiated with more than 1 field is very low.
PiperOrigin-RevId: 557132505
Mike Kruskal
Joshua Haberman
Protobuf Team Bot
Eric Salo
Adam Cozzette
Protobuf Team Bot
Martijn Vels
Jonas Kvinge
Stan Hu
Sandy Zhang
Matt Kulukundis