This updates all our text parsers and serializers to better handle tag-delimited fields under editions. Under proto2, groups were the only tag-delimited fields possible, and the group name (i.e. the message type) was guaranteed to be unique. Text-format and various generators used this instead of the synthetic field name (lower-cased group name) to represent these fields.
Under editions, we've removed group syntax and allowed any message field to be tag-delimited. This breaks those cases when adding new tag-delimited fields where the message type might not be unique or correspond to the field name. Code generators have already been fixed to treat "group-like" fields using the old behavior, and treat new fields like any other sub-message.
This change addresses the text-format issue. Text parsers will accept *either* the type or field name for "group-like" fields, and only the field name for every other message field. Text serializers will continue to emit the message name for "group-like" fields, but also use the field name for everything else.
This creates some awkward capitalization behavior for fields that happen to *look* like proto2 groups, but it won't lead to any conflicts or invalid encodings. A feature will likely be added to edition 2024 to allow for migration off this legacy behavior.
PiperOrigin-RevId: 622260327
For now, these are limited to tests of text-form for delimited fields that locks down our problematic behavior in editions. Follow up changes will adjust the behavior to behave better under editions.
PiperOrigin-RevId: 622211473
They are not needed after the rules are move into protobuf repo.
Except for the reference to toolchain type, which is currently in rules_proto and can be moved after the implementation is moved into protobuf repo.
PiperOrigin-RevId: 622176865
Implementations should not rearrange unknown fields for the same field number, because some field types accept multiple wire types (eg. packed and unpacked).
All languages currently pass this test except Java "full" and Obj-C.
PiperOrigin-RevId: 619953711
Three of these runtimes are based on upb, and the fourth is based on the Java runtime. Both of these already have editions support, so this was mostly just a matter of:
- Advertising support to allow editions codegen
- Stripping features from the runtime options
- Hooking up conformance tests
- Adding some lightweight editions tests
There are also a few minor orthogonal fixes included here:
- Ruby's upb hack for treating all enums as open enums needed tweaking
- The `enable_editions` flag is no longer needed in our internal proto rules
PiperOrigin-RevId: 616256211
Until recently, these targets were dependencies of `//:protobuf` and could thus
be accessed through that target. But now that we are adhering to the layering
check, we need to provide proper access in a way that respects that check. This
change uses top-level alias targets following the existing pattern.
PiperOrigin-RevId: 605123979
# Motivation
https://github.com/protocolbuffers/protobuf/issues/7392#issuecomment-1884666885
This is for completeness - I am not blocked by this change in any way, so feel free to close the PR or treat is a low priority.
# Changes
Implement JsonIgnoreUnknownParsingTest in C# conformance test runner.
# Tested
Locally with:
```
bazel test //csharp:conformance_test \
--action_env=DOTNET_CLI_TELEMETRY_OPTOUT=1 --test_env=DOTNET_CLI_HOME=~ \
--action_env=DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1
```
Closes#15369
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15369 from noom:issue-7392/show-csharp-failures 3a652202a0
PiperOrigin-RevId: 602537927
This check enforces that each C++ build target has the correct dependencies for
all headers that it includes. We have many targets that were not correct with
respect to this check, so I fixed them up.
I also cleaned up the C++ targets related to the well-known types. I created a
cc_proto_library() target for each one and removed the :wkt_cc_protos target,
since this was necessary to satisfy the layering check. I deleted the
//src/google/protobuf:protobuf_nowkt target and deprecated :protobuf_nowkt,
because the distinction between the :protobuf and :protobuf_nowkt targets was
not really correct. Neither one exposed the headers for the well-known types in
a way that was valid with respect to the layering check, and the idea of
bundling all the well-known types together is not idiomatic in Bazel anyway.
This is a breaking change, because the //:protobuf target no longer bundles the
well-known types. From now on they should be accessed through the new
//:*_cc_proto aliases in our top-level package.
I renamed the :port_def target to :port, which simplifies things a bit by
matching our internal name.
The original motivation for this change was that to move utf8_range onto our CI
infrastructure, we needed to make its dependency rules_fuzzing compatible with
Bazel 6. The rules_fuzzing project builds with the layering check, and I found
that the process of upgrading it to Bazel 6 made it take a dependency on
protobuf, which caused it to break due to layering violations. I was able to
work around this, but it would still be nice to comply with the layering check
so that we don't have to worry about this kind of thing in the future.
PiperOrigin-RevId: 595516736
There is no canonical rules_ruby repo today, and we don't want our fork to become one. In order to unblock inclusion of Protobuf in the bzlmod registry, we're making this a dev dependency and dropping support for Bazel/Ruby.
Fixes#14569
PiperOrigin-RevId: 584393841
This also fixes a few minor bugs in the editions implementation that were caught in python/conformance tests, and adds a new SetFeatureSetDefaults API to the def pool for consistency with C++ and other python implementations.
PiperOrigin-RevId: 581384108
This change only covers pure python, and follow-up changes will handle C++/upb variants and actually enable editions support. The C++ one works (as evident from the conformance tests), but needs some APIs added to allow for testing.
PiperOrigin-RevId: 580304039
We transformed the proto2/proto3 protos to editions, and then run the same set of tests over both. This will verify that migrating to editions preserves the same proto2/proto3 behavior. These will not be enabled by default, and require a flag `--maximum_edition=2023`.
Future changes will:
- add more targeted editions-specific tests
- clean up our conformance test framework to allow for more targeted tests
- add wildcards to failure lists in limited cases to reduce noise
- add feature resolution conformance tests
PiperOrigin-RevId: 574570607
This doesn't change the tests at all, but refactors them to be more reusable in different contexts. Specifically, this will make it easier to add corresponding editions-based tests. This also splits the concept of "uses a proto3 message" and "runs proto3 tests", in preparation for that change.
PiperOrigin-RevId: 574310612
This doesn't change the tests at all, but refactors them to be more reusable in different contexts. Specifically, this will make it easier to add corresponding editions-based tests. This also splits the concept of "uses a proto3 message" and "runs proto3 tests", in preparation for that change.
PiperOrigin-RevId: 574299923
Mike Kruskal
Protobuf Team Bot
Eric Salo
Joshua Haberman
Anton Grbin
Sandy Zhang
Ryan Govostes
aiuto
Adam Cozzette
Marcel Hlopko
Thomas Van Lenten