This represents the future direction of protobuf, replacing proto2/proto3 syntax with editions. These will enable more incremental evolution of protobuf APIs through features, which are individual behaviors (such as whether field presence is explicit or implicit). For more details see https://protobuf.dev/editions/overview/.
This PR contains a working implementation of editions for the protoc frontend and C++ code generation, along with various infrastructure improvements to support it. It gives early access for anyone who wants to a preview of editions, but has no effect on proto2/proto3 syntax. It is flag-guarded behind the `--experimental_editions` flag, and is an experimental feature with no guarantees.
PiperOrigin-RevId: 544805690
Prior to this CL, asserts have no effect for Ruby 3+, because Ruby unconditionally defines `NDEBUG`: https://bugs.ruby-lang.org/issues/18777
We work around this by doing `#undef NDEBUG` right after including Ruby, if `NDEBUG` was not previously defined.
PiperOrigin-RevId: 535359088
An extension range is either of verification state "UNVERIFIED" or "DECLARATION". If "DECLARATION", all extension fields of the range must be declared, or build error otherwise. The current default is "UNVERIFIED", but we will flip the default later.
Deprecate `is_repeated` in favor of `repeated`.
PiperOrigin-RevId: 526184056
This can be useful when an extension field/declaration is deleted in schema to avoid data corruption.
It also adds a check for duplicate numbers in the declarations.
PiperOrigin-RevId: 523269837
Write barrier protected objects are allowed to be promoted to the old generation, which means they only get marked on major GC.
The downside is that the `RB_BJ_WRITE` macro MUST be used to set references, otherwise the referenced object may be garbaged collected.
But the `*Descriptor` classes and `Arena` have very few references and are only set in a few places, so it's relatively easy to implement.
cc @peterzhu2118Closes#11793
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11793 from casperisfine:descriptor-write-barrier 215e8fad4c
PiperOrigin-RevId: 511875342
This fixes a v22 regression for Ruby.
Windows builds of the Ruby package are broken in 3.22.0.rc.2. The reason is, `ruby-upb.h` no longer provides the full path when `#include`ing `utf8_range.h`. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/ruby-upb.h#L10479 (22.0-rc-2), cf. https://github.com/protocolbuffers/protobuf/blob/v21.12/ruby/ext/google/protobuf_c/ruby-upb.h#L5365 (21.12). The `extconf.rb` build configuration tries to compensate by adding `third_party/utf8_range` to the include path, but does not do so on Windows (i.e. `mingw`) platforms. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/extconf.rb#L9-L10 (22.0-rc-2), cf. https://github.com/protocolbuffers/protobuf/blob/v21.12/ruby/ext/google/protobuf_c/extconf.rb#L9-L10 (21.12).
We could have fixed this by adding another clause to the if statement for the case `RUBY_PLATFORM =~ /mingw/` and adding the appropriate `-I` flag to `CFLAGS`. However, that `CFLAGS` hack is present in the first place due to a related problem: the usage of `$INCFLAGS` is incorrect.
The `$INCFLAGS` constant in Ruby's `mkmf` is a string in a similar format to `CFLAGS`. It's simply appended to compiler invocations. So when you append new flags to it, you have to (1) provide the flag itself, and (2) precede it with a space to delimit it from the previous entry. In 22.0-rc-2 (and indeed in all earlier versions) the usage is incorrect: it's appending a path to the string without the `-I` and without a space. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/extconf.rb#L22. Hence, not only does the intended include path not get appended correctly, it also clobbers the previous path in the string. Luckily, the previous path is only `-I$(srcdir)` which happens not to matter for our library. But it does mean that the apparent intent of that line, adding `$(srcdir)/third_party/utf8_range` to the include path, isn't working; hence the code that adds it to `CFLAGS` instead.
(Note that the previous line, adding the path to `$VPATH`, _is_ correct as is, because `$VPATH` is an array.)
So what this PR actually does is fix the `$INCFLAGS` usage so `$(srcdir)/third_party/utf8_range` gets added properly to compiler include paths, for all platforms including Windows. Since that should now be working as intended, we also remove the extra `-I` from CFLAGS. Builds for all platforms should now be able to handle the change to `ruby-upb.h`. This has been tested by running the prototype Ruby build Kokoro job against a patched 22.0-rc-2.
This also needs to be backported to the 22.x branch.
/attn @deannagarciaCloses#11882
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11882 from dazuma:pr/ruby-builds-fix ebb18e0004
PiperOrigin-RevId: 508550039
When generating Ruby clients for proto3 messages that have a oneof, we generate a hazzer for members of the oneof, not just a hazzer for the oneof itself.
In other words, for a proto like this:
```
syntax = "proto3";
message Foo {
oneof bar {
string baz = 1;
}
}
```
The generated `Foo` will now have a method called `has_baz?`, in addition to the (pre-existing) method `has_bar?`.
I updated the unit tests, and verified that all the tests under `//ruby/...` pass.
Fixes https://github.com/protocolbuffers/protobuf/issues/9561.
Closes#11655
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11655 from shaldengeki:test-ruby-oneof-hazzer a15e474da6
PiperOrigin-RevId: 506090930
Add bazel targets to create ruby release artifacts.
Should be run with:
```
bazel run ruby:release
bazel run ruby:jruby_release
```
Closes#11468
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11468 from deannagarcia:rubyTargets b7b7eb6865
PiperOrigin-RevId: 503277136
This uses https://github.com/protocolbuffers/rules_ruby to fully Bazelify our ruby runtime code. The Rakefile is left in place for now and is still used by our aarch64 tests. With the current implementation ruby behaves similarly to our python wrapper, which selects whatever version is installed in the system. Future enhancements will allow for more hermetic builds via Bazel flags to pin a specific version
Closes#10525
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10525 from mkruskal-google:rules_ruby 97fa1f70ab
PiperOrigin-RevId: 499283908
We would like for upb_Map_Delete() to optionally return the deleted value.
Unfortunately this will require several steps since we are crossing repos.
Step #4: advance the upb version used by protobuf and update PHP/Ruby accordingly.
PiperOrigin-RevId: 498426185
We would like for upb_Map_Delete() to optionally return the deleted value.
Unfortunately this will require several steps since we are crossing repos.
Step #2: Point PHP and Ruby at the new temporary function.
point the protobuf repo at latest upb
regenerate the amalgamation files
PiperOrigin-RevId: 497310441
Joshua Haberman
Matt Fowles Kulukundis
Sandy Zhang
Mike Kruskal
Jie Luo
Jean byroot Boussier
Adam Cozzette
Deanna Garcia
Protobuf Team Bot
Eric Salo
Jakob Buchgraber
Minh Quy
Daniel Azuma
deannagarcia
Charles OuGuo
zhangskz
Joshua Haberman