Write barrier protected objects are allowed to be promoted to the old generation, which means they only get marked on major GC.
The downside is that the `RB_BJ_WRITE` macro MUST be used to set references, otherwise the referenced object may be garbaged collected.
But the `*Descriptor` classes and `Arena` have very few references and are only set in a few places, so it's relatively easy to implement.
cc @peterzhu2118Closes#11793
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11793 from casperisfine:descriptor-write-barrier 215e8fad4c
PiperOrigin-RevId: 511875342
This fixes a v22 regression for Ruby.
Windows builds of the Ruby package are broken in 3.22.0.rc.2. The reason is, `ruby-upb.h` no longer provides the full path when `#include`ing `utf8_range.h`. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/ruby-upb.h#L10479 (22.0-rc-2), cf. https://github.com/protocolbuffers/protobuf/blob/v21.12/ruby/ext/google/protobuf_c/ruby-upb.h#L5365 (21.12). The `extconf.rb` build configuration tries to compensate by adding `third_party/utf8_range` to the include path, but does not do so on Windows (i.e. `mingw`) platforms. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/extconf.rb#L9-L10 (22.0-rc-2), cf. https://github.com/protocolbuffers/protobuf/blob/v21.12/ruby/ext/google/protobuf_c/extconf.rb#L9-L10 (21.12).
We could have fixed this by adding another clause to the if statement for the case `RUBY_PLATFORM =~ /mingw/` and adding the appropriate `-I` flag to `CFLAGS`. However, that `CFLAGS` hack is present in the first place due to a related problem: the usage of `$INCFLAGS` is incorrect.
The `$INCFLAGS` constant in Ruby's `mkmf` is a string in a similar format to `CFLAGS`. It's simply appended to compiler invocations. So when you append new flags to it, you have to (1) provide the flag itself, and (2) precede it with a space to delimit it from the previous entry. In 22.0-rc-2 (and indeed in all earlier versions) the usage is incorrect: it's appending a path to the string without the `-I` and without a space. See https://github.com/protocolbuffers/protobuf/blob/v22.0-rc2/ruby/ext/google/protobuf_c/extconf.rb#L22. Hence, not only does the intended include path not get appended correctly, it also clobbers the previous path in the string. Luckily, the previous path is only `-I$(srcdir)` which happens not to matter for our library. But it does mean that the apparent intent of that line, adding `$(srcdir)/third_party/utf8_range` to the include path, isn't working; hence the code that adds it to `CFLAGS` instead.
(Note that the previous line, adding the path to `$VPATH`, _is_ correct as is, because `$VPATH` is an array.)
So what this PR actually does is fix the `$INCFLAGS` usage so `$(srcdir)/third_party/utf8_range` gets added properly to compiler include paths, for all platforms including Windows. Since that should now be working as intended, we also remove the extra `-I` from CFLAGS. Builds for all platforms should now be able to handle the change to `ruby-upb.h`. This has been tested by running the prototype Ruby build Kokoro job against a patched 22.0-rc-2.
This also needs to be backported to the 22.x branch.
/attn @deannagarciaCloses#11882
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11882 from dazuma:pr/ruby-builds-fix ebb18e0004
PiperOrigin-RevId: 508550039
This is a proof of concept that we can experiment with before migrating all of our Kokoro tests to GHA. In addition to the migration, this builds out infrastructure for safe handling of both external and internal contributions. Two of the existing GHA workflows have also been migrated to this system to unify how we handle testing.
To test the new workflow introduced here, you can open PRs to the `gha` branch. This feature branch acts as a staging area for GHA. PRs opened up from this repo will automatically have all tests run on each commit. We will also run them all as post-submits, and continuously on a daily schedule. PRs opened from forked repos will need per-commit approval for each test run, which can be given by adding the `safe for tests` label.
Examples (failures are intentional to show that the PR code is being tested):
- Internal PR: https://github.com/protocolbuffers/protobuf/pull/11679
- Fork PR (approved): https://github.com/protocolbuffers/protobuf/pull/11685
- Fork PR (rejected): https://github.com/protocolbuffers/protobuf/pull/11683
- External PR: https://github.com/protocolbuffers/protobuf/pull/11695Closes#11702
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11702 from protocolbuffers:gha f3a2663896
PiperOrigin-RevId: 506169444
When generating Ruby clients for proto3 messages that have a oneof, we generate a hazzer for members of the oneof, not just a hazzer for the oneof itself.
In other words, for a proto like this:
```
syntax = "proto3";
message Foo {
oneof bar {
string baz = 1;
}
}
```
The generated `Foo` will now have a method called `has_baz?`, in addition to the (pre-existing) method `has_bar?`.
I updated the unit tests, and verified that all the tests under `//ruby/...` pass.
Fixes https://github.com/protocolbuffers/protobuf/issues/9561.
Closes#11655
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11655 from shaldengeki:test-ruby-oneof-hazzer a15e474da6
PiperOrigin-RevId: 506090930
Add bazel targets to create ruby release artifacts.
Should be run with:
```
bazel run ruby:release
bazel run ruby:jruby_release
```
Closes#11468
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11468 from deannagarcia:rubyTargets b7b7eb6865
PiperOrigin-RevId: 503277136
This uses https://github.com/protocolbuffers/rules_ruby to fully Bazelify our ruby runtime code. The Rakefile is left in place for now and is still used by our aarch64 tests. With the current implementation ruby behaves similarly to our python wrapper, which selects whatever version is installed in the system. Future enhancements will allow for more hermetic builds via Bazel flags to pin a specific version
Closes#10525
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10525 from mkruskal-google:rules_ruby 97fa1f70ab
PiperOrigin-RevId: 499283908
We would like for upb_Map_Delete() to optionally return the deleted value.
Unfortunately this will require several steps since we are crossing repos.
Step #4: advance the upb version used by protobuf and update PHP/Ruby accordingly.
PiperOrigin-RevId: 498426185
We would like for upb_Map_Delete() to optionally return the deleted value.
Unfortunately this will require several steps since we are crossing repos.
Step #2: Point PHP and Ruby at the new temporary function.
point the protobuf repo at latest upb
regenerate the amalgamation files
PiperOrigin-RevId: 497310441
This was silently breaking because we didn't fail on errors in the test script. The upb version is also bumped to pull in improvements for system_python that fix the underlying issue (python 3.5 doesn't work with our pip dependencies).
PiperOrigin-RevId: 492526855
This pulls in updates to system_python, which allows for better handling of our non-hermetic setup. Specifically, our python tests are marked incompatible on systems without a python installation. This will cause them to get skipped by wildcard expansions (which is why we add 1 explicit target to the kokoro builds).
PiperOrigin-RevId: 492286518
They probably should be hidden on every ELF platform, but for now only do it on Darwin, Linux, and FreeBSD.
This prevents some load-time errors that have been seen in GitLab's gitaly server, and in a few other places.
Closes#10832
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10832 from dankm:hide-symbols 4cb59f2691
PiperOrigin-RevId: 490522474
Jean byroot Boussier
Mike Kruskal
Deanna Garcia
Protobuf Team Bot
zhangskz
Daniel Azuma
deannagarcia
Charles OuGuo
zhangskz
Adam Cozzette
Protobuf Team Bot
Eric Salo
Matt Fowles Kulukundis
Joshua Haberman
dankm