Write barrier protected objects are allowed to be promoted to the old generation, which means they only get marked on major GC.
The downside is that the `RB_BJ_WRITE` macro MUST be used to set references, otherwise the referenced object may be garbaged collected.
But the `*Descriptor` classes and `Arena` have very few references and are only set in a few places, so it's relatively easy to implement.
cc @peterzhu2118Closes#11793
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11793 from casperisfine:descriptor-write-barrier 215e8fad4c
PiperOrigin-RevId: 511875342
When generating Ruby clients for proto3 messages that have a oneof, we generate a hazzer for members of the oneof, not just a hazzer for the oneof itself.
In other words, for a proto like this:
```
syntax = "proto3";
message Foo {
oneof bar {
string baz = 1;
}
}
```
The generated `Foo` will now have a method called `has_baz?`, in addition to the (pre-existing) method `has_bar?`.
I updated the unit tests, and verified that all the tests under `//ruby/...` pass.
Fixes https://github.com/protocolbuffers/protobuf/issues/9561.
Closes#11655
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11655 from shaldengeki:test-ruby-oneof-hazzer a15e474da6
PiperOrigin-RevId: 506090930
* Added staleness test for ruby-upb.{c,h} and updated.
* Removed file comment markers, too much trouble for too little benefit.
* Ran clang-format.
* Updated ruby-upb.{c,h}.
* Added missing table code to amalgamation.
* Updated to latest upb, patch no longer needed.
* Reverted changes to third_party sub-modules.
* Added missing unicode file.
* Removed conformance failures for Ruby.
The only difference between all the `Message` classes is just
their `descriptor` instance variable.
So rather than create an entirely new class from scratch every time
we can simply inherit from an abstract class.
This shink each `Message` class from `1456` bytes to `944` bytes, and
the singleton class of each from `960` to `792`, for a total of
`680` bytes saved per message class, so a ~28% reduction.
Setting an instance variable without an `@` prefix makes it
unnacessible from the Ruby side, which prevent introspection
by various tools.
I see no reason to prevent this.
* Message.decode/encode: Add max_recursion_depth option
This allows increasing the recursing depth from the default of 64, by
setting the "max_recursion_depth" to the desired integer value. This is
useful to encode or decode complex nested protobuf messages that otherwise
error out with a RuntimeError or "Error occurred during parsing".
Fixes#1493
* Address review comments
Co-authored-by: Adam Cozzette <acozzette@google.com>
* Fixed memory bug: properly root repeated/map field when assigning.
Previously the protobuf extension would not properly root
memory from a repeated field or map when assigning to a
message field (see the attached test case). This could cause
crashes if the repeated field is subsequently accessed.
* Add accidentally-deleted Ruby test.
* Some more updates to PHP testing infrastructure (#8576)
* WIP.
* Added build config for all of the tests.
* Use ../src/protoc if it is available, for cases where Bazel isn't available.
* Added test_php.sh.
* Fix for the broken macOS tests.
* Move all jobs to use php80 instead of lots of separate jobs.
* Only pass -t flag if we are running in a terminal.
* Updated php_all job to use new Docker stuff.
* Fixed PHP memory leaks and arginfo errors (#8614)
* Fixed a bunch of incorrect arginfo and a few incorrect error messages.
* Passes mem check test with no leaks!
* WIP.
* Fix build warning that was causing Bazel build to fail.
* Added compatibility code for PHP <8.0.
* Added test_valgrind target and made tests Valgrind-clean.
* Updated Valgrind test to fail if memory leaks are detected.
* Removed intermediate shell script so commands are easier to cut, paste, and modify.
* Passing all Valgrind tests!
* Hoist addref into ObjCache_Get().
* Removed special case of map descriptors by keying object map on upb_msgdef.
* Removed all remaining RETURN_ZVAL() macros.
* Removed all explicit reference add/del operations.
* Added REFCOUNTING.md to Makefile.am.
* Updated upb version and fixed PHP to not get unset message field. (#8621)
* Updated upb version and fixed PHP to not get unset message field.
* Updated changelog.
* Fixed preproc test to handle old versions of Clang withot __has_attribute().
* A second try at fixing __has_attribute().
* Copy __has_attribute() fix to cc file also.
* Updated failure list for PHP for fixed test.
* Updated version of upb for Ruby (#8624)
* Updated upb.
* Preserve legacy behavior for unset messages.
* Updated failure list.
* Updated CHANGES.txt.
* Added erroneously-deleted test file.
* Fixed condition on compatibility code.
* Re-introduced deleted file again, and fixed Rakefile to not delete it.
* Fix generation of test protos.
* fix(ruby): Fix crash when calculating Message hash values on 64-bit Windows
* Better mapping for values outside the fixnum range
* Simpler downcasting of hash values
* Fix precedence
* Fix bundle on Ruby 2.4
Ruby <2.7 does not allow non-finalizable objects to be WeakMap
keys: https://bugs.ruby-lang.org/issues/16035
We work around this by using a secondary map for Ruby <2.7 which
maps the non-finalizable integer to a distinct object.
For now we accept that the entries in the secondary map wil never
be collected. If this becomes a problem we can perform a GC pass
every so often that looks at the contents of the object cache to
decide what can be deleted from the secondary map.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* Added some missing files.
* WIP.
* WIP.
* Updated upb.
* Extension loads, but crashes immediately.
* Gets through the test suite without SEGV!
Still a lot of bugs to fix, but it is a major step!
214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed
* Test and build for Ruby 3.0
* Fixed a few more bugs, efficient #inspect is almost done.
214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed
* Fixed message hash initialization and encode depth checking.
214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed
* A bunch of fixes to failing tests, now 70% passing.
214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed
* More than 80% of tests are passing now.
214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed
Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.
* Add linux/ruby30 and macos/ruby30
* Use rvm master for 3.0.0-preview2
* Over 90% of tests are passing!
214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed
* Passes all tests!
214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
* A bunch of cleanup.
1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.
* Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure.
Intermittent failure:
ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed
* Removed a few more things from protobuf.h.
* Ruby 3.0.0-preview2 to 3.0.0
* Require rake-compiler-dock >= 1.1.0
* More progress, fighting with the object cache.
* Passes on all Ruby versions!
* Updated and clarified comment regarding WeakMap.
* Fixed the wyhash compile.
* Fixed conformance tests for Ruby.
Conformance results now look like:
RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb
CONFORMANCE TEST BEGIN ====================================
CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.
CONFORMANCE TEST BEGIN ====================================
CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.
Fixes include:
- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
JSON.
- Fixed conformance test to properly report serialization errors.
* Removed debug printf and fixed #inspect for floats.
* Fixed compatibility test to have proper semantics for #to_json.
* Updated Makefile.am with new file list.
* Don't try to copy wyhash when inside Docker.
* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.
* Avoid exporting unneeded symbols and refactored a bit of code.
* Some more refactoring.
* Simplified and added more comments.
* Some more comments and simplification. Added a missing license block.
Co-authored-by: Masaki Hara <hara@wantedly.com>
* WIP.
* WIP.
* Builds and runs. Tests need to be updated to test presence.
* Ruby: proto3 presence is passing all tests.
* Fixed a bug where empty messages has the wrong oneof count.
Sandy Zhang
Joshua Haberman
Jason Lunn
Matt Fowles Kulukundis
Deanna Garcia
Minh Quy
Jean byroot Boussier
Charles OuGuo
Eric Salo
Mike Kruskal
Derek Perez
tison
Joshua Haberman
Mike Kruskal
Jean Boussier
Adam Cozzette
Luka Dornhecker
Lukas Fittl
Peter Zhu
Bo Yang
Daniel Azuma
listonjesse