This PR implements lookup of service descriptor and method descriptor objects in Ruby as described in issue https://github.com/protocolbuffers/protobuf/issues/14891.
It contains three implementations - one for the CRuby extension API, one for JRuby, and one for FFI.
With this patch,
* `DescriptorPool#lookup('fully.qualified.service.name')` works and returns a `Google::Protobuf::ServiceDescriptor` object
* You can call `#options` on that to get the service options
* You can call `#methods` on that to get the services' methods as `Google::Protobuf::MethodDescriptor` objects,
* You can call `MethodDescriptor#options` to get method options
* You can also get the streaming flags & input/output types of the method with `#input_type`, `#output_type`, `#client_streaming`, and `#server_streaming`.
In order to make the FFI implementation work, I had to mark some more methods in the UPB header as exported - I guess that's something which will have to be done on the UPB side, like this 01fed1cc1b
CC @dazuma & @haberman from the original issue, and @JasonLunn (since you work on protobuf it seems - small world!)
I apologies for the large volume of copy-pasta'd code from the existing descriptor class implementations into the new ones - I felt this was probably better than designing new abstractions to reduce it off the bat though; this feels like it "fits in" with the existing implementation.
Closes#15817
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15817 from KJTsanaktsidis:ktsanaktsidis/add_service_method_descriptors 54d7218431
PiperOrigin-RevId: 618221016
Three of these runtimes are based on upb, and the fourth is based on the Java runtime. Both of these already have editions support, so this was mostly just a matter of:
- Advertising support to allow editions codegen
- Stripping features from the runtime options
- Hooking up conformance tests
- Adding some lightweight editions tests
There are also a few minor orthogonal fixes included here:
- Ruby's upb hack for treating all enums as open enums needed tweaking
- The `enable_editions` flag is no longer needed in our internal proto rules
PiperOrigin-RevId: 616256211
These tests were put into the wrong scope, and never actually ran. Fixing them turned up a bug in JRuby FFI where this fix wasn't applied.
PiperOrigin-RevId: 616172197
This was unintentionally disabling a test in repeated_field_test that is broken in JRuby FFI. It appears to be a bug in Enumeration, rather than our code.
PiperOrigin-RevId: 616164907
Coming from [[protobuf/issues/15180] [Ruby] Support for currently ignored Array methods in `RepeatedField`](https://github.com/protocolbuffers/protobuf/issues/15180)
This adds a couple of `Array` methods to what gets delegated from `RepeatedField`.
- `intersection`, because `|` was already delegated
- `union`, because `&` was already delegated
- `difference`, because `-` was already delegated
Closes#15652
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15652 from marianosimone:delegate_rb_array_methods 2971981338
PiperOrigin-RevId: 604534655
This bug arises only in the uncommon case where there is more than one DescriptorPool. In such a case, JSON encode/decode should always use the pool of the message being encoded/decoded, not the generated pool.
Closes#15281
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15281 from protocolbuffers:ruby-json-pool-fix 91e2dc55dc
PiperOrigin-RevId: 596027770
Currently we're aliasing `each_index` to `each_with_index`, incorrectly passing both the index and the value of a repeated field to the block.
What we want is to just pass the index. Luckily this is a method on Ruby arrays, so we just wrap the native Ruby array method.
Fixes https://github.com/protocolbuffers/protobuf/issues/7806Closes#11767
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11767 from shaldengeki:shaldengeki-repeated-field-each-index-returns-actual-index 874916c21d
PiperOrigin-RevId: 593835025
## Issue
When an object that is an instance of a string-derived class is passed to a string field in a protobuf message in Ruby, it results in a `Google::Protobuf::TypeError`.
### Steps to reproduce
```rb
~/src/github.com/protocolbuffers/protobuf/ruby/tests ❯❯❯ irb -I .
irb(main):001:0> require 'basic_test_pb'
=> true
irb(main):002:0> myString = Class.new(String)
=> #<Class:0x00000001531540d8>
irb(main):003:0> str = myString.new("foo")
=> "foo"
irb(main):004:0> BasicTest::TestMessage.new(optional_string: "foo")
=> <BasicTest::TestMessage: optional_string: "foo", repeated_int32: [], repeated_int64: [], repeated_uint32: [], repeated_uint64: [], repeated_bool: [], repeated_float: [], repeated_double: [], repeated_string: [], repeated_bytes: [], repeated_msg: [], repeated_enum: []>
irb(main):005:0> BasicTest::TestMessage.new(optional_string: str)
(irb):5:in `initialize': Invalid argument for string field 'optional_string' (given #<Class:0x00000001531540d8>). (Google::Protobuf::TypeError)
irb(main):006:0>
```
## Fix
The issue appears to be caused by the field checking mechanism not properly handling instances of classes that inherit from basic types like String. My proposed solution is to improve the type checking for string fields to consider not just String instances but also instances of subclasses of String.
## Impact
The changes will allow instances of classes derived from String to be passed to string fields without any error.
This is a backwards-compatible change and will not affect the existing behaviour with standard String instances.
Closes#13818
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13818 from NotFounds:support-string-subclass-for-ruby 2d2796c4f9
PiperOrigin-RevId: 590941235
Rewrrte and extension of #12828, with additional work for JRuby. Partially fixes#1198 by adding support for custom options. Handling of extensions will be handled in a follow up.
Also includes these unrelated fixes:
* Removes code echo between `google/protobuf/repeated_field.rb` and `google/protobuf/ffi/repeated_field.rb` by `require`'ing the former in the latter.
* Adds missing calles to `testFrozen()` from methods of `RepeatedField` under JRuby that mutate.
* Various typos in comments.
Closes#14594
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14594 from protocolbuffers:add-support-for-options-in-ruby 16cc9e35b8
PiperOrigin-RevId: 580848874
Unblocks execution of these tests on JRuby FFI.
Automated reformatting of WORKSPACE with documentary comment to ease future debugging of JRuby tests locally.
Closes#13293
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/13293 from JasonLunn:object_cache_test_update 7db211a342
PiperOrigin-RevId: 547614702
When generating Ruby clients for proto3 messages that have a oneof, we generate a hazzer for members of the oneof, not just a hazzer for the oneof itself.
In other words, for a proto like this:
```
syntax = "proto3";
message Foo {
oneof bar {
string baz = 1;
}
}
```
The generated `Foo` will now have a method called `has_baz?`, in addition to the (pre-existing) method `has_bar?`.
I updated the unit tests, and verified that all the tests under `//ruby/...` pass.
Fixes https://github.com/protocolbuffers/protobuf/issues/9561.
Closes#11655
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/11655 from shaldengeki:test-ruby-oneof-hazzer a15e474da6
PiperOrigin-RevId: 506090930
This uses https://github.com/protocolbuffers/rules_ruby to fully Bazelify our ruby runtime code. The Rakefile is left in place for now and is still used by our aarch64 tests. With the current implementation ruby behaves similarly to our python wrapper, which selects whatever version is installed in the system. Future enhancements will allow for more hermetic builds via Bazel flags to pin a specific version
Closes#10525
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10525 from mkruskal-google:rules_ruby 97fa1f70ab
PiperOrigin-RevId: 499283908
* Bazelfying conformance tests
Adding infrastructure to "Bazelify" languages other than Java and C++
* Delete benchmarks for languages supported by other repositories
* Bazelfying benchmark tests
* Bazelfying python
Use upb's system python rule instead of branching tensorflow
* Bazelfying Ruby
* Bazelfying C#
* Bazelfying Objective-c
* Bazelfying Kokoro mac builds
* Bazelfying Kokoro linux builds
* Deleting all deprecated files from autotools cleanup
This boils down to Makefile.am and tests.sh and all of their remaining references
* Cleanup after PR reorganizing
- Enable 32 bit tests
- Move conformance tests back
- Use select statements to select alternate runtimes
- Add internal prefixes to proto library macros
* Updating READMEs to use bazel instead of autotools.
* Bazelfying Kokoro release builds
* First round of review fixes
* Second round of review fixes
* Third round of review fixes
* Filtering out conformance tests from Bazel on Windows (b/241484899)
* Add version metadata that was previously scraped from configure.ac
* fixing typo from previous fix
* Adding ruby version tests
* Bumping pinned upb version, and adding tests to python CI
All synthetic methods implemented by `method_missing` are now supported by `respond_to?`.
Fixes issue #9202.
* Fix null pointer exceptions exposed by new regression tests.
* Fix clear_ on oneofs so that it is safe to call repeatedly and so that respond_to? does not depend on whether the oneof is currently cleared.
* Code cleanup: reenable more tests on JRuby.
* Align JRuby behavior with CRuby by throwing a RuntimeError when attempting to assign to a oneof.
(cherry picked from commit 8e7f936696)
All synthetic methods implemented by `method_missing` are now supported by `respond_to?`.
Fixes issue #9202.
* Fix null pointer exceptions exposed by new regression tests.
* Fix clear_ on oneofs so that it is safe to call repeatedly and so that respond_to? does not depend on whether the oneof is currently cleared.
* Code cleanup: reenable more tests on JRuby.
* Align JRuby behavior with CRuby by throwing a RuntimeError when attempting to assign to a oneof.
* Allow pre-compiled binaries for ruby 3.1.0 (#9566)
* Allow pre-compiled binaries for ruby 3.1.1
* add comment
* fix build and use ruby 3.1.0
* add ruby31 to build CI for tests and release
* trying to fix ci
* install ruby 3.1.0 in ruby_build_environment.sh
* use head for rvm to install 3.1.0
* just install master version of rvm in prepare_build_macos_rc
* force install of master rvm in ruby_build_environment.sh
* Use coroutine=universal when compiling ruby31
* use ucontext
* fix filename
* fix coroutine name
* use git head for rake-compiler-dock
* use newest rake-compiler-dock version
* Updated CHANGES.txt for Ruby changes.
* Fixed Ruby 3.1 tests by marking intersect? as unimplemented. (#9645)
* Fixed Ruby 3.1 tests by marking intersect? as unimplemented.
* Updated compatibility tests.
Co-authored-by: Marco Concetto Rudilosso <marcoconcettorudilosso@gmail.com>
* Add ruby-specific upb_alloc using xrealloc/xfree for use in Arena_alloc so Ruby GC is aware of allocated memory.
* Add RB_GC_GUARD to DescriptorPool_add_serialized_file to ensure ruby does not aggressively garbage collect arena_rb due to lack of references.
* Updated PHP to the new version of upb.
This is a large change, as the upb API surface has been
renamed to follow Google style more closely.
* Fixed utf8_range.
* Updated Ruby for new utf8_range.
* Picked up new upb for PHP, with spelling fixes.
* Fixed the 32-bit build.
Joshua Haberman
Eric Salo
KJ Tsanaktsidis
Mike Kruskal
Jean byroot Boussier
aiuto
Mariano Simone
Jason Lunn
Joshua Haberman
Charles OuGuo
Iori IKEDA
Matt Fowles Kulukundis
Minh Quy
tison
Mike Kruskal
Luc Thevenard
Adam Cozzette
Luka Dornhecker
zhangskz
Anthony Dmitriyev