Chiebot-Mirror
/
protobuf
mirror of https://github.com/protocolbuffers/protobuf.git
8
0
Fork 0
Code Issues Projects Releases Wiki Activity

Ported Ruby extension to upb_msg (#8184)

Browse Source 
* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* WIP.

* Added some missing files.

* WIP.

* WIP.

* Updated upb.

* Extension loads, but crashes immediately.

* Gets through the test suite without SEGV!

Still a lot of bugs to fix, but it is a major step!

214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed

* Test and build for Ruby 3.0

* Fixed a few more bugs, efficient #inspect is almost done.

214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed

* Fixed message hash initialization and encode depth checking.

214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed

* A bunch of fixes to failing tests, now 70% passing.

214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed

* More than 80% of tests are passing now.

214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed

Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.

* Add linux/ruby30 and macos/ruby30

* Use rvm master for 3.0.0-preview2

* Over 90% of tests are passing!

214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed

* Passes all tests!

214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed

* A bunch of cleanup.

1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.

* Removed a bunch more stuff from protobuf.h.  There is an intermittent assert failure.

Intermittent failure:

ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed

* Removed a few more things from protobuf.h.

* Ruby 3.0.0-preview2 to 3.0.0

* Require rake-compiler-dock >= 1.1.0

* More progress, fighting with the object cache.

* Passes on all Ruby versions!

* Updated and clarified comment regarding WeakMap.

* Fixed the wyhash compile.

* Fixed conformance tests for Ruby.

Conformance results now look like:

RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.

CONFORMANCE TEST BEGIN ====================================

CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.

Fixes include:

- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
  present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
  instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
  JSON.
- Fixed conformance test to properly report serialization errors.

* Removed debug printf and fixed #inspect for floats.

* Fixed compatibility test to have proper semantics for #to_json.

* Updated Makefile.am with new file list.

* Don't try to copy wyhash when inside Docker.

* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.

* Avoid exporting unneeded symbols and refactored a bit of code.

* Some more refactoring.

* Simplified and added more comments.

* Some more comments and simplification. Added a missing license block.

Co-authored-by: Masaki Hara <hara@wantedly.com>
pull/8209/head
Joshua Haberman 4 years ago committed by GitHub
parent 468bc193ec
commit 9abf6e2ab0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
42 changed files with 14071 additions and 23328 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 12
      Makefile.am
  2. 6
      conformance/Makefile.am
  3. 53
      conformance/conformance_ruby.rb
  4. 166
      conformance/failure_list_ruby.txt
  5. 3
      kokoro/linux/dockerfile/test/ruby/Dockerfile
  6. 18
      kokoro/linux/ruby30/build.sh
  7. 11
      kokoro/linux/ruby30/continuous.cfg
  8. 11
      kokoro/linux/ruby30/presubmit.cfg
  9. 2
      kokoro/macos/prepare_build_macos_rc
  10. 13
      kokoro/macos/ruby30/build.sh
  11. 5
      kokoro/macos/ruby30/continuous.cfg
  12. 5
      kokoro/macos/ruby30/presubmit.cfg
  13. 17
      kokoro/release/ruby/macos/ruby/ruby_build_environment.sh
  14. 10
      ruby/Rakefile
  15. 6
      ruby/compatibility_tests/v3.0.0/tests/basic.rb
  16. 349
      ruby/ext/google/protobuf_c/convert.c
  17. 72
      ruby/ext/google/protobuf_c/convert.h
  18. 1655
      ruby/ext/google/protobuf_c/defs.c
  19. 107
      ruby/ext/google/protobuf_c/defs.h
  20. 1795
      ruby/ext/google/protobuf_c/encode_decode.c
  21. 9
      ruby/ext/google/protobuf_c/extconf.rb
  22. 764
      ruby/ext/google/protobuf_c/map.c
  23. 66
      ruby/ext/google/protobuf_c/map.h
  24. 1322
      ruby/ext/google/protobuf_c/message.c
  25. 98
      ruby/ext/google/protobuf_c/message.h
  26. 390
      ruby/ext/google/protobuf_c/protobuf.c
  27. 639
      ruby/ext/google/protobuf_c/protobuf.h
  28. 612
      ruby/ext/google/protobuf_c/repeated_field.c
  29. 62
      ruby/ext/google/protobuf_c/repeated_field.h
  30. 8913
      ruby/ext/google/protobuf_c/ruby-upb.c
  31. 5049
      ruby/ext/google/protobuf_c/ruby-upb.h
  32. 1198
      ruby/ext/google/protobuf_c/storage.c
  33. 13817
      ruby/ext/google/protobuf_c/upb.c
  34. 2
      ruby/google-protobuf.gemspec
  35. 33
      ruby/tests/basic.rb
  36. 17
      ruby/tests/basic_proto2.rb
  37. 4
      ruby/tests/basic_test.proto
  38. 73
      ruby/tests/common_tests.rb
  39. 2
      ruby/travis-test.sh
  40. 6
      src/google/protobuf/compiler/ruby/ruby_generator.cc
  41. 5
      tests.sh
  42. 2
      third_party/wyhash/wyhash.h

12
Makefile.am
Unescape View File

@ -1094,17 +1094,21 @@ ruby_EXTRA_DIST= \
ruby/compatibility_tests/v3.0.0/test.sh \
ruby/compatibility_tests/v3.0.0/Rakefile \
ruby/compatibility_tests/v3.0.0/README.md \
ruby/ext/google/protobuf_c/convert.c \
ruby/ext/google/protobuf_c/convert.h \
ruby/ext/google/protobuf_c/defs.c \
ruby/ext/google/protobuf_c/encode_decode.c \
ruby/ext/google/protobuf_c/defs.h \
ruby/ext/google/protobuf_c/extconf.rb \
ruby/ext/google/protobuf_c/map.c \
ruby/ext/google/protobuf_c/map.h \
ruby/ext/google/protobuf_c/message.c \
ruby/ext/google/protobuf_c/message.h \
ruby/ext/google/protobuf_c/protobuf.c \
ruby/ext/google/protobuf_c/protobuf.h \
ruby/ext/google/protobuf_c/repeated_field.c \
ruby/ext/google/protobuf_c/storage.c \
ruby/ext/google/protobuf_c/upb.c \
ruby/ext/google/protobuf_c/upb.h \
ruby/ext/google/protobuf_c/repeated_field.h \
ruby/ext/google/protobuf_c/ruby-upb.c \
ruby/ext/google/protobuf_c/ruby-upb.h \
ruby/ext/google/protobuf_c/wrap_memcpy.c \
ruby/google-protobuf.gemspec \
ruby/lib/google/protobuf/message_exts.rb \

6
conformance/Makefile.am
Unescape View File

@ -5,7 +5,7 @@ conformance_protoc_inputs = \
$(top_srcdir)/src/google/protobuf/test_messages_proto3.proto
# proto2 input files, should be separated with proto3, as we
# can't generate proto2 files for ruby, php and objc
# can't generate proto2 files for php.
conformance_proto2_protoc_inputs = \
$(top_srcdir)/src/google/protobuf/test_messages_proto2.proto
@ -261,7 +261,7 @@ if USE_EXTERNAL_PROTOC
# Some implementations include pre-generated versions of well-known types.
protoc_middleman: $(conformance_protoc_inputs) $(conformance_proto2_protoc_inputs) $(well_known_type_protoc_inputs) google-protobuf
$(PROTOC) -I$(srcdir) -I$(top_srcdir) --cpp_out=. --java_out=. --ruby_out=. --objc_out=. --python_out=. --php_out=. --js_out=import_style=commonjs,binary:. $(conformance_protoc_inputs)
$(PROTOC) -I$(srcdir) -I$(top_srcdir) --cpp_out=. --java_out=. --objc_out=. --python_out=. --js_out=import_style=commonjs,binary:. $(conformance_proto2_protoc_inputs)
$(PROTOC) -I$(srcdir) -I$(top_srcdir) --cpp_out=. --java_out=. --ruby_out=. --objc_out=. --python_out=. --js_out=import_style=commonjs,binary:. $(conformance_proto2_protoc_inputs)
$(PROTOC) -I$(srcdir) -I$(top_srcdir) --cpp_out=. --java_out=. --ruby_out=. --python_out=. --js_out=import_style=commonjs,binary:google-protobuf $(well_known_type_protoc_inputs)
## $(PROTOC) -I$(srcdir) -I$(top_srcdir) --java_out=lite:lite $(conformance_protoc_inputs) $(well_known_type_protoc_inputs)
touch protoc_middleman
@ -273,7 +273,7 @@ else
# building out-of-tree.
protoc_middleman: $(top_srcdir)/src/protoc$(EXEEXT) $(conformance_protoc_inputs) $(conformance_proto2_protoc_inputs) $(well_known_type_protoc_inputs) google-protobuf
oldpwd=`pwd` && ( cd $(srcdir) && $$oldpwd/../src/protoc$(EXEEXT) -I. -I$(top_srcdir)/src --cpp_out=$$oldpwd --java_out=$$oldpwd --ruby_out=$$oldpwd --objc_out=$$oldpwd --python_out=$$oldpwd --php_out=$$oldpwd --js_out=import_style=commonjs,binary:$$oldpwd $(conformance_protoc_inputs) )
oldpwd=`pwd` && ( cd $(srcdir) && $$oldpwd/../src/protoc$(EXEEXT) -I. -I$(top_srcdir)/src --cpp_out=$$oldpwd --java_out=$$oldpwd --objc_out=. --python_out=$$oldpwd --js_out=import_style=commonjs,binary:$$oldpwd $(conformance_proto2_protoc_inputs) )
oldpwd=`pwd` && ( cd $(srcdir) && $$oldpwd/../src/protoc$(EXEEXT) -I. -I$(top_srcdir)/src --cpp_out=$$oldpwd --java_out=$$oldpwd --ruby_out=$$oldpwd --objc_out=$$oldpwd --python_out=$$oldpwd --js_out=import_style=commonjs,binary:$$oldpwd $(conformance_proto2_protoc_inputs) )
oldpwd=`pwd` && ( cd $(srcdir) && $$oldpwd/../src/protoc$(EXEEXT) -I. -I$(top_srcdir)/src --cpp_out=$$oldpwd --java_out=$$oldpwd --ruby_out=$$oldpwd --python_out=$$oldpwd --js_out=import_style=commonjs,binary:$$oldpwd/google-protobuf $(well_known_type_protoc_inputs) )
## @mkdir -p lite
## oldpwd=`pwd` && ( cd $(srcdir) && $$oldpwd/../src/protoc$(EXEEXT) -I. -I$(top_srcdir)/src --java_out=lite:$$oldpwd/lite $(conformance_protoc_inputs) $(well_known_type_protoc_inputs) )

53
conformance/conformance_ruby.rb
Unescape View File

@ -32,6 +32,7 @@
require 'conformance_pb'
require 'google/protobuf/test_messages_proto3_pb'
require 'google/protobuf/test_messages_proto2_pb'
$test_count = 0
$verbose = false
@ -39,39 +40,39 @@ $verbose = false
def do_test(request)
test_message = ProtobufTestMessages::Proto3::TestAllTypesProto3.new
response = Conformance::ConformanceResponse.new
descriptor = Google::Protobuf::DescriptorPool.generated_pool.lookup(request.message_type)
unless descriptor
response.skipped = "Unknown message type: " + request.message_type
end
begin
case request.payload
when :protobuf_payload
if request.message_type.eql?('protobuf_test_messages.proto3.TestAllTypesProto3')
begin
test_message = ProtobufTestMessages::Proto3::TestAllTypesProto3.decode(
request.protobuf_payload)
rescue Google::Protobuf::ParseError => err
response.parse_error = err.message.encode('utf-8')
return response
end
elsif request.message_type.eql?('protobuf_test_messages.proto2.TestAllTypesProto2')
response.skipped = "Ruby doesn't support proto2"
begin
test_message = descriptor.msgclass.decode(request.protobuf_payload)
rescue Google::Protobuf::ParseError => err
response.parse_error = err.message.encode('utf-8')
return response
else
fail "Protobuf request doesn't have specific payload type"
end
when :json_payload
begin
test_message = ProtobufTestMessages::Proto3::TestAllTypesProto3.decode_json(
request.json_payload)
options = {}
if request.test_category == :JSON_IGNORE_UNKNOWN_PARSING_TEST
options[:ignore_unknown_fields] = true
end
test_message = descriptor.msgclass.decode_json(request.json_payload, options)
rescue Google::Protobuf::ParseError => err
response.parse_error = err.message.encode('utf-8')
return response
end
when :text_payload
begin
response.skipped = "Ruby doesn't support proto2"
return response
end
when :text_payload
begin
response.skipped = "Ruby doesn't support text format"
return response
end
when nil
fail "Request didn't have payload"
@ -82,10 +83,18 @@ def do_test(request)
fail 'Unspecified output format'
when :PROTOBUF
response.protobuf_payload = test_message.to_proto
begin
response.protobuf_payload = test_message.to_proto
rescue Google::Protobuf::ParseError => err
response.serialize_error = err.message.encode('utf-8')
end
when :JSON
response.json_payload = test_message.to_json
begin
response.json_payload = test_message.to_json
rescue Google::Protobuf::ParseError => err
response.serialize_error = err.message.encode('utf-8')
end
when nil
fail "Request didn't have requested output format"

166
conformance/failure_list_ruby.txt
Unescape View File

@ -1,114 +1,58 @@
Recommended.FieldMaskNumbersDontRoundTrip.JsonOutput
Recommended.FieldMaskPathsDontRoundTrip.JsonOutput
Recommended.FieldMaskTooManyUnderscore.JsonOutput
Recommended.Proto2.JsonInput.FieldNameExtension.Validator
Recommended.Proto3.JsonInput.BytesFieldBase64Url.JsonOutput
Recommended.Proto3.JsonInput.BytesFieldBase64Url.ProtobufOutput
Recommended.Proto3.JsonInput.DurationHas3FractionalDigits.Validator
Recommended.Proto3.JsonInput.DurationHas6FractionalDigits.Validator
Recommended.Proto3.JsonInput.FieldMaskInvalidCharacter
Recommended.Proto3.JsonInput.MapFieldValueIsNull
Recommended.Proto3.JsonInput.NullValueInNormalMessage.Validator
Recommended.Proto3.JsonInput.NullValueInOtherOneofNewFormat.Validator
Recommended.Proto3.JsonInput.NullValueInOtherOneofOldFormat.Validator
Recommended.Proto3.JsonInput.RepeatedFieldMessageElementIsNull
Recommended.Proto3.JsonInput.RepeatedFieldPrimitiveElementIsNull
Recommended.Proto3.JsonInput.StringEndsWithEscapeChar
Recommended.Proto3.JsonInput.StringFieldSurrogateInWrongOrder
Recommended.Proto3.JsonInput.StringFieldUnpairedHighSurrogate
Recommended.Proto3.JsonInput.StringFieldUnpairedLowSurrogate
Recommended.Proto3.JsonInput.TimestampHas3FractionalDigits.Validator
Recommended.Proto3.JsonInput.TimestampHas6FractionalDigits.Validator
Recommended.Proto3.ProtobufInput.ValidDataRepeated.BOOL.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.BOOL.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.BOOL.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.BOOL.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.DOUBLE.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.DOUBLE.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.DOUBLE.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.DOUBLE.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.ENUM.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.ENUM.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.BOOL.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.BOOL.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.DOUBLE.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.DOUBLE.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.ENUM.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.ENUM.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.FIXED32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.FIXED32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.FIXED64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.FIXED64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.FLOAT.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.FLOAT.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.INT32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.INT32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.INT64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.INT64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SFIXED32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SFIXED32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SFIXED64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SFIXED64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SINT32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SINT32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SINT64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.SINT64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.UINT32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.UINT32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.UINT64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto2.ProtobufInput.ValidDataRepeated.UINT64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataOneofBinary.MESSAGE.Merge.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.BOOL.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.BOOL.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.DOUBLE.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.DOUBLE.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.ENUM.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.ENUM.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.ENUM.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.ENUM.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED32.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED32.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED64.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED64.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT32.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT32.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT64.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT64.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED32.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED32.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED64.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED64.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT32.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT32.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT64.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT64.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.PackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.PackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.UnpackedInput.DefaultOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.UnpackedInput.PackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataScalarBinary.ENUM[3].ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataScalarBinary.ENUM[4].ProtobufOutput
Required.DurationProtoInputTooLarge.JsonOutput
Required.DurationProtoInputTooSmall.JsonOutput
Required.Proto2.JsonInput.StoresDefaultPrimitive.Validator
Required.Proto3.JsonInput.DoubleFieldMaxNegativeValue.JsonOutput
Required.Proto3.JsonInput.DoubleFieldMaxNegativeValue.ProtobufOutput
Required.Proto3.JsonInput.DoubleFieldMinPositiveValue.JsonOutput
Required.Proto3.JsonInput.DoubleFieldMinPositiveValue.ProtobufOutput
Required.Proto3.JsonInput.DoubleFieldNan.JsonOutput
Required.Proto3.JsonInput.DurationMinValue.JsonOutput
Required.Proto3.JsonInput.DurationRepeatedValue.JsonOutput
Required.Proto3.JsonInput.FloatFieldInfinity.JsonOutput
Required.Proto3.JsonInput.FloatFieldNan.JsonOutput
Required.Proto3.JsonInput.FloatFieldNegativeInfinity.JsonOutput
Required.Proto3.JsonInput.IgnoreUnknownJsonFalse.ProtobufOutput
Required.Proto3.JsonInput.IgnoreUnknownJsonNull.ProtobufOutput
Required.Proto3.JsonInput.IgnoreUnknownJsonNumber.ProtobufOutput
Required.Proto3.JsonInput.IgnoreUnknownJsonObject.ProtobufOutput
Required.Proto3.JsonInput.IgnoreUnknownJsonString.ProtobufOutput
Required.Proto3.JsonInput.IgnoreUnknownJsonTrue.ProtobufOutput
Required.Proto3.JsonInput.OneofFieldDuplicate
Required.Proto3.JsonInput.RejectTopLevelNull
Required.Proto3.JsonInput.StringFieldSurrogatePair.JsonOutput
Required.Proto3.JsonInput.StringFieldSurrogatePair.ProtobufOutput
Required.Proto3.ProtobufInput.DoubleFieldNormalizeQuietNan.JsonOutput
Required.Proto3.ProtobufInput.DoubleFieldNormalizeSignalingNan.JsonOutput
Required.Proto3.ProtobufInput.FloatFieldNormalizeQuietNan.JsonOutput
Required.Proto3.ProtobufInput.FloatFieldNormalizeSignalingNan.JsonOutput
Required.Proto3.ProtobufInput.ValidDataMap.STRING.MESSAGE.MissingDefault.JsonOutput
Required.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.PackedInput.JsonOutput
Required.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.UnpackedInput.JsonOutput
Required.Proto3.ProtobufInput.ValidDataScalar.FLOAT[2].JsonOutput
Required.TimestampProtoInputTooLarge.JsonOutput
Required.TimestampProtoInputTooSmall.JsonOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED32.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED32.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED64.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FIXED64.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.FLOAT.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT32.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT32.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT64.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.INT64.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED32.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED32.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED64.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SFIXED64.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT32.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT32.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT64.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.SINT64.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT32.UnpackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.PackedInput.UnpackedOutput.ProtobufOutput
Recommended.Proto3.ProtobufInput.ValidDataRepeated.UINT64.UnpackedInput.UnpackedOutput.ProtobufOutput

3
kokoro/linux/dockerfile/test/ruby/Dockerfile
Unescape View File

@ -26,13 +26,14 @@ RUN apt-get update && apt-get install -y \
RUN gpg --keyserver hkp://keys.gnupg.net --recv-keys \
409B6B1796C275462A1703113804BB82D39DC0E3 \
7D2BAF1CF37B13E2069D6956105BD0E739499BDB
RUN \curl -sSL https://get.rvm.io | bash -s stable
RUN \curl -sSL https://get.rvm.io | bash -s master
RUN /bin/bash -l -c "rvm install 2.3.8"
RUN /bin/bash -l -c "rvm install 2.4.5"
RUN /bin/bash -l -c "rvm install 2.5.1"
RUN /bin/bash -l -c "rvm install 2.6.0"
RUN /bin/bash -l -c "rvm install 2.7.0"
RUN /bin/bash -l -c "rvm install 3.0.0"
RUN /bin/bash -l -c "echo 'gem: --no-ri --no-rdoc' > ~/.gemrc"
RUN /bin/bash -l -c "echo 'export PATH=/usr/local/rvm/bin:$PATH' >> ~/.bashrc"

18
kokoro/linux/ruby30/build.sh
Unescape View File

@ -0,0 +1,18 @@
#!/bin/bash
#
# This is the top-level script we give to Kokoro as the entry point for
# running the "pull request" project:
#
# This script selects a specific Dockerfile (for building a Docker image) and
# a script to run inside that image. Then we delegate to the general
# build_and_run_docker.sh script.
# Change to repo root
cd $(dirname $0)/../../..
export DOCKERHUB_ORGANIZATION=protobuftesting
export DOCKERFILE_DIR=kokoro/linux/dockerfile/test/ruby
export DOCKER_RUN_SCRIPT=kokoro/linux/pull_request_in_docker.sh
export OUTPUT_DIR=testoutput
export TEST_SET="ruby30"
./kokoro/linux/build_and_run_docker.sh

11
kokoro/linux/ruby30/continuous.cfg
Unescape View File

@ -0,0 +1,11 @@
# Config file for running tests in Kokoro
# Location of the build script in repository
build_file: "protobuf/kokoro/linux/ruby30/build.sh"
timeout_mins: 120
action {
define_artifacts {
regex: "**/sponge_log.xml"
}
}

11
kokoro/linux/ruby30/presubmit.cfg
Unescape View File

@ -0,0 +1,11 @@
# Config file for running tests in Kokoro
# Location of the build script in repository
build_file: "protobuf/kokoro/linux/ruby30/build.sh"
timeout_mins: 120
action {
define_artifacts {
regex: "**/sponge_log.xml"
}
}

2
kokoro/macos/prepare_build_macos_rc
Unescape View File

@ -80,5 +80,5 @@ if [[ "${KOKORO_INSTALL_RVM:-}" == "yes" ]] ; then
curl -sSL https://rvm.io/mpapis.asc | gpg --import -
curl -sSL https://rvm.io/pkuczynski.asc | gpg --import -
curl -sSL https://get.rvm.io | bash -s stable --ruby
curl -sSL https://get.rvm.io | bash -s master --ruby
fi

13
kokoro/macos/ruby30/build.sh
Unescape View File

@ -0,0 +1,13 @@
#!/bin/bash
#
# Build file to set up and run tests
# Change to repo root
cd $(dirname $0)/../../..
# Prepare worker environment to run tests
KOKORO_INSTALL_RUBY=yes
KOKORO_INSTALL_RVM=yes
source kokoro/macos/prepare_build_macos_rc
./tests.sh ruby30

5
kokoro/macos/ruby30/continuous.cfg
Unescape View File

@ -0,0 +1,5 @@
# Config file for running tests in Kokoro
# Location of the build script in repository
build_file: "protobuf/kokoro/macos/ruby30/build.sh"
timeout_mins: 1440

5
kokoro/macos/ruby30/presubmit.cfg
Unescape View File

@ -0,0 +1,5 @@
# Config file for running tests in Kokoro
# Location of the build script in repository
build_file: "protobuf/kokoro/macos/ruby30/build.sh"
timeout_mins: 1440

17
kokoro/release/ruby/macos/ruby/ruby_build_environment.sh
Unescape View File

@ -21,13 +21,13 @@ rm -rf ~/.rake-compiler
CROSS_RUBY=$(mktemp tmpfile.XXXXXXXX)
curl https://raw.githubusercontent.com/rake-compiler/rake-compiler/v1.1.0/tasks/bin/cross-ruby.rake > "$CROSS_RUBY"
curl https://raw.githubusercontent.com/rake-compiler/rake-compiler/72184e51779b6a3b9b8580b036a052fdc3181ced/tasks/bin/cross-ruby.rake > "$CROSS_RUBY"
# See https://github.com/grpc/grpc/issues/12161 for verconf.h patch details
patch "$CROSS_RUBY" << EOF
--- cross-ruby.rake 2018-04-10 11:32:16.000000000 -0700
+++ patched 2018-04-10 11:40:25.000000000 -0700
@@ -141,8 +141,10 @@
--- cross-ruby.rake 2020-12-11 11:17:53.000000000 +0900
+++ patched 2020-12-11 11:18:52.000000000 +0900
@@ -111,10 +111,12 @@
"--host=#{MINGW_HOST}",
"--target=#{MINGW_TARGET}",
"--build=#{RUBY_BUILD}",
@ -36,10 +36,13 @@ patch "$CROSS_RUBY" << EOF
+ '--disable-shared',
'--disable-install-doc',
+ '--without-gmp',
'--with-ext='
'--with-ext=',
- 'LDFLAGS=-pipe -s',
+ 'LDFLAGS=-pipe',
]
@@ -159,6 +161,7 @@
# Force Winsock2 for Ruby 1.8, 1.9 defaults to it
@@ -130,6 +132,7 @@
# make
file "#{build_dir}/ruby.exe" => ["#{build_dir}/Makefile"] do |t|
chdir File.dirname(t.prerequisites.first) do
@ -55,7 +58,7 @@ set +x # rvm commands are very verbose
rvm use 2.7.0
set -x
ruby --version | grep 'ruby 2.7.0'
for v in 2.7.0 ; do
for v in 3.0.0 2.7.0 ; do
ccache -c
rake -f "$CROSS_RUBY" cross-ruby VERSION="$v" HOST=x86_64-darwin11 MAKE="$MAKE"
done

10
ruby/Rakefile
Unescape View File

@ -51,6 +51,12 @@ if RUBY_PLATFORM == "java"
system("mvn --batch-mode package")
end
else
unless ENV['IN_DOCKER'] == 'true'
# We need wyhash in-tree.
FileUtils.mkdir_p("ext/google/protobuf_c/third_party/wyhash")
FileUtils.cp("../third_party/wyhash/wyhash.h", "ext/google/protobuf_c/third_party/wyhash/wyhash.h")
end
Rake::ExtensionTask.new("protobuf_c", spec) do |ext|
unless RUBY_PLATFORM =~ /darwin/
# TODO: also set "no_native to true" for mac if possible. As is,
@ -73,7 +79,7 @@ else
['x86-mingw32', 'x64-mingw32', 'x86_64-linux', 'x86-linux'].each do |plat|
RakeCompilerDock.sh <<-"EOT", platform: plat
bundle && \
IN_DOCKER=true rake native:#{plat} pkg/#{spec.full_name}-#{plat}.gem RUBY_CC_VERSION=2.7.0:2.6.0:2.5.0:2.4.0:2.3.0
IN_DOCKER=true rake native:#{plat} pkg/#{spec.full_name}-#{plat}.gem RUBY_CC_VERSION=3.0.0:2.7.0:2.6.0:2.5.0:2.4.0:2.3.0
EOT
end
end
@ -81,7 +87,7 @@ else
if RUBY_PLATFORM =~ /darwin/
task 'gem:native' do
system "rake genproto"
system "rake cross native gem RUBY_CC_VERSION=2.7.0:2.6.0:2.5.1:2.4.0:2.3.0"
system "rake cross native gem RUBY_CC_VERSION=3.0.0:2.7.0:2.6.0:2.5.1:2.4.0:2.3.0"
end
else
task 'gem:native' => [:genproto, 'gem:windows']

6
ruby/compatibility_tests/v3.0.0/tests/basic.rb
Unescape View File

@ -1264,10 +1264,10 @@ module BasicTest
m = MapMessage.new(:map_string_int32 => {"a" => 1})
expected = '{"mapStringInt32":{"a":1},"mapStringMsg":{}}'
expected_preserve = '{"map_string_int32":{"a":1},"map_string_msg":{}}'
assert MapMessage.encode_json(m) == expected
assert_equal expected, MapMessage.encode_json(m, :emit_defaults => true)
json = MapMessage.encode_json(m, :preserve_proto_fieldnames => true)
assert json == expected_preserve
json = MapMessage.encode_json(m, :preserve_proto_fieldnames => true, :emit_defaults => true)
assert_equal expected_preserve, json
m2 = MapMessage.decode_json(MapMessage.encode_json(m))
assert m == m2

349
ruby/ext/google/protobuf_c/convert.c
Unescape View File

@ -0,0 +1,349 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// -----------------------------------------------------------------------------
// Ruby <-> upb data conversion functions.
//
// This file Also contains a few other assorted algorithms on upb_msgval.
//
// None of the algorithms in this file require any access to the internal
// representation of Ruby or upb objects.
// -----------------------------------------------------------------------------
#include "convert.h"
#include "message.h"
#include "protobuf.h"
#include "third_party/wyhash/wyhash.h"
static upb_strview Convert_StringData(VALUE str, upb_arena *arena) {
upb_strview ret;
if (arena) {
char *ptr = upb_arena_malloc(arena, RSTRING_LEN(str));
memcpy(ptr, RSTRING_PTR(str), RSTRING_LEN(str));
ret.data = ptr;
} else {
// Data is only needed temporarily (within map lookup).
ret.data = RSTRING_PTR(str);
}
ret.size = RSTRING_LEN(str);
return ret;
}
static bool is_ruby_num(VALUE value) {
return (TYPE(value) == T_FLOAT ||
TYPE(value) == T_FIXNUM ||
TYPE(value) == T_BIGNUM);
}
static void Convert_CheckInt(const char* name, upb_fieldtype_t type,
VALUE val) {
if (!is_ruby_num(val)) {
rb_raise(cTypeError,
"Expected number type for integral field '%s' (given %s).", name,
rb_class2name(CLASS_OF(val)));
}
// NUM2{INT,UINT,LL,ULL} macros do the appropriate range checks on upper
// bound; we just need to do precision checks (i.e., disallow rounding) and
// check for < 0 on unsigned types.
if (TYPE(val) == T_FLOAT) {
double dbl_val = NUM2DBL(val);
if (floor(dbl_val) != dbl_val) {
rb_raise(rb_eRangeError,
"Non-integral floating point value assigned to integer field "
"'%s' (given %s).",
name, rb_class2name(CLASS_OF(val)));
}
}
if (type == UPB_TYPE_UINT32 || type == UPB_TYPE_UINT64) {
if (NUM2DBL(val) < 0) {
rb_raise(
rb_eRangeError,
"Assigning negative value to unsigned integer field '%s' (given %s).",
name, rb_class2name(CLASS_OF(val)));
}
}
}
static int32_t Convert_ToEnum(VALUE value, const char* name,
const upb_enumdef* e) {
int32_t val;
switch (TYPE(value)) {
case T_FLOAT:
case T_FIXNUM:
case T_BIGNUM:
Convert_CheckInt(name, UPB_TYPE_INT32, value);
val = NUM2INT(value);
break;
case T_STRING:
if (!upb_enumdef_ntoi(e, RSTRING_PTR(value), RSTRING_LEN(value), &val)) {
goto unknownval;
}
break;
case T_SYMBOL:
if (!upb_enumdef_ntoiz(e, rb_id2name(SYM2ID(value)), &val)) {
goto unknownval;
}
break;
default:
rb_raise(cTypeError,
"Expected number or symbol type for enum field '%s'.", name);
}
return val;
unknownval:
rb_raise(rb_eRangeError, "Unknown symbol value for enum field '%s'.", name);
}
upb_msgval Convert_RubyToUpb(VALUE value, const char* name, TypeInfo type_info,
upb_arena* arena) {
upb_msgval ret;
switch (type_info.type) {
case UPB_TYPE_FLOAT:
if (!is_ruby_num(value)) {
rb_raise(cTypeError, "Expected number type for float field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
ret.float_val = NUM2DBL(value);
break;
case UPB_TYPE_DOUBLE:
if (!is_ruby_num(value)) {
rb_raise(cTypeError, "Expected number type for double field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
ret.double_val = NUM2DBL(value);
break;
case UPB_TYPE_BOOL: {
if (value == Qtrue) {
ret.bool_val = 1;
} else if (value == Qfalse) {
ret.bool_val = 0;
} else {
rb_raise(cTypeError, "Invalid argument for boolean field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
break;
}
case UPB_TYPE_STRING: {
VALUE utf8 = rb_enc_from_encoding(rb_utf8_encoding());
if (CLASS_OF(value) == rb_cSymbol) {
value = rb_funcall(value, rb_intern("to_s"), 0);
} else if (CLASS_OF(value) != rb_cString) {
rb_raise(cTypeError, "Invalid argument for string field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
if (rb_obj_encoding(value) != utf8) {
// Note: this will not duplicate underlying string data unless necessary.
value = rb_str_encode(value, utf8, 0, Qnil);
if (rb_enc_str_coderange(value) == ENC_CODERANGE_BROKEN) {
rb_raise(rb_eEncodingError, "String is invalid UTF-8");
}
}
ret.str_val = Convert_StringData(value, arena);
break;
}
case UPB_TYPE_BYTES: {
VALUE bytes = rb_enc_from_encoding(rb_ascii8bit_encoding());
if (CLASS_OF(value) != rb_cString) {
rb_raise(cTypeError, "Invalid argument for bytes field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
if (rb_obj_encoding(value) != bytes) {
// Note: this will not duplicate underlying string data unless necessary.
// TODO(haberman): is this really necessary to get raw bytes?
value = rb_str_encode(value, bytes, 0, Qnil);
}
ret.str_val = Convert_StringData(value, arena);
break;
}
case UPB_TYPE_MESSAGE:
ret.msg_val =
Message_GetUpbMessage(value, type_info.def.msgdef, name, arena);
break;
case UPB_TYPE_ENUM:
ret.int32_val = Convert_ToEnum(value, name, type_info.def.enumdef);
break;
case UPB_TYPE_INT32:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT32:
case UPB_TYPE_UINT64:
Convert_CheckInt(name, type_info.type, value);
switch (type_info.type) {
case UPB_TYPE_INT32:
ret.int32_val = NUM2INT(value);
break;
case UPB_TYPE_INT64:
ret.int64_val = NUM2LL(value);
break;
case UPB_TYPE_UINT32:
ret.uint32_val = NUM2UINT(value);
break;
case UPB_TYPE_UINT64:
ret.uint64_val = NUM2ULL(value);
break;
default:
break;
}
break;
default:
break;
}
return ret;
}
VALUE Convert_UpbToRuby(upb_msgval upb_val, TypeInfo type_info, VALUE arena) {
switch (type_info.type) {
case UPB_TYPE_FLOAT:
return DBL2NUM(upb_val.float_val);
case UPB_TYPE_DOUBLE:
return DBL2NUM(upb_val.double_val);
case UPB_TYPE_BOOL:
return upb_val.bool_val ? Qtrue : Qfalse;
case UPB_TYPE_INT32:
return INT2NUM(upb_val.int32_val);
case UPB_TYPE_INT64:
return LL2NUM(upb_val.int64_val);
case UPB_TYPE_UINT32:
return UINT2NUM(upb_val.uint32_val);
case UPB_TYPE_UINT64:
return ULL2NUM(upb_val.int64_val);
case UPB_TYPE_ENUM: {
const char* name =
upb_enumdef_iton(type_info.def.enumdef, upb_val.int32_val);
if (name) {
return ID2SYM(rb_intern(name));
} else {
return INT2NUM(upb_val.int32_val);
}
}
case UPB_TYPE_STRING: {
VALUE str_rb = rb_str_new(upb_val.str_val.data, upb_val.str_val.size);
rb_enc_associate(str_rb, rb_utf8_encoding());
rb_obj_freeze(str_rb);
return str_rb;
}
case UPB_TYPE_BYTES: {
VALUE str_rb = rb_str_new(upb_val.str_val.data, upb_val.str_val.size);
rb_enc_associate(str_rb, rb_ascii8bit_encoding());
rb_obj_freeze(str_rb);
return str_rb;
}
case UPB_TYPE_MESSAGE:
return Message_GetRubyWrapper((upb_msg*)upb_val.msg_val,
type_info.def.msgdef, arena);
default:
rb_raise(rb_eRuntimeError, "Convert_UpbToRuby(): Unexpected type %d",
(int)type_info.type);
}
}
upb_msgval Msgval_DeepCopy(upb_msgval msgval, TypeInfo type_info,
upb_arena* arena) {
upb_msgval new_msgval;
switch (type_info.type) {
default:
memcpy(&new_msgval, &msgval, sizeof(msgval));
break;
case UPB_TYPE_STRING:
case UPB_TYPE_BYTES: {
size_t n = msgval.str_val.size;
char *mem = upb_arena_malloc(arena, n);
new_msgval.str_val.data = mem;
new_msgval.str_val.size = n;
memcpy(mem, msgval.str_val.data, n);
break;
}
case UPB_TYPE_MESSAGE:
new_msgval.msg_val =
Message_deep_copy(msgval.msg_val, type_info.def.msgdef, arena);
break;
}
return new_msgval;
}
bool Msgval_IsEqual(upb_msgval val1, upb_msgval val2, TypeInfo type_info) {
switch (type_info.type) {
case UPB_TYPE_BOOL:
return memcmp(&val1, &val2, 1) == 0;
case UPB_TYPE_FLOAT:
case UPB_TYPE_INT32:
case UPB_TYPE_UINT32:
case UPB_TYPE_ENUM:
return memcmp(&val1, &val2, 4) == 0;
case UPB_TYPE_DOUBLE:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT64:
return memcmp(&val1, &val2, 8) == 0;
case UPB_TYPE_STRING:
case UPB_TYPE_BYTES:
return val1.str_val.size != val2.str_val.size ||
memcmp(val1.str_val.data, val2.str_val.data,
val1.str_val.size) == 0;
case UPB_TYPE_MESSAGE:
return Message_Equal(val1.msg_val, val2.msg_val, type_info.def.msgdef);
default:
rb_raise(rb_eRuntimeError, "Internal error, unexpected type");
}
}
uint64_t Msgval_GetHash(upb_msgval val, TypeInfo type_info, uint64_t seed) {
switch (type_info.type) {
case UPB_TYPE_BOOL:
return wyhash(&val, 1, seed, _wyp);
case UPB_TYPE_FLOAT:
case UPB_TYPE_INT32:
case UPB_TYPE_UINT32:
case UPB_TYPE_ENUM:
return wyhash(&val, 4, seed, _wyp);
case UPB_TYPE_DOUBLE:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT64:
return wyhash(&val, 8, seed, _wyp);
case UPB_TYPE_STRING:
case UPB_TYPE_BYTES:
return wyhash(val.str_val.data, val.str_val.size, seed, _wyp);
case UPB_TYPE_MESSAGE:
return Message_Hash(val.msg_val, type_info.def.msgdef, seed);
default:
rb_raise(rb_eRuntimeError, "Internal error, unexpected type");
}
}

72
ruby/ext/google/protobuf_c/convert.h
Unescape View File

@ -0,0 +1,72 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef RUBY_PROTOBUF_CONVERT_H_
#define RUBY_PROTOBUF_CONVERT_H_
#include <ruby/ruby.h>
#include "protobuf.h"
#include "ruby-upb.h"
// Converts |ruby_val| to a upb_msgval according to |type_info|.
//
// The |arena| parameter indicates the lifetime of the container where this
// value will be assigned. It is used as follows:
// - If type is string or bytes, the string data will be copied into |arena|.
// - If type is message, and we need to auto-construct a message due to implicit
// conversions (eg. Time -> Google::Protobuf::Timestamp), the new message
// will be created in |arena|.
// - If type is message and the Ruby value is a message instance, we will fuse
// the message's arena into |arena|, to ensure that this message outlives the
// container.
upb_msgval Convert_RubyToUpb(VALUE ruby_val, const char *name,
TypeInfo type_info, upb_arena *arena);
// Converts |upb_val| to a Ruby VALUE according to |type_info|. This may involve
// creating a Ruby wrapper object.
//
// The |arena| parameter indicates the arena that owns the lifetime of
// |upb_val|. Any Ruby wrapper object that is created will reference |arena|
// and ensure it outlives the wrapper.
VALUE Convert_UpbToRuby(upb_msgval upb_val, TypeInfo type_info, VALUE arena);
// Creates a deep copy of |msgval| in |arena|.
upb_msgval Msgval_DeepCopy(upb_msgval msgval, TypeInfo type_info,
upb_arena *arena);
// Returns true if |val1| and |val2| are equal. Their type is given by
// |type_info|.
bool Msgval_IsEqual(upb_msgval val1, upb_msgval val2, TypeInfo type_info);
// Returns a hash value for the given upb_msgval.
uint64_t Msgval_GetHash(upb_msgval val, TypeInfo type_info, uint64_t seed);
#endif // RUBY_PROTOBUF_CONVERT_H_

1655
ruby/ext/google/protobuf_c/defs.c
View File

File diff suppressed because it is too large Load Diff

107
ruby/ext/google/protobuf_c/defs.h
Unescape View File

@ -0,0 +1,107 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef RUBY_PROTOBUF_DEFS_H_
#define RUBY_PROTOBUF_DEFS_H_
#include <ruby/ruby.h>
#include "protobuf.h"
#include "ruby-upb.h"
// -----------------------------------------------------------------------------
// TypeInfo
// -----------------------------------------------------------------------------
// This bundles a upb_fieldtype_t and msgdef/enumdef when appropriate. This is
// convenient for functions that need type information but cannot necessarily
// assume a upb_fielddef will be available.
//
// For example, Google::Protobuf::Map and Google::Protobuf::RepeatedField can
// be constructed with type information alone:
//
// # RepeatedField will internally store the type information in a TypeInfo.
// Google::Protobuf::RepeatedField.new(:message, FooMessage)
typedef struct {
upb_fieldtype_t type;
union {
const upb_msgdef* msgdef; // When type == UPB_TYPE_MESSAGE
const upb_enumdef* enumdef; // When type == UPB_TYPE_ENUM
} def;
} TypeInfo;
static inline TypeInfo TypeInfo_get(const upb_fielddef *f) {
TypeInfo ret = {upb_fielddef_type(f), {NULL}};
switch (ret.type) {
case UPB_TYPE_MESSAGE:
ret.def.msgdef = upb_fielddef_msgsubdef(f);
break;
case UPB_TYPE_ENUM:
ret.def.enumdef = upb_fielddef_enumsubdef(f);
break;
default:
break;
}
return ret;
}
TypeInfo TypeInfo_FromClass(int argc, VALUE* argv, int skip_arg,
VALUE* type_class, VALUE* init_arg);
static inline TypeInfo TypeInfo_from_type(upb_fieldtype_t type) {
TypeInfo ret = {type};
assert(type != UPB_TYPE_MESSAGE && type != UPB_TYPE_ENUM);
return ret;
}
// -----------------------------------------------------------------------------
// Other utilities
// -----------------------------------------------------------------------------
VALUE Descriptor_DefToClass(const upb_msgdef *m);
// Returns the underlying msgdef, enumdef, or symtab (respectively) for the
// given Descriptor, EnumDescriptor, or DescriptorPool Ruby object.
const upb_enumdef *EnumDescriptor_GetEnumDef(VALUE enum_desc_rb);
const upb_symtab *DescriptorPool_GetSymtab(VALUE desc_pool_rb);
const upb_msgdef *Descriptor_GetMsgDef(VALUE desc_rb);
// Returns a upb field type for the given Ruby symbol
// (eg. :float => UPB_TYPE_FLOAT).
upb_fieldtype_t ruby_to_fieldtype(VALUE type);
// The singleton generated pool (a DescriptorPool object).
extern VALUE generated_pool;
// Call at startup to register all types in this module.
void Defs_register(VALUE module);
#endif // RUBY_PROTOBUF_DEFS_H_

1795
ruby/ext/google/protobuf_c/encode_decode.c
View File

File diff suppressed because it is too large Load Diff

9
ruby/ext/google/protobuf_c/extconf.rb
Unescape View File

@ -3,9 +3,9 @@
require 'mkmf'
if RUBY_PLATFORM =~ /darwin/ || RUBY_PLATFORM =~ /linux/
$CFLAGS += " -std=gnu90 -O3 -DNDEBUG -Wall -Wdeclaration-after-statement -Wsign-compare"
$CFLAGS += " -std=gnu99 -O3 -DNDEBUG -fvisibility=hidden -Wall -Wsign-compare -Wno-declaration-after-statement"
else
$CFLAGS += " -std=gnu90 -O3 -DNDEBUG"
$CFLAGS += " -std=gnu99 -O3 -DNDEBUG"
end
@ -14,8 +14,7 @@ if RUBY_PLATFORM =~ /linux/
$LDFLAGS += " -Wl,-wrap,memcpy"
end
$objs = ["protobuf.o", "defs.o", "storage.o", "message.o",
"repeated_field.o", "map.o", "encode_decode.o", "upb.o",
"wrap_memcpy.o"]
$objs = ["protobuf.o", "convert.o", "defs.o", "message.o",
"repeated_field.o", "map.o", "ruby-upb.o", "wrap_memcpy.o"]
create_makefile("google/protobuf_c")

764
ruby/ext/google/protobuf_c/map.c
Unescape View File

@ -28,170 +28,231 @@
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "convert.h"
#include "defs.h"
#include "message.h"
#include "protobuf.h"
// -----------------------------------------------------------------------------
// Basic map operations on top of upb's strtable.
// Basic map operations on top of upb_map.
//
// Note that we roll our own `Map` container here because, as for
// `RepeatedField`, we want a strongly-typed container. This is so that any user
// errors due to incorrect map key or value types are raised as close as
// possible to the error site, rather than at some deferred point (e.g.,
// serialization).
//
// We build our `Map` on top of upb_strtable so that we're able to take
// advantage of the native_slot storage abstraction, as RepeatedField does.
// (This is not quite a perfect mapping -- see the key conversions below -- but
// gives us full support and error-checking for all value types for free.)
// -----------------------------------------------------------------------------
// Map values are stored using the native_slot abstraction (as with repeated
// field values), but keys are a bit special. Since we use a strtable, we need
// to store keys as sequences of bytes such that equality of those bytes maps
// one-to-one to equality of keys. We store strings directly (i.e., they map to
// their own bytes) and integers as native integers (using the native_slot
// abstraction).
// Note that there is another tradeoff here in keeping string keys as native
// strings rather than Ruby strings: traversing the Map requires conversion to
// Ruby string values on every traversal, potentially creating more garbage. We
// should consider ways to cache a Ruby version of the key if this becomes an
// issue later.
// Forms a key to use with the underlying strtable from a Ruby key value. |buf|
// must point to TABLE_KEY_BUF_LENGTH bytes of temporary space, used to
// construct a key byte sequence if needed. |out_key| and |out_length| provide
// the resulting key data/length.
#define TABLE_KEY_BUF_LENGTH 8 // sizeof(uint64_t)
static VALUE table_key(Map* self, VALUE key,
char* buf,
const char** out_key,
size_t* out_length) {
switch (self->key_type) {
case UPB_TYPE_BYTES:
case UPB_TYPE_STRING:
// Strings: use string content directly.
if (TYPE(key) == T_SYMBOL) {
key = rb_id2str(SYM2ID(key));
}
Check_Type(key, T_STRING);
key = native_slot_encode_and_freeze_string(self->key_type, key);
*out_key = RSTRING_PTR(key);
*out_length = RSTRING_LEN(key);
break;
case UPB_TYPE_BOOL:
case UPB_TYPE_INT32:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT32:
case UPB_TYPE_UINT64:
native_slot_set("", self->key_type, Qnil, buf, key);
*out_key = buf;
*out_length = native_slot_size(self->key_type);
break;
default:
// Map constructor should not allow a Map with another key type to be
// constructed.
assert(false);
break;
}
return key;
}
static VALUE table_key_to_ruby(Map* self, upb_strview key) {
switch (self->key_type) {
case UPB_TYPE_BYTES:
case UPB_TYPE_STRING: {
VALUE ret = rb_str_new(key.data, key.size);
rb_enc_associate(ret,
(self->key_type == UPB_TYPE_BYTES) ?
kRubyString8bitEncoding : kRubyStringUtf8Encoding);
return ret;
}
case UPB_TYPE_BOOL:
case UPB_TYPE_INT32:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT32:
case UPB_TYPE_UINT64:
return native_slot_get(self->key_type, Qnil, key.data);
default:
assert(false);
return Qnil;
}
}
static void* value_memory(upb_value* v) {
return (void*)(&v->val);
}
// -----------------------------------------------------------------------------
// Map container type.
// -----------------------------------------------------------------------------
typedef struct {
const upb_map *map; // Can convert to mutable when non-frozen.
upb_fieldtype_t key_type;
TypeInfo value_type_info;
VALUE value_type_class;
VALUE arena;
} Map;
static void Map_mark(void* _self) {
Map* self = _self;
rb_gc_mark(self->value_type_class);
rb_gc_mark(self->arena);
}
const rb_data_type_t Map_type = {
"Google::Protobuf::Map",
{ Map_mark, Map_free, NULL },
{ Map_mark, RUBY_DEFAULT_FREE, NULL },
.flags = RUBY_TYPED_FREE_IMMEDIATELY,
};
VALUE cMap;
Map* ruby_to_Map(VALUE _self) {
static Map* ruby_to_Map(VALUE _self) {
Map* self;
TypedData_Get_Struct(_self, Map, &Map_type, self);
return self;
}
void Map_mark(void* _self) {
Map* self = _self;
static VALUE Map_alloc(VALUE klass) {
Map* self = ALLOC(Map);
self->map = NULL;
self->value_type_class = Qnil;
self->value_type_info.def.msgdef = NULL;
self->arena = Qnil;
return TypedData_Wrap_Struct(klass, &Map_type, self);
}
rb_gc_mark(self->value_type_class);
rb_gc_mark(self->parse_frame);
if (self->value_type == UPB_TYPE_STRING ||
self->value_type == UPB_TYPE_BYTES ||
self->value_type == UPB_TYPE_MESSAGE) {
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
native_slot_mark(self->value_type, mem);
VALUE Map_GetRubyWrapper(upb_map* map, upb_fieldtype_t key_type,
TypeInfo value_type, VALUE arena) {
PBRUBY_ASSERT(map);
VALUE val = ObjectCache_Get(map);
if (val == Qnil) {
val = Map_alloc(cMap);
Map* self;
ObjectCache_Add(map, val, Arena_get(arena));
TypedData_Get_Struct(val, Map, &Map_type, self);
self->map = map;
self->arena = arena;
self->key_type = key_type;
self->value_type_info = value_type;
if (self->value_type_info.type == UPB_TYPE_MESSAGE) {
const upb_msgdef *val_m = self->value_type_info.def.msgdef;
self->value_type_class = Descriptor_DefToClass(val_m);
}
}
return val;
}
void Map_free(void* _self) {
Map* self = _self;
upb_strtable_uninit(&self->table);
xfree(self);
static VALUE Map_new_this_type(Map *from) {
VALUE arena_rb = Arena_new();
upb_map* map = upb_map_new(Arena_get(arena_rb), from->key_type,
from->value_type_info.type);
VALUE ret =
Map_GetRubyWrapper(map, from->key_type, from->value_type_info, arena_rb);
PBRUBY_ASSERT(ruby_to_Map(ret)->value_type_class == from->value_type_class);
return ret;
}
VALUE Map_alloc(VALUE klass) {
Map* self = ALLOC(Map);
memset(self, 0, sizeof(Map));
self->value_type_class = Qnil;
return TypedData_Wrap_Struct(klass, &Map_type, self);
static TypeInfo Map_keyinfo(Map* self) {
TypeInfo ret;
ret.type = self->key_type;
ret.def.msgdef = NULL;
return ret;
}
VALUE Map_set_frame(VALUE map, VALUE val) {
Map* self = ruby_to_Map(map);
self->parse_frame = val;
return val;
static upb_map *Map_GetMutable(VALUE _self) {
rb_check_frozen(_self);
return (upb_map*)ruby_to_Map(_self)->map;
}
static bool needs_typeclass(upb_fieldtype_t type) {
switch (type) {
case UPB_TYPE_MESSAGE:
case UPB_TYPE_ENUM:
return true;
default:
return false;
VALUE Map_CreateHash(const upb_map* map, upb_fieldtype_t key_type,
TypeInfo val_info) {
VALUE hash = rb_hash_new();
size_t iter = UPB_MAP_BEGIN;
TypeInfo key_info = TypeInfo_from_type(key_type);
if (!map) return hash;
while (upb_mapiter_next(map, &iter)) {
upb_msgval key = upb_mapiter_key(map, iter);
upb_msgval val = upb_mapiter_value(map, iter);
VALUE key_val = Convert_UpbToRuby(key, key_info, Qnil);
VALUE val_val = Scalar_CreateHash(val, val_info);
rb_hash_aset(hash, key_val, val_val);
}
return hash;
}
VALUE Map_deep_copy(VALUE obj) {
Map* self = ruby_to_Map(obj);
VALUE new_arena_rb = Arena_new();
upb_arena *arena = Arena_get(new_arena_rb);
upb_map* new_map =
upb_map_new(arena, self->key_type, self->value_type_info.type);
size_t iter = UPB_MAP_BEGIN;
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval key = upb_mapiter_key(self->map, iter);
upb_msgval val = upb_mapiter_value(self->map, iter);
upb_msgval val_copy = Msgval_DeepCopy(val, self->value_type_info, arena);
upb_map_set(new_map, key, val_copy, arena);
}
return Map_GetRubyWrapper(new_map, self->key_type, self->value_type_info,
new_arena_rb);
}
const upb_map* Map_GetUpbMap(VALUE val, const upb_fielddef *field) {
const upb_fielddef* key_field = map_field_key(field);
const upb_fielddef* value_field = map_field_value(field);
TypeInfo value_type_info = TypeInfo_get(value_field);
Map* self;
if (!RB_TYPE_P(val, T_DATA) || !RTYPEDDATA_P(val) ||
RTYPEDDATA_TYPE(val) != &Map_type) {
rb_raise(cTypeError, "Expected Map instance");
}
self = ruby_to_Map(val);
if (self->key_type != upb_fielddef_type(key_field)) {
rb_raise(cTypeError, "Map key type does not match field's key type");
}
if (self->value_type_info.type != value_type_info.type) {
rb_raise(cTypeError, "Map value type does not match field's value type");
}
if (self->value_type_info.def.msgdef != value_type_info.def.msgdef) {
rb_raise(cTypeError, "Map value type has wrong message/enum class");
}
return self->map;
}
void Map_Inspect(StringBuilder* b, const upb_map* map, upb_fieldtype_t key_type,
TypeInfo val_type) {
bool first = true;
TypeInfo key_type_info = {key_type};
StringBuilder_Printf(b, "{");
if (map) {
size_t iter = UPB_MAP_BEGIN;
while (upb_mapiter_next(map, &iter)) {
upb_msgval key = upb_mapiter_key(map, iter);
upb_msgval val = upb_mapiter_value(map, iter);
if (first) {
first = false;
} else {
StringBuilder_Printf(b, ", ");
}
StringBuilder_PrintMsgval(b, key, key_type_info);
StringBuilder_Printf(b, "=>");
StringBuilder_PrintMsgval(b, val, val_type);
}
}
StringBuilder_Printf(b, "}");
}
static int merge_into_self_callback(VALUE key, VALUE val, VALUE _self) {
Map* self = ruby_to_Map(_self);
upb_arena *arena = Arena_get(self->arena);
upb_msgval key_val = Convert_RubyToUpb(key, "", Map_keyinfo(self), arena);
upb_msgval val_val = Convert_RubyToUpb(val, "", self->value_type_info, arena);
upb_map_set(Map_GetMutable(_self), key_val, val_val, arena);
return ST_CONTINUE;
}
// Used only internally -- shared by #merge and #initialize.
static VALUE Map_merge_into_self(VALUE _self, VALUE hashmap) {
if (TYPE(hashmap) == T_HASH) {
rb_hash_foreach(hashmap, merge_into_self_callback, _self);
} else if (RB_TYPE_P(hashmap, T_DATA) && RTYPEDDATA_P(hashmap) &&
RTYPEDDATA_TYPE(hashmap) == &Map_type) {
Map* self = ruby_to_Map(_self);
Map* other = ruby_to_Map(hashmap);
upb_arena *arena = Arena_get(self->arena);
upb_msg *self_msg = Map_GetMutable(_self);
size_t iter = UPB_MAP_BEGIN;
upb_arena_fuse(arena, Arena_get(other->arena));
if (self->key_type != other->key_type ||
self->value_type_info.type != other->value_type_info.type ||
self->value_type_class != other->value_type_class) {
rb_raise(rb_eArgError, "Attempt to merge Map with mismatching types");
}
while (upb_mapiter_next(other->map, &iter)) {
upb_msgval key = upb_mapiter_key(other->map, iter);
upb_msgval val = upb_mapiter_value(other->map, iter);
upb_map_set(self_msg, key, val, arena);
}
} else {
rb_raise(rb_eArgError, "Unknown type merging into Map");
}
return _self;
}
/*
@ -224,9 +285,9 @@ static bool needs_typeclass(upb_fieldtype_t type) {
* references to underlying objects will be shared if the value type is a
* message type.
*/
VALUE Map_init(int argc, VALUE* argv, VALUE _self) {
static VALUE Map_init(int argc, VALUE* argv, VALUE _self) {
Map* self = ruby_to_Map(_self);
int init_value_arg;
VALUE init_arg;
// We take either two args (:key_type, :value_type), three args (:key_type,
// :value_type, "ValueMessageType"), or four args (the above plus an initial
@ -236,8 +297,9 @@ VALUE Map_init(int argc, VALUE* argv, VALUE _self) {
}
self->key_type = ruby_to_fieldtype(argv[0]);
self->value_type = ruby_to_fieldtype(argv[1]);
self->parse_frame = Qnil;
self->value_type_info =
TypeInfo_FromClass(argc, argv, 1, &self->value_type_class, &init_arg);
self->arena = Arena_new();
// Check that the key type is an allowed type.
switch (self->key_type) {
@ -254,21 +316,12 @@ VALUE Map_init(int argc, VALUE* argv, VALUE _self) {
rb_raise(rb_eArgError, "Invalid key type for map.");
}
init_value_arg = 2;
if (needs_typeclass(self->value_type) && argc > 2) {
self->value_type_class = argv[2];
validate_type_class(self->value_type, self->value_type_class);
init_value_arg = 3;
}
// Table value type is always UINT64: this ensures enough space to store the
// native_slot value.
if (!upb_strtable_init(&self->table, UPB_CTYPE_UINT64)) {
rb_raise(rb_eRuntimeError, "Could not allocate table.");
}
self->map = upb_map_new(Arena_get(self->arena), self->key_type,
self->value_type_info.type);
ObjectCache_Add(self->map, _self, Arena_get(self->arena));
if (argc > init_value_arg) {
Map_merge_into_self(_self, argv[init_value_arg]);
if (init_arg != Qnil) {
Map_merge_into_self(_self, init_arg);
}
return Qnil;
@ -282,22 +335,16 @@ VALUE Map_init(int argc, VALUE* argv, VALUE _self) {
* Note that Map also includes Enumerable; map thus acts like a normal Ruby
* sequence.
*/
VALUE Map_each(VALUE _self) {
static VALUE Map_each(VALUE _self) {
Map* self = ruby_to_Map(_self);
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
VALUE key = table_key_to_ruby(self, upb_strtable_iter_key(&it));
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
VALUE value = native_slot_get(self->value_type,
self->value_type_class,
mem);
rb_yield_values(2, key, value);
size_t iter = UPB_MAP_BEGIN;
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval key = upb_mapiter_key(self->map, iter);
upb_msgval val = upb_mapiter_value(self->map, iter);
VALUE key_val = Convert_UpbToRuby(key, Map_keyinfo(self), self->arena);
VALUE val_val = Convert_UpbToRuby(val, self->value_type_info, self->arena);
rb_yield_values(2, key_val, val_val);
}
return Qnil;
@ -309,17 +356,15 @@ VALUE Map_each(VALUE _self) {
*
* Returns the list of keys contained in the map, in unspecified order.
*/
VALUE Map_keys(VALUE _self) {
static VALUE Map_keys(VALUE _self) {
Map* self = ruby_to_Map(_self);
size_t iter = UPB_MAP_BEGIN;
VALUE ret = rb_ary_new();
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
VALUE key = table_key_to_ruby(self, upb_strtable_iter_key(&it));
rb_ary_push(ret, key);
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval key = upb_mapiter_key(self->map, iter);
VALUE key_val = Convert_UpbToRuby(key, Map_keyinfo(self), self->arena);
rb_ary_push(ret, key_val);
}
return ret;
@ -331,22 +376,15 @@ VALUE Map_keys(VALUE _self) {
*
* Returns the list of values contained in the map, in unspecified order.
*/
VALUE Map_values(VALUE _self) {
static VALUE Map_values(VALUE _self) {
Map* self = ruby_to_Map(_self);
size_t iter = UPB_MAP_BEGIN;
VALUE ret = rb_ary_new();
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
VALUE value = native_slot_get(self->value_type,
self->value_type_class,
mem);
rb_ary_push(ret, value);
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval val = upb_mapiter_value(self->map, iter);
VALUE val_val = Convert_UpbToRuby(val, self->value_type_info, self->arena);
rb_ary_push(ret, val_val);
}
return ret;
@ -359,18 +397,13 @@ VALUE Map_values(VALUE _self) {
* Accesses the element at the given key. Throws an exception if the key type is
* incorrect. Returns nil when the key is not present in the map.
*/
VALUE Map_index(VALUE _self, VALUE key) {
static VALUE Map_index(VALUE _self, VALUE key) {
Map* self = ruby_to_Map(_self);
upb_msgval key_upb = Convert_RubyToUpb(key, "", Map_keyinfo(self), NULL);
upb_msgval val;
char keybuf[TABLE_KEY_BUF_LENGTH];
const char* keyval = NULL;
size_t length = 0;
upb_value v;
key = table_key(self, key, keybuf, &keyval, &length);
if (upb_strtable_lookup2(&self->table, keyval, length, &v)) {
void* mem = value_memory(&v);
return native_slot_get(self->value_type, self->value_type_class, mem);
if (upb_map_get(self->map, key_upb, &val)) {
return Convert_UpbToRuby(val, self->value_type_info, self->arena);
} else {
return Qnil;
}
@ -384,33 +417,15 @@ VALUE Map_index(VALUE _self, VALUE key) {
* Throws an exception if the key type is incorrect. Returns the new value that
* was just inserted.
*/
VALUE Map_index_set(VALUE _self, VALUE key, VALUE value) {
static VALUE Map_index_set(VALUE _self, VALUE key, VALUE val) {
Map* self = ruby_to_Map(_self);
char keybuf[TABLE_KEY_BUF_LENGTH];
const char* keyval = NULL;
size_t length = 0;
upb_value v;
void* mem;
key = table_key(self, key, keybuf, &keyval, &length);
rb_check_frozen(_self);
upb_arena *arena = Arena_get(self->arena);
upb_msgval key_upb = Convert_RubyToUpb(key, "", Map_keyinfo(self), NULL);
upb_msgval val_upb = Convert_RubyToUpb(val, "", self->value_type_info, arena);
if (TYPE(value) == T_HASH) {
VALUE args[1] = { value };
value = rb_class_new_instance(1, args, self->value_type_class);
}
mem = value_memory(&v);
native_slot_set("", self->value_type, self->value_type_class, mem, value);
// Replace any existing value by issuing a 'remove' operation first.
upb_strtable_remove2(&self->table, keyval, length, NULL);
if (!upb_strtable_insert2(&self->table, keyval, length, v)) {
rb_raise(rb_eRuntimeError, "Could not insert into table");
}
upb_map_set(Map_GetMutable(_self), key_upb, val_upb, arena);
// Ruby hashmap's :[]= method also returns the inserted value.
return value;
return val;
}
/*
@ -420,15 +435,11 @@ VALUE Map_index_set(VALUE _self, VALUE key, VALUE value) {
* Returns true if the given key is present in the map. Throws an exception if
* the key has the wrong type.
*/
VALUE Map_has_key(VALUE _self, VALUE key) {
static VALUE Map_has_key(VALUE _self, VALUE key) {
Map* self = ruby_to_Map(_self);
upb_msgval key_upb = Convert_RubyToUpb(key, "", Map_keyinfo(self), NULL);
char keybuf[TABLE_KEY_BUF_LENGTH];
const char* keyval = NULL;
size_t length = 0;
key = table_key(self, key, keybuf, &keyval, &length);
if (upb_strtable_lookup2(&self->table, keyval, length, NULL)) {
if (upb_map_get(self->map, key_upb, NULL)) {
return Qtrue;
} else {
return Qfalse;
@ -442,22 +453,25 @@ VALUE Map_has_key(VALUE _self, VALUE key) {
* Deletes the value at the given key, if any, returning either the old value or
* nil if none was present. Throws an exception if the key is of the wrong type.
*/
VALUE Map_delete(VALUE _self, VALUE key) {
static VALUE Map_delete(VALUE _self, VALUE key) {
Map* self = ruby_to_Map(_self);
char keybuf[TABLE_KEY_BUF_LENGTH];
const char* keyval = NULL;
size_t length = 0;
upb_value v;
key = table_key(self, key, keybuf, &keyval, &length);
upb_msgval key_upb = Convert_RubyToUpb(key, "", Map_keyinfo(self), NULL);
upb_msgval val_upb;
VALUE ret;
rb_check_frozen(_self);
if (upb_strtable_remove2(&self->table, keyval, length, &v)) {
void* mem = value_memory(&v);
return native_slot_get(self->value_type, self->value_type_class, mem);
// TODO(haberman): make upb_map_delete() also capable of returning the deleted
// value.
if (upb_map_get(self->map, key_upb, &val_upb)) {
ret = Convert_UpbToRuby(val_upb, self->value_type_info, self->arena);
} else {
return Qnil;
ret = Qnil;
}
upb_map_delete(Map_GetMutable(_self), key_upb);
return ret;
}
/*
@ -466,17 +480,8 @@ VALUE Map_delete(VALUE _self, VALUE key) {
*
* Removes all entries from the map.
*/
VALUE Map_clear(VALUE _self) {
Map* self = ruby_to_Map(_self);
rb_check_frozen(_self);
// Uninit and reinit the table -- this is faster than iterating and doing a
// delete-lookup on each key.
upb_strtable_uninit(&self->table);
if (!upb_strtable_init(&self->table, UPB_CTYPE_INT64)) {
rb_raise(rb_eRuntimeError, "Unable to re-initialize table");
}
static VALUE Map_clear(VALUE _self) {
upb_map_clear(Map_GetMutable(_self));
return Qnil;
}
@ -486,24 +491,9 @@ VALUE Map_clear(VALUE _self) {
*
* Returns the number of entries (key-value pairs) in the map.
*/
VALUE Map_length(VALUE _self) {
static VALUE Map_length(VALUE _self) {
Map* self = ruby_to_Map(_self);
return ULL2NUM(upb_strtable_count(&self->table));
}
VALUE Map_new_this_type(VALUE _self) {
Map* self = ruby_to_Map(_self);
VALUE new_map = Qnil;
VALUE key_type = fieldtype_to_ruby(self->key_type);
VALUE value_type = fieldtype_to_ruby(self->value_type);
if (self->value_type_class != Qnil) {
new_map = rb_funcall(CLASS_OF(_self), rb_intern("new"), 3,
key_type, value_type, self->value_type_class);
} else {
new_map = rb_funcall(CLASS_OF(_self), rb_intern("new"), 2,
key_type, value_type);
}
return new_map;
return ULL2NUM(upb_map_size(self->map));
}
/*
@ -513,54 +503,23 @@ VALUE Map_new_this_type(VALUE _self) {
* Duplicates this map with a shallow copy. References to all non-primitive
* element objects (e.g., submessages) are shared.
*/
VALUE Map_dup(VALUE _self) {
Map* self = ruby_to_Map(_self);
VALUE new_map = Map_new_this_type(_self);
Map* new_self = ruby_to_Map(new_map);
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
upb_strview k = upb_strtable_iter_key(&it);
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
upb_value dup;
void* dup_mem = value_memory(&dup);
native_slot_dup(self->value_type, dup_mem, mem);
if (!upb_strtable_insert2(&new_self->table, k.data, k.size, dup)) {
rb_raise(rb_eRuntimeError, "Error inserting value into new table");
}
}
return new_map;
}
// Used by Google::Protobuf.deep_copy but not exposed directly.
VALUE Map_deep_copy(VALUE _self) {
static VALUE Map_dup(VALUE _self) {
Map* self = ruby_to_Map(_self);
VALUE new_map = Map_new_this_type(_self);
Map* new_self = ruby_to_Map(new_map);
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
upb_strview k = upb_strtable_iter_key(&it);
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
upb_value dup;
void* dup_mem = value_memory(&dup);
native_slot_deep_copy(self->value_type, self->value_type_class, dup_mem,
mem);
if (!upb_strtable_insert2(&new_self->table, k.data, k.size, dup)) {
rb_raise(rb_eRuntimeError, "Error inserting value into new table");
}
VALUE new_map_rb = Map_new_this_type(self);
Map* new_self = ruby_to_Map(new_map_rb);
size_t iter = UPB_MAP_BEGIN;
upb_arena *arena = Arena_get(new_self->arena);
upb_map *new_map = Map_GetMutable(new_map_rb);
upb_arena_fuse(arena, Arena_get(self->arena));
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval key = upb_mapiter_key(self->map, iter);
upb_msgval val = upb_mapiter_value(self->map, iter);
upb_map_set(new_map, key, val, arena);
}
return new_map;
return new_map_rb;
}
/*
@ -579,12 +538,11 @@ VALUE Map_deep_copy(VALUE _self) {
VALUE Map_eq(VALUE _self, VALUE _other) {
Map* self = ruby_to_Map(_self);
Map* other;
upb_strtable_iter it;
// Allow comparisons to Ruby hashmaps by converting to a temporary Map
// instance. Slow, but workable.
if (TYPE(_other) == T_HASH) {
VALUE other_map = Map_new_this_type(_self);
VALUE other_map = Map_new_this_type(self);
Map_merge_into_self(other_map, _other);
_other = other_map;
}
@ -595,33 +553,27 @@ VALUE Map_eq(VALUE _self, VALUE _other) {
return Qtrue;
}
if (self->key_type != other->key_type ||
self->value_type != other->value_type ||
self->value_type_info.type != other->value_type_info.type ||
self->value_type_class != other->value_type_class) {
return Qfalse;
}
if (upb_strtable_count(&self->table) != upb_strtable_count(&other->table)) {
if (upb_map_size(self->map) != upb_map_size(other->map)) {
return Qfalse;
}
// For each member of self, check that an equal member exists at the same key
// in other.
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
upb_strview k = upb_strtable_iter_key(&it);
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
upb_value other_v;
void* other_mem = value_memory(&other_v);
if (!upb_strtable_lookup2(&other->table, k.data, k.size, &other_v)) {
size_t iter = UPB_MAP_BEGIN;
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval key = upb_mapiter_key(self->map, iter);
upb_msgval val = upb_mapiter_value(self->map, iter);
upb_msgval other_val;
if (!upb_map_get(other->map, key, &other_val)) {
// Not present in other map.
return Qfalse;
}
if (!native_slot_eq(self->value_type, self->value_type_class, mem,
other_mem)) {
// Present, but value not equal.
if (!Msgval_IsEqual(val, other_val, self->value_type_info)) {
// Present but different value.
return Qfalse;
}
}
@ -629,6 +581,21 @@ VALUE Map_eq(VALUE _self, VALUE _other) {
return Qtrue;
}
/*
* call-seq:
* Message.freeze => self
*
* Freezes the message object. We have to intercept this so we can pin the
* Ruby object into memory so we don't forget it's frozen.
*/
static VALUE Map_freeze(VALUE _self) {
Map* self = ruby_to_Map(_self);
ObjectCache_Pin(self->map, _self, Arena_get(self->arena));
RB_OBJ_FREEZE(_self);
return _self;
}
/*
* call-seq:
* Map.hash => hash_value
@ -637,26 +604,18 @@ VALUE Map_eq(VALUE _self, VALUE _other) {
*/
VALUE Map_hash(VALUE _self) {
Map* self = ruby_to_Map(_self);
st_index_t h = rb_hash_start(0);
VALUE hash_sym = rb_intern("hash");
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table); !upb_strtable_done(&it);
upb_strtable_next(&it)) {
VALUE key = table_key_to_ruby(self, upb_strtable_iter_key(&it));
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
VALUE value = native_slot_get(self->value_type,
self->value_type_class,
mem);
h = rb_hash_uint(h, NUM2LONG(rb_funcall(key, hash_sym, 0)));
h = rb_hash_uint(h, NUM2LONG(rb_funcall(value, hash_sym, 0)));
uint64_t hash = 0;
size_t iter = UPB_MAP_BEGIN;
TypeInfo key_info = {self->key_type};
while (upb_mapiter_next(self->map, &iter)) {
upb_msgval key = upb_mapiter_key(self->map, iter);
upb_msgval val = upb_mapiter_value(self->map, iter);
hash = Msgval_GetHash(key, key_info, hash);
hash = Msgval_GetHash(val, self->value_type_info, hash);
}
return INT2FIX(h);
return LL2NUM(hash);
}
/*
@ -667,24 +626,7 @@ VALUE Map_hash(VALUE _self) {
*/
VALUE Map_to_h(VALUE _self) {
Map* self = ruby_to_Map(_self);
VALUE hash = rb_hash_new();
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
VALUE key = table_key_to_ruby(self, upb_strtable_iter_key(&it));
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
VALUE value = native_slot_get(self->value_type,
self->value_type_class,
mem);
if (self->value_type == UPB_TYPE_MESSAGE) {
value = Message_to_h(value);
}
rb_hash_aset(hash, key, value);
}
return hash;
return Map_CreateHash(self->map, self->key_type, self->value_type_info);
}
/*
@ -698,34 +640,11 @@ VALUE Map_to_h(VALUE _self) {
VALUE Map_inspect(VALUE _self) {
Map* self = ruby_to_Map(_self);
VALUE str = rb_str_new2("{");
bool first = true;
VALUE inspect_sym = rb_intern("inspect");
upb_strtable_iter it;
for (upb_strtable_begin(&it, &self->table); !upb_strtable_done(&it);
upb_strtable_next(&it)) {
VALUE key = table_key_to_ruby(self, upb_strtable_iter_key(&it));
upb_value v = upb_strtable_iter_value(&it);
void* mem = value_memory(&v);
VALUE value = native_slot_get(self->value_type,
self->value_type_class,
mem);
if (!first) {
str = rb_str_cat2(str, ", ");
} else {
first = false;
}
str = rb_str_append(str, rb_funcall(key, inspect_sym, 0));
str = rb_str_cat2(str, "=>");
str = rb_str_append(str, rb_funcall(value, inspect_sym, 0));
}
str = rb_str_cat2(str, "}");
return str;
StringBuilder* builder = StringBuilder_New();
Map_Inspect(builder, self->map, self->key_type, self->value_type_info);
VALUE ret = StringBuilder_ToRubyString(builder);
StringBuilder_Free(builder);
return ret;
}
/*
@ -737,79 +656,11 @@ VALUE Map_inspect(VALUE _self) {
* in the new copy of this map. Returns the new copy of this map with merged
* contents.
*/
VALUE Map_merge(VALUE _self, VALUE hashmap) {
static VALUE Map_merge(VALUE _self, VALUE hashmap) {
VALUE dupped = Map_dup(_self);
return Map_merge_into_self(dupped, hashmap);
}
static int merge_into_self_callback(VALUE key, VALUE value, VALUE self) {
Map_index_set(self, key, value);
return ST_CONTINUE;
}
// Used only internally -- shared by #merge and #initialize.
VALUE Map_merge_into_self(VALUE _self, VALUE hashmap) {
if (TYPE(hashmap) == T_HASH) {
rb_hash_foreach(hashmap, merge_into_self_callback, _self);
} else if (RB_TYPE_P(hashmap, T_DATA) && RTYPEDDATA_P(hashmap) &&
RTYPEDDATA_TYPE(hashmap) == &Map_type) {
Map* self = ruby_to_Map(_self);
Map* other = ruby_to_Map(hashmap);
upb_strtable_iter it;
if (self->key_type != other->key_type ||
self->value_type != other->value_type ||
self->value_type_class != other->value_type_class) {
rb_raise(rb_eArgError, "Attempt to merge Map with mismatching types");
}
for (upb_strtable_begin(&it, &other->table);
!upb_strtable_done(&it);
upb_strtable_next(&it)) {
upb_strview k = upb_strtable_iter_key(&it);
// Replace any existing value by issuing a 'remove' operation first.
upb_value v;
upb_value oldv;
upb_strtable_remove2(&self->table, k.data, k.size, &oldv);
v = upb_strtable_iter_value(&it);
upb_strtable_insert2(&self->table, k.data, k.size, v);
}
} else {
rb_raise(rb_eArgError, "Unknown type merging into Map");
}
return _self;
}
// Internal method: map iterator initialization (used for serialization).
void Map_begin(VALUE _self, Map_iter* iter) {
Map* self = ruby_to_Map(_self);
iter->self = self;
upb_strtable_begin(&iter->it, &self->table);
}
void Map_next(Map_iter* iter) {
upb_strtable_next(&iter->it);
}
bool Map_done(Map_iter* iter) {
return upb_strtable_done(&iter->it);
}
VALUE Map_iter_key(Map_iter* iter) {
return table_key_to_ruby(iter->self, upb_strtable_iter_key(&iter->it));
}
VALUE Map_iter_value(Map_iter* iter) {
upb_value v = upb_strtable_iter_value(&iter->it);
void* mem = value_memory(&v);
return native_slot_get(iter->self->value_type,
iter->self->value_type_class,
mem);
}
void Map_register(VALUE module) {
VALUE klass = rb_define_class_under(module, "Map", rb_cObject);
rb_define_alloc_func(klass, Map_alloc);
@ -828,6 +679,7 @@ void Map_register(VALUE module) {
rb_define_method(klass, "length", Map_length, 0);
rb_define_method(klass, "dup", Map_dup, 0);
rb_define_method(klass, "==", Map_eq, 1);
rb_define_method(klass, "freeze", Map_freeze, 0);
rb_define_method(klass, "hash", Map_hash, 0);
rb_define_method(klass, "to_h", Map_to_h, 0);
rb_define_method(klass, "inspect", Map_inspect, 0);

66
ruby/ext/google/protobuf_c/map.h
Unescape View File

@ -0,0 +1,66 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef RUBY_PROTOBUF_MAP_H_
#define RUBY_PROTOBUF_MAP_H_
#include <ruby/ruby.h>
#include "protobuf.h"
#include "ruby-upb.h"
// Returns a Ruby wrapper object for the given map, which will be created if
// one does not exist already.
VALUE Map_GetRubyWrapper(upb_map *map, upb_fieldtype_t key_type,
TypeInfo value_type, VALUE arena);
// Gets the underlying upb_map for this Ruby map object, which must have
// key/value type that match |field|. If this is not a map or the type doesn't
// match, raises an exception.
const upb_map *Map_GetUpbMap(VALUE val, const upb_fielddef *field);
// Implements #inspect for this map by appending its contents to |b|.
void Map_Inspect(StringBuilder *b, const upb_map *map, upb_fieldtype_t key_type,
TypeInfo val_type);
// Returns a new Hash object containing the contents of this Map.
VALUE Map_CreateHash(const upb_map* map, upb_fieldtype_t key_type,
TypeInfo val_info);
// Returns a deep copy of this Map object.
VALUE Map_deep_copy(VALUE obj);
// Ruby class of Google::Protobuf::Map.
extern VALUE cMap;
// Call at startup to register all types in this module.
void Map_register(VALUE module);
#endif // RUBY_PROTOBUF_MAP_H_

1322
ruby/ext/google/protobuf_c/message.c
View File

File diff suppressed because it is too large Load Diff

98
ruby/ext/google/protobuf_c/message.h
Unescape View File

@ -0,0 +1,98 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef RUBY_PROTOBUF_MESSAGE_H_
#define RUBY_PROTOBUF_MESSAGE_H_
#include <ruby/ruby.h>
#include "protobuf.h"
#include "ruby-upb.h"
// Gets the underlying upb_msg* and upb_msgdef for the given Ruby message
// wrapper. Requires that |value| is indeed a message object.
const upb_msg *Message_Get(VALUE value, const upb_msgdef **m);
// Like Message_Get(), but checks that the object is not frozen and returns a
// mutable pointer.
upb_msg *Message_GetMutable(VALUE value, const upb_msgdef **m);
// Returns the Arena object for this message.
VALUE Message_GetArena(VALUE value);
// Converts |value| into a upb_msg value of the expected upb_msgdef type,
// raising an error if this is not possible. Used when assigning |value| to a
// field of another message, which means the message must be of a particular
// type.
//
// This will perform automatic conversions in some cases (for example, Time ->
// Google::Protobuf::Timestamp). If any new message is created, it will be
// created on |arena|, and any existing message will have its arena fused with
// |arena|.
const upb_msg* Message_GetUpbMessage(VALUE value, const upb_msgdef* m,
const char* name, upb_arena* arena);
// Gets or constructs a Ruby wrapper object for the given message. The wrapper
// object will reference |arena| and ensure that it outlives this object.
VALUE Message_GetRubyWrapper(upb_msg* msg, const upb_msgdef* m, VALUE arena);
// Implements #inspect for this message, printing the text to |b|.
void Message_PrintMessage(StringBuilder* b, const upb_msg* msg,
const upb_msgdef* m);
// Returns a hash value for the given message.
uint64_t Message_Hash(const upb_msg *msg, const upb_msgdef *m, uint64_t seed);
// Returns a deep copy of the given message.
upb_msg* Message_deep_copy(const upb_msg* msg, const upb_msgdef* m,
upb_arena *arena);
// Returns true if these two messages are equal.
bool Message_Equal(const upb_msg *m1, const upb_msg *m2, const upb_msgdef *m);
// Checks that this Ruby object is a message, and raises an exception if not.
void Message_CheckClass(VALUE klass);
// Returns a new Hash object containing the contents of this message.
VALUE Scalar_CreateHash(upb_msgval val, TypeInfo type_info);
// Creates a message class or enum module for this descriptor, respectively.
VALUE build_class_from_descriptor(VALUE descriptor);
VALUE build_module_from_enumdesc(VALUE _enumdesc);
// Returns the Descriptor/EnumDescriptor for the given message class or enum
// module, respectively. Returns nil if this is not a message class or enum
// module.
VALUE MessageOrEnum_GetDescriptor(VALUE klass);
// Call at startup to register all types in this module.
void Message_register(VALUE protobuf);
#endif // RUBY_PROTOBUF_MESSAGE_H_

390
ruby/ext/google/protobuf_c/protobuf.c
Unescape View File

@ -30,62 +30,342 @@
#include "protobuf.h"
#include <ruby/version.h>
#include "defs.h"
#include "map.h"
#include "message.h"
#include "repeated_field.h"
VALUE cError;
VALUE cParseError;
VALUE cTypeError;
VALUE c_only_cookie = Qnil;
static VALUE cached_empty_string = Qnil;
static VALUE cached_empty_bytes = Qnil;
const upb_fielddef* map_field_key(const upb_fielddef* field) {
const upb_msgdef *entry = upb_fielddef_msgsubdef(field);
return upb_msgdef_itof(entry, 1);
}
const upb_fielddef* map_field_value(const upb_fielddef* field) {
const upb_msgdef *entry = upb_fielddef_msgsubdef(field);
return upb_msgdef_itof(entry, 2);
}
static VALUE create_frozen_string(const char* str, size_t size, bool binary) {
VALUE str_rb = rb_str_new(str, size);
// -----------------------------------------------------------------------------
// StringBuilder, for inspect
// -----------------------------------------------------------------------------
struct StringBuilder {
size_t size;
size_t cap;
char *data;
};
typedef struct StringBuilder StringBuilder;
static size_t StringBuilder_SizeOf(size_t cap) {
return sizeof(StringBuilder) + cap;
}
StringBuilder* StringBuilder_New() {
const size_t cap = 128;
StringBuilder* builder = malloc(sizeof(*builder));
builder->size = 0;
builder->cap = cap;
builder->data = malloc(builder->cap);
return builder;
}
void StringBuilder_Free(StringBuilder* b) {
free(b->data);
free(b);
}
void StringBuilder_Printf(StringBuilder* b, const char *fmt, ...) {
size_t have = b->cap - b->size;
size_t n;
va_list args;
va_start(args, fmt);
n = vsnprintf(&b->data[b->size], have, fmt, args);
va_end(args);
if (have <= n) {
while (have <= n) {
b->cap *= 2;
have = b->cap - b->size;
}
b->data = realloc(b->data, StringBuilder_SizeOf(b->cap));
va_start(args, fmt);
n = vsnprintf(&b->data[b->size], have, fmt, args);
va_end(args);
PBRUBY_ASSERT(n < have);
}
b->size += n;
}
rb_enc_associate(str_rb,
binary ? kRubyString8bitEncoding : kRubyStringUtf8Encoding);
rb_obj_freeze(str_rb);
return str_rb;
VALUE StringBuilder_ToRubyString(StringBuilder* b) {
VALUE ret = rb_str_new(b->data, b->size);
rb_enc_associate(ret, rb_utf8_encoding());
return ret;
}
VALUE get_frozen_string(const char* str, size_t size, bool binary) {
if (size == 0) {
return binary ? cached_empty_bytes : cached_empty_string;
static void StringBuilder_PrintEnum(StringBuilder* b, int32_t val,
const upb_enumdef* e) {
const char *name = upb_enumdef_iton(e, val);
if (name) {
StringBuilder_Printf(b, ":%s", name);
} else {
// It is harder to memoize non-empty strings. The obvious approach would be
// to use a Ruby hash keyed by string as memo table, but looking up in such a table
// requires constructing a string (the very thing we're trying to avoid).
//
// Since few fields have defaults, we will just optimize the empty string
// case for now.
return create_frozen_string(str, size, binary);
StringBuilder_Printf(b, "%" PRId32, val);
}
}
void StringBuilder_PrintMsgval(StringBuilder* b, upb_msgval val,
TypeInfo info) {
switch (info.type) {
case UPB_TYPE_BOOL:
StringBuilder_Printf(b, "%s", val.bool_val ? "true" : "false");
break;
case UPB_TYPE_FLOAT: {
VALUE str = rb_inspect(DBL2NUM(val.float_val));
StringBuilder_Printf(b, "%s", RSTRING_PTR(str));
break;
}
case UPB_TYPE_DOUBLE: {
VALUE str = rb_inspect(DBL2NUM(val.double_val));
StringBuilder_Printf(b, "%s", RSTRING_PTR(str));
break;
}
case UPB_TYPE_INT32:
StringBuilder_Printf(b, "%" PRId32, val.int32_val);
break;
case UPB_TYPE_UINT32:
StringBuilder_Printf(b, "%" PRIu32, val.uint32_val);
break;
case UPB_TYPE_INT64:
StringBuilder_Printf(b, "%" PRId64, val.int64_val);
break;
case UPB_TYPE_UINT64:
StringBuilder_Printf(b, "%" PRIu64, val.uint64_val);
break;
case UPB_TYPE_STRING:
StringBuilder_Printf(b, "\"%.*s\"", (int)val.str_val.size, val.str_val.data);
break;
case UPB_TYPE_BYTES:
StringBuilder_Printf(b, "\"%.*s\"", (int)val.str_val.size, val.str_val.data);
break;
case UPB_TYPE_ENUM:
StringBuilder_PrintEnum(b, val.int32_val, info.def.enumdef);
break;
case UPB_TYPE_MESSAGE:
Message_PrintMessage(b, val.msg_val, info.def.msgdef);
break;
}
}
// -----------------------------------------------------------------------------
// Utilities.
// Arena
// -----------------------------------------------------------------------------
// Raises a Ruby error if |status| is not OK, using its error message.
void check_upb_status(const upb_status* status, const char* msg) {
if (!upb_ok(status)) {
rb_raise(rb_eRuntimeError, "%s: %s\n", msg, upb_status_errmsg(status));
}
void Arena_free(void* data) { upb_arena_free(data); }
static VALUE cArena;
const rb_data_type_t Arena_type = {
"Google::Protobuf::Internal::Arena",
{ NULL, Arena_free, NULL },
};
static VALUE Arena_alloc(VALUE klass) {
upb_arena *arena = upb_arena_new();
return TypedData_Wrap_Struct(klass, &Arena_type, arena);
}
upb_arena *Arena_get(VALUE _arena) {
upb_arena *arena;
TypedData_Get_Struct(_arena, upb_arena, &Arena_type, arena);
return arena;
}
VALUE Arena_new() {
return Arena_alloc(cArena);
}
void Arena_register(VALUE module) {
VALUE internal = rb_define_module_under(module, "Internal");
VALUE klass = rb_define_class_under(internal, "Arena", rb_cObject);
rb_define_alloc_func(klass, Arena_alloc);
rb_gc_register_address(&cArena);
cArena = klass;
}
// String encodings: we look these up once, at load time, and then cache them
// here.
rb_encoding* kRubyStringUtf8Encoding;
rb_encoding* kRubyStringASCIIEncoding;
rb_encoding* kRubyString8bitEncoding;
// -----------------------------------------------------------------------------
// Object Cache
// -----------------------------------------------------------------------------
// Ruby-interned string: "descriptor". We use this identifier to store an
// instance variable on message classes we create in order to link them back to
// their descriptors.
// A pointer -> Ruby Object cache that keeps references to Ruby wrapper
// objects. This allows us to look up any Ruby wrapper object by the address
// of the object it is wrapping. That way we can avoid ever creating two
// different wrapper objects for the same C object, which saves memory and
// preserves object identity.
//
// We intern this once at module load time then use the interned identifier at
// runtime in order to avoid the cost of repeatedly interning in hot paths.
const char* kDescriptorInstanceVar = "descriptor";
ID descriptor_instancevar_interned;
// We use Hash and/or WeakMap for the cache. WeakMap is faster overall
// (probably due to removal being integrated with GC) but doesn't work for Ruby
// <2.7 (see note below). We need Hash for Ruby <2.7 and for cases where we
// need to GC-root the object (notably when the object has been frozen).
#if RUBY_API_VERSION_CODE >= 20700
#define USE_WEAK_MAP 1
#else
#define USE_WEAK_MAP 0
#endif
static VALUE ObjectCache_GetKey(const void* key) {
char buf[sizeof(key)];
memcpy(&buf, &key, sizeof(key));
intptr_t key_int = (intptr_t)key;
PBRUBY_ASSERT((key_int & 3) == 0);
return LL2NUM(key_int >> 2);
}
// Strong object cache, uses regular Hash and GC-roots objects.
// - For Ruby <2.7, used for all objects.
// - For Ruby >=2.7, used only for frozen objects, so we preserve the "frozen"
// bit (since this information is not preserved at the upb level).
VALUE strong_obj_cache = Qnil;
static void StrongObjectCache_Init() {
rb_gc_register_address(&strong_obj_cache);
strong_obj_cache = rb_hash_new();
}
static void StrongObjectCache_Remove(void* key) {
VALUE key_rb = ObjectCache_GetKey(key);
PBRUBY_ASSERT(rb_hash_lookup(strong_obj_cache, key_rb) != Qnil);
rb_hash_delete(strong_obj_cache, key_rb);
}
static VALUE StrongObjectCache_Get(const void* key) {
VALUE key_rb = ObjectCache_GetKey(key);
return rb_hash_lookup(strong_obj_cache, key_rb);
}
static void StrongObjectCache_Add(const void* key, VALUE val,
upb_arena* arena) {
PBRUBY_ASSERT(StrongObjectCache_Get(key) == Qnil);
VALUE key_rb = ObjectCache_GetKey(key);
rb_hash_aset(strong_obj_cache, key_rb, val);
upb_arena_addcleanup(arena, (void*)key, StrongObjectCache_Remove);
}
// Weak object cache. This speeds up the test suite significantly, so we
// presume it speeds up real code also. However we can only use it in Ruby
// >=2.7 due to:
// https://bugs.ruby-lang.org/issues/16035
#if USE_WEAK_MAP
VALUE weak_obj_cache = Qnil;
static void WeakObjectCache_Init() {
rb_gc_register_address(&weak_obj_cache);
VALUE klass = rb_eval_string("ObjectSpace::WeakMap");
weak_obj_cache = rb_class_new_instance(0, NULL, klass);
}
static VALUE WeakObjectCache_Get(const void* key) {
VALUE key_rb = ObjectCache_GetKey(key);
VALUE ret = rb_funcall(weak_obj_cache, rb_intern("[]"), 1, key_rb);
return ret;
}
static void WeakObjectCache_Add(const void* key, VALUE val) {
PBRUBY_ASSERT(WeakObjectCache_Get(key) == Qnil);
VALUE key_rb = ObjectCache_GetKey(key);
rb_funcall(weak_obj_cache, rb_intern("[]="), 2, key_rb, val);
PBRUBY_ASSERT(WeakObjectCache_Get(key) == val);
}
#endif
// Public ObjectCache API.
static void ObjectCache_Init() {
StrongObjectCache_Init();
#if USE_WEAK_MAP
WeakObjectCache_Init();
#endif
}
void ObjectCache_Add(const void* key, VALUE val, upb_arena *arena) {
#if USE_WEAK_MAP
(void)arena;
WeakObjectCache_Add(key, val);
#else
StrongObjectCache_Add(key, val, arena);
#endif
}
// Returns the cached object for this key, if any. Otherwise returns Qnil.
VALUE ObjectCache_Get(const void* key) {
#if USE_WEAK_MAP
return WeakObjectCache_Get(key);
#else
return StrongObjectCache_Get(key);
#endif
}
void ObjectCache_Pin(const void* key, VALUE val, upb_arena *arena) {
#if USE_WEAK_MAP
PBRUBY_ASSERT(WeakObjectCache_Get(key) == val);
// This will GC-root the object, but we'll still use the weak map for
// actual lookup.
StrongObjectCache_Add(key, val, arena);
#else
// Value is already pinned, nothing to do.
#endif
}
/*
* call-seq:
* Google::Protobuf.discard_unknown(msg)
*
* Discard unknown fields in the given message object and recursively discard
* unknown fields in submessages.
*/
static VALUE Google_Protobuf_discard_unknown(VALUE self, VALUE msg_rb) {
const upb_msgdef *m;
upb_msg *msg = Message_GetMutable(msg_rb, &m);
if (!upb_msg_discardunknown(msg, m, 128)) {
rb_raise(rb_eRuntimeError, "Messages nested too deeply.");
}
return Qnil;
}
/*
* call-seq:
* Google::Protobuf.deep_copy(obj) => copy_of_obj
*
* Performs a deep copy of a RepeatedField instance, a Map instance, or a
* message object, recursively copying its members.
*/
VALUE Google_Protobuf_deep_copy(VALUE self, VALUE obj) {
VALUE klass = CLASS_OF(obj);
if (klass == cRepeatedField) {
return RepeatedField_deep_copy(obj);
} else if (klass == cMap) {
return Map_deep_copy(obj);
} else {
VALUE new_arena_rb = Arena_new();
upb_arena *new_arena = Arena_get(new_arena_rb);
const upb_msgdef *m;
const upb_msg *msg = Message_Get(obj, &m);
upb_msg* new_msg = Message_deep_copy(msg, m, new_arena);
return Message_GetRubyWrapper(new_msg, m, new_arena_rb);
}
}
// -----------------------------------------------------------------------------
// Initialization/entry point.
@ -93,44 +373,24 @@ ID descriptor_instancevar_interned;
// This must be named "Init_protobuf_c" because the Ruby module is named
// "protobuf_c" -- the VM looks for this symbol in our .so.
__attribute__ ((visibility ("default")))
void Init_protobuf_c() {
ObjectCache_Init();
VALUE google = rb_define_module("Google");
VALUE protobuf = rb_define_module_under(google, "Protobuf");
VALUE internal = rb_define_module_under(protobuf, "Internal");
descriptor_instancevar_interned = rb_intern(kDescriptorInstanceVar);
DescriptorPool_register(protobuf);
Descriptor_register(protobuf);
FileDescriptor_register(protobuf);
FieldDescriptor_register(protobuf);
OneofDescriptor_register(protobuf);
EnumDescriptor_register(protobuf);
MessageBuilderContext_register(internal);
OneofBuilderContext_register(internal);
EnumBuilderContext_register(internal);
FileBuilderContext_register(internal);
Builder_register(internal);
Arena_register(protobuf);
Defs_register(protobuf);
RepeatedField_register(protobuf);
Map_register(protobuf);
Message_register(protobuf);
cError = rb_const_get(protobuf, rb_intern("Error"));
cParseError = rb_const_get(protobuf, rb_intern("ParseError"));
cTypeError = rb_const_get(protobuf, rb_intern("TypeError"));
rb_define_singleton_method(protobuf, "discard_unknown",
Google_Protobuf_discard_unknown, 1);
rb_define_singleton_method(protobuf, "deep_copy",
Google_Protobuf_deep_copy, 1);
kRubyStringUtf8Encoding = rb_utf8_encoding();
kRubyStringASCIIEncoding = rb_usascii_encoding();
kRubyString8bitEncoding = rb_ascii8bit_encoding();
rb_gc_register_address(&c_only_cookie);
c_only_cookie = rb_class_new_instance(0, NULL, rb_cObject);
rb_gc_register_address(&cached_empty_string);
rb_gc_register_address(&cached_empty_bytes);
cached_empty_string = create_frozen_string("", 0, false);
cached_empty_bytes = create_frozen_string("", 0, true);
}

639
ruby/ext/google/protobuf_c/protobuf.h
Unescape View File

@ -35,631 +35,76 @@
#include <ruby/vm.h>
#include <ruby/encoding.h>
#include "upb.h"
// Forward decls.
struct DescriptorPool;
struct Descriptor;
struct FileDescriptor;
struct FieldDescriptor;
struct EnumDescriptor;
struct MessageLayout;
struct MessageField;
struct MessageHeader;
struct MessageBuilderContext;
struct EnumBuilderContext;
struct FileBuilderContext;
struct Builder;
typedef struct DescriptorPool DescriptorPool;
typedef struct Descriptor Descriptor;
typedef struct FileDescriptor FileDescriptor;
typedef struct FieldDescriptor FieldDescriptor;
typedef struct OneofDescriptor OneofDescriptor;
typedef struct EnumDescriptor EnumDescriptor;
typedef struct MessageLayout MessageLayout;
typedef struct MessageField MessageField;
typedef struct MessageOneof MessageOneof;
typedef struct MessageHeader MessageHeader;
typedef struct MessageBuilderContext MessageBuilderContext;
typedef struct OneofBuilderContext OneofBuilderContext;
typedef struct EnumBuilderContext EnumBuilderContext;
typedef struct FileBuilderContext FileBuilderContext;
typedef struct Builder Builder;
/*
It can be a bit confusing how the C structs defined below and the Ruby
objects interact and hold references to each other. First, a few principles:
- Ruby's "TypedData" abstraction lets a Ruby VALUE hold a pointer to a C
struct (or arbitrary memory chunk), own it, and free it when collected.
Thus, each struct below will have a corresponding Ruby object
wrapping/owning it.
- To get back from an underlying upb {msg,enum}def to the Ruby object, we
keep a global hashmap, accessed by get_def_obj/add_def_obj below.
The in-memory structure is then something like:
Ruby | upb
|
DescriptorPool ------------|-----------> upb_symtab____________________
| | (message types) \
| v \
Descriptor ---------------|-----------> upb_msgdef (enum types)|
|--> msgclass | | ^ |
| (dynamically built) | | | (submsg fields) |
|--> MessageLayout | | | /
|--------------------------|> decoder method| | /
\--------------------------|> serialize | | /
| handlers v | /
FieldDescriptor -----------|-----------> upb_fielddef /
| | /
| v (enum fields) /
EnumDescriptor ------------|-----------> upb_enumdef <----------'
|
|
^ | \___/
`---------------|-----------------' (get_def_obj map)
*/
// -----------------------------------------------------------------------------
// Ruby class structure definitions.
// -----------------------------------------------------------------------------
struct DescriptorPool {
VALUE def_to_descriptor; // Hash table of def* -> Ruby descriptor.
upb_symtab* symtab;
upb_handlercache* fill_handler_cache;
upb_handlercache* pb_serialize_handler_cache;
upb_handlercache* json_serialize_handler_cache;
upb_handlercache* json_serialize_handler_preserve_cache;
upb_pbcodecache* fill_method_cache;
upb_json_codecache* json_fill_method_cache;
};
struct Descriptor {
const upb_msgdef* msgdef;
MessageLayout* layout;
VALUE klass;
VALUE descriptor_pool;
};
struct FileDescriptor {
const upb_filedef* filedef;
VALUE descriptor_pool; // Owns the upb_filedef.
};
struct FieldDescriptor {
const upb_fielddef* fielddef;
VALUE descriptor_pool; // Owns the upb_fielddef.
};
struct OneofDescriptor {
const upb_oneofdef* oneofdef;
VALUE descriptor_pool; // Owns the upb_oneofdef.
};
struct EnumDescriptor {
const upb_enumdef* enumdef;
VALUE module; // begins as nil
VALUE descriptor_pool; // Owns the upb_enumdef.
};
struct MessageBuilderContext {
google_protobuf_DescriptorProto* msg_proto;
VALUE file_builder;
};
struct OneofBuilderContext {
int oneof_index;
VALUE message_builder;
};
struct EnumBuilderContext {
google_protobuf_EnumDescriptorProto* enum_proto;
VALUE file_builder;
};
struct FileBuilderContext {
upb_arena *arena;
google_protobuf_FileDescriptorProto* file_proto;
VALUE descriptor_pool;
};
struct Builder {
VALUE descriptor_pool;
VALUE default_file_builder;
};
extern VALUE cDescriptorPool;
extern VALUE cDescriptor;
extern VALUE cFileDescriptor;
extern VALUE cFieldDescriptor;
extern VALUE cEnumDescriptor;
extern VALUE cMessageBuilderContext;
extern VALUE cOneofBuilderContext;
extern VALUE cEnumBuilderContext;
extern VALUE cFileBuilderContext;
extern VALUE cBuilder;
extern VALUE cError;
extern VALUE cParseError;
extern VALUE cTypeError;
// We forward-declare all of the Ruby method implementations here because we
// sometimes call the methods directly across .c files, rather than going
// through Ruby's method dispatching (e.g. during message parse). It's cleaner
// to keep the list of object methods together than to split them between
// static-in-file definitions and header declarations.
void DescriptorPool_mark(void* _self);
void DescriptorPool_free(void* _self);
VALUE DescriptorPool_alloc(VALUE klass);
void DescriptorPool_register(VALUE module);
DescriptorPool* ruby_to_DescriptorPool(VALUE value);
VALUE DescriptorPool_build(int argc, VALUE* argv, VALUE _self);
VALUE DescriptorPool_lookup(VALUE _self, VALUE name);
VALUE DescriptorPool_generated_pool(VALUE _self);
extern VALUE generated_pool;
void Descriptor_mark(void* _self);
void Descriptor_free(void* _self);
VALUE Descriptor_alloc(VALUE klass);
void Descriptor_register(VALUE module);
Descriptor* ruby_to_Descriptor(VALUE value);
VALUE Descriptor_initialize(VALUE _self, VALUE cookie, VALUE descriptor_pool,
VALUE ptr);
VALUE Descriptor_name(VALUE _self);
VALUE Descriptor_each(VALUE _self);
VALUE Descriptor_lookup(VALUE _self, VALUE name);
VALUE Descriptor_each_oneof(VALUE _self);
VALUE Descriptor_lookup_oneof(VALUE _self, VALUE name);
VALUE Descriptor_msgclass(VALUE _self);
VALUE Descriptor_file_descriptor(VALUE _self);
extern const rb_data_type_t _Descriptor_type;
void FileDescriptor_mark(void* _self);
void FileDescriptor_free(void* _self);
VALUE FileDescriptor_alloc(VALUE klass);
void FileDescriptor_register(VALUE module);
FileDescriptor* ruby_to_FileDescriptor(VALUE value);
VALUE FileDescriptor_initialize(VALUE _self, VALUE cookie,
VALUE descriptor_pool, VALUE ptr);
VALUE FileDescriptor_name(VALUE _self);
VALUE FileDescriptor_syntax(VALUE _self);
void FieldDescriptor_mark(void* _self);
void FieldDescriptor_free(void* _self);
VALUE FieldDescriptor_alloc(VALUE klass);
void FieldDescriptor_register(VALUE module);
FieldDescriptor* ruby_to_FieldDescriptor(VALUE value);
VALUE FieldDescriptor_initialize(VALUE _self, VALUE cookie,
VALUE descriptor_pool, VALUE ptr);
VALUE FieldDescriptor_name(VALUE _self);
VALUE FieldDescriptor_type(VALUE _self);
VALUE FieldDescriptor_default(VALUE _self);
VALUE FieldDescriptor_label(VALUE _self);
VALUE FieldDescriptor_number(VALUE _self);
VALUE FieldDescriptor_submsg_name(VALUE _self);
VALUE FieldDescriptor_subtype(VALUE _self);
VALUE FieldDescriptor_has(VALUE _self, VALUE msg_rb);
VALUE FieldDescriptor_clear(VALUE _self, VALUE msg_rb);
VALUE FieldDescriptor_get(VALUE _self, VALUE msg_rb);
VALUE FieldDescriptor_set(VALUE _self, VALUE msg_rb, VALUE value);
upb_fieldtype_t ruby_to_fieldtype(VALUE type);
VALUE fieldtype_to_ruby(upb_fieldtype_t type);
void OneofDescriptor_mark(void* _self);
void OneofDescriptor_free(void* _self);
VALUE OneofDescriptor_alloc(VALUE klass);
void OneofDescriptor_register(VALUE module);
OneofDescriptor* ruby_to_OneofDescriptor(VALUE value);
VALUE OneofDescriptor_initialize(VALUE _self, VALUE cookie,
VALUE descriptor_pool, VALUE ptr);
VALUE OneofDescriptor_name(VALUE _self);
VALUE OneofDescriptor_each(VALUE _self);
void EnumDescriptor_mark(void* _self);
void EnumDescriptor_free(void* _self);
VALUE EnumDescriptor_alloc(VALUE klass);
VALUE EnumDescriptor_initialize(VALUE _self, VALUE cookie,
VALUE descriptor_pool, VALUE ptr);
void EnumDescriptor_register(VALUE module);
EnumDescriptor* ruby_to_EnumDescriptor(VALUE value);
VALUE EnumDescriptor_file_descriptor(VALUE _self);
VALUE EnumDescriptor_name(VALUE _self);
VALUE EnumDescriptor_lookup_name(VALUE _self, VALUE name);
VALUE EnumDescriptor_lookup_value(VALUE _self, VALUE number);
VALUE EnumDescriptor_each(VALUE _self);
VALUE EnumDescriptor_enummodule(VALUE _self);
extern const rb_data_type_t _EnumDescriptor_type;
void MessageBuilderContext_mark(void* _self);
void MessageBuilderContext_free(void* _self);
VALUE MessageBuilderContext_alloc(VALUE klass);
void MessageBuilderContext_register(VALUE module);
MessageBuilderContext* ruby_to_MessageBuilderContext(VALUE value);
VALUE MessageBuilderContext_initialize(VALUE _self,
VALUE _file_builder,
VALUE name);
VALUE MessageBuilderContext_optional(int argc, VALUE* argv, VALUE _self);
VALUE MessageBuilderContext_proto3_optional(int argc, VALUE* argv, VALUE _self);
VALUE MessageBuilderContext_required(int argc, VALUE* argv, VALUE _self);
VALUE MessageBuilderContext_repeated(int argc, VALUE* argv, VALUE _self);
VALUE MessageBuilderContext_map(int argc, VALUE* argv, VALUE _self);
VALUE MessageBuilderContext_oneof(VALUE _self, VALUE name);
void OneofBuilderContext_mark(void* _self);
void OneofBuilderContext_free(void* _self);
VALUE OneofBuilderContext_alloc(VALUE klass);
void OneofBuilderContext_register(VALUE module);
OneofBuilderContext* ruby_to_OneofBuilderContext(VALUE value);
VALUE OneofBuilderContext_initialize(VALUE _self,
VALUE descriptor,
VALUE builder);
VALUE OneofBuilderContext_optional(int argc, VALUE* argv, VALUE _self);
void EnumBuilderContext_mark(void* _self);
void EnumBuilderContext_free(void* _self);
VALUE EnumBuilderContext_alloc(VALUE klass);
void EnumBuilderContext_register(VALUE module);
EnumBuilderContext* ruby_to_EnumBuilderContext(VALUE value);
VALUE EnumBuilderContext_initialize(VALUE _self, VALUE _file_builder,
VALUE name);
VALUE EnumBuilderContext_value(VALUE _self, VALUE name, VALUE number);
void FileBuilderContext_mark(void* _self);
void FileBuilderContext_free(void* _self);
VALUE FileBuilderContext_alloc(VALUE klass);
void FileBuilderContext_register(VALUE module);
FileBuilderContext* ruby_to_FileBuilderContext(VALUE _self);
upb_strview FileBuilderContext_strdup(VALUE _self, VALUE rb_str);
upb_strview FileBuilderContext_strdup_name(VALUE _self, VALUE rb_str);
upb_strview FileBuilderContext_strdup_sym(VALUE _self, VALUE rb_sym);
VALUE FileBuilderContext_initialize(VALUE _self, VALUE descriptor_pool,
VALUE name, VALUE options);
VALUE FileBuilderContext_add_message(VALUE _self, VALUE name);
VALUE FileBuilderContext_add_enum(VALUE _self, VALUE name);
VALUE FileBuilderContext_pending_descriptors(VALUE _self);
void Builder_mark(void* _self);
void Builder_free(void* _self);
VALUE Builder_alloc(VALUE klass);
void Builder_register(VALUE module);
Builder* ruby_to_Builder(VALUE value);
VALUE Builder_build(VALUE _self);
VALUE Builder_initialize(VALUE _self, VALUE descriptor_pool);
VALUE Builder_add_file(int argc, VALUE *argv, VALUE _self);
VALUE Builder_add_message(VALUE _self, VALUE name);
VALUE Builder_add_enum(VALUE _self, VALUE name);
VALUE Builder_finalize_to_pool(VALUE _self, VALUE pool_rb);
// -----------------------------------------------------------------------------
// Native slot storage abstraction.
// -----------------------------------------------------------------------------
#define NATIVE_SLOT_MAX_SIZE sizeof(uint64_t)
size_t native_slot_size(upb_fieldtype_t type);
void native_slot_set(const char* name,
upb_fieldtype_t type,
VALUE type_class,
void* memory,
VALUE value);
// Atomically (with respect to Ruby VM calls) either update the value and set a
// oneof case, or do neither. If |case_memory| is null, then no case value is
// set.
void native_slot_set_value_and_case(const char* name,
upb_fieldtype_t type,
VALUE type_class,
void* memory,
VALUE value,
uint32_t* case_memory,
uint32_t case_number);
VALUE native_slot_get(upb_fieldtype_t type,
VALUE type_class,
const void* memory);
void native_slot_init(upb_fieldtype_t type, void* memory);
void native_slot_mark(upb_fieldtype_t type, void* memory);
void native_slot_dup(upb_fieldtype_t type, void* to, void* from);
void native_slot_deep_copy(upb_fieldtype_t type, VALUE type_class, void* to,
void* from);
bool native_slot_eq(upb_fieldtype_t type, VALUE type_class, void* mem1,
void* mem2);
VALUE native_slot_encode_and_freeze_string(upb_fieldtype_t type, VALUE value);
void native_slot_check_int_range_precision(const char* name, upb_fieldtype_t type, VALUE value);
uint32_t slot_read_oneof_case(MessageLayout* layout, const void* storage,
const upb_oneofdef* oneof);
bool is_value_field(const upb_fielddef* f);
extern rb_encoding* kRubyStringUtf8Encoding;
extern rb_encoding* kRubyStringASCIIEncoding;
extern rb_encoding* kRubyString8bitEncoding;
VALUE field_type_class(const MessageLayout* layout, const upb_fielddef* field);
#define MAP_KEY_FIELD 1
#define MAP_VALUE_FIELD 2
// Oneof case slot value to indicate that no oneof case is set. The value `0` is
// safe because field numbers are used as case identifiers, and no field can
// have a number of 0.
#define ONEOF_CASE_NONE 0
#include "ruby-upb.h"
#include "defs.h"
// These operate on a map field (i.e., a repeated field of submessages whose
// submessage type is a map-entry msgdef).
bool is_map_field(const upb_fielddef* field);
const upb_fielddef* map_field_key(const upb_fielddef* field);
const upb_fielddef* map_field_value(const upb_fielddef* field);
// These operate on a map-entry msgdef.
const upb_fielddef* map_entry_key(const upb_msgdef* msgdef);
const upb_fielddef* map_entry_value(const upb_msgdef* msgdef);
// -----------------------------------------------------------------------------
// Repeated field container type.
// -----------------------------------------------------------------------------
typedef struct {
upb_fieldtype_t field_type;
VALUE field_type_class;
void* elements;
int size;
int capacity;
} RepeatedField;
void RepeatedField_mark(void* self);
void RepeatedField_free(void* self);
VALUE RepeatedField_alloc(VALUE klass);
VALUE RepeatedField_init(int argc, VALUE* argv, VALUE self);
void RepeatedField_register(VALUE module);
extern const rb_data_type_t RepeatedField_type;
extern VALUE cRepeatedField;
RepeatedField* ruby_to_RepeatedField(VALUE value);
VALUE RepeatedField_new_this_type(VALUE _self);
VALUE RepeatedField_each(VALUE _self);
VALUE RepeatedField_index(int argc, VALUE* argv, VALUE _self);
void* RepeatedField_index_native(VALUE _self, int index);
int RepeatedField_size(VALUE _self);
VALUE RepeatedField_index_set(VALUE _self, VALUE _index, VALUE val);
void RepeatedField_reserve(RepeatedField* self, int new_size);
VALUE RepeatedField_push(VALUE _self, VALUE val);
void RepeatedField_push_native(VALUE _self, void* data);
VALUE RepeatedField_pop_one(VALUE _self);
VALUE RepeatedField_insert(int argc, VALUE* argv, VALUE _self);
VALUE RepeatedField_replace(VALUE _self, VALUE list);
VALUE RepeatedField_clear(VALUE _self);
VALUE RepeatedField_length(VALUE _self);
VALUE RepeatedField_dup(VALUE _self);
VALUE RepeatedField_deep_copy(VALUE _self);
VALUE RepeatedField_to_ary(VALUE _self);
VALUE RepeatedField_eq(VALUE _self, VALUE _other);
VALUE RepeatedField_hash(VALUE _self);
VALUE RepeatedField_inspect(VALUE _self);
VALUE RepeatedField_plus(VALUE _self, VALUE list);
// Defined in repeated_field.c; also used by Map.
void validate_type_class(upb_fieldtype_t type, VALUE klass);
// -----------------------------------------------------------------------------
// Map container type.
// Arena
// -----------------------------------------------------------------------------
typedef struct {
upb_fieldtype_t key_type;
upb_fieldtype_t value_type;
VALUE value_type_class;
VALUE parse_frame;
upb_strtable table;
} Map;
void Map_mark(void* self);
void Map_free(void* self);
VALUE Map_alloc(VALUE klass);
VALUE Map_init(int argc, VALUE* argv, VALUE self);
void Map_register(VALUE module);
VALUE Map_set_frame(VALUE self, VALUE val);
extern const rb_data_type_t Map_type;
extern VALUE cMap;
Map* ruby_to_Map(VALUE value);
VALUE Map_new_this_type(VALUE _self);
VALUE Map_each(VALUE _self);
VALUE Map_keys(VALUE _self);
VALUE Map_values(VALUE _self);
VALUE Map_index(VALUE _self, VALUE key);
VALUE Map_index_set(VALUE _self, VALUE key, VALUE value);
VALUE Map_has_key(VALUE _self, VALUE key);
VALUE Map_delete(VALUE _self, VALUE key);
VALUE Map_clear(VALUE _self);
VALUE Map_length(VALUE _self);
VALUE Map_dup(VALUE _self);
VALUE Map_deep_copy(VALUE _self);
VALUE Map_eq(VALUE _self, VALUE _other);
VALUE Map_hash(VALUE _self);
VALUE Map_to_h(VALUE _self);
VALUE Map_inspect(VALUE _self);
VALUE Map_merge(VALUE _self, VALUE hashmap);
VALUE Map_merge_into_self(VALUE _self, VALUE hashmap);
// A Ruby object that wraps an underlying upb_arena. Any objects that are
// allocated from this arena should reference the Arena in rb_gc_mark(), to
// ensure that the object's underlying memory outlives any Ruby object that can
// reach it.
typedef struct {
Map* self;
upb_strtable_iter it;
} Map_iter;
void Map_begin(VALUE _self, Map_iter* iter);
void Map_next(Map_iter* iter);
bool Map_done(Map_iter* iter);
VALUE Map_iter_key(Map_iter* iter);
VALUE Map_iter_value(Map_iter* iter);
VALUE Arena_new();
upb_arena *Arena_get(VALUE arena);
// -----------------------------------------------------------------------------
// Message layout / storage.
// ObjectCache
// -----------------------------------------------------------------------------
#define MESSAGE_FIELD_NO_HASBIT ((uint32_t)-1)
struct MessageField {
uint32_t offset;
uint32_t hasbit;
};
struct MessageOneof {
uint32_t offset;
uint32_t case_offset;
};
// MessageLayout is owned by the enclosing Descriptor, which must outlive us.
struct MessageLayout {
const Descriptor* desc;
const upb_msgdef* msgdef;
void* empty_template; // Can memcpy() onto a layout to clear it.
MessageField* fields;
MessageOneof* oneofs;
uint32_t size;
uint32_t value_offset;
int value_count;
int repeated_count;
int map_count;
};
#define ONEOF_CASE_MASK 0x80000000
void create_layout(Descriptor* desc);
void free_layout(MessageLayout* layout);
bool field_contains_hasbit(MessageLayout* layout,
const upb_fielddef* field);
VALUE layout_get_default(const upb_fielddef* field);
VALUE layout_get(MessageLayout* layout,
const void* storage,
const upb_fielddef* field);
void layout_set(MessageLayout* layout,
void* storage,
const upb_fielddef* field,
VALUE val);
VALUE layout_has(MessageLayout* layout,
const void* storage,
const upb_fielddef* field);
void layout_clear(MessageLayout* layout,
const void* storage,
const upb_fielddef* field);
void layout_init(MessageLayout* layout, void* storage);
void layout_mark(MessageLayout* layout, void* storage);
void layout_dup(MessageLayout* layout, void* to, void* from);
void layout_deep_copy(MessageLayout* layout, void* to, void* from);
VALUE layout_eq(MessageLayout* layout, void* msg1, void* msg2);
VALUE layout_hash(MessageLayout* layout, void* storage);
VALUE layout_inspect(MessageLayout* layout, void* storage);
bool is_wrapper_type_field(const upb_fielddef* field);
VALUE ruby_wrapper_type(VALUE type_class, VALUE value);
// Global object cache from upb array/map/message/symtab to wrapper object.
//
// This is a conceptually "weak" cache, in that it does not prevent "val" from
// being collected (though in Ruby <2.7 is it effectively strong, due to
// implementation limitations).
// Adds an entry to the cache. The "arena" parameter must give the arena that
// "key" was allocated from. In Ruby <2.7.0, it will be used to remove the key
// from the cache when the arena is destroyed.
void ObjectCache_Add(const void* key, VALUE val, upb_arena *arena);
// Returns the cached object for this key, if any. Otherwise returns Qnil.
VALUE ObjectCache_Get(const void* key);
// Pins the previously added object so it is GC-rooted. This turns the
// reference to "val" from weak to strong. We use this to guarantee that the
// "frozen" bit on the object will be remembered, even if the user drops their
// reference to this precise object.
//
// The "arena" parameter must give the arena that "key" was allocated from.
void ObjectCache_Pin(const void* key, VALUE val, upb_arena *arena);
// -----------------------------------------------------------------------------
// Message class creation.
// StringBuilder, for inspect
// -----------------------------------------------------------------------------
// This should probably be factored into a common upb component.
typedef struct {
upb_byteshandler handler;
upb_bytessink sink;
char *ptr;
size_t len, size;
} stringsink;
void stringsink_uninit(stringsink *sink);
struct MessageHeader {
Descriptor* descriptor; // kept alive by self.class.descriptor reference.
stringsink* unknown_fields; // store unknown fields in decoding.
// Data comes after this.
};
extern rb_data_type_t Message_type;
VALUE build_class_from_descriptor(VALUE descriptor);
void* Message_data(void* msg);
void Message_mark(void* self);
void Message_free(void* self);
VALUE Message_alloc(VALUE klass);
VALUE Message_method_missing(int argc, VALUE* argv, VALUE _self);
VALUE Message_initialize(int argc, VALUE* argv, VALUE _self);
VALUE Message_dup(VALUE _self);
VALUE Message_deep_copy(VALUE _self);
VALUE Message_eq(VALUE _self, VALUE _other);
VALUE Message_hash(VALUE _self);
VALUE Message_inspect(VALUE _self);
VALUE Message_to_h(VALUE _self);
VALUE Message_index(VALUE _self, VALUE field_name);
VALUE Message_index_set(VALUE _self, VALUE field_name, VALUE value);
VALUE Message_descriptor(VALUE klass);
VALUE Message_decode(VALUE klass, VALUE data);
VALUE Message_encode(VALUE klass, VALUE msg_rb);
VALUE Message_decode_json(int argc, VALUE* argv, VALUE klass);
VALUE Message_encode_json(int argc, VALUE* argv, VALUE klass);
VALUE Google_Protobuf_discard_unknown(VALUE self, VALUE msg_rb);
VALUE Google_Protobuf_deep_copy(VALUE self, VALUE obj);
VALUE build_module_from_enumdesc(VALUE _enumdesc);
VALUE enum_lookup(VALUE self, VALUE number);
VALUE enum_resolve(VALUE self, VALUE sym);
VALUE enum_descriptor(VALUE self);
struct StringBuilder;
typedef struct StringBuilder StringBuilder;
const upb_pbdecodermethod *new_fillmsg_decodermethod(
Descriptor* descriptor, const void *owner);
void add_handlers_for_message(const void *closure, upb_handlers *h);
StringBuilder* StringBuilder_New();
void StringBuilder_Free(StringBuilder* b);
void StringBuilder_Printf(StringBuilder* b, const char *fmt, ...);
VALUE StringBuilder_ToRubyString(StringBuilder* b);
// Maximum depth allowed during encoding, to avoid stack overflows due to
// cycles.
#define ENCODE_MAX_NESTING 63
// -----------------------------------------------------------------------------
// A cache of frozen string objects to use as field defaults.
// -----------------------------------------------------------------------------
VALUE get_frozen_string(const char* data, size_t size, bool binary);
// -----------------------------------------------------------------------------
// Global map from upb {msg,enum}defs to wrapper Descriptor/EnumDescriptor
// instances.
// -----------------------------------------------------------------------------
VALUE get_msgdef_obj(VALUE descriptor_pool, const upb_msgdef* def);
VALUE get_enumdef_obj(VALUE descriptor_pool, const upb_enumdef* def);
VALUE get_fielddef_obj(VALUE descriptor_pool, const upb_fielddef* def);
VALUE get_filedef_obj(VALUE descriptor_pool, const upb_filedef* def);
VALUE get_oneofdef_obj(VALUE descriptor_pool, const upb_oneofdef* def);
void StringBuilder_PrintMsgval(StringBuilder* b, upb_msgval val, TypeInfo info);
// -----------------------------------------------------------------------------
// Utilities.
// -----------------------------------------------------------------------------
void check_upb_status(const upb_status* status, const char* msg);
#define CHECK_UPB(code, msg) do { \
upb_status status = UPB_STATUS_INIT; \
code; \
check_upb_status(&status, msg); \
} while (0)
extern ID descriptor_instancevar_interned;
// A distinct object that is not accessible from Ruby. We use this as a
// constructor argument to enforce that certain objects cannot be created from
// Ruby.
extern VALUE c_only_cookie;
extern VALUE cTypeError;
#ifdef NDEBUG
#define UPB_ASSERT(expr) do {} while (false && (expr))
#define PBRUBY_ASSERT(expr) do {} while (false && (expr))
#else
#define UPB_ASSERT(expr) assert(expr)
#define PBRUBY_ASSERT(expr) assert(expr)
#endif
#define UPB_UNUSED(var) (void)var

612
ruby/ext/google/protobuf_c/repeated_field.c
Unescape View File

@ -28,49 +28,162 @@
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#include "repeated_field.h"
#include "convert.h"
#include "defs.h"
#include "message.h"
#include "protobuf.h"
#include "third_party/wyhash/wyhash.h"
// -----------------------------------------------------------------------------
// Repeated field container type.
// -----------------------------------------------------------------------------
typedef struct {
const upb_array *array; // Can get as mutable when non-frozen.
TypeInfo type_info;
VALUE type_class; // To GC-root the msgdef/enumdef in type_info.
VALUE arena; // To GC-root the upb_array.
} RepeatedField;
VALUE cRepeatedField;
static void RepeatedField_mark(void* _self) {
RepeatedField* self = (RepeatedField*)_self;
rb_gc_mark(self->type_class);
rb_gc_mark(self->arena);
}
const rb_data_type_t RepeatedField_type = {
"Google::Protobuf::RepeatedField",
{ RepeatedField_mark, RepeatedField_free, NULL },
{ RepeatedField_mark, RUBY_DEFAULT_FREE, NULL },
.flags = RUBY_TYPED_FREE_IMMEDIATELY,
};
VALUE cRepeatedField;
RepeatedField* ruby_to_RepeatedField(VALUE _self) {
static RepeatedField* ruby_to_RepeatedField(VALUE _self) {
RepeatedField* self;
TypedData_Get_Struct(_self, RepeatedField, &RepeatedField_type, self);
return self;
}
void* RepeatedField_memoryat(RepeatedField* self, int index, int element_size) {
return ((uint8_t *)self->elements) + index * element_size;
static upb_array *RepeatedField_GetMutable(VALUE _self) {
rb_check_frozen(_self);
return (upb_array*)ruby_to_RepeatedField(_self)->array;
}
VALUE RepeatedField_alloc(VALUE klass) {
RepeatedField* self = ALLOC(RepeatedField);
self->arena = Qnil;
self->type_class = Qnil;
self->array = NULL;
return TypedData_Wrap_Struct(klass, &RepeatedField_type, self);
}
VALUE RepeatedField_GetRubyWrapper(upb_array* array, TypeInfo type_info,
VALUE arena) {
PBRUBY_ASSERT(array);
VALUE val = ObjectCache_Get(array);
if (val == Qnil) {
val = RepeatedField_alloc(cRepeatedField);
RepeatedField* self;
ObjectCache_Add(array, val, Arena_get(arena));
TypedData_Get_Struct(val, RepeatedField, &RepeatedField_type, self);
self->array = array;
self->arena = arena;
self->type_info = type_info;
if (self->type_info.type == UPB_TYPE_MESSAGE) {
self->type_class = Descriptor_DefToClass(type_info.def.msgdef);
}
}
PBRUBY_ASSERT(ruby_to_RepeatedField(val)->type_info.type == type_info.type);
PBRUBY_ASSERT(ruby_to_RepeatedField(val)->type_info.def.msgdef ==
type_info.def.msgdef);
return val;
}
static VALUE RepeatedField_new_this_type(RepeatedField* from) {
VALUE arena_rb = Arena_new();
upb_array *array = upb_array_new(Arena_get(arena_rb), from->type_info.type);
VALUE ret = RepeatedField_GetRubyWrapper(array, from->type_info, arena_rb);
PBRUBY_ASSERT(ruby_to_RepeatedField(ret)->type_class == from->type_class);
return ret;
}
void RepeatedField_Inspect(StringBuilder* b, const upb_array* array,
TypeInfo info) {
bool first = true;
StringBuilder_Printf(b, "[");
size_t n = array ? upb_array_size(array) : 0;
for (size_t i = 0; i < n; i++) {
if (first) {
first = false;
} else {
StringBuilder_Printf(b, ", ");
}
StringBuilder_PrintMsgval(b, upb_array_get(array, i), info);
}
StringBuilder_Printf(b, "]");
}
VALUE RepeatedField_deep_copy(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
VALUE new_rptfield = RepeatedField_new_this_type(self);
RepeatedField* new_self = ruby_to_RepeatedField(new_rptfield);
VALUE arena_rb = new_self->arena;
upb_array *new_array = RepeatedField_GetMutable(new_rptfield);
upb_arena *arena = Arena_get(arena_rb);
size_t elements = upb_array_size(self->array);
upb_array_resize(new_array, elements, arena);
size_t size = upb_array_size(self->array);
for (size_t i = 0; i < size; i++) {
upb_msgval msgval = upb_array_get(self->array, i);
upb_msgval copy = Msgval_DeepCopy(msgval, self->type_info, arena);
upb_array_set(new_array, i, copy);
}
return new_rptfield;
}
const upb_array* RepeatedField_GetUpbArray(VALUE val, const upb_fielddef *field) {
RepeatedField* self;
TypeInfo type_info = TypeInfo_get(field);
if (!RB_TYPE_P(val, T_DATA) || !RTYPEDDATA_P(val) ||
RTYPEDDATA_TYPE(val) != &RepeatedField_type) {
rb_raise(cTypeError, "Expected repeated field array");
}
self = ruby_to_RepeatedField(val);
if (self->type_info.type != type_info.type) {
rb_raise(cTypeError, "Repeated field array has wrong element type");
}
if (self->type_info.def.msgdef != type_info.def.msgdef) {
rb_raise(cTypeError, "Repeated field array has wrong message/enum class");
}
return self->array;
}
static int index_position(VALUE _index, RepeatedField* repeated_field) {
int index = NUM2INT(_index);
if (index < 0 && repeated_field->size > 0) {
index = repeated_field->size + index;
}
if (index < 0) index += upb_array_size(repeated_field->array);
return index;
}
VALUE RepeatedField_subarray(VALUE _self, long beg, long len) {
RepeatedField* self = ruby_to_RepeatedField(_self);
int element_size = native_slot_size(self->field_type);
upb_fieldtype_t field_type = self->field_type;
VALUE field_type_class = self->field_type_class;
size_t off = beg * element_size;
VALUE ary = rb_ary_new2(len);
int i;
static VALUE RepeatedField_subarray(RepeatedField* self, long beg, long len) {
size_t size = upb_array_size(self->array);
VALUE ary = rb_ary_new2(size);
long i;
for (i = beg; i < beg + len; i++, off += element_size) {
void* mem = ((uint8_t *)self->elements) + off;
VALUE elem = native_slot_get(field_type, field_type_class, mem);
for (i = beg; i < beg + len; i++) {
upb_msgval msgval = upb_array_get(self->array, i);
VALUE elem = Convert_UpbToRuby(msgval, self->type_info, self->arena);
rb_ary_push(ary, elem);
}
return ary;
@ -84,17 +197,14 @@ VALUE RepeatedField_subarray(VALUE _self, long beg, long len) {
* also includes Enumerable; combined with this method, the repeated field thus
* acts like an ordinary Ruby sequence.
*/
VALUE RepeatedField_each(VALUE _self) {
static VALUE RepeatedField_each(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
VALUE field_type_class = self->field_type_class;
int element_size = native_slot_size(field_type);
size_t off = 0;
int size = upb_array_size(self->array);
int i;
for (i = 0; i < self->size; i++, off += element_size) {
void* memory = (void *) (((uint8_t *)self->elements) + off);
VALUE val = native_slot_get(field_type, field_type_class, memory);
for (i = 0; i < size; i++) {
upb_msgval msgval = upb_array_get(self->array, i);
VALUE val = Convert_UpbToRuby(msgval, self->type_info, self->arena);
rb_yield(val);
}
return _self;
@ -107,11 +217,9 @@ VALUE RepeatedField_each(VALUE _self) {
*
* Accesses the element at the given index. Returns nil on out-of-bounds
*/
VALUE RepeatedField_index(int argc, VALUE* argv, VALUE _self) {
static VALUE RepeatedField_index(int argc, VALUE* argv, VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
int element_size = native_slot_size(self->field_type);
upb_fieldtype_t field_type = self->field_type;
VALUE field_type_class = self->field_type_class;
long size = upb_array_size(self->array);
VALUE arg = argv[0];
long beg, len;
@ -119,35 +227,36 @@ VALUE RepeatedField_index(int argc, VALUE* argv, VALUE _self) {
if (argc == 1){
if (FIXNUM_P(arg)) {
/* standard case */
void* memory;
upb_msgval msgval;
int index = index_position(argv[0], self);
if (index < 0 || index >= self->size) {
if (index < 0 || (size_t)index >= upb_array_size(self->array)) {
return Qnil;
}
memory = RepeatedField_memoryat(self, index, element_size);
return native_slot_get(field_type, field_type_class, memory);
}else{
msgval = upb_array_get(self->array, index);
return Convert_UpbToRuby(msgval, self->type_info, self->arena);
} else {
/* check if idx is Range */
switch (rb_range_beg_len(arg, &beg, &len, self->size, 0)) {
switch (rb_range_beg_len(arg, &beg, &len, size, 0)) {
case Qfalse:
break;
case Qnil:
return Qnil;
default:
return RepeatedField_subarray(_self, beg, len);
return RepeatedField_subarray(self, beg, len);
}
}
}
/* assume 2 arguments */
beg = NUM2LONG(argv[0]);
len = NUM2LONG(argv[1]);
if (beg < 0) {
beg += self->size;
beg += size;
}
if (beg >= self->size) {
if (beg >= size) {
return Qnil;
}
return RepeatedField_subarray(_self, beg, len);
return RepeatedField_subarray(self, beg, len);
}
/*
@ -157,128 +266,88 @@ VALUE RepeatedField_index(int argc, VALUE* argv, VALUE _self) {
* Sets the element at the given index. On out-of-bounds assignments, extends
* the array and fills the hole (if any) with default values.
*/
VALUE RepeatedField_index_set(VALUE _self, VALUE _index, VALUE val) {
static VALUE RepeatedField_index_set(VALUE _self, VALUE _index, VALUE val) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
VALUE field_type_class = self->field_type_class;
int element_size = native_slot_size(field_type);
void* memory;
int size = upb_array_size(self->array);
upb_array *array = RepeatedField_GetMutable(_self);
upb_arena *arena = Arena_get(self->arena);
upb_msgval msgval = Convert_RubyToUpb(val, "", self->type_info, arena);
int index = index_position(_index, self);
if (index < 0 || index >= (INT_MAX - 1)) {
return Qnil;
}
if (index >= self->size) {
upb_fieldtype_t field_type = self->field_type;
int element_size = native_slot_size(field_type);
int i;
RepeatedField_reserve(self, index + 1);
for (i = self->size; i <= index; i++) {
void* elem = RepeatedField_memoryat(self, i, element_size);
native_slot_init(field_type, elem);
if (index >= size) {
upb_array_resize(array, index + 1, arena);
upb_msgval fill;
memset(&fill, 0, sizeof(fill));
for (int i = size; i < index; i++) {
// Fill default values.
// TODO(haberman): should this happen at the upb level?
upb_array_set(array, i, fill);
}
self->size = index + 1;
}
memory = RepeatedField_memoryat(self, index, element_size);
native_slot_set("", field_type, field_type_class, memory, val);
upb_array_set(array, index, msgval);
return Qnil;
}
static int kInitialSize = 8;
void RepeatedField_reserve(RepeatedField* self, int new_size) {
void* old_elems = self->elements;
int elem_size = native_slot_size(self->field_type);
if (new_size <= self->capacity) {
return;
}
if (self->capacity == 0) {
self->capacity = kInitialSize;
}
while (self->capacity < new_size) {
self->capacity *= 2;
}
self->elements = ALLOC_N(uint8_t, elem_size * self->capacity);
if (old_elems != NULL) {
memcpy(self->elements, old_elems, self->size * elem_size);
xfree(old_elems);
}
}
/*
* call-seq:
* RepeatedField.push(value)
* RepeatedField.push(value, ...)
*
* Adds a new element to the repeated field.
*/
VALUE RepeatedField_push(VALUE _self, VALUE val) {
static VALUE RepeatedField_push_vararg(int argc, VALUE* argv, VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
int element_size = native_slot_size(field_type);
void* memory;
RepeatedField_reserve(self, self->size + 1);
memory = (void *) (((uint8_t *)self->elements) + self->size * element_size);
native_slot_set("", field_type, self->field_type_class, memory, val);
// native_slot_set may raise an error; bump size only after set.
self->size++;
return _self;
}
VALUE RepeatedField_push_vararg(VALUE _self, VALUE args) {
upb_arena *arena = Arena_get(self->arena);
upb_array *array = RepeatedField_GetMutable(_self);
int i;
for (i = 0; i < RARRAY_LEN(args); i++) {
RepeatedField_push(_self, rb_ary_entry(args, i));
for (i = 0; i < argc; i++) {
upb_msgval msgval = Convert_RubyToUpb(argv[i], "", self->type_info, arena);
upb_array_append(array, msgval, arena);
}
return _self;
}
// Used by parsing handlers.
void RepeatedField_push_native(VALUE _self, void* data) {
/*
* call-seq:
* RepeatedField.<<(value)
*
* Adds a new element to the repeated field.
*/
static VALUE RepeatedField_push(VALUE _self, VALUE val) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
int element_size = native_slot_size(field_type);
void* memory;
RepeatedField_reserve(self, self->size + 1);
memory = (void *) (((uint8_t *)self->elements) + self->size * element_size);
memcpy(memory, data, element_size);
self->size++;
}
upb_arena *arena = Arena_get(self->arena);
upb_array *array = RepeatedField_GetMutable(_self);
void* RepeatedField_index_native(VALUE _self, int index) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
int element_size = native_slot_size(field_type);
return RepeatedField_memoryat(self, index, element_size);
}
upb_msgval msgval = Convert_RubyToUpb(val, "", self->type_info, arena);
upb_array_append(array, msgval, arena);
int RepeatedField_size(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
return self->size;
return _self;
}
/*
* Private ruby method, used by RepeatedField.pop
*/
VALUE RepeatedField_pop_one(VALUE _self) {
static VALUE RepeatedField_pop_one(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
VALUE field_type_class = self->field_type_class;
int element_size = native_slot_size(field_type);
int index;
void* memory;
size_t size = upb_array_size(self->array);
upb_array *array = RepeatedField_GetMutable(_self);
upb_msgval last;
VALUE ret;
if (self->size == 0) {
if (size == 0) {
return Qnil;
}
index = self->size - 1;
memory = RepeatedField_memoryat(self, index, element_size);
ret = native_slot_get(field_type, field_type_class, memory);
self->size--;
last = upb_array_get(self->array, size - 1);
ret = Convert_UpbToRuby(last, self->type_info, self->arena);
upb_array_resize(array, size - 1, Arena_get(self->arena));
return ret;
}
@ -288,15 +357,18 @@ VALUE RepeatedField_pop_one(VALUE _self) {
*
* Replaces the contents of the repeated field with the given list of elements.
*/
VALUE RepeatedField_replace(VALUE _self, VALUE list) {
static VALUE RepeatedField_replace(VALUE _self, VALUE list) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_array *array = RepeatedField_GetMutable(_self);
int i;
Check_Type(list, T_ARRAY);
self->size = 0;
upb_array_resize(array, 0, Arena_get(self->arena));
for (i = 0; i < RARRAY_LEN(list); i++) {
RepeatedField_push(_self, rb_ary_entry(list, i));
}
return list;
}
@ -306,9 +378,10 @@ VALUE RepeatedField_replace(VALUE _self, VALUE list) {
*
* Clears (removes all elements from) this repeated field.
*/
VALUE RepeatedField_clear(VALUE _self) {
static VALUE RepeatedField_clear(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
self->size = 0;
upb_array *array = RepeatedField_GetMutable(_self);
upb_array_resize(array, 0, Arena_get(self->arena));
return _self;
}
@ -318,23 +391,9 @@ VALUE RepeatedField_clear(VALUE _self) {
*
* Returns the length of this repeated field.
*/
VALUE RepeatedField_length(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
return INT2NUM(self->size);
}
VALUE RepeatedField_new_this_type(VALUE _self) {
static VALUE RepeatedField_length(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
VALUE new_rptfield = Qnil;
VALUE element_type = fieldtype_to_ruby(self->field_type);
if (self->field_type_class != Qnil) {
new_rptfield = rb_funcall(CLASS_OF(_self), rb_intern("new"), 2,
element_type, self->field_type_class);
} else {
new_rptfield = rb_funcall(CLASS_OF(_self), rb_intern("new"), 1,
element_type);
}
return new_rptfield;
return INT2NUM(upb_array_size(self->array));
}
/*
@ -344,42 +403,20 @@ VALUE RepeatedField_new_this_type(VALUE _self) {
* Duplicates this repeated field with a shallow copy. References to all
* non-primitive element objects (e.g., submessages) are shared.
*/
VALUE RepeatedField_dup(VALUE _self) {
static VALUE RepeatedField_dup(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
VALUE new_rptfield = RepeatedField_new_this_type(_self);
VALUE new_rptfield = RepeatedField_new_this_type(self);
RepeatedField* new_rptfield_self = ruby_to_RepeatedField(new_rptfield);
upb_fieldtype_t field_type = self->field_type;
size_t elem_size = native_slot_size(field_type);
size_t off = 0;
upb_array *new_array = RepeatedField_GetMutable(new_rptfield);
upb_arena* arena = Arena_get(new_rptfield_self->arena);
int size = upb_array_size(self->array);
int i;
RepeatedField_reserve(new_rptfield_self, self->size);
for (i = 0; i < self->size; i++, off += elem_size) {
void* to_mem = (uint8_t *)new_rptfield_self->elements + off;
void* from_mem = (uint8_t *)self->elements + off;
native_slot_dup(field_type, to_mem, from_mem);
new_rptfield_self->size++;
}
return new_rptfield;
}
upb_arena_fuse(arena, Arena_get(self->arena));
// Internal only: used by Google::Protobuf.deep_copy.
VALUE RepeatedField_deep_copy(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
VALUE new_rptfield = RepeatedField_new_this_type(_self);
RepeatedField* new_rptfield_self = ruby_to_RepeatedField(new_rptfield);
upb_fieldtype_t field_type = self->field_type;
size_t elem_size = native_slot_size(field_type);
size_t off = 0;
int i;
RepeatedField_reserve(new_rptfield_self, self->size);
for (i = 0; i < self->size; i++, off += elem_size) {
void* to_mem = (uint8_t *)new_rptfield_self->elements + off;
void* from_mem = (uint8_t *)self->elements + off;
native_slot_deep_copy(field_type, self->field_type_class, to_mem, from_mem);
new_rptfield_self->size++;
for (i = 0; i < size; i++) {
upb_msgval msgval = upb_array_get(self->array, i);
upb_array_append(new_array, msgval, arena);
}
return new_rptfield;
@ -394,17 +431,16 @@ VALUE RepeatedField_deep_copy(VALUE _self) {
*/
VALUE RepeatedField_to_ary(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_fieldtype_t field_type = self->field_type;
size_t elem_size = native_slot_size(field_type);
size_t off = 0;
VALUE ary = rb_ary_new2(self->size);
int size = upb_array_size(self->array);
VALUE ary = rb_ary_new2(size);
int i;
for (i = 0; i < self->size; i++, off += elem_size) {
void* mem = ((uint8_t *)self->elements) + off;
VALUE elem = native_slot_get(field_type, self->field_type_class, mem);
rb_ary_push(ary, elem);
for (i = 0; i < size; i++) {
upb_msgval msgval = upb_array_get(self->array, i);
VALUE val = Convert_UpbToRuby(msgval, self->type_info, self->arena);
rb_ary_push(ary, val);
}
return ary;
}
@ -436,28 +472,38 @@ VALUE RepeatedField_eq(VALUE _self, VALUE _other) {
self = ruby_to_RepeatedField(_self);
other = ruby_to_RepeatedField(_other);
if (self->field_type != other->field_type ||
self->field_type_class != other->field_type_class ||
self->size != other->size) {
size_t n = upb_array_size(self->array);
if (self->type_info.type != other->type_info.type ||
self->type_class != other->type_class ||
upb_array_size(other->array) != n) {
return Qfalse;
}
{
upb_fieldtype_t field_type = self->field_type;
size_t elem_size = native_slot_size(field_type);
size_t off = 0;
int i;
for (i = 0; i < self->size; i++, off += elem_size) {
void* self_mem = ((uint8_t *)self->elements) + off;
void* other_mem = ((uint8_t *)other->elements) + off;
if (!native_slot_eq(field_type, self->field_type_class, self_mem,
other_mem)) {
return Qfalse;
}
for (size_t i = 0; i < n; i++) {
upb_msgval val1 = upb_array_get(self->array, i);
upb_msgval val2 = upb_array_get(other->array, i);
if (!Msgval_IsEqual(val1, val2, self->type_info)) {
return Qfalse;
}
return Qtrue;
}
return Qtrue;
}
/*
* call-seq:
* RepeatedField.freeze => self
*
* Freezes the repeated field. We have to intercept this so we can pin the Ruby
* object into memory so we don't forget it's frozen.
*/
static VALUE RepeatedField_freeze(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
ObjectCache_Pin(self->array, _self, Arena_get(self->arena));
RB_OBJ_FREEZE(_self);
return _self;
}
/*
@ -468,22 +514,15 @@ VALUE RepeatedField_eq(VALUE _self, VALUE _other) {
*/
VALUE RepeatedField_hash(VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
st_index_t h = rb_hash_start(0);
VALUE hash_sym = rb_intern("hash");
upb_fieldtype_t field_type = self->field_type;
VALUE field_type_class = self->field_type_class;
size_t elem_size = native_slot_size(field_type);
size_t off = 0;
int i;
uint64_t hash = 0;
size_t n = upb_array_size(self->array);
for (i = 0; i < self->size; i++, off += elem_size) {
void* mem = ((uint8_t *)self->elements) + off;
VALUE elem = native_slot_get(field_type, field_type_class, mem);
h = rb_hash_uint(h, NUM2LONG(rb_funcall(elem, hash_sym, 0)));
for (size_t i = 0; i < n; i++) {
upb_msgval val = upb_array_get(self->array, i);
hash = Msgval_GetHash(val, self->type_info, hash);
}
h = rb_hash_end(h);
return INT2FIX(h);
return LL2NUM(hash);
}
/*
@ -495,34 +534,39 @@ VALUE RepeatedField_hash(VALUE _self) {
* be either another repeated field or a Ruby array.
*/
VALUE RepeatedField_plus(VALUE _self, VALUE list) {
VALUE dupped = RepeatedField_dup(_self);
VALUE dupped_ = RepeatedField_dup(_self);
if (TYPE(list) == T_ARRAY) {
int i;
for (i = 0; i < RARRAY_LEN(list); i++) {
VALUE elem = rb_ary_entry(list, i);
RepeatedField_push(dupped, elem);
RepeatedField_push(dupped_, elem);
}
} else if (RB_TYPE_P(list, T_DATA) && RTYPEDDATA_P(list) &&
RTYPEDDATA_TYPE(list) == &RepeatedField_type) {
RepeatedField* self = ruby_to_RepeatedField(_self);
RepeatedField* list_rptfield = ruby_to_RepeatedField(list);
RepeatedField* dupped = ruby_to_RepeatedField(dupped_);
upb_array *dupped_array = RepeatedField_GetMutable(dupped_);
upb_arena* arena = Arena_get(dupped->arena);
int size = upb_array_size(list_rptfield->array);
int i;
if (self->field_type != list_rptfield->field_type ||
self->field_type_class != list_rptfield->field_type_class) {
if (self->type_info.type != list_rptfield->type_info.type ||
self->type_class != list_rptfield->type_class) {
rb_raise(rb_eArgError,
"Attempt to append RepeatedField with different element type.");
}
for (i = 0; i < list_rptfield->size; i++) {
void* mem = RepeatedField_index_native(list, i);
RepeatedField_push_native(dupped, mem);
for (i = 0; i < size; i++) {
upb_msgval msgval = upb_array_get(list_rptfield->array, i);
upb_array_append(dupped_array, msgval, arena);
}
} else {
rb_raise(rb_eArgError, "Unknown type appending to RepeatedField");
}
return dupped;
return dupped_;
}
/*
@ -541,116 +585,41 @@ VALUE RepeatedField_concat(VALUE _self, VALUE list) {
return _self;
}
void validate_type_class(upb_fieldtype_t type, VALUE klass) {
if (rb_ivar_get(klass, descriptor_instancevar_interned) == Qnil) {
rb_raise(rb_eArgError,
"Type class has no descriptor. Please pass a "
"class or enum as returned by the DescriptorPool.");
}
if (type == UPB_TYPE_MESSAGE) {
VALUE desc = rb_ivar_get(klass, descriptor_instancevar_interned);
if (!RB_TYPE_P(desc, T_DATA) || !RTYPEDDATA_P(desc) ||
RTYPEDDATA_TYPE(desc) != &_Descriptor_type) {
rb_raise(rb_eArgError, "Descriptor has an incorrect type.");
}
if (rb_get_alloc_func(klass) != &Message_alloc) {
rb_raise(rb_eArgError,
"Message class was not returned by the DescriptorPool.");
}
} else if (type == UPB_TYPE_ENUM) {
VALUE enumdesc = rb_ivar_get(klass, descriptor_instancevar_interned);
if (!RB_TYPE_P(enumdesc, T_DATA) || !RTYPEDDATA_P(enumdesc) ||
RTYPEDDATA_TYPE(enumdesc) != &_EnumDescriptor_type) {
rb_raise(rb_eArgError, "Descriptor has an incorrect type.");
}
}
}
void RepeatedField_init_args(int argc, VALUE* argv,
VALUE _self) {
/*
* call-seq:
* RepeatedField.new(type, type_class = nil, initial_elems = [])
*
* Creates a new repeated field. The provided type must be a Ruby symbol, and
* can take on the same values as those accepted by FieldDescriptor#type=. If
* the type is :message or :enum, type_class must be non-nil, and must be the
* Ruby class or module returned by Descriptor#msgclass or
* EnumDescriptor#enummodule, respectively. An initial list of elements may also
* be provided.
*/
VALUE RepeatedField_init(int argc, VALUE* argv, VALUE _self) {
RepeatedField* self = ruby_to_RepeatedField(_self);
upb_arena *arena;
VALUE ary = Qnil;
self->arena = Arena_new();
arena = Arena_get(self->arena);
if (argc < 1) {
rb_raise(rb_eArgError, "Expected at least 1 argument.");
}
self->field_type = ruby_to_fieldtype(argv[0]);
if (self->field_type == UPB_TYPE_MESSAGE ||
self->field_type == UPB_TYPE_ENUM) {
if (argc < 2) {
rb_raise(rb_eArgError, "Expected at least 2 arguments for message/enum.");
}
self->field_type_class = argv[1];
if (argc > 2) {
ary = argv[2];
}
validate_type_class(self->field_type, self->field_type_class);
} else {
if (argc > 2) {
rb_raise(rb_eArgError, "Too many arguments: expected 1 or 2.");
}
if (argc > 1) {
ary = argv[1];
}
}
self->type_info = TypeInfo_FromClass(argc, argv, 0, &self->type_class, &ary);
self->array = upb_array_new(arena, self->type_info.type);
ObjectCache_Add(self->array, _self, arena);
if (ary != Qnil) {
int i;
if (!RB_TYPE_P(ary, T_ARRAY)) {
rb_raise(rb_eArgError, "Expected array as initialize argument");
}
for (i = 0; i < RARRAY_LEN(ary); i++) {
for (int i = 0; i < RARRAY_LEN(ary); i++) {
RepeatedField_push(_self, rb_ary_entry(ary, i));
}
}
}
// Mark, free, alloc, init and class setup functions.
void RepeatedField_mark(void* _self) {
RepeatedField* self = (RepeatedField*)_self;
upb_fieldtype_t field_type = self->field_type;
int element_size = native_slot_size(field_type);
int i;
rb_gc_mark(self->field_type_class);
for (i = 0; i < self->size; i++) {
void* memory = (((uint8_t *)self->elements) + i * element_size);
native_slot_mark(self->field_type, memory);
}
}
void RepeatedField_free(void* _self) {
RepeatedField* self = (RepeatedField*)_self;
xfree(self->elements);
xfree(self);
}
/*
* call-seq:
* RepeatedField.new(type, type_class = nil, initial_elems = [])
*
* Creates a new repeated field. The provided type must be a Ruby symbol, and
* can take on the same values as those accepted by FieldDescriptor#type=. If
* the type is :message or :enum, type_class must be non-nil, and must be the
* Ruby class or module returned by Descriptor#msgclass or
* EnumDescriptor#enummodule, respectively. An initial list of elements may also
* be provided.
*/
VALUE RepeatedField_alloc(VALUE klass) {
RepeatedField* self = ALLOC(RepeatedField);
self->elements = NULL;
self->size = 0;
self->capacity = 0;
self->field_type = -1;
self->field_type_class = Qnil;
return TypedData_Wrap_Struct(klass, &RepeatedField_type, self);
}
VALUE RepeatedField_init(int argc, VALUE* argv, VALUE self) {
RepeatedField_init_args(argc, argv, self);
return Qnil;
}
@ -667,7 +636,7 @@ void RepeatedField_register(VALUE module) {
rb_define_method(klass, "[]", RepeatedField_index, -1);
rb_define_method(klass, "at", RepeatedField_index, -1);
rb_define_method(klass, "[]=", RepeatedField_index_set, 2);
rb_define_method(klass, "push", RepeatedField_push_vararg, -2);
rb_define_method(klass, "push", RepeatedField_push_vararg, -1);
rb_define_method(klass, "<<", RepeatedField_push, 1);
rb_define_private_method(klass, "pop_one", RepeatedField_pop_one, 0);
rb_define_method(klass, "replace", RepeatedField_replace, 1);
@ -679,6 +648,7 @@ void RepeatedField_register(VALUE module) {
rb_define_method(klass, "clone", RepeatedField_dup, 0);
rb_define_method(klass, "==", RepeatedField_eq, 1);
rb_define_method(klass, "to_ary", RepeatedField_to_ary, 0);
rb_define_method(klass, "freeze", RepeatedField_freeze, 0);
rb_define_method(klass, "hash", RepeatedField_hash, 0);
rb_define_method(klass, "+", RepeatedField_plus, 1);
rb_define_method(klass, "concat", RepeatedField_concat, 1);

62
ruby/ext/google/protobuf_c/repeated_field.h
Unescape View File

@ -0,0 +1,62 @@
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#ifndef RUBY_PROTOBUF_REPEATED_FIELD_H_
#define RUBY_PROTOBUF_REPEATED_FIELD_H_
#include <ruby/ruby.h>
#include "protobuf.h"
#include "ruby-upb.h"
// Returns a Ruby wrapper object for the given upb_array, which will be created
// if one does not exist already.
VALUE RepeatedField_GetRubyWrapper(upb_array* msg, TypeInfo type_info,
VALUE arena);
// Gets the underlying upb_array for this Ruby RepeatedField object, which must
// have a type that matches |f|. If this is not a repeated field or the type
// doesn't match, raises an exception.
const upb_array* RepeatedField_GetUpbArray(VALUE value, const upb_fielddef* f);
// Implements #inspect for this repeated field by appending its contents to |b|.
void RepeatedField_Inspect(StringBuilder* b, const upb_array* array,
TypeInfo info);
// Returns a deep copy of this RepeatedField object.
VALUE RepeatedField_deep_copy(VALUE obj);
// Ruby class of Google::Protobuf::RepeatedField.
extern VALUE cRepeatedField;
// Call at startup to register all types in this module.
void RepeatedField_register(VALUE module);
#endif // RUBY_PROTOBUF_REPEATED_FIELD_H_

8913
ruby/ext/google/protobuf_c/ruby-upb.c
View File

File diff suppressed because it is too large Load Diff

5049
ruby/ext/google/protobuf_c/upb.h → ruby/ext/google/protobuf_c/ruby-upb.h
View File

File diff suppressed because it is too large Load Diff

1198
ruby/ext/google/protobuf_c/storage.c
View File

File diff suppressed because it is too large Load Diff

13817
ruby/ext/google/protobuf_c/upb.c
View File

File diff suppressed because it is too large Load Diff

2
ruby/google-protobuf.gemspec
Unescape View File

@ -17,7 +17,7 @@ Gem::Specification.new do |s|
else
s.files += Dir.glob('ext/**/*')
s.extensions= ["ext/google/protobuf_c/extconf.rb"]
s.add_development_dependency "rake-compiler-dock", ">= 1.0.1", "< 2.0"
s.add_development_dependency "rake-compiler-dock", ">= 1.1.0", "< 2.0"
end
s.test_files = ["tests/basic.rb",
"tests/stress.rb",

33
ruby/tests/basic.rb
Unescape View File

@ -216,7 +216,7 @@ module BasicTest
m.map_string_int32 = {}
end
assert_raise TypeError do
assert_raise Google::Protobuf::TypeError do
m = MapMessage.new(:map_string_int32 => { 1 => "I am not a number" })
end
end
@ -447,6 +447,7 @@ module BasicTest
:optional_int32=>0,
:optional_int64=>0,
:optional_msg=>nil,
:optional_msg2=>nil,
:optional_string=>"foo",
:optional_uint32=>0,
:optional_uint64=>0,
@ -484,9 +485,9 @@ module BasicTest
m = MapMessage.new(:map_string_int32 => {"a" => 1})
expected = {mapStringInt32: {a: 1}, mapStringMsg: {}, mapStringEnum: {}}
expected_preserve = {map_string_int32: {a: 1}, map_string_msg: {}, map_string_enum: {}}
assert_equal JSON.parse(MapMessage.encode_json(m), :symbolize_names => true), expected
assert_equal JSON.parse(MapMessage.encode_json(m, :emit_defaults=>true), :symbolize_names => true), expected
json = MapMessage.encode_json(m, :preserve_proto_fieldnames => true)
json = MapMessage.encode_json(m, :preserve_proto_fieldnames => true, :emit_defaults=>true)
assert_equal JSON.parse(json, :symbolize_names => true), expected_preserve
m2 = MapMessage.decode_json(MapMessage.encode_json(m))
@ -496,7 +497,7 @@ module BasicTest
def test_json_maps_emit_defaults_submsg
# TODO: Fix JSON in JRuby version.
return if RUBY_PLATFORM == "java"
m = MapMessage.new(:map_string_msg => {"a" => TestMessage2.new})
m = MapMessage.new(:map_string_msg => {"a" => TestMessage2.new(foo: 0)})
expected = {mapStringInt32: {}, mapStringMsg: {a: {foo: 0}}, mapStringEnum: {}}
actual = MapMessage.encode_json(m, :emit_defaults => true)
@ -504,6 +505,30 @@ module BasicTest
assert_equal JSON.parse(actual, :symbolize_names => true), expected
end
def test_json_emit_defaults_submsg
# TODO: Fix JSON in JRuby version.
return if RUBY_PLATFORM == "java"
m = TestSingularFields.new(singular_msg: proto_module::TestMessage2.new)
expected = {
singularInt32: 0,
singularInt64: "0",
singularUint32: 0,
singularUint64: "0",
singularBool: false,
singularFloat: 0,
singularDouble: 0,
singularString: "",
singularBytes: "",
singularMsg: {},
singularEnum: "Default",
}
actual = proto_module::TestMessage.encode_json(m, :emit_defaults => true)
assert_equal expected, JSON.parse(actual, :symbolize_names => true)
end
def test_respond_to
# This test fails with JRuby 1.7.23, likely because of an old JRuby bug.
return if RUBY_PLATFORM == "java"

17
ruby/tests/basic_proto2.rb
Unescape View File

@ -237,23 +237,6 @@ module BasicTestProto2
assert_equal expected_result, m.to_h
end
def test_map_keyword_disabled
pool = Google::Protobuf::DescriptorPool.new
e = assert_raise ArgumentError do
pool.build do
add_file 'test_file.proto', syntax: :proto2 do
add_message "MapMessage" do
map :map_string_int32, :string, :int32, 1
map :map_string_msg, :string, :message, 2, "TestMessage2"
end
end
end
end
assert_match(/Cannot add a native map/, e.message)
end
def test_respond_to
# This test fails with JRuby 1.7.23, likely because of an old JRuby bug.
return if RUBY_PLATFORM == "java"

4
ruby/tests/basic_test.proto
Unescape View File

@ -44,6 +44,8 @@ message TestMessage {
repeated bytes repeated_bytes = 20;
repeated TestMessage2 repeated_msg = 21;
repeated TestEnum repeated_enum = 22;
optional TestSingularFields optional_msg2 = 23;
}
message TestSingularFields {
@ -61,7 +63,7 @@ message TestSingularFields {
}
message TestMessage2 {
int32 foo = 1;
optional int32 foo = 1;
}
enum TestEnum {

73
ruby/tests/common_tests.rb
Unescape View File

@ -115,14 +115,14 @@ module CommonTests
m = proto_module::TestMessage.new(
:optional_int32 => -42,
:optional_enum => :A,
:optional_msg => proto_module::TestMessage2.new,
:optional_msg => proto_module::TestMessage2.new(foo: 0),
:repeated_string => ["hello", "there", "world"])
expected = "<#{proto_module}::TestMessage: optional_int32: -42, optional_int64: 0, optional_uint32: 0, optional_uint64: 0, optional_bool: false, optional_float: 0.0, optional_double: 0.0, optional_string: \"\", optional_bytes: \"\", optional_msg: <#{proto_module}::TestMessage2: foo: 0>, optional_enum: :A, repeated_int32: [], repeated_int64: [], repeated_uint32: [], repeated_uint64: [], repeated_bool: [], repeated_float: [], repeated_double: [], repeated_string: [\"hello\", \"there\", \"world\"], repeated_bytes: [], repeated_msg: [], repeated_enum: []>"
expected = "<#{proto_module}::TestMessage: optional_int32: -42, optional_msg: <#{proto_module}::TestMessage2: foo: 0>, optional_enum: :A, repeated_int32: [], repeated_int64: [], repeated_uint32: [], repeated_uint64: [], repeated_bool: [], repeated_float: [], repeated_double: [], repeated_string: [\"hello\", \"there\", \"world\"], repeated_bytes: [], repeated_msg: [], repeated_enum: []>"
assert_equal expected, m.inspect
assert_equal expected, m.to_s
m = proto_module::OneofMessage.new(:b => -42)
expected = "<#{proto_module}::OneofMessage: a: \"\", b: -42, c: nil, d: :Default>"
expected = "<#{proto_module}::OneofMessage: b: -42>"
assert_equal expected, m.inspect
assert_equal expected, m.to_s
end
@ -428,7 +428,7 @@ module CommonTests
assert m.length == 0
assert m == {}
assert_raise TypeError do
assert_raise Google::Protobuf::TypeError do
m[1] = 1
end
assert_raise RangeError do
@ -492,7 +492,7 @@ module CommonTests
m = Google::Protobuf::Map.new(:string, :int32)
m["asdf"] = 1
assert_raise TypeError do
assert_raise Google::Protobuf::TypeError do
m[1] = 1
end
assert_raise Encoding::UndefinedConversionError do
@ -505,7 +505,7 @@ module CommonTests
m[bytestring] = 1
# Allowed -- we will automatically convert to ASCII-8BIT.
m["asdf"] = 1
assert_raise TypeError do
assert_raise Google::Protobuf::TypeError do
m[1] = 1
end
end
@ -547,6 +547,7 @@ module CommonTests
"b" => proto_module::TestMessage.new(:optional_int32 => 84) })
m2 = m.dup
assert m.to_h == m2.to_h
assert m == m2
assert m.object_id != m2.object_id
assert m["a"].object_id == m2["a"].object_id
@ -1103,16 +1104,6 @@ module CommonTests
m = proto_module::TestMessage.new
expected = {
optionalInt32: 0,
optionalInt64: "0",
optionalUint32: 0,
optionalUint64: "0",
optionalBool: false,
optionalFloat: 0,
optionalDouble: 0,
optionalString: "",
optionalBytes: "",
optionalEnum: "Default",
repeatedInt32: [],
repeatedInt64: [],
repeatedUint32: [],
@ -1137,17 +1128,7 @@ module CommonTests
m = proto_module::TestMessage.new(optional_msg: proto_module::TestMessage2.new)
expected = {
optionalInt32: 0,
optionalInt64: "0",
optionalUint32: 0,
optionalUint64: "0",
optionalBool: false,
optionalFloat: 0,
optionalDouble: 0,
optionalString: "",
optionalBytes: "",
optionalMsg: {foo: 0},
optionalEnum: "Default",
optionalMsg: {},
repeatedInt32: [],
repeatedInt64: [],
repeatedUint32: [],
@ -1172,16 +1153,6 @@ module CommonTests
m = proto_module::TestMessage.new(repeated_msg: [proto_module::TestMessage2.new])
expected = {
optionalInt32: 0,
optionalInt64: "0",
optionalUint32: 0,
optionalUint64: "0",
optionalBool: false,
optionalFloat: 0,
optionalDouble: 0,
optionalString: "",
optionalBytes: "",
optionalEnum: "Default",
repeatedInt32: [],
repeatedInt64: [],
repeatedUint32: [],
@ -1191,7 +1162,7 @@ module CommonTests
repeatedDouble: [],
repeatedString: [],
repeatedBytes: [],
repeatedMsg: [{foo: 0}],
repeatedMsg: [{}],
repeatedEnum: []
}
@ -1256,8 +1227,9 @@ module CommonTests
assert_equal json, struct.to_json
assert_raise(RuntimeError, "Maximum recursion depth exceeded during encoding") do
proto_module::MyRepeatedStruct.encode(
proto_module::MyRepeatedStruct.new(structs: [proto_module::MyStruct.new(struct: struct)]))
struct = Google::Protobuf::Struct.new
struct.fields["foobar"] = Google::Protobuf::Value.new(struct_value: struct)
Google::Protobuf::Struct.encode(struct)
end
end
@ -1786,4 +1758,25 @@ module CommonTests
assert m1.hash != m2.hash
assert_nil h[m2]
end
def test_object_gc
m = proto_module::TestMessage.new(optional_msg: proto_module::TestMessage2.new)
m.optional_msg
GC.start(full_mark: true, immediate_sweep: true)
m.optional_msg.inspect
end
def test_object_gc_freeze
m = proto_module::TestMessage.new
m.repeated_float.freeze
GC.start(full_mark: true)
# Make sure we remember that the object is frozen.
# The wrapper object contains this information, so we need to ensure that
# the previous GC did not collect it.
assert m.repeated_float.frozen?
GC.start(full_mark: true, immediate_sweep: true)
assert m.repeated_float.frozen?
end
end

2
ruby/travis-test.sh
Unescape View File

@ -18,7 +18,7 @@ test_version() {
rake gc_test &&
cd ../conformance && make test_jruby &&
cd ../ruby/compatibility_tests/v3.0.0 && ./test.sh"
elif [ "$version" == "ruby-2.6.0" -o "$version" == "ruby-2.7.0" ] ; then
elif [ "$version" == "ruby-2.6.0" -o "$version" == "ruby-2.7.0" -o "$version" == "ruby-3.0.0" ] ; then
bash --login -c \
"rvm install $version && rvm use $version && \
which ruby && \

6
src/google/protobuf/compiler/ruby/ruby_generator.cc
Unescape View File

@ -242,8 +242,7 @@ void GenerateOneof(const OneofDescriptor* oneof, io::Printer* printer) {
bool GenerateMessage(const Descriptor* message, io::Printer* printer,
std::string* error) {
if (message->extension_range_count() > 0 || message->extension_count() > 0) {
*error = "Extensions are not yet supported for proto2 .proto files.";
return false;
GOOGLE_LOG(WARNING) << "Extensions are not yet supported for proto2 .proto files.";
}
// Don't generate MapEntry messages -- we use the Ruby extension's native
@ -543,8 +542,7 @@ bool GenerateFile(const FileDescriptor* file, io::Printer* printer,
// TODO: Remove this when ruby supports extensions for proto2 syntax.
if (file->syntax() == FileDescriptor::SYNTAX_PROTO2 &&
file->extension_count() > 0) {
*error = "Extensions are not yet supported for proto2 .proto files.";
return false;
GOOGLE_LOG(WARNING) << "Extensions are not yet supported for proto2 .proto files.";
}
printer->Print("Google::Protobuf::DescriptorPool.generated_pool.build do\n");

5
tests.sh
Unescape View File

@ -441,6 +441,10 @@ build_ruby27() {
internal_build_cpp # For conformance tests.
cd ruby && bash travis-test.sh ruby-2.7.0 && cd ..
}
build_ruby30() {
internal_build_cpp # For conformance tests.
cd ruby && bash travis-test.sh ruby-3.0.0 && cd ..
}
build_jruby() {
internal_build_cpp # For conformance tests.
@ -723,6 +727,7 @@ Usage: $0 { cpp |
ruby25 |
ruby26 |
ruby27 |
ruby30 |
jruby |
ruby_all |
php7.0 |

2
third_party/wyhash/wyhash.h vendored
Unescape View File

@ -116,7 +116,7 @@ static inline uint64_t wyhash(const void *key, uint64_t len, uint64_t seed, cons
return _wyfinish(p,len,seed,secret,i);
}
//utility functions
const uint64_t _wyp[5] = {0xa0761d6478bd642full, 0xe7037ed1a0b428dbull, 0x8ebc6af09c88c6e3ull, 0x589965cc75374cc3ull, 0x1d8e4e27c47d124full};
static const uint64_t _wyp[5] = {0xa0761d6478bd642full, 0xe7037ed1a0b428dbull, 0x8ebc6af09c88c6e3ull, 0x589965cc75374cc3ull, 0x1d8e4e27c47d124full};
static inline uint64_t wyhash64(uint64_t A, uint64_t B){ A^=_wyp[0]; B^=_wyp[1]; _wymum(&A,&B); return _wymix(A^_wyp[0],B^_wyp[1]);}
static inline uint64_t wyrand(uint64_t *seed){ *seed+=_wyp[0]; return _wymix(*seed,*seed^_wyp[1]);}
static inline double wy2u01(uint64_t r){ const double _wynorm=1.0/(1ull<<52); return (r>>12)*_wynorm;}

Write Preview
Loading…
Cancel
Save