Protocol Buffers - Google's data interchange format (grpc依赖) https://developers.google.com/protocol-buffers/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

349 lines
11 KiB

Ported Ruby extension to upb_msg (#8184) * WIP. * WIP. * WIP. * WIP. * WIP. * WIP. * Added some missing files. * WIP. * WIP. * Updated upb. * Extension loads, but crashes immediately. * Gets through the test suite without SEGV! Still a lot of bugs to fix, but it is a major step! 214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications 14.0187% passed * Test and build for Ruby 3.0 * Fixed a few more bugs, efficient #inspect is almost done. 214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications 18.6916% passed * Fixed message hash initialization and encode depth checking. 214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications 42.5234% passed * A bunch of fixes to failing tests, now 70% passing. 214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications 70.0935% passed * More than 80% of tests are passing now. 214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications 81.7757% passed Unfortunately there is also a sporadic bug/segfault hanging around that appears to be GC-related. * Add linux/ruby30 and macos/ruby30 * Use rvm master for 3.0.0-preview2 * Over 90% of tests are passing! 214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications 92.5234% passed * Passes all tests! 214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed * A bunch of cleanup. 1. Removed a bunch of internal-only symbols from headers. 2. Required a frozen check to get a non-const pointer to a map or array. 3. De-duplicated the code to get a type argument for Map/RepeatedField. * Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure. Intermittent failure: ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]") ), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed * Removed a few more things from protobuf.h. * Ruby 3.0.0-preview2 to 3.0.0 * Require rake-compiler-dock >= 1.1.0 * More progress, fighting with the object cache. * Passes on all Ruby versions! * Updated and clarified comment regarding WeakMap. * Fixed the wyhash compile. * Fixed conformance tests for Ruby. Conformance results now look like: RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures. CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures. Fixes include: - Changed Ruby compiler to no longer reject proto2 maps. - Changed Ruby compiler to emit a warning when proto2 extensions are present instead of rejecting the .proto file completely. - Fixed conformance tests to allow proto2 and look up message by name instead of hardcoding a specific list of messages. - Fixed conformance test to support the "ignore unknown" option for JSON. - Fixed conformance test to properly report serialization errors. * Removed debug printf and fixed #inspect for floats. * Fixed compatibility test to have proper semantics for #to_json. * Updated Makefile.am with new file list. * Don't try to copy wyhash when inside Docker. * Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7. * Avoid exporting unneeded symbols and refactored a bit of code. * Some more refactoring. * Simplified and added more comments. * Some more comments and simplification. Added a missing license block. Co-authored-by: Masaki Hara <hara@wantedly.com>
4 years ago
// Protocol Buffers - Google's data interchange format
// Copyright 2008 Google Inc. All rights reserved.
// https://developers.google.com/protocol-buffers/
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
// * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// -----------------------------------------------------------------------------
// Ruby <-> upb data conversion functions.
//
// This file Also contains a few other assorted algorithms on upb_msgval.
//
// None of the algorithms in this file require any access to the internal
// representation of Ruby or upb objects.
// -----------------------------------------------------------------------------
#include "convert.h"
#include "message.h"
#include "protobuf.h"
static upb_strview Convert_StringData(VALUE str, upb_arena *arena) {
upb_strview ret;
if (arena) {
char *ptr = upb_arena_malloc(arena, RSTRING_LEN(str));
memcpy(ptr, RSTRING_PTR(str), RSTRING_LEN(str));
ret.data = ptr;
} else {
// Data is only needed temporarily (within map lookup).
ret.data = RSTRING_PTR(str);
}
ret.size = RSTRING_LEN(str);
return ret;
}
static bool is_ruby_num(VALUE value) {
return (TYPE(value) == T_FLOAT ||
TYPE(value) == T_FIXNUM ||
TYPE(value) == T_BIGNUM);
}
static void Convert_CheckInt(const char* name, upb_fieldtype_t type,
VALUE val) {
if (!is_ruby_num(val)) {
rb_raise(cTypeError,
"Expected number type for integral field '%s' (given %s).", name,
rb_class2name(CLASS_OF(val)));
}
// NUM2{INT,UINT,LL,ULL} macros do the appropriate range checks on upper
// bound; we just need to do precision checks (i.e., disallow rounding) and
// check for < 0 on unsigned types.
if (TYPE(val) == T_FLOAT) {
double dbl_val = NUM2DBL(val);
if (floor(dbl_val) != dbl_val) {
rb_raise(rb_eRangeError,
"Non-integral floating point value assigned to integer field "
"'%s' (given %s).",
name, rb_class2name(CLASS_OF(val)));
}
}
if (type == UPB_TYPE_UINT32 || type == UPB_TYPE_UINT64) {
if (NUM2DBL(val) < 0) {
rb_raise(
rb_eRangeError,
"Assigning negative value to unsigned integer field '%s' (given %s).",
name, rb_class2name(CLASS_OF(val)));
}
}
}
static int32_t Convert_ToEnum(VALUE value, const char* name,
const upb_enumdef* e) {
int32_t val;
switch (TYPE(value)) {
case T_FLOAT:
case T_FIXNUM:
case T_BIGNUM:
Convert_CheckInt(name, UPB_TYPE_INT32, value);
val = NUM2INT(value);
break;
case T_STRING:
if (!upb_enumdef_ntoi(e, RSTRING_PTR(value), RSTRING_LEN(value), &val)) {
goto unknownval;
}
break;
case T_SYMBOL:
if (!upb_enumdef_ntoiz(e, rb_id2name(SYM2ID(value)), &val)) {
goto unknownval;
}
break;
default:
rb_raise(cTypeError,
"Expected number or symbol type for enum field '%s'.", name);
}
return val;
unknownval:
rb_raise(rb_eRangeError, "Unknown symbol value for enum field '%s'.", name);
}
upb_msgval Convert_RubyToUpb(VALUE value, const char* name, TypeInfo type_info,
upb_arena* arena) {
upb_msgval ret;
switch (type_info.type) {
case UPB_TYPE_FLOAT:
if (!is_ruby_num(value)) {
rb_raise(cTypeError, "Expected number type for float field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
ret.float_val = NUM2DBL(value);
break;
case UPB_TYPE_DOUBLE:
if (!is_ruby_num(value)) {
rb_raise(cTypeError, "Expected number type for double field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
ret.double_val = NUM2DBL(value);
break;
case UPB_TYPE_BOOL: {
if (value == Qtrue) {
ret.bool_val = 1;
} else if (value == Qfalse) {
ret.bool_val = 0;
} else {
rb_raise(cTypeError, "Invalid argument for boolean field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
break;
}
case UPB_TYPE_STRING: {
VALUE utf8 = rb_enc_from_encoding(rb_utf8_encoding());
if (CLASS_OF(value) == rb_cSymbol) {
value = rb_funcall(value, rb_intern("to_s"), 0);
} else if (CLASS_OF(value) != rb_cString) {
rb_raise(cTypeError, "Invalid argument for string field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
if (rb_obj_encoding(value) != utf8) {
// Note: this will not duplicate underlying string data unless necessary.
value = rb_str_encode(value, utf8, 0, Qnil);
if (rb_enc_str_coderange(value) == ENC_CODERANGE_BROKEN) {
rb_raise(rb_eEncodingError, "String is invalid UTF-8");
}
}
ret.str_val = Convert_StringData(value, arena);
break;
}
case UPB_TYPE_BYTES: {
VALUE bytes = rb_enc_from_encoding(rb_ascii8bit_encoding());
if (CLASS_OF(value) != rb_cString) {
rb_raise(cTypeError, "Invalid argument for bytes field '%s' (given %s).",
name, rb_class2name(CLASS_OF(value)));
}
if (rb_obj_encoding(value) != bytes) {
// Note: this will not duplicate underlying string data unless necessary.
// TODO(haberman): is this really necessary to get raw bytes?
value = rb_str_encode(value, bytes, 0, Qnil);
}
ret.str_val = Convert_StringData(value, arena);
break;
}
case UPB_TYPE_MESSAGE:
ret.msg_val =
Message_GetUpbMessage(value, type_info.def.msgdef, name, arena);
break;
case UPB_TYPE_ENUM:
ret.int32_val = Convert_ToEnum(value, name, type_info.def.enumdef);
break;
case UPB_TYPE_INT32:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT32:
case UPB_TYPE_UINT64:
Convert_CheckInt(name, type_info.type, value);
switch (type_info.type) {
case UPB_TYPE_INT32:
ret.int32_val = NUM2INT(value);
break;
case UPB_TYPE_INT64:
ret.int64_val = NUM2LL(value);
break;
case UPB_TYPE_UINT32:
ret.uint32_val = NUM2UINT(value);
break;
case UPB_TYPE_UINT64:
ret.uint64_val = NUM2ULL(value);
break;
default:
break;
}
break;
default:
break;
}
return ret;
}
VALUE Convert_UpbToRuby(upb_msgval upb_val, TypeInfo type_info, VALUE arena) {
switch (type_info.type) {
case UPB_TYPE_FLOAT:
return DBL2NUM(upb_val.float_val);
case UPB_TYPE_DOUBLE:
return DBL2NUM(upb_val.double_val);
case UPB_TYPE_BOOL:
return upb_val.bool_val ? Qtrue : Qfalse;
case UPB_TYPE_INT32:
return INT2NUM(upb_val.int32_val);
case UPB_TYPE_INT64:
return LL2NUM(upb_val.int64_val);
case UPB_TYPE_UINT32:
return UINT2NUM(upb_val.uint32_val);
case UPB_TYPE_UINT64:
return ULL2NUM(upb_val.int64_val);
case UPB_TYPE_ENUM: {
const char* name =
upb_enumdef_iton(type_info.def.enumdef, upb_val.int32_val);
if (name) {
return ID2SYM(rb_intern(name));
} else {
return INT2NUM(upb_val.int32_val);
}
}
case UPB_TYPE_STRING: {
VALUE str_rb = rb_str_new(upb_val.str_val.data, upb_val.str_val.size);
rb_enc_associate(str_rb, rb_utf8_encoding());
rb_obj_freeze(str_rb);
return str_rb;
}
case UPB_TYPE_BYTES: {
VALUE str_rb = rb_str_new(upb_val.str_val.data, upb_val.str_val.size);
rb_enc_associate(str_rb, rb_ascii8bit_encoding());
rb_obj_freeze(str_rb);
return str_rb;
}
case UPB_TYPE_MESSAGE:
return Message_GetRubyWrapper((upb_msg*)upb_val.msg_val,
type_info.def.msgdef, arena);
default:
rb_raise(rb_eRuntimeError, "Convert_UpbToRuby(): Unexpected type %d",
(int)type_info.type);
}
}
upb_msgval Msgval_DeepCopy(upb_msgval msgval, TypeInfo type_info,
upb_arena* arena) {
upb_msgval new_msgval;
switch (type_info.type) {
default:
memcpy(&new_msgval, &msgval, sizeof(msgval));
break;
case UPB_TYPE_STRING:
case UPB_TYPE_BYTES: {
size_t n = msgval.str_val.size;
char *mem = upb_arena_malloc(arena, n);
new_msgval.str_val.data = mem;
new_msgval.str_val.size = n;
memcpy(mem, msgval.str_val.data, n);
break;
}
case UPB_TYPE_MESSAGE:
new_msgval.msg_val =
Message_deep_copy(msgval.msg_val, type_info.def.msgdef, arena);
break;
}
return new_msgval;
}
bool Msgval_IsEqual(upb_msgval val1, upb_msgval val2, TypeInfo type_info) {
switch (type_info.type) {
case UPB_TYPE_BOOL:
return memcmp(&val1, &val2, 1) == 0;
case UPB_TYPE_FLOAT:
case UPB_TYPE_INT32:
case UPB_TYPE_UINT32:
case UPB_TYPE_ENUM:
return memcmp(&val1, &val2, 4) == 0;
case UPB_TYPE_DOUBLE:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT64:
return memcmp(&val1, &val2, 8) == 0;
case UPB_TYPE_STRING:
case UPB_TYPE_BYTES:
return val1.str_val.size == val2.str_val.size &&
Ported Ruby extension to upb_msg (#8184) * WIP. * WIP. * WIP. * WIP. * WIP. * WIP. * Added some missing files. * WIP. * WIP. * Updated upb. * Extension loads, but crashes immediately. * Gets through the test suite without SEGV! Still a lot of bugs to fix, but it is a major step! 214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications 14.0187% passed * Test and build for Ruby 3.0 * Fixed a few more bugs, efficient #inspect is almost done. 214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications 18.6916% passed * Fixed message hash initialization and encode depth checking. 214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications 42.5234% passed * A bunch of fixes to failing tests, now 70% passing. 214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications 70.0935% passed * More than 80% of tests are passing now. 214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications 81.7757% passed Unfortunately there is also a sporadic bug/segfault hanging around that appears to be GC-related. * Add linux/ruby30 and macos/ruby30 * Use rvm master for 3.0.0-preview2 * Over 90% of tests are passing! 214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications 92.5234% passed * Passes all tests! 214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed * A bunch of cleanup. 1. Removed a bunch of internal-only symbols from headers. 2. Required a frozen check to get a non-const pointer to a map or array. 3. De-duplicated the code to get a type argument for Map/RepeatedField. * Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure. Intermittent failure: ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]") ), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed * Removed a few more things from protobuf.h. * Ruby 3.0.0-preview2 to 3.0.0 * Require rake-compiler-dock >= 1.1.0 * More progress, fighting with the object cache. * Passes on all Ruby versions! * Updated and clarified comment regarding WeakMap. * Fixed the wyhash compile. * Fixed conformance tests for Ruby. Conformance results now look like: RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures. CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures. Fixes include: - Changed Ruby compiler to no longer reject proto2 maps. - Changed Ruby compiler to emit a warning when proto2 extensions are present instead of rejecting the .proto file completely. - Fixed conformance tests to allow proto2 and look up message by name instead of hardcoding a specific list of messages. - Fixed conformance test to support the "ignore unknown" option for JSON. - Fixed conformance test to properly report serialization errors. * Removed debug printf and fixed #inspect for floats. * Fixed compatibility test to have proper semantics for #to_json. * Updated Makefile.am with new file list. * Don't try to copy wyhash when inside Docker. * Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7. * Avoid exporting unneeded symbols and refactored a bit of code. * Some more refactoring. * Simplified and added more comments. * Some more comments and simplification. Added a missing license block. Co-authored-by: Masaki Hara <hara@wantedly.com>
4 years ago
memcmp(val1.str_val.data, val2.str_val.data,
val1.str_val.size) == 0;
case UPB_TYPE_MESSAGE:
return Message_Equal(val1.msg_val, val2.msg_val, type_info.def.msgdef);
default:
rb_raise(rb_eRuntimeError, "Internal error, unexpected type");
}
}
uint64_t Msgval_GetHash(upb_msgval val, TypeInfo type_info, uint64_t seed) {
switch (type_info.type) {
case UPB_TYPE_BOOL:
return Wyhash(&val, 1, seed, kWyhashSalt);
Ported Ruby extension to upb_msg (#8184) * WIP. * WIP. * WIP. * WIP. * WIP. * WIP. * Added some missing files. * WIP. * WIP. * Updated upb. * Extension loads, but crashes immediately. * Gets through the test suite without SEGV! Still a lot of bugs to fix, but it is a major step! 214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications 14.0187% passed * Test and build for Ruby 3.0 * Fixed a few more bugs, efficient #inspect is almost done. 214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications 18.6916% passed * Fixed message hash initialization and encode depth checking. 214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications 42.5234% passed * A bunch of fixes to failing tests, now 70% passing. 214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications 70.0935% passed * More than 80% of tests are passing now. 214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications 81.7757% passed Unfortunately there is also a sporadic bug/segfault hanging around that appears to be GC-related. * Add linux/ruby30 and macos/ruby30 * Use rvm master for 3.0.0-preview2 * Over 90% of tests are passing! 214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications 92.5234% passed * Passes all tests! 214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed * A bunch of cleanup. 1. Removed a bunch of internal-only symbols from headers. 2. Required a frozen check to get a non-const pointer to a map or array. 3. De-duplicated the code to get a type argument for Map/RepeatedField. * Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure. Intermittent failure: ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]") ), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed * Removed a few more things from protobuf.h. * Ruby 3.0.0-preview2 to 3.0.0 * Require rake-compiler-dock >= 1.1.0 * More progress, fighting with the object cache. * Passes on all Ruby versions! * Updated and clarified comment regarding WeakMap. * Fixed the wyhash compile. * Fixed conformance tests for Ruby. Conformance results now look like: RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures. CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures. Fixes include: - Changed Ruby compiler to no longer reject proto2 maps. - Changed Ruby compiler to emit a warning when proto2 extensions are present instead of rejecting the .proto file completely. - Fixed conformance tests to allow proto2 and look up message by name instead of hardcoding a specific list of messages. - Fixed conformance test to support the "ignore unknown" option for JSON. - Fixed conformance test to properly report serialization errors. * Removed debug printf and fixed #inspect for floats. * Fixed compatibility test to have proper semantics for #to_json. * Updated Makefile.am with new file list. * Don't try to copy wyhash when inside Docker. * Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7. * Avoid exporting unneeded symbols and refactored a bit of code. * Some more refactoring. * Simplified and added more comments. * Some more comments and simplification. Added a missing license block. Co-authored-by: Masaki Hara <hara@wantedly.com>
4 years ago
case UPB_TYPE_FLOAT:
case UPB_TYPE_INT32:
case UPB_TYPE_UINT32:
case UPB_TYPE_ENUM:
return Wyhash(&val, 4, seed, kWyhashSalt);
Ported Ruby extension to upb_msg (#8184) * WIP. * WIP. * WIP. * WIP. * WIP. * WIP. * Added some missing files. * WIP. * WIP. * Updated upb. * Extension loads, but crashes immediately. * Gets through the test suite without SEGV! Still a lot of bugs to fix, but it is a major step! 214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications 14.0187% passed * Test and build for Ruby 3.0 * Fixed a few more bugs, efficient #inspect is almost done. 214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications 18.6916% passed * Fixed message hash initialization and encode depth checking. 214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications 42.5234% passed * A bunch of fixes to failing tests, now 70% passing. 214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications 70.0935% passed * More than 80% of tests are passing now. 214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications 81.7757% passed Unfortunately there is also a sporadic bug/segfault hanging around that appears to be GC-related. * Add linux/ruby30 and macos/ruby30 * Use rvm master for 3.0.0-preview2 * Over 90% of tests are passing! 214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications 92.5234% passed * Passes all tests! 214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed * A bunch of cleanup. 1. Removed a bunch of internal-only symbols from headers. 2. Required a frozen check to get a non-const pointer to a map or array. 3. De-duplicated the code to get a type argument for Map/RepeatedField. * Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure. Intermittent failure: ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]") ), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed * Removed a few more things from protobuf.h. * Ruby 3.0.0-preview2 to 3.0.0 * Require rake-compiler-dock >= 1.1.0 * More progress, fighting with the object cache. * Passes on all Ruby versions! * Updated and clarified comment regarding WeakMap. * Fixed the wyhash compile. * Fixed conformance tests for Ruby. Conformance results now look like: RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures. CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures. Fixes include: - Changed Ruby compiler to no longer reject proto2 maps. - Changed Ruby compiler to emit a warning when proto2 extensions are present instead of rejecting the .proto file completely. - Fixed conformance tests to allow proto2 and look up message by name instead of hardcoding a specific list of messages. - Fixed conformance test to support the "ignore unknown" option for JSON. - Fixed conformance test to properly report serialization errors. * Removed debug printf and fixed #inspect for floats. * Fixed compatibility test to have proper semantics for #to_json. * Updated Makefile.am with new file list. * Don't try to copy wyhash when inside Docker. * Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7. * Avoid exporting unneeded symbols and refactored a bit of code. * Some more refactoring. * Simplified and added more comments. * Some more comments and simplification. Added a missing license block. Co-authored-by: Masaki Hara <hara@wantedly.com>
4 years ago
case UPB_TYPE_DOUBLE:
case UPB_TYPE_INT64:
case UPB_TYPE_UINT64:
return Wyhash(&val, 8, seed, kWyhashSalt);
Ported Ruby extension to upb_msg (#8184) * WIP. * WIP. * WIP. * WIP. * WIP. * WIP. * Added some missing files. * WIP. * WIP. * Updated upb. * Extension loads, but crashes immediately. * Gets through the test suite without SEGV! Still a lot of bugs to fix, but it is a major step! 214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications 14.0187% passed * Test and build for Ruby 3.0 * Fixed a few more bugs, efficient #inspect is almost done. 214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications 18.6916% passed * Fixed message hash initialization and encode depth checking. 214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications 42.5234% passed * A bunch of fixes to failing tests, now 70% passing. 214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications 70.0935% passed * More than 80% of tests are passing now. 214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications 81.7757% passed Unfortunately there is also a sporadic bug/segfault hanging around that appears to be GC-related. * Add linux/ruby30 and macos/ruby30 * Use rvm master for 3.0.0-preview2 * Over 90% of tests are passing! 214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications 92.5234% passed * Passes all tests! 214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed * A bunch of cleanup. 1. Removed a bunch of internal-only symbols from headers. 2. Required a frozen check to get a non-const pointer to a map or array. 3. De-duplicated the code to get a type argument for Map/RepeatedField. * Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure. Intermittent failure: ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]") ), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed * Removed a few more things from protobuf.h. * Ruby 3.0.0-preview2 to 3.0.0 * Require rake-compiler-dock >= 1.1.0 * More progress, fighting with the object cache. * Passes on all Ruby versions! * Updated and clarified comment regarding WeakMap. * Fixed the wyhash compile. * Fixed conformance tests for Ruby. Conformance results now look like: RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures. CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures. Fixes include: - Changed Ruby compiler to no longer reject proto2 maps. - Changed Ruby compiler to emit a warning when proto2 extensions are present instead of rejecting the .proto file completely. - Fixed conformance tests to allow proto2 and look up message by name instead of hardcoding a specific list of messages. - Fixed conformance test to support the "ignore unknown" option for JSON. - Fixed conformance test to properly report serialization errors. * Removed debug printf and fixed #inspect for floats. * Fixed compatibility test to have proper semantics for #to_json. * Updated Makefile.am with new file list. * Don't try to copy wyhash when inside Docker. * Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7. * Avoid exporting unneeded symbols and refactored a bit of code. * Some more refactoring. * Simplified and added more comments. * Some more comments and simplification. Added a missing license block. Co-authored-by: Masaki Hara <hara@wantedly.com>
4 years ago
case UPB_TYPE_STRING:
case UPB_TYPE_BYTES:
return Wyhash(val.str_val.data, val.str_val.size, seed, kWyhashSalt);
Ported Ruby extension to upb_msg (#8184) * WIP. * WIP. * WIP. * WIP. * WIP. * WIP. * Added some missing files. * WIP. * WIP. * Updated upb. * Extension loads, but crashes immediately. * Gets through the test suite without SEGV! Still a lot of bugs to fix, but it is a major step! 214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications 14.0187% passed * Test and build for Ruby 3.0 * Fixed a few more bugs, efficient #inspect is almost done. 214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications 18.6916% passed * Fixed message hash initialization and encode depth checking. 214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications 42.5234% passed * A bunch of fixes to failing tests, now 70% passing. 214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications 70.0935% passed * More than 80% of tests are passing now. 214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications 81.7757% passed Unfortunately there is also a sporadic bug/segfault hanging around that appears to be GC-related. * Add linux/ruby30 and macos/ruby30 * Use rvm master for 3.0.0-preview2 * Over 90% of tests are passing! 214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications 92.5234% passed * Passes all tests! 214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications 100% passed * A bunch of cleanup. 1. Removed a bunch of internal-only symbols from headers. 2. Required a frozen check to get a non-const pointer to a map or array. 3. De-duplicated the code to get a type argument for Map/RepeatedField. * Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure. Intermittent failure: ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]") ), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed * Removed a few more things from protobuf.h. * Ruby 3.0.0-preview2 to 3.0.0 * Require rake-compiler-dock >= 1.1.0 * More progress, fighting with the object cache. * Passes on all Ruby versions! * Updated and clarified comment regarding WeakMap. * Fixed the wyhash compile. * Fixed conformance tests for Ruby. Conformance results now look like: RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures. CONFORMANCE TEST BEGIN ==================================== CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures. Fixes include: - Changed Ruby compiler to no longer reject proto2 maps. - Changed Ruby compiler to emit a warning when proto2 extensions are present instead of rejecting the .proto file completely. - Fixed conformance tests to allow proto2 and look up message by name instead of hardcoding a specific list of messages. - Fixed conformance test to support the "ignore unknown" option for JSON. - Fixed conformance test to properly report serialization errors. * Removed debug printf and fixed #inspect for floats. * Fixed compatibility test to have proper semantics for #to_json. * Updated Makefile.am with new file list. * Don't try to copy wyhash when inside Docker. * Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7. * Avoid exporting unneeded symbols and refactored a bit of code. * Some more refactoring. * Simplified and added more comments. * Some more comments and simplification. Added a missing license block. Co-authored-by: Masaki Hara <hara@wantedly.com>
4 years ago
case UPB_TYPE_MESSAGE:
return Message_Hash(val.msg_val, type_info.def.msgdef, seed);
default:
rb_raise(rb_eRuntimeError, "Internal error, unexpected type");
}
}