* WIP.
* WIP.
* Builds and runs. Tests need to be updated to test presence.
* Ruby: proto3 presence is passing all tests.
* Fixed a bug where empty messages has the wrong oneof count.
This patch has almost no change in behaviour where users have not
patched the implementation of new on either a specific proto object
class, or `Google::Protobuf::MessageExts::ClassMethods`. The default
implementation of `new`, and `rb_class_new_instance` have the same
behaviour.
By default when we call `new` on a class in Ruby, it goes to the `Class`
class's implementation:
```ruby
class Foo
end
>> Foo.method(:new).owner
=> Class
```
the `Class` implementation of `new` is (pseudocode, it's actually in c):
```ruby
class Class
def new(*args, &blk)
instance = alloc
instance.initialize(*args, &blk)
instance
end
end
```
`rb_class_new_instance` does the same thing, it calls down to
[`rb_class_s_new`](https://github.com/ruby/ruby/blob/v2_5_5/object.c#L2147),
which calls `rb_class_alloc`, then `rb_obj_call_init`.
`rb_funcall` is a variadic c function for calling a ruby method on an object,
it takes:
* A `VALUE` on to which the method should be called
* An `ID` which should be an ID of a method, usually created with `rb_intern`,
to get an ID from a string
* An integer, the number of arguments calling the method with,
* A number of `VALUE`s, to send to the method call.
`rb_funcall` is the same as calling a method directly in Ruby, and will perform
ancestor chain respecting method lookup.
This means that in C extensions, if nobody has defined the `new` method on any
classes or modules in a class's inheritance chain calling
`rb_class_new_instance` is the same as calling `rb_funcall(klass,
rb_intern("new"))`, *however* this removes the ability for users to define or
monkey patch their own constructors in to the objects created by the C
extension.
In Ads, we define [`new`](https://git.io/JvFC9) on
`Google::Protobuf::MessageExts::ClassMethods` to allow us to insert a
monkeypatch which makes it possible to assign primitive values to wrapper type
fields (e.g. Google::Protobuf::StringValue). The monkeypatch we apply works for
objects that we create for the user via the `new` method. Before this commit,
however, the patch does not work for the `decode` method, for the reasons
outlined above. Before this commit, protobuf uses `rb_class_new_instance`.
After this commit, we use `rb_funcall(klass, rb_intern("new"), 0);` to construct
protobuf objects during decoding. While I haven't measured it this will have
a very minor performance impact for decoding (`rb_funcall` will have to go to the
method cache, which `rb_class_new_instance` will not). This does however do
the "more rubyish" thing of respecting the protobuf object's inheritance chain
to construct them during decoding.
I have run both Ads and Cloud's test suites for Ruby libraries against this
patch, as well as the protobuf Ruby gem's test suite locally.
* Add failing tests for issues with wrapped values where the value is the default
* Add test for wrapped values without a value set
* Bugfix for wrapper types with default values.
The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.
In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.
The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.
Co-authored-by: Joshua Haberman <jhaberman@gmail.com>
Co-authored-by: Dan Quan <dan@quan.io>
* Add failing tests for issues with wrapped values where the value is the default
* Add test for wrapped values without a value set
* Bugfix for wrapper types with default values.
The previous optimizations for wrapper types had a bug that prevented
wrappers from registering as "present" if the "value" field was not
present on the wire.
In practice the "value" field will not be serialized when it is zero,
according to proto3 semantics, but due to the optimization this
prevented it from creating a new object to represent the presence of the
field.
The fix is to ensure that if the wrapper message is present on the wire,
we always initialize its value to zero.
Co-authored-by: Dan Quan <dan@quan.io>
The only case that doesn't work is decoding a wrapper type from JSON
at the top level. This doesn't make sense and probably no users do it
I changed it to throw.
We were creating a map decoding frame when starting the *map*,
but clearing the GC slot when finishing each *map entry*. This
means that the decoding frame could be collected in the meantime.
* Rolled forward again with "Updated upb from defcleanup branch..."
Revert "Revert "Updated upb from defcleanup branch and modified Ruby to use it (#5539)" (#5848)"
This reverts commit 1568deab40.
* A few more merge fixes.
* Updated for defcleanup2 branch.
* Fixed upb to define upb_decode().
* Fixed names of nested messages.
* Revert submodule.
* Set -std=gnu90 and fixed warnings/errors.
Some of our Kokoro tests seem to run with this level of warnings,
and the source strives to be gnu90 compatible. Enforcing it for
every build removes the possibility of some errors showing up in
Kokoro/Travis tests only.
* Fixed remaining warnings with gnu90 mode.
I tried to match warning flags with what Ruby appears to do
in our Kokoro tests.
* Initialize values registered by rb_gc_register_address().
* Fixed subtle GC bug.
We need to initialize this marked value before creating the instance.
* Truly fix the GC bug.
* Updated upb for mktime() fix.
* Removed XOPEN_SOURCE as we are not using strptime().
* Removed fixed tests from the conformance failure list for Ruby.
* Fixed memory error related to oneof def names.
* Picked up new upb changes re: JSON printing.
* Uncomment concurrent decoding test.
value_field_typeclass should be a enum module, not EnumDescriptor
object.
Also expanding tests for enum valued maps.
Fixes#4580
Signed-off-by: Sorah Fukumori <her@sorah.jp>
* Revert "Revert "Enable the ignore_unknown_field option in the Ruby unmarshal options" (#5511)"
This reverts commit be1716a6d0.
* Separate ruby conformance test on Mac
* Fix shell syntax
* Fix test
This adds the ability for the MRI Ruby library to optionally pass in a
ignore_unknown_fields option when decoding JSON. The functionality was
added upstream in upb, this change exposes that option.
This change only adds basic proto2 support without advanced features
like extensions, custom options, maps, etc.
The protoc binary now generates ruby code for proto2 syntax.
However, for now, it is restricted to proto2 files without advanced features
like extensions, in which case it still errors out.
This change also modifies the DSL to add proto messages to the DescriptorPool.
There is a new DSL Builder#add_file to create a new FileDescriptor. With this,
the generated ruby DSL looks something like:
Google::Protobuf::DescriptorPool.generated_pool.build do
add_file "test.proto" do
add_message "foo" do
optional :val, :int32, 1
end
end
end
* Add discard unknown API in ruby.
* Add test for oneof message field.
* Add TestUnknown to represent unknown field data clearly.
* Only serialize the message with unknown fields itself in test.
* Move discard_unknown from Message to Google.Protobuf
* Reserve unknown in ruby
* Revert ruby tests. Wait for cpp impl for conformance test
* Add conformance test for preserving unknown
* Add unknown field conformance test to csharp failure list.
* Fix comments
* Fix comment
* Fix comments
* Fix typo
* Use stringsink_string directly
* Mark hd unused
* Remove unused encodeunknown_handlerfunc
* Down-integrate internal changes to github.
* Update conformance test failure list.
* Explicitly import used class in nano test to avoid random test fail.
* Update _GNUC_VER to use the correct implementation of atomic operation
on Mac.
* maps_test.js: check whether Symbol is defined before using it (#2524)
Symbol is not yet available on older versions of Node.js and so this
test fails with them. This change just directly checks whether Symbol is
available before we try to use it.
* Added well_known_types_embed.cc to CLEANFILES so that it gets cleaned up
* Updated Makefile.am to fix out-of-tree builds
* Added Bazel genrule for generating well_known_types_embed.cc
In pull request #2517 I made this change for the CMake and autotools
builds but forgot to do it for the Bazel build.
* Update _GNUC_VER to use the correct implementation of atomic operation on Mac.
* Add new js file in extra dist.
* Bump version number to 3.2.0
* Fixed issue with autoloading - Invalid paths (#2538)
* PHP fix int64 decoding (#2516)
* fix int64 decoding
* fix int64 decoding + tests
* Fix int64 decoding on 32-bit machines.
* Fix warning in compiler/js/embed.cc
embed.cc: In function ‘std::string CEscape(const string&)’:
embed.cc:51:32: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int i = 0; i < str.size(); ++i) {
^
* Fix include in auto-generated well_known_types_embed.cc
Restore include style fix (e3da722) that has been trampled by
auto-generation of well_known_types_embed.cc
* Fixed cross compilations with the Autotools build
Pull request #2517 caused cross compilations to start failing, because
the js_embed binary was being built to run on the target platform
instead of on the build machine. This change updates the Autotools build
to use the AX_PROG_CXX_FOR_BUILD macro to find a suitable compiler for
the build machine and always use that when building js_embed.
* Minor fix for autocreated object repeated fields and maps.
- If setting/clearing a repeated field/map that was objects, check the class
before checking the autocreator.
- Just to be paranoid, don’t mutate within copy/mutableCopy for the autocreated
classes to ensure there is less chance of issues if someone does something
really crazy threading wise.
- Some more tests for the internal AutocreatedArray/AutocreatedDictionary
classes to ensure things are working as expected.
- Add Xcode 8.2 to the full_mac_build.sh supported list.
* Fix generation of extending nested messages in JavaScript (#2439)
* Fix generation of extending nested messages in JavaScript
* Added missing test8.proto to build
* Fix generated code when there is no namespace but there is enum definition.
* Decoding unknown field should succeed.
* Add embed.cc in src/Makefile.am to fix dist check.
* Fixed "make distcheck" for the Autotools build
To make the test pass I needed to fix out-of-tree builds and update
EXTRA_DIST and CLEANFILES.
* Remove redundent embed.cc from src/Makefile.am
* Update version number to 3.2.0-rc.1 (#2578)
* Change protoc-artifacts version to 3.2.0-rc.1
* Update version number to 3.2.0rc2
* Update change logs for 3.2.0 release.
* Update php README
* Update upb, fixes some bugs (including a hash table problem). (#2611)
* Update upb, fixes some bugs (including a hash table problem).
* Ruby: added a test for the previous hash table corruption.
Verified that this triggers the bug in the currently released
version.
* Ruby: bugfix for SEGV.
* Ruby: removed old code for dup'ing defs.
* Reverting deployment target to 7.0 (#2618)
The Protobuf library doesn’t require the 7.1 deployment target so
reverting it back to 7.0
* Fix typo that breaks builds on big-endian (#2632)
* Bump version number to 3.2.0
For JSON encoding we provide a new option to decide at
encode time whether to use camelCase or original proto field
names:
json = MapMessage.encode_json(m, :preserve_proto_fieldnames => true)
listonjesse
Joshua Haberman
Penelope Phippen
Rafi Kamal
Sorah Fukumori
Paul Yang
bmoyles0117
Joe Bolinger
Erik Benoist
Harshit Chopra
Ewout
Aaron Patterson