Rewrrte and extension of #12828, with additional work for JRuby. Partially fixes#1198 by adding support for custom options. Handling of extensions will be handled in a follow up.
Also includes these unrelated fixes:
* Removes code echo between `google/protobuf/repeated_field.rb` and `google/protobuf/ffi/repeated_field.rb` by `require`'ing the former in the latter.
* Adds missing calles to `testFrozen()` from methods of `RepeatedField` under JRuby that mutate.
* Various typos in comments.
Closes#14594
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/14594 from protocolbuffers:add-support-for-options-in-ruby 16cc9e35b8
PiperOrigin-RevId: 580848874
Prior to this CL, asserts have no effect for Ruby 3+, because Ruby unconditionally defines `NDEBUG`: https://bugs.ruby-lang.org/issues/18777
We work around this by doing `#undef NDEBUG` right after including Ruby, if `NDEBUG` was not previously defined.
PiperOrigin-RevId: 535359088
* Fixed memory bug: properly root repeated/map field when assigning.
Previously the protobuf extension would not properly root
memory from a repeated field or map when assigning to a
message field (see the attached test case). This could cause
crashes if the repeated field is subsequently accessed.
* Add accidentally-deleted Ruby test.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* WIP.
* Added some missing files.
* WIP.
* WIP.
* Updated upb.
* Extension loads, but crashes immediately.
* Gets through the test suite without SEGV!
Still a lot of bugs to fix, but it is a major step!
214 tests, 378 assertions, 37 failures, 147 errors, 0 pendings, 0 omissions, 0 notifications
14.0187% passed
* Test and build for Ruby 3.0
* Fixed a few more bugs, efficient #inspect is almost done.
214 tests, 134243 assertions, 30 failures, 144 errors, 0 pendings, 0 omissions, 0 notifications
18.6916% passed
* Fixed message hash initialization and encode depth checking.
214 tests, 124651 assertions, 53 failures, 70 errors, 0 pendings, 0 omissions, 0 notifications
42.5234% passed
* A bunch of fixes to failing tests, now 70% passing.
214 tests, 202091 assertions, 41 failures, 23 errors, 0 pendings, 0 omissions, 0 notifications
70.0935% passed
* More than 80% of tests are passing now.
214 tests, 322331 assertions, 30 failures, 9 errors, 0 pendings, 0 omissions, 0 notifications
81.7757% passed
Unfortunately there is also a sporadic bug/segfault hanging around
that appears to be GC-related.
* Add linux/ruby30 and macos/ruby30
* Use rvm master for 3.0.0-preview2
* Over 90% of tests are passing!
214 tests, 349898 assertions, 15 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
92.5234% passed
* Passes all tests!
214 tests, 369388 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
* A bunch of cleanup.
1. Removed a bunch of internal-only symbols from headers.
2. Required a frozen check to get a non-const pointer to a map or array.
3. De-duplicated the code to get a type argument for Map/RepeatedField.
* Removed a bunch more stuff from protobuf.h. There is an intermittent assert failure.
Intermittent failure:
ruby: ../../../../ext/google/protobuf_c/protobuf.c:263: ObjectCache_Add: Assertion `rb_funcall(obj_cache2, (__builtin_constant_p("[]") ? __extension__ ({ static ID rb_intern_id_cache; if (!rb_intern_id_cache) rb_intern_id_cache = rb_intern2((("[]")
), (long)strlen(("[]"))); (ID) rb_intern_id_cache; }) : rb_intern("[]")), 1, key_rb) == val' failed
* Removed a few more things from protobuf.h.
* Ruby 3.0.0-preview2 to 3.0.0
* Require rake-compiler-dock >= 1.1.0
* More progress, fighting with the object cache.
* Passes on all Ruby versions!
* Updated and clarified comment regarding WeakMap.
* Fixed the wyhash compile.
* Fixed conformance tests for Ruby.
Conformance results now look like:
RUBYLIB=../ruby/lib:. ./conformance-test-runner --enforce_recommended --failure_list failure_list_ruby.txt --text_format_failure_list text_format_failure_list_ruby.txt ./conformance_ruby.rb
CONFORMANCE TEST BEGIN ====================================
CONFORMANCE SUITE PASSED: 1955 successes, 0 skipped, 58 expected failures, 0 unexpected failures.
CONFORMANCE TEST BEGIN ====================================
CONFORMANCE SUITE PASSED: 0 successes, 111 skipped, 8 expected failures, 0 unexpected failures.
Fixes include:
- Changed Ruby compiler to no longer reject proto2 maps.
- Changed Ruby compiler to emit a warning when proto2 extensions are
present instead of rejecting the .proto file completely.
- Fixed conformance tests to allow proto2 and look up message by name
instead of hardcoding a specific list of messages.
- Fixed conformance test to support the "ignore unknown" option for
JSON.
- Fixed conformance test to properly report serialization errors.
* Removed debug printf and fixed #inspect for floats.
* Fixed compatibility test to have proper semantics for #to_json.
* Updated Makefile.am with new file list.
* Don't try to copy wyhash when inside Docker.
* Fixed bug where we would forget that a sub-object is frozen in Ruby >=2.7.
* Avoid exporting unneeded symbols and refactored a bit of code.
* Some more refactoring.
* Simplified and added more comments.
* Some more comments and simplification. Added a missing license block.
Co-authored-by: Masaki Hara <hara@wantedly.com>
- Add more to the ObjC dir readme.
- Merge the ExtensionField and ExtensionDescriptor to reduce overhead.
- Fix an initialization race.
- Clean up the Xcode schemes.
- Remove the class/enum filter.
- Remove some forced inline that were bloating things without proof of performance wins.
- Rename some internal types to avoid conflicts with the well know types protos.
- Drop the use of ApplyFunctions to the compiler/optimizer can do what it wants.
- Better document some possible future improvements.
- Add missing support for parsing repeated primitive fields in packed or unpacked forms.
- Improve -hash.
- Add *Count for repeated and map<> fields to avoid auto create when checking for them being set.
- Style fixups in the code.
- map<> serialization fixes and more tests.
- Autocreation of map<> fields (to match repeated fields).
- @@protoc_insertion_point(global_scope|imports).
- Fixup proto2 syntax extension support.
- Move all startup code to +initialize so it happen on class usage and not app startup.
- Have generated headers use forward declarations and move imports into generated code, reduces what is need at compile time to speed up compiled and avoid pointless rippling of rebuilds.
You can use the processor option store_unknown_fields to switch
this support on:
aprotoc --javanano_out=store_unknown_fields=true:/tmp/out
A separate option for extensions isn't required. Support
for unknown fields must be turned on to allow storing and
retrieving extensions, because they are just stored as
unknown fields. If unknown fields are switched on, extension
related code will be generated when a proto message includes
an extension range, or an extension is encountered.
By default, store_unknown_fields is false. No additional
code is generated, and the generator will error out if protos
contain extension ranges or extensions.
Change-Id: I1e034c9e8f3305612953f72438189a7da6ed2167
Like micro protobufs except:
- No setter/getter/hazzer functions.
- Has state is not available. Outputs all fields != their default.
- CodedInputStream can only take byte[] (not InputStream).
- Repeated fields are in arrays, not ArrayList or Vector.
- Unset messages/groups are null, not "defaultInstance()".
- Required fields are always serialized.
To use:
- Link libprotobuf-java-2.3.0-nano runtime.
- Use LOCAL_PROTOC_OPTIMIZE_TYPE := nano
Change-Id: I7429015b3c5f7f38b7be01eb2d4927f7a9999c80
Like micro protobufs except:
- No setter/getter/hazzer functions.
- Has state is not available. Outputs all fields != their default.
- CodedInputStream can only take byte[] (not InputStream).
- Repeated fields are in arrays, not ArrayList or Vector.
- Unset messages/groups are null, not "defaultInstance()".
- Required fields are always serialized.
To use:
- Link libprotobuf-java-2.3.0-nano runtime.
- Use LOCAL_PROTOC_OPTIMIZE_TYPE := nano
Change-Id: I7429015b3c5f7f38b7be01eb2d4927f7a9999c80
Joshua Haberman
Jason Lunn
Joshua Haberman
Adam Cozzette
listonjesse
Brian Wignall
Hao Nguyen
Feng Xiao
Jisi Liu
Thomas Van Lenten
Brian Duff
Ulas Kirazci