- A new PHP-specific upb amalgamation. It contains everything related to upb_msg, but leaves out all of the old handlers-related interfaces and encoders/decoders.
# Schema/Defs Changes
- Changed `upb_fielddef_msgsubdef()` and `upb_fielddef_enumsubdef()` to return `NULL` instead of assert-failing if the field is not a message or enum.
- Added `upb_msgdef_iswrapper()`, to test whether this is a wrapper well-known type.
# Decoder
- Decoder bugfix: when we parse a submessage inside a oneof, we need to clear out any previous data, so we don't misinterpret it as a pointer to an existing submessage.
# JSON Decoder
- Allowed well-known types at the top level to have their special processing.
- Fixed a bug that could occur when parsing nested empty lists/objects, eg `[[]]`.
- Made the "ignore unknown" option also be permissive about unknown enumerators by setting them to 0.
# JSON Encoder
- Allowed well-known types at the top level to have their special processing.
- Removed all spaces after `:` and `,` characters, to match the old encoder and pass goldenfile tests.
# Message / Reflection
- Changed `upb_msg_hasoneof()` -> `upb_msg_whichoneof()`. The new function returns the `upb_fielddef*` of whichever oneof is set.
- Implemented `upb_msg_clearfield()` and added/implemented `upb_msg_clear()`.
- Added `upb_msg_discardunknown()`. Part of me thinks this should go in a util library instead of core reflection since it is a recursive algorithm.
# Compiler
- Always emit descriptors as an array instead of as a string, to avoid exceeding maximum string lengths. If this becomes a speed issue later we can go back to two separate paths.
* resolvename is declared to return a bool value, but instead can return NULL. MSVC 2019 does not like that an throws a compile error. Fixed by returning false instead of NULL.
* When compiling with MSVC 2019, the UPB_ASSUME macro expands out to:
do {} if (false && (ok))
That isn't valid C code. Fixed by adding an elif for MSVC that uses __assume(0), which is similar to gcc's __builtin_unreachable according to http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0627r0.pdf.
* Created amalgamation with upb_msg but no handlers.
* Bugfix for upb_array_resize().
* Renamed "lite" amalgamation to "core", to avoid confusion.
Traditionally "lite" has meant "without reflection", but here we
mean it as "without handlers-based code."
* Build fixes from CI tests.
* Removed some more C++-style comments.
* Fix for out-of-order statements.
* Simplified epoch day algorithm.
* A nicer version that divides by 2048, thanks to Gerben. for the suggestion.
* A little less cleverness, for the sake of readability.
* A bit more tweaking for readability.
* Updated CMake copy.
* Add tests for proto2
* Fix ignore unknown enum value
* Fix CheckConsistency
* Add note to mention clear field for unknown enum value
* Update Comment
* Fix generated test
* Fix generated test
* WIP.
* WIP.
* Tests are passing.
* Recover some perf: LIKELY doesn't propagate through functions. :(
* Added some more benchmarks.
* Simplify & optimize upb_arena_realloc().
* Only add owned blocks to the freelist.
* More optimization/simplification.
* Re-fixed the bug.
* Revert unintentional changes to parser.rl.
* Revert Lua changes for now.
* Revert the arena fuse changes for now.
* Added last_size to the arena representation.
* Re-applied Lua changes.
* Implemented upb_arena_fuse().
* Fix the compile by re-ordering statements.
* Improve comments.
* WIP.
* WIP.
* Tests are passing.
* Recover some perf: LIKELY doesn't propagate through functions. :(
* Added some more benchmarks.
* Simplify & optimize upb_arena_realloc().
* Only add owned blocks to the freelist.
* More optimization/simplification.
* Re-fixed the bug.
* Revert unintentional changes to parser.rl.
* Revert Lua changes for now.
* Revert the arena fuse changes for now.
* Added last_size to the arena representation.
* Fixed compile errors.
* Fixed compile error and changed benchmarks to do one allocation.
* Added support for proto3 optional to defs.
* Added proto3 optional support. Untested at the moment.
* Changes to support proto3 optional.
* Fixed real oneof count for messages with no fields.
* Fixed compile error and test.
* Added comment about why I'm commenting out the assert.
This makes both the C (.h) and C++ (.hpp) files read nicer
and keeps the core of upb C-only.
Existing users of the C++ wrappers will have to add manual
#includes of the .hpp files.
* WIP.
* Passes most tests.
* A few fixes.
* A few optimizations.
* Some more optimiation.
* Update Protobuf to v3.11.4 and Abseil to LTS 2020-02-25
* Use longjmp instead of explicit error checks at every level.
* Used macros for better documentation of ops.
* Fixed bug with map parsing. All tests are passing except a few conformance tests.
* Fixed remaining bugs, all conformance tests pass.
Also ported all of upb to a single UPB_PTR_AT() macro instead of
having multiple .c files define their own.
* Formatted with clang-format.
* Fixes to compile on Linux.
* A few more compile fixes.
* Script to benchmark changes.
* Fixed parenthesis bug in op calculation.
* Updated generated descriptor files.
* WIP.
* Removed trailing enum to fix the Linux build.
* Respect packed=false to fix conformance failures in new protobuf version.
* Small simplification.
* Fixes to decoder.
* Removed stray comment.
Co-authored-by: Yannic Bonenberger <contact@yannic-bonenberger.com>
New code is smaller (in both source size and compiled size) and faster.
# Speed
The decoder speeds up on all machines I tested, though the amount of speedup varies. I was only able to test Intel CPUs.
### Linux Desktop
```
CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
OS: Linux
name old time/op new time/op delta
CreateArena 4.72ns ± 0% 4.93ns ± 0% +4.47% (p=0.000 n=11+11)
ParseDescriptor 12.4µs ± 1% 9.1µs ± 1% -26.65% (p=0.000 n=11+11)
```
### Mac Laptop
```
CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
OS: macOS
name old time/op new time/op delta
CreateArena 5.33ns ± 3% 5.58ns ± 2% +4.69% (p=0.000 n=12+12)
ParseDescriptor 15.0µs ± 2% 11.9µs ± 2% -20.20% (p=0.000 n=12+12)
```
### Linux Workstation
```
CPU: Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
OS: Linux
name old time/op new time/op delta
CreateArena 5.29ns ± 0% 5.52ns ± 0% +4.37% (p=0.000 n=10+12)
ParseDescriptor 18.6µs ± 0% 16.4µs ± 0% -11.54% (p=0.000 n=12+12)
```
# Size
A few source files grow marginally because of some arena functionality moved inline. But `upb/decode.c` shrinks by 30% on Linux:
```
VM SIZE
--------------
+2.1% +283 upb/json_decode.c
+24% +205 upb/msg.c
+8.4% +115 upb/upb.c
+0.9% +28 upb/reflection.c
[ = ] 0 upb/def.c
[ = ] 0 upb/encode.c
[ = ] 0 upb/json_encode.c
[ = ] 0 upb/table.c
-30.3% -1.51Ki upb/decode.c
-0.7% -738 TOTAL
```
* [textformat]: added missing newline when a message opens.
* Added tostring() support to Lua that prints to text format.
Also fixed a gnarly bug that this exposed.
Map parsing/serializing relies on map entries always
having a predictable order. The code that generates
layout was not respecting this in the case of string
keys and primitive values.
* WIP, first version of encoder.
* More progress on text encoder.
* A lot of progress on the text printer.
* Added textencode header file.
* Text encoder now passes conformance tests.
These aren't very stringent though, and more testing is needed.
* Print text into static buffer. Passes all conformance tests.
* Fixed kokoro errors.
* Fix for indent depth when printing map fields.
upb/json/parser.rl: In function 'end_member.isra.150':
bazel-out/k8-opt/bin/upb.c:5536:13: error: 'sel' may be used uninitialized in this function [-Werror=maybe-uninitialized]
upb_func *ret = (upb_func *)h->table[s].func;
Joshua Haberman
Charlie Savage
Paul Yang
Nicolas Noble
Anna R