Eric Salo
e4f7b39c06
upb: create upb/wire/internal/
...
This is prep work for splitting out the :wire targets into their own BUILD
PiperOrigin-RevId: 552294501
1 year ago
Joshua Haberman
a48af3f824
Moved aliasing logic for string field parsing into EpsCopyInputStream.
...
Moving the logic down to EpsCopyInputStream makes it easier to test and reuse this functionality.
We also implement aliasing for the final bytes of the patch buffer, which has never been supported before. We used to always force a copy for any data parsed out of the patch buffer at the end of the stream.
Much of this logic is ported directly from the C++ EpsCopyInputStream class.
PiperOrigin-RevId: 498091644
2 years ago
Eric Salo
02cf7aaa1d
upb_Array_Resize() now correctly clears new values
...
PiperOrigin-RevId: 496255949
2 years ago
Joshua Haberman
68d1d91475
Separated out buffering code into upb_EpsCopyInputStream.
...
This mirrors the structure of C++ protobuf, which has an EpsCopyInputStream class.
This will lay the foundation for making EpsCopyInputStream capable of true streaming, by reading its input from a ZeroCopyInputStream. It also lets us test EpsCopyInputStream separately from the decoder: see the new unit test that fuzzes upb_EpsCopyInputStream.
After this CL is submitted, the two decoders (the normal decoder and the fast decoder) should no longer be accessing the members of upb_EpsCopyInputStream.
PiperOrigin-RevId: 494400285
2 years ago
Eric Salo
f6307877d3
move portability stuff into upb/port/
...
Also delete redundant system #includes that are already pulled in by port/def.inc
PiperOrigin-RevId: 486398989
2 years ago
Eric Salo
fd14316f38
create collections/ subdir for all array and map code
...
PiperOrigin-RevId: 486013554
2 years ago
Eric Salo
e137175092
create wire/ subdir for all wire-format code
...
PiperOrigin-RevId: 485898080
2 years ago
Joshua Haberman
ba7603b7c1
Updated decoder internal symbols to new naming scheme.
...
This is a naming change only, with no functional change. We did replace one inline function that had only one caller, but the net effect should be a no-op.
I did remove several `return` statements for calls to certain `noreturn` functions. These had no effect but were intended to improve readability. However the unused "return" caused ClangTidy to throw warnings, so these have been removed.
PiperOrigin-RevId: 472619191
2 years ago
Protobuf Team Bot
8c44f04697
create and lock down upb/internal/array.h
...
Internal array functions are now implemented in upb/internal/array.c and declared in
upb/internal/array.h, which only has local visibility.
PiperOrigin-RevId: 458260144
2 years ago
Protobuf Team Bot
1695cb2788
rename the upb_Array 'len' field as 'size'
...
Now that 'size' has been renamed as 'capacity' we are free to rename 'len' as
'size', so upb_Array_Size() is actually returning the 'size' field.
PiperOrigin-RevId: 456865972
3 years ago
Protobuf Team Bot
83f4988561
rename the upb_Array 'size' field as 'capacity'
...
The current field/function names for upb_Array are quite confusing.
We will fix them in two steps, this being the first step.
PiperOrigin-RevId: 456687224
3 years ago
Protobuf Team Bot
e4635f223e
match file names to type names
...
Lots of changes but it's all just moving things around.
Backward-compatible stub #include's have been provided for now.
upb_Arena/upb_Status have been split out from upb/upb.?
upb_Array/upb_Map/upb_MessageValue have been split out from upb/collections.?
upb_ExtensionRegistry has been split out from upb/msg.?
upb/decode_internal.h is now upb/internal/decode.h
upb/mini_table_accessors_internal.h is now upb/internal/mini_table_accessors.h
upb/table_internal.h is now upb/internal/table.h
upb/upb_internal.h is now upb/internal/upb.h
PiperOrigin-RevId: 456297617
3 years ago
Joshua Haberman
3692e6659f
Reverted decode_fast.c changes. They are not necessary for this PR.
3 years ago
Joshua Haberman
5b711f286b
WIP.
3 years ago
Joshua Haberman
499c2cc8b1
upb_extreg, upb_msg
3 years ago
Joshua Haberman
1c955f37ce
Mass API rename and clang-reformat ( #485 )
...
* Wave 1: upb_fielddef.
* upb_fielddef itself.
* upb_oneofdef.
* upb_msgdef.
* ExtensionRange.
* upb_enumdef
* upb_enumvaldef
* upb_filedef
* upb_methoddef
* upb_servicedef
* upb_symtab
* upb_defpool_init
* upb_wellknown and upb_syntax_t
* Some constants.
* upb_status
* upb_strview
* upb_arena
* upb.h constants
* reflection
* encode
* JSON decode.
* json encode.
* msg_internal.
* Formatted with clang-format.
* Some naming fixups and comment reformatting.
* More refinements.
* A few more stragglers.
* Fixed PyObject_HEAD with semicolon. Removed TODO entries.
3 years ago
Joshua Haberman
c5a69b79aa
Updated protobuf version.
...
This also required fixing a bug in the fast parser.
3 years ago
Joshua Haberman
f7980b7ed1
Restructured for simplicity and fixed fasttable parser.
3 years ago
Joshua Haberman
3d437bbcab
Some pre-PR fixes.
3 years ago
Joshua Haberman
b23ec9a59c
Fixed fasttable parser.
3 years ago
Joshua Haberman
4132034634
Addressed PR comment.
4 years ago
Joshua Haberman
ed708fcd5d
Addressed PR comments.
4 years ago
Joshua Haberman
876abae2db
Removed some debug printing and simplified checktag slightly.
4 years ago
Joshua Haberman
286441afa7
Fixed a size regression due to inlining UTF-8 verification.
...
Overall size/speed impact on fasttable decoder is now:
name old time/op new time/op delta
ArenaOneAlloc 21.5ns ± 0% 21.5ns ± 0% ~ (p=0.060 n=12+12)
ArenaInitialBlockOneAlloc 6.33ns ± 0% 6.33ns ± 0% ~ (p=0.413 n=11+12)
LoadDescriptor_Upb 43.4µs ± 1% 45.5µs ± 1% +4.79% (p=0.000 n=12+12)
LoadAdsDescriptor_Upb 2.50ms ± 0% 2.51ms ± 2% ~ (p=0.512 n=10+11)
LoadDescriptor_Proto2 240µs ± 0% 240µs ± 0% -0.25% (p=0.000 n=12+12)
LoadAdsDescriptor_Proto2 12.9ms ± 0% 12.9ms ± 0% +0.20% (p=0.014 n=10+12)
Parse_Upb_FileDesc<UseArena,Copy> 4.99µs ± 0% 5.04µs ± 0% +0.98% (p=0.000 n=11+10)
Parse_Upb_FileDesc<UseArena,Alias> 4.02µs ± 0% 4.18µs ± 0% +4.16% (p=0.000 n=10+12)
Parse_Upb_FileDesc<InitBlock,Copy> 4.49µs ± 0% 4.54µs ± 0% +1.16% (p=0.000 n=11+10)
Parse_Upb_FileDesc<InitBlock,Alias> 3.60µs ± 0% 3.80µs ± 0% +5.73% (p=0.000 n=12+11)
Parse_Proto2<FileDesc,NoArena,Copy> 29.3µs ± 0% 29.3µs ± 0% ~ (p=0.069 n=11+12)
Parse_Proto2<FileDesc,UseArena,Copy> 20.2µs ± 3% 20.3µs ± 2% ~ (p=0.880 n=12+11)
Parse_Proto2<FileDesc,InitBlock,Copy> 16.5µs ± 0% 16.5µs ± 0% ~ (p=1.000 n=12+12)
Parse_Proto2<FileDescSV,InitBlock,Alias> 16.4µs ± 0% 16.4µs ± 1% ~ (p=0.590 n=12+12)
SerializeDescriptor_Proto2 5.31µs ± 1% 6.65µs ±29% +25.07% (p=0.000 n=12+12)
SerializeDescriptor_Upb 12.4µs ± 0% 12.5µs ± 0% +1.23% (p=0.000 n=12+12)
FILE SIZE VM SIZE
-------------- --------------
+16% +128 [ = ] 0 [Unmapped]
-1.2% -4 -1.2% -4 [section .text]
[NEW] +2 [NEW] +2 fastdecode_isdonefallback
[DEL] -6 [DEL] -6 fastdecode_longstring_noutf8
-0.2% -124 -0.2% -124 upb/decode_fast.c
+5.8% +64 +6.0% +64 upb_pom_1bt_max64b
+2.7% +64 +2.7% +64 upb_ppv8_2bt
+2.7% +32 +2.8% +32 upb_psm_1bt_max256b
+2.8% +32 +3.0% +32 upb_psm_1bt_max64b
+2.8% +32 +3.0% +32 upb_psm_2bt_max64b
+4.0% +24 +4.2% +24 upb_psv8_1bt
+2.0% +16 +2.1% +16 upb_prf4_2bt
+1.3% +16 +1.4% +16 upb_prz8_2bt
-0.3% -4 -0.3% -4 [3 Others]
-1.6% -8 -1.7% -8 upb_cob_1bt
-1.6% -8 -1.7% -8 upb_csb_1bt
-2.5% -16 -2.6% -16 upb_pov4_1bt
-1.3% -16 -1.3% -16 upb_prv8_2bt
-2.5% -16 -2.7% -16 upb_psv4_1bt
-2.5% -16 -2.6% -16 upb_psv4_2bt
-3.0% -32 -3.1% -32 upb_prs_2bt
-2.6% -32 -2.6% -32 upb_prv4_2bt
-4.9% -48 -5.1% -48 upb_prb_2bt
-3.9% -48 -4.0% -48 upb_prv4_1bt
-7.2% -72 -7.5% -72 upb_prb_1bt
-7.8% -88 -8.0% -88 upb_prs_1bt
[ = ] 0 -0.1% -128 TOTAL
There is a bit of speed regression, but it appears there were bigger
CPU regressions prior to this. We probably need some separate
optimization attention again to get back to the performance numbers
we had when fasttable was first submitted.
4 years ago
Joshua Haberman
e84793dd73
Cleaned up debugging artifacts.
4 years ago
Joshua Haberman
a4b35aa388
Everything passes except 4 conformance tests.
4 years ago
Joshua Haberman
3881393907
Renamed .int.h to _internal.h, for greater clarity.
4 years ago
Joshua Haberman
823eb09694
Update all 2011 dates to 2021.
4 years ago
Joshua Haberman
e59d2c8fa7
Added license headers to all files.
4 years ago
Joshua Haberman
e4343f0fa3
Update comment for ARM64.
4 years ago
Joshua Haberman
65d166a6ba
Added API for copy vs. alias and added benchmarks to test both.
...
Benchmark output:
$ bazel-bin/benchmarks/benchmark '--benchmark_filter=BM_Parse'
2020-11-11 15:39:04
Running bazel-bin/benchmarks/benchmark
Run on (72 X 3700 MHz CPU s)
CPU Caches:
L1 Data 32K (x36)
L1 Instruction 32K (x36)
L2 Unified 1024K (x36)
L3 Unified 25344K (x2)
-------------------------------------------------------------------------------------
Benchmark Time CPU Iterations
-------------------------------------------------------------------------------------
BM_Parse_Upb_FileDesc<UseArena, Copy> 4134 ns 4134 ns 168714 1.69152GB/s
BM_Parse_Upb_FileDesc<UseArena, Alias> 3487 ns 3487 ns 199509 2.00526GB/s
BM_Parse_Upb_FileDesc<InitBlock, Copy> 3727 ns 3726 ns 187581 1.87643GB/s
BM_Parse_Upb_FileDesc<InitBlock, Alias> 3110 ns 3110 ns 224970 2.24866GB/s
BM_Parse_Proto2<FileDesc, NoArena, Copy> 31132 ns 31132 ns 22437 229.995MB/s
BM_Parse_Proto2<FileDesc, UseArena, Copy> 21011 ns 21009 ns 33922 340.812MB/s
BM_Parse_Proto2<FileDesc, InitBlock, Copy> 17976 ns 17975 ns 38808 398.337MB/s
BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 17357 ns 17356 ns 40244 412.539MB/s
4 years ago
Joshua Haberman
982b634bc5
Fixed a few minor bugs found by fuzzing.
4 years ago
Joshua Haberman
1eb7bd39e7
Some formatting fixes.
4 years ago
Joshua Haberman
154f2c25f4
Added UTF-8 validation for proto3 string fields.
4 years ago
Joshua Haberman
e8f9eac68c
Added #defines UPB_ENABLE_FASTTABLE and UPB_TRY_ENABLE_FASTTABLE.
...
These control whether fasttable decoding is on.
4 years ago
Joshua Haberman
e86541ac1d
Fixed the build after the merge.
4 years ago
Joshua Haberman
efd576b698
Added -std=gnu99 for fastdecode and ran Buildifier.
4 years ago
Joshua Haberman
b928696942
A few more fixes, and test fastdecode under Kokoro.
4 years ago
Joshua Haberman
55f3569cd2
A few minor fixes and more assertions.
4 years ago
Joshua Haberman
46eb82467a
Added comment to decode_fast.h.
4 years ago
Joshua Haberman
bd9f8f580d
Fixed a few bugs with the fast decoder.
...
1. For long tags we were putting table entries in the wrong slot.
2. For repeated strings, when the buffer flipped to no longer alias we
were failing to notice and kept aliasing anyway.
4 years ago
Joshua Haberman
021db6fcd5
Allow larger tags into the table if they are unique mod 31.
...
Also fixed a bug with fixed packed in decode_fast.c.
4 years ago
Joshua Haberman
86d9908c55
Fastdecode support for packed fields.
...
This is not very optimized yet. There is a lot of room to
optimize it further.
4 years ago
Joshua Haberman
e3e797b680
Added fasttable support for oneofs.
4 years ago
Joshua Haberman
7ffa9c181a
Fixed some small bugs and performance problems in string copying.
...
Before this CL, with alias=false:
------------------------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------------------------
BM_Parse_Upb_FileDesc_WithInitialBlock 3715 ns 3715 ns 188916 1.88206GB/s
Performance counter stats for 'bazel-bin/benchmarks/benchmark --benchmark_filter=BM_Parse_Upb_FileDesc_WithInitialBlock':
1,122.92 msec task-clock # 0.979 CPUs utilized
3 context-switches # 0.003 K/sec
0 cpu-migrations # 0.000 K/sec
196 page-faults # 0.175 K/sec
4,144,746,717 cycles # 3.691 GHz
15,351,966,804 instructions # 3.70 insn per cycle
2,590,281,905 branches # 2306.728 M/sec
2,996,157 branch-misses # 0.12% of all branches
1.146615328 seconds time elapsed
1.115578000 seconds user
0.008025000 seconds sys
After this CL:
------------------------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------------------------
BM_Parse_Upb_FileDesc_WithInitialBlock 3554 ns 3554 ns 197527 1.9674GB/s
Performance counter stats for 'bazel-bin/benchmarks/benchmark --benchmark_filter=BM_Parse_Upb_FileDesc_WithInitialBlock':
1,105.34 msec task-clock # 0.982 CPUs utilized
3 context-switches # 0.003 K/sec
0 cpu-migrations # 0.000 K/sec
197 page-faults # 0.178 K/sec
4,077,736,892 cycles # 3.689 GHz
15,442,709,352 instructions # 3.79 insn per cycle
2,435,131,301 branches # 2203.068 M/sec
2,643,775 branch-misses # 0.11% of all branches
1.125393845 seconds time elapsed
1.097770000 seconds user
0.008012000 seconds sys
4 years ago
Joshua Haberman
e2c709e047
Repeated string and primitive support.
...
Much of the code was adapted from Gerben's code in:
6333031195
4 years ago
Joshua Haberman
d81ba58215
Optimized short string copying.
...
This sped up the alias=false case:
Before:
------------------------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------------------------
BM_Parse_Upb_FileDesc_WithInitialBlock 4562 ns 4562 ns 153251 1.53276GB/s
Performance counter stats for 'bazel-bin/benchmarks/benchmark --benchmark_filter=BM_Parse_Upb_FileDesc_WithInitialBlock':
1,216.65 msec task-clock # 0.936 CPUs utilized
6 context-switches # 0.005 K/sec
0 cpu-migrations # 0.000 K/sec
200 page-faults # 0.164 K/sec
4,490,925,650 cycles # 3.691 GHz
16,516,403,731 instructions # 3.68 insn per cycle
2,828,536,650 branches # 2324.861 M/sec
5,425,830 branch-misses # 0.19% of all branches
1.300178903 seconds time elapsed
1.211475000 seconds user
0.072207000 seconds sys
After:
------------------------------------------------------------------------------
Benchmark Time CPU Iterations
------------------------------------------------------------------------------
BM_Parse_Upb_FileDesc_WithInitialBlock 3587 ns 3587 ns 195749 1.94935GB/s
Performance counter stats for 'bazel-bin/benchmarks/benchmark --benchmark_filter=BM_Parse_Upb_FileDesc_WithInitialBlock':
1,109.69 msec task-clock # 0.930 CPUs utilized
5 context-switches # 0.005 K/sec
0 cpu-migrations # 0.000 K/sec
198 page-faults # 0.178 K/sec
4,094,010,257 cycles # 3.689 GHz
15,672,677,812 instructions # 3.83 insn per cycle
2,589,291,160 branches # 2333.346 M/sec
3,306,386 branch-misses # 0.13% of all branches
1.193221789 seconds time elapsed
1.102538000 seconds user
0.072166000 seconds sys
4 years ago
Joshua Haberman
f3a2a79349
More optimization, back up to 2.56GB/s.
4 years ago
Joshua Haberman
199c914295
Simplify push/pop when msg fits in the current buffer.
4 years ago
Joshua Haberman
d5f5db2729
Put string-copying field parser into a separate function.
...
This helps to regain a bit of lost perf. Now at 2.3GB/s.
4 years ago