protobuf

Commit Graph

Author	SHA1	Message	Date
Joshua Haberman	c7787cbaa1	Fixed a bunch of Clang warnings. Unfortunately a few of the Clang warnings did not have easy fixes: ../../../../ext/google/protobuf_c/ruby-upb.c: In function ‘fastdecode_err’: ../../../../ext/google/protobuf_c/ruby-upb.c:353:13: warning: function might be candidate for attribute ‘noreturn’ [-Wsuggest-attribute=noreturn] 353 \| const char fastdecode_err(upb_decstate d) { \| ^~~~~~~~~~~~~~ ../../../../ext/google/protobuf_c/ruby-upb.c: In function ‘_upb_decode’: ../../../../ext/google/protobuf_c/ruby-upb.c:867:30: warning: argument ‘buf’ might be clobbered by ‘longjmp’ or ‘vfork’ [-Wclobbered] 867 \| bool _upb_decode(const char buf, size_t size, void msg, I even tried to suppress the first error, but it still shows up.	4 years ago
Joshua Haberman	9175989431	Bugfix for arena cleanup list when passing to upb_decode().	4 years ago
Joshua Haberman	8d670d8aea	Renamed decode_varint32() to decode_tag().	4 years ago
Joshua Haberman	9abf8e043f	Clamp 32-bit varints to 5 bytes to fix a fuzz failure.	4 years ago
Joshua Haberman	358fa14d0e	Fixed headers and updated benchmark script.	4 years ago
Joshua Haberman	bc200451ce	Use a macro instead of an inline function for setjmp/longjmp.	4 years ago
Joshua Haberman	fbc0639b07	Use _setjmp on mac to avoid saving/restoring the signal mask.	4 years ago
Joshua Haberman	65d166a6ba	Added API for copy vs. alias and added benchmarks to test both. Benchmark output: $ bazel-bin/benchmarks/benchmark '--benchmark_filter=BM_Parse' 2020-11-11 15:39:04 Running bazel-bin/benchmarks/benchmark Run on (72 X 3700 MHz CPU s) CPU Caches: L1 Data 32K (x36) L1 Instruction 32K (x36) L2 Unified 1024K (x36) L3 Unified 25344K (x2) ------------------------------------------------------------------------------------- Benchmark Time CPU Iterations ------------------------------------------------------------------------------------- BM_Parse_Upb_FileDesc<UseArena, Copy> 4134 ns 4134 ns 168714 1.69152GB/s BM_Parse_Upb_FileDesc<UseArena, Alias> 3487 ns 3487 ns 199509 2.00526GB/s BM_Parse_Upb_FileDesc<InitBlock, Copy> 3727 ns 3726 ns 187581 1.87643GB/s BM_Parse_Upb_FileDesc<InitBlock, Alias> 3110 ns 3110 ns 224970 2.24866GB/s BM_Parse_Proto2<FileDesc, NoArena, Copy> 31132 ns 31132 ns 22437 229.995MB/s BM_Parse_Proto2<FileDesc, UseArena, Copy> 21011 ns 21009 ns 33922 340.812MB/s BM_Parse_Proto2<FileDesc, InitBlock, Copy> 17976 ns 17975 ns 38808 398.337MB/s BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 17357 ns 17356 ns 40244 412.539MB/s	4 years ago
Joshua Haberman	982b634bc5	Fixed a few minor bugs found by fuzzing.	4 years ago
Joshua Haberman	0497f8deed	Fixed a critical bug on 32-bit builds, and added much more Kokoro testing. There was a bug in our arena code where we assumed that sizeof(upb_array) would be a multiple of 8. On i386 it was not, and this was causing memory corruption on 32-bit builds.	4 years ago
Joshua Haberman	154f2c25f4	Added UTF-8 validation for proto3 string fields.	4 years ago
Joshua Haberman	e8f9eac68c	Added #defines UPB_ENABLE_FASTTABLE and UPB_TRY_ENABLE_FASTTABLE. These control whether fasttable decoding is on.	4 years ago
Joshua Haberman	55f3569cd2	A few minor fixes and more assertions.	4 years ago
Joshua Haberman	e3e797b680	Added fasttable support for oneofs.	4 years ago
Joshua Haberman	f3a2a79349	More optimization, back up to 2.56GB/s.	4 years ago
Joshua Haberman	199c914295	Simplify push/pop when msg fits in the current buffer.	4 years ago
Joshua Haberman	f4adbe0698	Optimized varint decoding from Gerben. This speeds things up but costs some code size. name old time/op new time/op delta ArenaOneAlloc 21.1ns ± 0% 21.3ns ± 0% +1.33% (p=0.000 n=12+12) ArenaInitialBlockOneAlloc 6.02ns ± 0% 6.02ns ± 0% ~ (p=0.579 n=10+10) LoadDescriptor_Upb 111µs ± 1% 110µs ± 1% -0.91% (p=0.003 n=11+12) LoadDescriptor_Proto2 258µs ± 1% 258µs ± 1% ~ (p=0.674 n=10+12) Parse_Upb_FileDesc_WithArena 11.2µs ± 0% 10.4µs ± 0% -6.67% (p=0.000 n=12+12) Parse_Upb_FileDesc_WithInitialBlock 10.6µs ± 0% 10.1µs ± 0% -4.48% (p=0.000 n=12+11) SerializeDescriptor_Proto2 5.36µs ± 5% 5.36µs ± 3% ~ (p=0.880 n=12+11) SerializeDescriptor_Upb 11.9µs ± 0% 12.0µs ± 0% +0.81% (p=0.000 n=12+12) FILE SIZE VM SIZE -------------- -------------- +23% +1.11Ki +24% +1.06Ki upb/decode.c +15% +560 +15% +560 decode_msg +140% +240 +188% +240 decode_longvarint64 [NEW] +174 [NEW] +128 decode_isdonefallback +56% +160 +65% +160 upb_decode -49.7% -1.06Ki [ = ] 0 [Unmapped] +0.0% +48 +0.9% +1.06Ki TOTAL	4 years ago
Joshua Haberman	48689df72e	Eliminated bounds checks inside parsing a field. Each field parser gets 16 bytes of slop. This requires using a patch buffer at end-of-buffer. This addes 80% of what is needed to support a pull parser with a data callback, since the main parser is now tolerant to buffer flips. There is a ~4% performance regression and 12% code size regression in upb/decode.c: name old time/op new time/op delta ArenaOneAlloc 21.0ns ± 0% 21.6ns ± 0% +2.87% (p=0.000 n=12+12) ArenaInitialBlockOneAlloc 6.02ns ± 0% 6.02ns ± 0% +0.09% (p=0.001 n=11+12) LoadDescriptor_Upb 114µs ± 1% 115µs ± 1% +0.96% (p=0.000 n=11+12) LoadDescriptor_Proto2 260µs ± 1% 261µs ± 1% +0.55% (p=0.033 n=12+12) Parse_Upb_FileDesc_WithArena 10.8µs ± 0% 11.2µs ± 0% +3.43% (p=0.000 n=11+11) Parse_Upb_FileDesc_WithInitialBlock 10.5µs ± 0% 10.9µs ± 0% +3.68% (p=0.000 n=12+12) SerializeDescriptor_Proto2 5.25µs ± 3% 5.42µs ± 5% +3.39% (p=0.007 n=12+12) SerializeDescriptor_Upb 12.0µs ± 0% 12.5µs ± 0% +4.14% (p=0.000 n=12+11) FILE SIZE VM SIZE -------------- -------------- +12% +606 +12% +560 upb/decode.c +7.9% +288 +7.9% +288 decode_msg [NEW] +174 [NEW] +128 decode_isdonefallback +56% +160 +65% +160 upb_decode -9.3% -16 -12.5% -16 decode_longvarint64 -25.5% -558 [ = ] 0 [Unmapped] +0.0% +48 +0.4% +560 TOTAL	4 years ago
gerben-s	9e68ec033f	Add repeated varints and fixed parsers	4 years ago
Joshua Haberman	bca7edac8c	Cleaned up table compression a bit.	4 years ago
Joshua Haberman	8ed6b2fe85	Stored mask in the table pointer.	4 years ago
Joshua Haberman	f2ddc15d76	Bugfix: initialize fastlimit and fastend.	4 years ago
Joshua Haberman	2339fc779c	Updated obsolete comment.	4 years ago
Joshua Haberman	b393849bbd	Updated obsolete comment.	4 years ago
Joshua Haberman	71749b7caf	Implemented inline array allocation, and moved type->lg2 map to reflection.	4 years ago
Joshua Haberman	9557b97acc	Implemented inline array allocation, and moved type->lg2 map to reflection.	4 years ago
Joshua Haberman	b58d2a0ee6	Shrink overhead of message representation.	4 years ago
Joshua Haberman	0bf063a2ca	Shrink overhead of message representation.	4 years ago
Joshua Haberman	d87ceeacab	Shave off one more store.	4 years ago
Joshua Haberman	ddc52ab9d6	Shave off one more store.	4 years ago
Joshua Haberman	c25d895adf	Shrunk the arena state that needs to be synced.	4 years ago
Joshua Haberman	7f67f68c1c	Shrunk the arena state that needs to be synced.	4 years ago
Joshua Haberman	cbcd635917	Fixed memory leak.	4 years ago
Joshua Haberman	bcbcdadbd2	Fixed memory leak.	4 years ago
Joshua Haberman	746f64692c	Moved arena inline for decoder.	4 years ago
Joshua Haberman	7363b91ac3	Moved arena inline for decoder.	4 years ago
Joshua Haberman	9e5c5ce089	Optimized memset() with cutoff and fixed group & unknown message bugs.	4 years ago
Joshua Haberman	405e7934b1	Handle 2-byte submessage lengths.	4 years ago
Joshua Haberman	f173642db4	Handle non-repeated submessages.	4 years ago
Joshua Haberman	7ec2c52346	Donate/steal from arena to accelerate decoding.	4 years ago
Joshua Haberman	fac992db83	Cleanup for showing.	4 years ago
Joshua Haberman	3937874a85	We have a properly structured algorithm, but perf regresses by 20%.	4 years ago
Joshua Haberman	438ecaeb5a	Give all field parsers a generic table entry.	4 years ago
Joshua Haberman	26abaa2345	WIP.	4 years ago
Joshua Haberman	763a3f6293	WIP.	4 years ago
Joshua Haberman	a202ce9629	Add UPB_FORCEINLINE for varint32 decoding. This speeds up the decoder by >20% and also reduces code size slightly! name old time/op new time/op delta ArenaOneAlloc 20.4ns ± 0% 20.2ns ± 0% -1.10% (p=0.000 n=12+11) ArenaInitialBlockOneAlloc 5.25ns ± 0% 5.25ns ± 0% ~ (p=0.786 n=11+12) ParseDescriptorNoHeap 17.1µs ± 0% 13.1µs ± 0% -23.29% (p=0.000 n=11+12) ParseDescriptor 17.4µs ± 1% 13.5µs ± 1% -22.51% (p=0.000 n=12+12) SerializeDescriptor 10.7µs ± 0% 10.9µs ± 0% +1.95% (p=0.000 n=12+12) FILE SIZE VM SIZE -------------- -------------- +2.7% +16 +2.7% +16 [LOAD #2 [RX]] +0.5% +16 [ = ] 0 [Unmapped] -1.4% -72 -0.7% -32 upb/decode.c +3.1% +98 +3.1% +98 decode_msg [DEL] -170 [DEL] -130 decode_varint32 -0.0% -40 -0.0% -16 TOTAL	4 years ago
Joshua Haberman	efefbffc80	Fixed binary encoding and decoding for big-endian machines.	4 years ago
Joshua Haberman	8e26a33bcb	Added a test for UTF-8 parse checking and added missing error reporting.	4 years ago
Joshua Haberman	a77ea639d5	Verify UTF-8 when parsing proto3 string fields.	4 years ago
Joshua Haberman	363e39c171	Fix for extra compiler warnings. (#290 )	5 years ago

1 2 3

121 Commits (53f5d9f69b4d6ba6ca3149e1913bd9834f4d6c78)