protobuf

Commit Graph

Author	SHA1	Message	Date
Eric Salo	85534bbb1e	remove unused #includes from reflection.h The functions declared in reflection.h use neither arrays nor maps, so (a) stop including the array and map definitions, and (b) update the handful of other source files which were relying on that transitivity. PiperOrigin-RevId: 472627278	2 years ago
Joshua Haberman	ba7603b7c1	Updated decoder internal symbols to new naming scheme. This is a naming change only, with no functional change. We did replace one inline function that had only one caller, but the net effect should be a no-op. I did remove several `return` statements for calls to certain `noreturn` functions. These had no effect but were intended to improve readability. However the unused "return" caused ClangTidy to throw warnings, so these have been removed. PiperOrigin-RevId: 472619191	2 years ago
Joshua Haberman	5a7644b2d0	Fixed fuzz bug in upb. Extending a MessageSet with a non-message extension was causing crashes that would manifest in various ways. PiperOrigin-RevId: 472496259	2 years ago
Protobuf Team Bot	8827a09490	Add a comment to remind ASAN users to compile interpreters with ASAN as well. PiperOrigin-RevId: 471898444	2 years ago
Eric Salo	111249d085	continue preparing def.c for the big split: - opt_default is now kUpbDefOptDefault - upb_AddDefCtx is now upb_DefBuilder - shortdefname() is now _upb_DefBuilder_FullToShort() - pack_def() is now _upb_DefBuilder_Pack() - unpack_def() is now _upb_DefBuilder_Unpack() - UPB_ASSERT() checks on def struct size moved up in the call chain - remove/expand CHK_OOM PiperOrigin-RevId: 471862679	2 years ago
Copybara-Service	ada907989e	Merge pull request #761 from protocolbuffers:perezd-patch-1 PiperOrigin-RevId: 471305820	2 years ago
Derek Perez	fa5e4cfd31	Updating rules_python to 0.12.0	2 years ago
Eric Salo	b987ef6249	use _upb_DefBuilder instead of _upb_AddDexCtx (Because it is a better name and we can't properly bikeshed it without changing it a few times first.) Also removed it as an arg from a function that doesn't actually need it. PiperOrigin-RevId: 470978322	2 years ago
Eric Salo	02ef81247a	rename some symtab functions to follow the upb style guide: - check_ident() is now _upb_AddDefCtx_CheckIdent() - makefullname() is now _upb_AddDefCtx_MakeFullName() - symtab_add() is now _upb_AddDefCtx_Add() - symtab_alloc() is now _upb_AddDefCtx_Alloc() - symtab_errf() is now _upb_AddDefCtx_Errf() - symtab_oomerr() is now _upb_AddDefCtx_OomErr() PiperOrigin-RevId: 470806376	2 years ago
Eric Salo	250321e63b	more misc tweaks to def.c: - renamed symtab_addctx as upb_AddDefCtx - simplified _upb_DefPool_AddFile() slightly - replaced upb_Status_setoom() with kOutOfMemory - deleted all references to UPB_DEFTYPE_LAYOUT and UPB_DEFTYPE_FILE PiperOrigin-RevId: 470592649	2 years ago
Copybara-Service	a694d2369b	Merge pull request #755 from haberman:fix-prs PiperOrigin-RevId: 470588004	2 years ago
Joshua Haberman	682b519aac	Fixed GitHub Actions tests for pull requests that come from other repos. Previously GitHub Actions would fail if they did not have access to repository secrets. The code attempted to have graceful fallback, but the graceful fallback was not working properly. This PR fixes the graceful fallback: - Bazel tests will use read-only caching instead of read-write. - We unfortunately need to skip the Python wheel tests, as they require our private Docker image to run.	2 years ago
Joshua Haberman	15b2402144	Removed unused `_upb_DefPool_registerlayout()` function. This function was introduced in https://github.com/protocolbuffers/upb/pull/426 but it appears it was never used. I am not sure what the purpose was, but in any case it is not needed. With this function removed, we no longer need to tag pointers for the DefPool "files" table. PiperOrigin-RevId: 470567000	2 years ago
Eric Salo	daac8dce8f	add more constructors to def.c PiperOrigin-RevId: 470566093	2 years ago
Eric Salo	11aa037bfd	add some more accessor calls to def.c PiperOrigin-RevId: 470404251	2 years ago
Protobuf Team Bot	52c9b98692	Resolve field name/accessor name conflicts. PiperOrigin-RevId: 470335824	2 years ago
Eric Salo	5b46a55a46	try to be better about using accessors in def.c There are many places within def.c where a function which implements a method for struct A ends up directly accessing fields within struct B even though accessor functions for these fields are defined. So, this is a first pass at trying to clean that up a bit. Yes, we are adding function calls to a lot of code paths by doing this but in the unlikely event that this adds unacceptable overhead we can deal with it then. PiperOrigin-RevId: 470321129	2 years ago
Eric Salo	1135746e42	start replumbing def.c Give the def types their own array allocators and also implement some simple array constructors for the enum and enum value defs. PiperOrigin-RevId: 470042521	2 years ago
Protobuf Team Bot	4e979b8d13	Add support for dashes in proto file names. PiperOrigin-RevId: 469853045	2 years ago
Protobuf Team Bot	ce32d9d68f	Fix code generation for infinity default value on float/double fields. PiperOrigin-RevId: 469843901	2 years ago
Eric Salo	f3316e2d7d	remove upb_String from the public tokenizer api upb_String is a hack which exists because the original C++ tokenizer got to assume the existence of C++ strings, so at least for now the C tokenizer needs a rough equivalent. But this should be a purely internal implementation detail, not part of the visible surface. PiperOrigin-RevId: 469814074	2 years ago
Eric Salo	0013c936ef	add upb_Status to the tokenizer PiperOrigin-RevId: 469721241	2 years ago
Eric Salo	c67021f84a	split out the json string-to-int functions for general use PiperOrigin-RevId: 469509635	2 years ago
Copybara-Service	12166431ed	Merge pull request #738 from Wyverald:patch-1 PiperOrigin-RevId: 469462667	2 years ago
Eric Salo	33114209dc	simplify the tokenizer - remove previous token from the public api - remove upb_Token type PiperOrigin-RevId: 469308543	2 years ago
Copybara-Service	e9f4211f3b	Merge pull request #740 from protocolbuffers:rule_python-update PiperOrigin-RevId: 469283641	2 years ago
Derek Perez	4135cd206b	Updating python to 0.11.0	2 years ago
Protobuf Team Bot	470f06cccb	Different message factories will return same message class for same descriptor in python. PiperOrigin-RevId: 468499201	2 years ago
Xùdōng Yáng	0aa5a453b8	install rsync on Bazel CI before running tests `//cmake:cmake_build` needs this: https://buildkite.com/bazel/upb/builds/1210#0182b0dc-2131-43f2-8b9d-0b9fba807be8	2 years ago
Eric Salo	922a858e5c	clean up tokenizer options and defaults - Disallow multiline strings. - Disallow a letter immediately following a number without intervening whitespace. - Replace distinct bool option flags with a single options int. PiperOrigin-RevId: 467829817	2 years ago
Eric Salo	6861966501	first stab at a Tokenizer api These functions are not yet part of the upb build but this is a good chunk of work so let's snapshot it now. PiperOrigin-RevId: 467733791	2 years ago
Protobuf Team Bot	0c6531378d	Merge GetEnum into GetInt32. Rename SetEnum to SetEnumProto2 to be clear that upb only treats Proto2 enum as enum. Proto3 enums should use SetInt32. PiperOrigin-RevId: 467000685	2 years ago
Mike Kruskal	63a4a7d74b	Bump pinned protobuf version PiperOrigin-RevId: 466825011	2 years ago
Eric Salo	3f4f7ab079	properly format extension names in text_encode() extension message names are now enclosed within square brackets PiperOrigin-RevId: 466499355	2 years ago
Joshua Haberman	fcb666d040	Fixed comparison of maps in Python. The `__eq__` method from our base class was not getting called, because we defined `tp_hash` as `NotImplemented`. Per [the docs on `tp_hash`](https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_hash), `tp_hash` and `tp_richcompare` are inherited together, so defining `tp_hash` was inhibiting the inheritance of `tp_richcompare`. PiperOrigin-RevId: 465184018	2 years ago
Protobuf Team Bot	79b735a7d7	Make upb numpy type checks consistent with pure python and cpp. PiperOrigin-RevId: 464907203	2 years ago
Protobuf Team Bot	e09d6fcb6d	Update mini table API comment PiperOrigin-RevId: 463868386	2 years ago
Protobuf Team Bot	33313a2891	Update python tests due to macos-10.15 deprecation. https://github.com/actions/virtual-environments/issues/5583 PiperOrigin-RevId: 463857895	2 years ago
Protobuf Team Bot	bf88f8ba97	Small typo fix (now -> not) PiperOrigin-RevId: 462158562	2 years ago
Protobuf Team Bot	f034bba2ed	fixed formatting and parsing of negative durations between -1s and 0s PiperOrigin-RevId: 462142321	2 years ago
Joshua Haberman	fcb5ef37f7	Fixed a bug in MiniTable construction for extensions. #fuzzing We were failing to assign the f->presence field, which resulted in a read of uninitialized memory. PiperOrigin-RevId: 462138061	2 years ago
Joshua Haberman	ececc21624	Fixed bug when parsing an unknown value in a proto2 enum extension. #fuzzing Proto2 enum parsing is the only case where we have to look at the wire value (not merely the tag) to decide whether the field is known or unknown. If the value is unknown, we need to put the value in the Unknown Fields, but for an extension we no longer have easy access to the message, because for extensions we replace the `msg` pointer with a pointer to the extension. The bug occurred when we were treating the fake `upb_Message` (which was actually a pointer to an extension) as a real `upb_Message` that can have unknown fields. This CL fixes the problem by preserving the true message pointer in `d->unknown_msg` when we are parsing an extension. This also required fixing a bug in MiniTable building when fasttables are enabled. We need to set the table_mask to `-1` to disable fasttable parsing, not `0`. For unknown reasons, this CL appears to speed up parsing somewhat significantly. Ideally we should be tracking parsing performance better over time, as it is possible this is merely regaining performance that was lost at a different time: ``` benchy --reference=srcfs third_party/upb/benchmarks:benchmark 10 / 10 [=================================================================================================================] 100.00% 2m32s (Generated by http://go/benchy. Settings: --runs 5 --reference "srcfs") name old cpu/op new cpu/op delta BM_ArenaOneAlloc 23.9ns ± 6% 23.7ns ± 4% ~ (p=0.180 n=53+51) BM_ArenaInitialBlockOneAlloc 7.62ns ± 4% 7.70ns ± 5% +0.99% (p=0.024 n=59+60) BM_LoadAdsDescriptor_Upb<NoLayout> 6.60ms ±10% 6.57ms ± 8% ~ (p=0.607 n=47+54) BM_LoadAdsDescriptor_Upb<WithLayout> 6.92ms ± 5% 6.88ms ± 8% ~ (p=0.257 n=54+54) BM_LoadAdsDescriptor_Proto2<NoLayout> 14.2ms ± 8% 14.0ms ± 7% -1.38% (p=0.025 n=58+59) BM_LoadAdsDescriptor_Proto2<WithLayout> 14.3ms ± 8% 14.2ms ± 8% -1.16% (p=0.031 n=58+57) BM_Parse_Upb_FileDesc<UseArena, Copy> 15.9µs ± 4% 14.6µs ± 4% -7.85% (p=0.000 n=57+59) BM_Parse_Upb_FileDesc<UseArena, Alias> 14.5µs ± 4% 13.3µs ± 5% -8.50% (p=0.000 n=57+60) BM_Parse_Upb_FileDesc<InitBlock, Copy> 15.7µs ± 4% 14.4µs ± 5% -7.99% (p=0.000 n=59+60) BM_Parse_Upb_FileDesc<InitBlock, Alias> 14.2µs ± 5% 13.0µs ± 4% -8.56% (p=0.000 n=57+58) BM_Parse_Proto2<FileDesc, NoArena, Copy> 26.3µs ± 4% 26.2µs ± 4% ~ (p=0.195 n=55+53) BM_Parse_Proto2<FileDesc, UseArena, Copy> 13.3µs ± 5% 13.2µs ± 4% ~ (p=0.085 n=59+59) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 12.9µs ± 4% 12.8µs ± 3% -0.66% (p=0.023 n=60+58) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 10.9µs ± 6% 10.9µs ± 4% ~ (p=0.063 n=59+58) BM_SerializeDescriptor_Proto2 7.57µs ± 6% 7.62µs ± 6% ~ (p=0.147 n=57+58) BM_SerializeDescriptor_Upb 12.8µs ± 4% 12.8µs ± 4% ~ (p=0.163 n=59+56) name old time/op new time/op delta BM_ArenaOneAlloc 23.9ns ± 5% 23.7ns ± 4% ~ (p=0.172 n=53+51) BM_ArenaInitialBlockOneAlloc 7.62ns ± 4% 7.70ns ± 5% +1.02% (p=0.017 n=59+60) BM_LoadAdsDescriptor_Upb<NoLayout> 6.60ms ±10% 6.58ms ± 8% ~ (p=0.727 n=47+55) BM_LoadAdsDescriptor_Upb<WithLayout> 6.92ms ± 5% 6.88ms ± 8% ~ (p=0.260 n=54+54) BM_LoadAdsDescriptor_Proto2<NoLayout> 14.2ms ± 7% 14.0ms ± 7% -1.40% (p=0.019 n=58+59) BM_LoadAdsDescriptor_Proto2<WithLayout> 14.3ms ± 8% 14.2ms ± 8% -1.13% (p=0.037 n=58+57) BM_Parse_Upb_FileDesc<UseArena, Copy> 15.9µs ± 4% 14.6µs ± 3% -7.88% (p=0.000 n=57+59) BM_Parse_Upb_FileDesc<UseArena, Alias> 14.5µs ± 4% 13.3µs ± 5% -8.46% (p=0.000 n=57+60) BM_Parse_Upb_FileDesc<InitBlock, Copy> 15.7µs ± 4% 14.4µs ± 5% -7.99% (p=0.000 n=59+60) BM_Parse_Upb_FileDesc<InitBlock, Alias> 14.2µs ± 5% 13.0µs ± 4% -8.56% (p=0.000 n=57+58) BM_Parse_Proto2<FileDesc, NoArena, Copy> 26.3µs ± 4% 26.2µs ± 4% ~ (p=0.224 n=55+53) BM_Parse_Proto2<FileDesc, UseArena, Copy> 13.3µs ± 5% 13.2µs ± 4% ~ (p=0.098 n=59+59) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 12.9µs ± 4% 12.8µs ± 3% -0.68% (p=0.015 n=60+58) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 10.9µs ± 6% 10.9µs ± 4% ~ (p=0.052 n=59+58) BM_SerializeDescriptor_Proto2 7.56µs ± 6% 7.62µs ± 6% ~ (p=0.111 n=58+58) BM_SerializeDescriptor_Upb 12.8µs ± 4% 12.8µs ± 4% ~ (p=0.241 n=56+56) name old allocs/op new allocs/op delta BM_ArenaOneAlloc 1.00 ± 0% 1.00 ± 0% ~ (all samples are equal) BM_ArenaInitialBlockOneAlloc 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_LoadAdsDescriptor_Upb<NoLayout> 5.98k ± 0% 5.98k ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Upb<WithLayout> 5.98k ± 0% 5.98k ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Proto2<NoLayout> 80.9k ± 0% 80.9k ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Proto2<WithLayout> 82.1k ± 0% 82.1k ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<UseArena, Copy> 7.00 ± 0% 7.00 ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<UseArena, Alias> 7.00 ± 0% 7.00 ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<InitBlock, Copy> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_Parse_Upb_FileDesc<InitBlock, Alias> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, NoArena, Copy> 765 ± 0% 765 ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, UseArena, Copy> 9.00 ± 0% 9.00 ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_SerializeDescriptor_Proto2 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_SerializeDescriptor_Upb 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) name old peak-mem(Bytes)/op new peak-mem(Bytes)/op delta BM_ArenaOneAlloc 344 ± 0% 344 ± 0% ~ (all samples are equal) BM_ArenaInitialBlockOneAlloc 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_LoadAdsDescriptor_Upb<NoLayout> 9.60M ± 0% 9.60M ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Upb<WithLayout> 9.68M ± 0% 9.68M ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Proto2<NoLayout> 6.41M ± 0% 6.41M ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Proto2<WithLayout> 6.44M ± 0% 6.44M ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<UseArena, Copy> 36.5k ± 0% 36.5k ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<UseArena, Alias> 36.5k ± 0% 36.5k ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<InitBlock, Copy> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_Parse_Upb_FileDesc<InitBlock, Alias> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, NoArena, Copy> 35.8k ± 0% 35.8k ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, UseArena, Copy> 40.7k ± 0% 40.7k ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_SerializeDescriptor_Proto2 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) BM_SerializeDescriptor_Upb 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal) name old speed new speed delta BM_LoadAdsDescriptor_Upb<NoLayout> 113MB/s ± 9% 113MB/s ± 8% ~ (p=0.712 n=47+55) BM_LoadAdsDescriptor_Upb<WithLayout> 107MB/s ± 8% 108MB/s ± 8% ~ (p=0.200 n=55+54) BM_LoadAdsDescriptor_Proto2<NoLayout> 52.5MB/s ± 8% 53.3MB/s ± 7% +1.51% (p=0.018 n=59+59) BM_LoadAdsDescriptor_Proto2<WithLayout> 51.9MB/s ± 7% 52.4MB/s ± 8% +1.01% (p=0.050 n=58+58) BM_Parse_Upb_FileDesc<UseArena, Copy> 473MB/s ± 4% 514MB/s ± 4% +8.52% (p=0.000 n=57+59) BM_Parse_Upb_FileDesc<UseArena, Alias> 518MB/s ± 4% 566MB/s ± 5% +9.30% (p=0.000 n=57+60) BM_Parse_Upb_FileDesc<InitBlock, Copy> 480MB/s ± 4% 521MB/s ± 5% +8.69% (p=0.000 n=59+60) BM_Parse_Upb_FileDesc<InitBlock, Alias> 528MB/s ± 4% 578MB/s ± 4% +9.36% (p=0.000 n=57+58) BM_Parse_Proto2<FileDesc, NoArena, Copy> 286MB/s ± 4% 287MB/s ± 4% ~ (p=0.195 n=55+53) BM_Parse_Proto2<FileDesc, UseArena, Copy> 566MB/s ± 5% 570MB/s ± 4% ~ (p=0.085 n=59+59) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 583MB/s ± 5% 587MB/s ± 3% +0.64% (p=0.023 n=60+58) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 688MB/s ± 6% 693MB/s ± 4% ~ (p=0.063 n=59+58) BM_SerializeDescriptor_Proto2 995MB/s ± 6% 988MB/s ± 5% ~ (p=0.147 n=57+58) BM_SerializeDescriptor_Upb 586MB/s ± 4% 589MB/s ± 4% ~ (p=0.163 n=59+56) ``` PiperOrigin-RevId: 462022073	2 years ago
Protobuf Team Bot	48d6764490	rolling back to fix some tests PiperOrigin-RevId: 461922243	2 years ago
Protobuf Team Bot	470d6322c9	fixed formatting and parsing of negative durations between -1s and 0s PiperOrigin-RevId: 461663523	2 years ago
Eric Salo	24f567b64a	clean up the public api for mini descriptors push the gory details down into internal/ where they belong PiperOrigin-RevId: 461264404	2 years ago
Eric Salo	28bc460dc9	create _upb_EnumDef_MiniDescriptor() delete upb_EnumDef_IsSorted() We now have a simple internal function for returning a mini descriptor directly from an enum def. PiperOrigin-RevId: 461208352	2 years ago
Joshua Haberman	b0ed763a41	Fixed some corner cases around empty packages in upb. 1. upb now tolerates a present-but-empty package name in a `FileDescriptorProto`. 2. upb now always returns a non-NULL string for `upb_FileDef_Package()`. If the package was empty or missing, the returned string is zero-length. This better matches the proto2 behavior: `proto2::FileDescriptor::package()` always returns a package string, which may be empty. PiperOrigin-RevId: 460831797	2 years ago
Mike Kruskal	17b6451684	Bumping protobuf dependency to newer commit PiperOrigin-RevId: 460811319	2 years ago
Eric Salo	eb66ab601f	update upb_Encode() to use arena api internally PiperOrigin-RevId: 460758866	2 years ago
Eric Salo	c2c6427f60	upb: fix NULL pointer bug in Python FFI Reference: https://github.com/protocolbuffers/protobuf/issues/10208 PiperOrigin-RevId: 460623221	2 years ago

1 2 3 4 5 ...

2454 Commits (85534bbb1e2119bc5dae045967386db9cad22aa7) All Branches Search

2454 Commits (85534bbb1e2119bc5dae045967386db9cad22aa7)

All Branches