protobuf

Commit Graph

Author	SHA1	Message	Date
Joshua Haberman	73489a9339	Removed the cleanup list It no longer has any users. If we need it later, we can add it back. This saves one pointer of memory from `sizeof(upb_Arena)`. Also, we now allow fuses if if the block allocators differ. This is made possible by cl/520144430, but was not take advantage of in that CL. PiperOrigin-RevId: 520174588	2 years ago
Joshua Haberman	a73852630f	Changed Arena representation so that fusing links arenas together instead of blocks. Previously when fusing, we would concatenate all blocks into a single list that lived in the arena root. From then on, all arenas would add their blocks to this single unified list. After this CL, arenas keep their distinct list of blocks even after being fused. Instead of unifying the block list, fuse now puts the arenas themselves into a list, so all arenas in the fused group can be iterated over at any time. This design makes it easier to keep each individual arena thread-compatible, because fuse and free are now the only mutating operations that touch state that is shared with the entire group. Read-only operations like `SpaceAllocated()` also iterate the list of arenas, but in a read-only fashion. (Note: we need tests for SpaceAllocated(), both single-threaded for correctness and multi-threaded for resilience to crashes and data races). Performance of fuse regresses by 5-20%. This is somewhat expected as we are performing more atomic operations during a fuse. ``` name old cpu/op new cpu/op delta BM_ArenaOneAlloc 18.4ns ± 6% 18.7ns ± 4% +2.00% (p=0.016 n=18+18) BM_ArenaInitialBlockOneAlloc 5.50ns ± 4% 6.57ns ± 4% +19.42% (p=0.000 n=16+17) BM_ArenaFuseUnbalanced/2 59.3ns ±10% 68.7ns ± 4% +15.85% (p=0.000 n=19+19) BM_ArenaFuseUnbalanced/8 479ns ± 5% 540ns ± 8% +12.57% (p=0.000 n=18+19) BM_ArenaFuseUnbalanced/64 4.50µs ± 4% 4.93µs ± 8% +9.59% (p=0.000 n=17+17) BM_ArenaFuseUnbalanced/128 9.24µs ± 3% 9.96µs ± 3% +7.81% (p=0.000 n=17+17) BM_ArenaFuseBalanced/2 63.3ns ±18% 71.0ns ± 4% +12.14% (p=0.000 n=19+18) BM_ArenaFuseBalanced/8 484ns ± 9% 543ns ±10% +12.11% (p=0.000 n=17+16) BM_ArenaFuseBalanced/64 4.50µs ± 6% 4.94µs ± 4% +9.62% (p=0.000 n=19+17) BM_ArenaFuseBalanced/128 9.20µs ± 4% 9.95µs ± 4% +8.12% (p=0.000 n=16+19) BM_LoadAdsDescriptor_Upb<NoLayout> 5.50ms ± 8% 5.69ms ±17% ~ (p=0.189 n=18+19) BM_LoadAdsDescriptor_Upb<WithLayout> 6.10ms ± 5% 6.05ms ± 4% ~ (p=0.258 n=17+18) BM_LoadAdsDescriptor_Proto2<NoLayout> 11.9ms ±15% 11.6ms ± 5% ~ (p=0.589 n=19+16) BM_LoadAdsDescriptor_Proto2<WithLayout> 11.8ms ± 5% 12.4ms ±17% ~ (p=0.604 n=16+20) BM_Parse_Upb_FileDesc<UseArena, Copy> 12.1µs ± 8% 12.1µs ± 4% ~ (p=1.000 n=18+18) BM_Parse_Upb_FileDesc<UseArena, Alias> 11.8µs ±17% 11.1µs ± 4% ~ (p=0.104 n=20+17) BM_Parse_Upb_FileDesc<InitBlock, Copy> 12.0µs ± 5% 11.9µs ± 4% ~ (p=0.134 n=18+19) BM_Parse_Upb_FileDesc<InitBlock, Alias> 10.9µs ± 7% 11.0µs ± 4% ~ (p=0.195 n=17+18) BM_Parse_Proto2<FileDesc, NoArena, Copy> 24.2µs ± 4% 24.4µs ± 7% ~ (p=0.767 n=18+18) BM_Parse_Proto2<FileDesc, UseArena, Copy> 11.6µs ± 5% 11.6µs ± 4% ~ (p=0.621 n=18+16) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 11.3µs ± 3% 11.3µs ± 3% ~ (p=0.743 n=18+18) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 12.7µs ± 8% 12.7µs ± 4% ~ (p=0.988 n=18+19) BM_SerializeDescriptor_Proto2 5.77µs ± 5% 5.71µs ± 5% ~ (p=0.433 n=17+17) BM_SerializeDescriptor_Upb 10.0µs ± 5% 10.1µs ± 7% ~ (p=0.102 n=19+16) name old time/op new time/op delta BM_ArenaOneAlloc 18.4ns ± 6% 18.8ns ± 4% +1.98% (p=0.019 n=18+18) BM_ArenaInitialBlockOneAlloc 5.51ns ± 4% 6.58ns ± 4% +19.42% (p=0.000 n=16+17) BM_ArenaFuseUnbalanced/2 59.5ns ±10% 68.9ns ± 4% +15.83% (p=0.000 n=19+19) BM_ArenaFuseUnbalanced/8 481ns ± 5% 541ns ± 8% +12.54% (p=0.000 n=18+19) BM_ArenaFuseUnbalanced/64 4.51µs ± 4% 4.94µs ± 8% +9.53% (p=0.000 n=17+17) BM_ArenaFuseUnbalanced/128 9.26µs ± 3% 9.98µs ± 3% +7.79% (p=0.000 n=17+17) BM_ArenaFuseBalanced/2 63.5ns ±19% 71.1ns ± 3% +12.07% (p=0.000 n=19+18) BM_ArenaFuseBalanced/8 485ns ± 9% 551ns ±20% +13.47% (p=0.000 n=17+17) BM_ArenaFuseBalanced/64 4.51µs ± 6% 4.95µs ± 4% +9.62% (p=0.000 n=19+17) BM_ArenaFuseBalanced/128 9.22µs ± 4% 9.97µs ± 4% +8.12% (p=0.000 n=16+19) BM_LoadAdsDescriptor_Upb<NoLayout> 5.52ms ± 8% 5.72ms ±18% ~ (p=0.199 n=18+19) BM_LoadAdsDescriptor_Upb<WithLayout> 6.12ms ± 5% 6.07ms ± 4% ~ (p=0.273 n=17+18) BM_LoadAdsDescriptor_Proto2<NoLayout> 11.9ms ±15% 11.6ms ± 5% ~ (p=0.589 n=19+16) BM_LoadAdsDescriptor_Proto2<WithLayout> 11.9ms ± 5% 12.5ms ±18% ~ (p=0.582 n=16+20) BM_Parse_Upb_FileDesc<UseArena, Copy> 12.2µs ± 8% 12.1µs ± 3% ~ (p=0.963 n=18+18) BM_Parse_Upb_FileDesc<UseArena, Alias> 11.8µs ±17% 11.1µs ± 4% ~ (p=0.104 n=20+17) BM_Parse_Upb_FileDesc<InitBlock, Copy> 12.0µs ± 5% 11.9µs ± 4% ~ (p=0.126 n=18+19) BM_Parse_Upb_FileDesc<InitBlock, Alias> 11.0µs ± 6% 11.1µs ± 4% ~ (p=0.195 n=17+18) BM_Parse_Proto2<FileDesc, NoArena, Copy> 24.3µs ± 4% 24.5µs ± 6% ~ (p=0.743 n=18+18) BM_Parse_Proto2<FileDesc, UseArena, Copy> 11.7µs ± 5% 11.6µs ± 4% ~ (p=0.574 n=18+16) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 11.3µs ± 3% 11.3µs ± 3% ~ (p=0.743 n=18+18) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 12.7µs ± 8% 12.7µs ± 4% ~ (p=0.988 n=18+19) BM_SerializeDescriptor_Proto2 5.78µs ± 5% 5.73µs ± 5% ~ (p=0.357 n=17+17) BM_SerializeDescriptor_Upb 10.0µs ± 5% 10.1µs ± 7% ~ (p=0.117 n=19+16) name old allocs/op new allocs/op delta BM_ArenaOneAlloc 1.00 ± 0% 1.00 ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/2 2.00 ± 0% 2.00 ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/8 8.00 ± 0% 8.00 ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/64 64.0 ± 0% 64.0 ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/128 128 ± 0% 128 ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/2 2.00 ± 0% 2.00 ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/8 8.00 ± 0% 8.00 ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/64 64.0 ± 0% 64.0 ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/128 128 ± 0% 128 ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Upb<NoLayout> 6.08k ± 0% 6.05k ± 0% -0.54% (p=0.000 n=20+20) BM_LoadAdsDescriptor_Upb<WithLayout> 6.39k ± 0% 6.36k ± 0% -0.55% (p=0.000 n=20+20) BM_LoadAdsDescriptor_Proto2<NoLayout> 83.4k ± 0% 83.4k ± 0% ~ (p=0.800 n=20+20) BM_LoadAdsDescriptor_Proto2<WithLayout> 84.4k ± 0% 84.4k ± 0% ~ (p=0.752 n=20+20) BM_Parse_Upb_FileDesc<UseArena, Copy> 7.00 ± 0% 7.00 ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<UseArena, Alias> 7.00 ± 0% 7.00 ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, NoArena, Copy> 765 ± 0% 765 ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, UseArena, Copy> 8.00 ± 0% 8.00 ± 0% ~ (all samples are equal) name old peak-mem(Bytes)/op new peak-mem(Bytes)/op delta BM_ArenaOneAlloc 336 ± 0% 336 ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/2 672 ± 0% 672 ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/8 2.69k ± 0% 2.69k ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/64 21.5k ± 0% 21.5k ± 0% ~ (all samples are equal) BM_ArenaFuseUnbalanced/128 43.0k ± 0% 43.0k ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/2 672 ± 0% 672 ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/8 2.69k ± 0% 2.69k ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/64 21.5k ± 0% 21.5k ± 0% ~ (all samples are equal) BM_ArenaFuseBalanced/128 43.0k ± 0% 43.0k ± 0% ~ (all samples are equal) BM_LoadAdsDescriptor_Upb<NoLayout> 9.89M ± 0% 9.95M ± 0% +0.65% (p=0.000 n=20+20) BM_LoadAdsDescriptor_Upb<WithLayout> 9.95M ± 0% 10.02M ± 0% +0.70% (p=0.000 n=20+20) BM_LoadAdsDescriptor_Proto2<NoLayout> 6.62M ± 0% 6.62M ± 0% ~ (p=0.800 n=20+20) BM_LoadAdsDescriptor_Proto2<WithLayout> 6.66M ± 0% 6.66M ± 0% ~ (p=0.752 n=20+20) BM_Parse_Upb_FileDesc<UseArena, Copy> 36.5k ± 0% 36.5k ± 0% ~ (all samples are equal) BM_Parse_Upb_FileDesc<UseArena, Alias> 36.5k ± 0% 36.5k ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, NoArena, Copy> 35.8k ± 0% 35.8k ± 0% ~ (all samples are equal) BM_Parse_Proto2<FileDesc, UseArena, Copy> 65.3k ± 0% 65.3k ± 0% ~ (all samples are equal) name old speed new speed delta BM_LoadAdsDescriptor_Upb<NoLayout> 138MB/s ± 7% 132MB/s ±15% ~ (p=0.126 n=18+20) BM_LoadAdsDescriptor_Upb<WithLayout> 124MB/s ± 5% 125MB/s ± 4% ~ (p=0.258 n=17+18) BM_LoadAdsDescriptor_Proto2<NoLayout> 63.9MB/s ±13% 65.2MB/s ± 5% ~ (p=0.589 n=19+16) BM_LoadAdsDescriptor_Proto2<WithLayout> 64.0MB/s ± 5% 61.3MB/s ±15% ~ (p=0.604 n=16+20) BM_Parse_Upb_FileDesc<UseArena, Copy> 620MB/s ± 8% 622MB/s ± 4% ~ (p=1.000 n=18+18) BM_Parse_Upb_FileDesc<UseArena, Alias> 644MB/s ±15% 679MB/s ± 4% ~ (p=0.104 n=20+17) BM_Parse_Upb_FileDesc<InitBlock, Copy> 627MB/s ± 4% 633MB/s ± 4% ~ (p=0.134 n=18+19) BM_Parse_Upb_FileDesc<InitBlock, Alias> 688MB/s ± 6% 682MB/s ± 4% ~ (p=0.195 n=17+18) BM_Parse_Proto2<FileDesc, NoArena, Copy> 310MB/s ± 4% 309MB/s ± 6% ~ (p=0.767 n=18+18) BM_Parse_Proto2<FileDesc, UseArena, Copy> 646MB/s ± 4% 649MB/s ± 4% ~ (p=0.621 n=18+16) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 666MB/s ± 3% 666MB/s ± 3% ~ (p=0.743 n=18+18) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 592MB/s ± 7% 593MB/s ± 4% ~ (p=0.988 n=18+19) BM_SerializeDescriptor_Proto2 1.30GB/s ± 5% 1.32GB/s ± 5% ~ (p=0.433 n=17+17) BM_SerializeDescriptor_Upb 756MB/s ± 5% 745MB/s ± 6% ~ (p=0.102 n=19+16) ``` PiperOrigin-RevId: 520144430	2 years ago
Joshua Haberman	bc7795702e	Fix the Lexan build. PiperOrigin-RevId: 520109290	2 years ago
Mike Kruskal	2d273b6d74	Internal Code Change PiperOrigin-RevId: 520076240	2 years ago
Joshua Haberman	9c0bcc0e90	Revamped portable atomic layer Two main improvements: 1. Memory order is an explicit parameter. 2. Functions are type-generic. PiperOrigin-RevId: 520074974	2 years ago
Joshua Haberman	6d6ab90ece	Enable arena test (it was previously #ifdef'd away accidentally) PiperOrigin-RevId: 519782179	2 years ago
deannagarcia	f69865183a	Use requirements.txt in GHA tests (#1237 ) We recently saw a failure pass through where requirements.txt was updated in a way that was not compatible with some python versions. This updates the tests to directly use requirements.txt to download requirements to catch issues like this in the future. Closes #1237 COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/upb/pull/1237 from protocolbuffers:useRequirementsGHA 22a963468e068535cbcf772ba41aee0b3a6e2eec PiperOrigin-RevId: 518962372	2 years ago
Deanna Garcia	e1c5a567ee	Delete UPB fuzz GHA tests. Our new fuzzer tests in upb/util make this obsolete. PiperOrigin-RevId: 518653168	2 years ago
Deanna Garcia	eda3f17ed3	Update numpy dependency to 1.22.0. GH advisory on versions < 1.22.0 https://github.com/advisories/GHSA-fpfv-jqm9-f5jm. PiperOrigin-RevId: 518601694	2 years ago
Joshua Haberman	d450990631	Allow for fuse/free races in `upb_Arena`. Implementation is by kfm@, I only added the portability code around it. `upb_Arena` was designed to be only thread-compatible. However, fusing of arenas muddies the waters somewhat, because two distinct `upb_Arena` objects will end up sharing state when fused. This causes a `upb_Arena_Free(a)` to interfere with `upb_Arena_Fuse(b, c)` if `a` and `b` were previously fused. It turns out that we can use atomics to fix this with about a 35% regression in fuse performance (see below). Arena create+free does not regress, thanks to special-case logic in Free(). `upb_Arena` is still a thread-compatible type, and it is still never safe to call `upb_Arena_xxx(a)` and `upb_Arena_yyy(a)` in parallel. However you can at least now call `upb_Arena_Free(a)` and `upb_Arena_Fuse(b, c)` in parallel, even if `a` and `b` were previously fused. Note that `upb_Arena_Fuse(a, b)` and `upb_Arena_Fuse(c, d)` is still not allowed if `b` and `c` were previously fused. In practice this means that fuses must still be single-threaded within a single fused group. Performance results: ``` name old cpu/op new cpu/op delta BM_ArenaOneAlloc 18.6ns ± 1% 18.6ns ± 1% ~ (p=0.726 n=18+17) BM_ArenaInitialBlockOneAlloc 6.28ns ± 1% 5.73ns ± 1% -8.68% (p=0.000 n=17+20) BM_ArenaFuseUnbalanced/2 44.1ns ± 2% 60.4ns ± 1% +37.05% (p=0.000 n=18+19) BM_ArenaFuseUnbalanced/8 370ns ± 2% 500ns ± 1% +35.12% (p=0.000 n=19+20) BM_ArenaFuseUnbalanced/64 3.52µs ± 1% 4.71µs ± 1% +33.80% (p=0.000 n=18+19) BM_ArenaFuseUnbalanced/128 7.20µs ± 1% 9.72µs ± 2% +34.93% (p=0.000 n=16+19) BM_ArenaFuseBalanced/2 44.4ns ± 2% 61.4ns ± 1% +38.23% (p=0.000 n=20+17) BM_ArenaFuseBalanced/8 373ns ± 2% 509ns ± 1% +36.57% (p=0.000 n=19+17) BM_ArenaFuseBalanced/64 3.55µs ± 2% 4.79µs ± 1% +34.80% (p=0.000 n=19+19) BM_ArenaFuseBalanced/128 7.26µs ± 1% 9.76µs ± 1% +34.45% (p=0.000 n=17+19) BM_LoadAdsDescriptor_Upb<NoLayout> 5.66ms ± 1% 5.69ms ± 1% +0.57% (p=0.013 n=18+20) BM_LoadAdsDescriptor_Upb<WithLayout> 6.30ms ± 1% 6.36ms ± 1% +0.90% (p=0.000 n=19+18) BM_LoadAdsDescriptor_Proto2<NoLayout> 12.1ms ± 1% 12.1ms ± 1% ~ (p=0.118 n=18+18) BM_LoadAdsDescriptor_Proto2<WithLayout> 12.2ms ± 1% 12.3ms ± 1% +0.50% (p=0.006 n=18+18) BM_Parse_Upb_FileDesc<UseArena, Copy> 12.7µs ± 1% 12.7µs ± 1% ~ (p=0.194 n=20+19) BM_Parse_Upb_FileDesc<UseArena, Alias> 11.6µs ± 1% 11.6µs ± 1% ~ (p=0.192 n=20+20) BM_Parse_Upb_FileDesc<InitBlock, Copy> 12.5µs ± 1% 12.5µs ± 0% ~ (p=0.750 n=18+14) BM_Parse_Upb_FileDesc<InitBlock, Alias> 11.4µs ± 1% 11.3µs ± 1% -0.34% (p=0.046 n=19+19) BM_Parse_Proto2<FileDesc, NoArena, Copy> 25.4µs ± 1% 25.7µs ± 2% +1.37% (p=0.000 n=18+18) BM_Parse_Proto2<FileDesc, UseArena, Copy> 12.1µs ± 2% 12.1µs ± 1% ~ (p=0.143 n=18+18) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 11.9µs ± 3% 11.9µs ± 1% ~ (p=0.076 n=17+19) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 13.2µs ± 1% 13.2µs ± 1% ~ (p=0.053 n=19+19) BM_SerializeDescriptor_Proto2 5.97µs ± 4% 5.90µs ± 4% ~ (p=0.093 n=17+19) BM_SerializeDescriptor_Upb 10.4µs ± 1% 10.4µs ± 1% ~ (p=0.909 n=17+18) name old time/op new time/op delta BM_ArenaOneAlloc 18.7ns ± 2% 18.6ns ± 0% ~ (p=0.607 n=18+17) BM_ArenaInitialBlockOneAlloc 6.29ns ± 1% 5.74ns ± 1% -8.71% (p=0.000 n=17+19) BM_ArenaFuseUnbalanced/2 44.1ns ± 1% 60.6ns ± 1% +37.21% (p=0.000 n=17+19) BM_ArenaFuseUnbalanced/8 371ns ± 2% 500ns ± 1% +35.02% (p=0.000 n=19+16) BM_ArenaFuseUnbalanced/64 3.53µs ± 1% 4.72µs ± 1% +33.85% (p=0.000 n=18+19) BM_ArenaFuseUnbalanced/128 7.22µs ± 1% 9.73µs ± 2% +34.87% (p=0.000 n=16+19) BM_ArenaFuseBalanced/2 44.5ns ± 2% 61.5ns ± 1% +38.22% (p=0.000 n=20+17) BM_ArenaFuseBalanced/8 373ns ± 2% 510ns ± 1% +36.58% (p=0.000 n=19+16) BM_ArenaFuseBalanced/64 3.56µs ± 2% 4.80µs ± 1% +34.87% (p=0.000 n=19+19) BM_ArenaFuseBalanced/128 7.27µs ± 1% 9.77µs ± 1% +34.40% (p=0.000 n=17+19) BM_LoadAdsDescriptor_Upb<NoLayout> 5.67ms ± 1% 5.71ms ± 1% +0.60% (p=0.011 n=18+20) BM_LoadAdsDescriptor_Upb<WithLayout> 6.32ms ± 1% 6.37ms ± 1% +0.87% (p=0.000 n=19+18) BM_LoadAdsDescriptor_Proto2<NoLayout> 12.1ms ± 1% 12.2ms ± 1% ~ (p=0.126 n=18+19) BM_LoadAdsDescriptor_Proto2<WithLayout> 12.2ms ± 1% 12.3ms ± 1% +0.51% (p=0.002 n=18+18) BM_Parse_Upb_FileDesc<UseArena, Copy> 12.7µs ± 1% 12.7µs ± 1% ~ (p=0.149 n=20+19) BM_Parse_Upb_FileDesc<UseArena, Alias> 11.6µs ± 1% 11.6µs ± 1% ~ (p=0.211 n=20+20) BM_Parse_Upb_FileDesc<InitBlock, Copy> 12.5µs ± 1% 12.5µs ± 1% ~ (p=0.986 n=18+15) BM_Parse_Upb_FileDesc<InitBlock, Alias> 11.4µs ± 1% 11.3µs ± 1% ~ (p=0.081 n=19+18) BM_Parse_Proto2<FileDesc, NoArena, Copy> 25.4µs ± 1% 25.8µs ± 2% +1.41% (p=0.000 n=18+18) BM_Parse_Proto2<FileDesc, UseArena, Copy> 12.1µs ± 2% 12.1µs ± 1% ~ (p=0.558 n=19+18) BM_Parse_Proto2<FileDesc, InitBlock, Copy> 12.0µs ± 3% 11.9µs ± 1% ~ (p=0.165 n=17+19) BM_Parse_Proto2<FileDescSV, InitBlock, Alias> 13.2µs ± 1% 13.2µs ± 1% ~ (p=0.070 n=19+19) BM_SerializeDescriptor_Proto2 5.98µs ± 4% 5.92µs ± 3% ~ (p=0.138 n=17+19) BM_SerializeDescriptor_Upb 10.4µs ± 1% 10.4µs ± 1% ~ (p=0.858 n=17+18) ``` PiperOrigin-RevId: 518573683	2 years ago
Eric Salo	ea4cb79f66	fix Python bug with required fields https://github.com/protocolbuffers/upb/issues/1220 There were two bugs here: Python was incorrectly mandating that a required field be set during assignment, and it was also incorrectly assuming a non-NULL return from an internal function call. PiperOrigin-RevId: 518561818	2 years ago
Eric Salo	4e25ba4575	consolidate the field type stuff PiperOrigin-RevId: 518553642	2 years ago
Joshua Haberman	8562ccc1f0	Use _fileno(stdin) instead of STDIN_FILENO on Windows The `*_FILENO` constants don't exist on Windows. PiperOrigin-RevId: 518553512	2 years ago
Deanna Garcia	b7437a1b0e	Update UPB main's protobuf dependency. This also requires some editing in the rewrites since importing plugin.proto shouldn't require the src prefix after https://github.com/protocolbuffers/protobuf/pull/11991. PiperOrigin-RevId: 518387501	2 years ago
Mike Kruskal	f4d045aa92	Only include utf8_range if it hasn't been included already See #1201 PiperOrigin-RevId: 518333273	2 years ago
Joshua Haberman	d520014cfa	Added benchmarks for `upb_Arena_Fuse()` I added two variants, one which is unbalanced (always fusing a single arena into the group) while the other is balanced (always fusing two groups of equal size). Unfortunately it is difficult to benchmark `upb_Arena_Free()` separately from fuse, since the only way to get a non-zero refcount is to fuse. Results on my machine: ``` CPU: Intel Skylake Xeon with HyperThreading (48 cores) dL1:32KB dL2:1024KB dL3:38MB Benchmark Time(ns) CPU(ns) Iterations ------------------------------------------------------------------- BM_ArenaFuseUnbalanced/2 52.2 52.2 1000000 38.300M items/s BM_ArenaFuseUnbalanced/8 416 416 142315 19.243M items/s BM_ArenaFuseUnbalanced/64 4034 4033 14306 15.868M items/s BM_ArenaFuseUnbalanced/128 8302 8301 6837 15.420M items/s BM_ArenaFuseBalanced/2 54.7 54.7 1000000 36.581M items/s BM_ArenaFuseBalanced/8 425 425 100000 18.845M items/s BM_ArenaFuseBalanced/64 4029 4029 14632 15.886M items/s BM_ArenaFuseBalanced/128 8050 8049 7176 15.902M items/s ``` PiperOrigin-RevId: 518292073	2 years ago
Deanna Garcia	666a28e6ac	Update numpy dependency to 1.22.0. GH advisory on versions < 1.22.0 https://github.com/advisories/GHSA-fpfv-jqm9-f5jm. PiperOrigin-RevId: 518103313	2 years ago
Matt Kulukundis	5bc3cae2d7	Add threading tests for arenas PiperOrigin-RevId: 518009578	2 years ago
Protobuf Team Bot	d5c22624fe	Fix cloning of repeated upb_Array. PiperOrigin-RevId: 517280344	2 years ago
Deanna Garcia	1043eee891	Update dependency on com_google_googletest to use the newly added googletest_deps to install transitive dependencies. PiperOrigin-RevId: 517217973	2 years ago
Joshua Haberman	2e9278de50	Fix for win32 binary i/o PiperOrigin-RevId: 516951027	2 years ago
Eric Salo	b7dcc1cf4c	implement upb_MiniTable_FindFieldByIndex() PiperOrigin-RevId: 516651714	2 years ago
Marcel Hlopko	05609bfac0	Add support for thunks to upb_proto_aspect. "Thunks" are .c files with definitions for the inline functions in the gencode C API. They are meant to be used from statically compiled languages that cannot use inline C functions such as Rust. Experimental, do not use. Our current plan is to delete this code in the order of months. Without any prior warning. PiperOrigin-RevId: 516522823	2 years ago
Protobuf Team Bot	fdb45f0da0	Quiet -Wsign-compare errors PiperOrigin-RevId: 516327145	2 years ago
Joshua Haberman	b9fb58bba5	Emit upbdev JSON using numeric representation for enums. PiperOrigin-RevId: 516273336	2 years ago
Joshua Haberman	bdee30b0a6	Added special case for INT64_MIN in the codegen PiperOrigin-RevId: 516236492	2 years ago
Protobuf Team Bot	3286f948f8	Implements upb_Message_DeepClone. PiperOrigin-RevId: 514723111	2 years ago
Eric Salo	a6ce73370f	upb: implement unsigned Int64 list PiperOrigin-RevId: 514312150	2 years ago
Eric Salo	f9c91e3a1b	lower the recursion limit again on fuzzer tests PiperOrigin-RevId: 514260720	2 years ago
Joshua Haberman	56c4a42cdd	Added new APIs for linking a MiniTable all at one time The new API upb_MiniTable_Link() links all sub-messages and sub-enums at a single time, by accepting an array of sub-tables and sub-enums. The order of these sub-tables can be queried using a separate function `upb_MiniTable_GetSubList()`, and this information is added to `CodeGeneratorRequest` as part of the upb-specific info. PiperOrigin-RevId: 513970874	2 years ago
Eric Salo	184a4758f2	fix an error path in the (still completely unused) tokenizer PiperOrigin-RevId: 513948062	2 years ago
Eric Salo	dfd5f176f4	implement Dart Int64 repeated fields PiperOrigin-RevId: 513854105	2 years ago
Eric Salo	c1727b730c	enforce a recursion limit on fuzzer tests PiperOrigin-RevId: 513690911	2 years ago
Protobuf Team Bot	ccdfaf7ea3	Fix a bug where message-level extension identifier should also be generated in the source file. PiperOrigin-RevId: 513687688	2 years ago
Marcel Hlopko	74905292da	Fix minor typos in cs-cpp-protos.md PiperOrigin-RevId: 512874930	2 years ago
Eric Salo	c4b98ddfb5	implement a function to register a single extension PiperOrigin-RevId: 512667332	2 years ago
Eric Salo	19c4e62875	add constructors for mini table extensions PiperOrigin-RevId: 512261931	2 years ago
Protobuf Team Bot	1b6beaa53d	[Upb C++] Update generator to create oneof_case() accessor and OneOfCase enumerations. PiperOrigin-RevId: 512197371	2 years ago
Protobuf Team Bot	a35491fa55	[Upb C++] Generate class scoped declarations for enumerations. PiperOrigin-RevId: 512191989	2 years ago
Eric Salo	9253c42843	add error check for duplicate oneof field names PiperOrigin-RevId: 512029347	2 years ago
Joshua Haberman	81d27ec0e5	Fixed MSAN warning by freeing the arena in ~InlinedArena(). This fixes an MSAN warning of the form: ``` WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x7fc964f2a597 in arena_findroot third_party/upb/upb/mem/arena.c:64:3 #1 0x7fc964f2a597 in upb_Arena_Free third_party/upb/upb/mem/arena.c:211:7 #2 0x7fc9d2af0028 in std::__msan::unique_ptr<upb_Arena, void ()(upb_Arena)>::~unique_ptr() third_party/crosstool/v18/stable/toolchain/bin/../include/c++/v1/__memory/unique_ptr.h:277:7 #3 0x7fc9d2aef7a2 in ~Arena third_party/upb/upb/upb.hpp:70:7 #4 0x7fc9d2aef7a2 in ~InlinedArena third_party/upb/upb/upb.hpp:97:7 #5 0x7fc9d2aef7a2 in Cpp_InlinedArena2_Test::TestBody() third_party/upb/upb/test/test_cpp.cc:187:1 #6 0x7fc97da78a57 in testing::Test::Run() third_party/googletest/googletest/src/gtest.cc:2695:5 #7 0x7fc97da7a3e8 in testing::TestInfo::Run() third_party/googletest/googletest/src/gtest.cc:2844:11 #8 0x7fc97da7b897 in testing::TestSuite::Run() third_party/googletest/googletest/src/gtest.cc:3003:30 #9 0x7fc97daa5136 in testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/googletest/src/gtest.cc:5899:44 #10 0x7fc97daa455c in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/googletest/src/gtest.cc #11 0x7fc97daa455c in testing::UnitTest::Run() third_party/googletest/googletest/src/gtest.cc:5464:10 #12 0x562a7fb876f0 in RUN_ALL_TESTS third_party/googletest/googletest/include/gtest/gtest.h:2329:73 #13 0x562a7fb876f0 in main testing/base/internal/gunit_main.cc:86:10 #14 0x7fc9ba9b7632 in __libc_start_main (/usr/grte/v5/lib64/libc.so.6+0x61632) (BuildId: 280088eab084c30a3992a9bce5c35b44) #15 0x562a7fafdbe9 in _start /build/work/ab393f4ac612f9027aae6b1a7226027ba2a2/google3/blaze-out/k8-opt/bin/third_party/grte/v5_src/grte-scratch/BUILD/src/csu/../sysdeps/x86_64/start.S:120 Member fields were destroyed #0 0x562a7fb0b13d in __sanitizer_dtor_callback_fields third_party/llvm/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:960:5 #1 0x7fc9d2aef79a in ~InlinedArena third_party/upb/upb/upb.hpp:105:8 #2 0x7fc9d2aef79a in ~InlinedArena third_party/upb/upb/upb.hpp:97:7 #3 0x7fc9d2aef79a in Cpp_InlinedArena2_Test::TestBody() third_party/upb/upb/test/test_cpp.cc:187:1 #4 0x7fc97da78a57 in testing::Test::Run() third_party/googletest/googletest/src/gtest.cc:2695:5 #5 0x7fc97da7a3e8 in testing::TestInfo::Run() third_party/googletest/googletest/src/gtest.cc:2844:11 #6 0x7fc97da7b897 in testing::TestSuite::Run() third_party/googletest/googletest/src/gtest.cc:3003:30 #7 0x7fc97daa5136 in testing::internal::UnitTestImpl::RunAllTests() third_party/googletest/googletest/src/gtest.cc:5899:44 #8 0x7fc97daa455c in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> third_party/googletest/googletest/src/gtest.cc #9 0x7fc97daa455c in testing::UnitTest::Run() third_party/googletest/googletest/src/gtest.cc:5464:10 #10 0x562a7fb876f0 in RUN_ALL_TESTS third_party/googletest/googletest/include/gtest/gtest.h:2329:73 #11 0x562a7fb876f0 in main testing/base/internal/gunit_main.cc:86:10 #12 0x7fc9ba9b7632 in __libc_start_main (/usr/grte/v5/lib64/libc.so.6+0x61632) (BuildId: 280088eab084c30a3992a9bce5c35b44) #13 0x562a7fafdbe9 in _start /build/work/ab393f4ac612f9027aae6b1a7226027ba2a2/google3/blaze-out/k8-opt/bin/third_party/grte/v5_src/grte-scratch/BUILD/src/csu/../sysdeps/x86_64/start.S:120 ``` PiperOrigin-RevId: 511849224	2 years ago
Joshua Haberman	2d40e1e2ee	Removed some special escapes from def_to_proto.c, to match C++ protos. This was not a bug -- the previous behavior was correct. However this change brings our implementation-specific details in line with C++. PiperOrigin-RevId: 511273853	2 years ago
Eric Salo	a7a097d443	remove generated hazzer for map fields PiperOrigin-RevId: 510449677	2 years ago
Eric Salo	e860867804	upb_MiniTableExtension_Build() -> upb_MiniTableExtension_Init() The upb convention is that "_Build()" means to also allocate, which this function does not do, so rename it as "_Init()" to free up the name for a future function that does allocate. PiperOrigin-RevId: 510282736	2 years ago
Eric Salo	9bd37b8413	turn down upb/fuzz/ New fuzzer tests in upb/test/ and upb/util/ make this one obsolete. PiperOrigin-RevId: 509953287	2 years ago
Eric Salo	a3f5ee3a5b	detect and reject repeated field numbers within a oneof PiperOrigin-RevId: 509707627	2 years ago
Protobuf Team Bot	a0fbbd6b2d	Avoid automatic variables in functions using setjmp. According to https://en.cppreference.com/w/c/program/setjmp automatic variables modified in a function calling setjmp can have indeterminate values. Instead, refactor all functions calling setjmp so that the function calling setjmp doesn’t have any local variables. Part II: Mini table decoder. PiperOrigin-RevId: 509644446	2 years ago
Protobuf Team Bot	0ccf6fd717	Fix upb_MiniTable_NextOneofField comment. PiperOrigin-RevId: 509416415	2 years ago
Joshua Haberman	57a79de7cc	Ensure that extensions respect deterministic serialization. Previously we were not sorting extensions at encode time, even in deterministic mode. PiperOrigin-RevId: 508217926	2 years ago
Protobuf Team Bot	28de62f463	Add ExtensionRegistry version of Parse to message templates. PiperOrigin-RevId: 508098749	2 years ago

1 2 3 4 5 ...

2930 Commits (f7e25ebfd1cd91afe202692e50d41026e48df9a9) All Branches Search

2930 Commits (f7e25ebfd1cd91afe202692e50d41026e48df9a9)

All Branches