Benchmarks: https://pastebin.com/tZ7dr67W. Works well especially on smaller ranges.
After a week on spending optimizing NEON SIMD where I almost managed to make hash tables work with NEON SIMD without performance hits (still 1 cycle to optimize and I gave up a little), I found an interesting optimization for aarch64 to use cls instruction (count leading sign bits).
The loop has a property that ctrl_ group is not matched against count when the first slot is empty or deleted.
```
void skip_empty_or_deleted() {
while (IsEmptyOrDeleted(*ctrl_)) {
uint32_t shift = Group{ctrl_}.CountLeadingEmptyOrDeleted();
ctrl_ += shift;
slot_ += shift;
}
...
}
```
However, `kEmpty` and `kDeleted` have format of `1xxxxxx0` and `~ctrl & (ctrl >> 7)` always sets the lowest bit to 1.
In naive implementation, it does +1 to start counting zero bits, however, in aarch64 we may start counting one bits immediately. This saves 1 cycle and 5% of iteration performance.
Then it becomes hard to find a supported and sustainable C++ version of it.
`__clsll` is not supported by GCC and was supported only since clang 8, `__builtin_clrsb` is not producing optimal codegen for clang. `__rbit` is not supported by GCC and there is no intrinsic to do that, however, in clang we have `__builtin_bitreverse{32,64}`. For now I decided to enable this only for clang, only if they have appropriate builtins.
PiperOrigin-RevId: 451168570
Change-Id: I7e9256a60aecdc88ced4e6eb15ebc257281b6664
When building pkg-config files, compute linker flags with a string
substitution rather than the JOIN generator expression. This ensures
that commas in linker flags don’t get treated as argument separators in
JOIN.
Bug: https://bugs.debian.org/1011294
PiperOrigin-RevId: 450675966
Change-Id: I61eacc46a468bae5ff3dae2b437a564f2f1042c2
Previously was disabled on iPhone, but still enabled for macOS.
The unscaled cycle clock does not work correctly when run on a VM.
PiperOrigin-RevId: 449876559
Change-Id: I679ade90b43462e8d2794b1a2b32569d59029ed9
Add a new (internal) feature test macro to detect whether the wrappers are no-ops on a given platform.
Note that one-arg __builtin_prefetch(x) is equivalent to __builtin_prefetch(x, 0, 3), per `man BUILTIN_PREFETCH(3)` and gcc docs.
PiperOrigin-RevId: 449508660
Change-Id: I144e750205eec0c956d8dd62bc72e10bdb87c4f7
Both Mutex and CondVar signal PerThreadSem/Waiter after satisfying the wait condition,
as the result the waiting thread may return w/o waiting on the
PerThreadSem/Waiter at all. If the waiting thread then exits, it currently
destroys Waiter object. As the result Waiter::Post can be called on
already destroyed object.
PerThreadSem/Waiter must be type-stable after creation and must not be destroyed.
The futex-based implementation is the only one that is not affected by the bug
since there is effectively nothing to destroy (maybe only UBSan/ASan
could complain about calling methods on a destroyed object).
Here is the problematic sequence of events:
1: void Mutex::Block(PerThreadSynch *s) {
2: while (s->state.load(std::memory_order_acquire) == PerThreadSynch::kQueued) {
3: if (!DecrementSynchSem(this, s, s->waitp->timeout)) {
4: PerThreadSynch *Mutex::Wakeup(PerThreadSynch *w) {
5: ...
6: w->state.store(PerThreadSynch::kAvailable, std::memory_order_release);
7: IncrementSynchSem(this, w);
8: ...
9: }
Consider line 6 is executed, then line 2 observes kAvailable and
line 3 is not called. The thread executing Mutex::Block returns from
the method, acquires the mutex, releases the mutex, exits and destroys
PerThreadSem/Waiter.
Now Mutex::Wakeup resumes and executes line 7 on the destroyed object. Boom!
CondVar uses a similar pattern.
Moreover the semaphore-based Waiter implementation is not even destruction-safe
(the Waiter cannot be used to signal own destruction). So even if Mutex/CondVar
would always pair Waiter::Post with Waiter::Wait before destroying PerThreadSem/Waiter,
it would still be subject to use-after-free bug on the semaphore.
PiperOrigin-RevId: 449159939
Change-Id: I497134fa8b6ce1294a422827c5f0de0e897cea31
CondVar::WaitWithTimeout can live-lock when timeout is racing with Signal/SignalAll
and Signal/SignalAll thread is not scheduled due to priorities, affinity or other
scheduler artifacts. This could lead to stalls of up to tens of seconds in some cases.
PiperOrigin-RevId: 449159670
Change-Id: I64bbd277c1f91964cfba3306ba8a80eeadf85f64
Stop the absl::Cord destructor from running on the constinit cord in
CordTest.ConstinitConstructor. This allows inspecting the cord at any point
while the test is exiting, which is important for the semantics of the test.
PiperOrigin-RevId: 448327386
Change-Id: Icef9faa2b63f1f0ae60b3430dcf6184f5dead885
* Avoid warnings due to deprecation of volatile return types.
Also fix up optional_test.cc due to ABSL_USES_STD_OPTIONAL always being
false in its body.
Bug: chromium:1284275
PiperOrigin-RevId: 447796238
Change-Id: If050206c979c6c08af22e71ff0ea91e7f7932f0c
The old analysis viewed it as birthday attack, which asks how often
there are multiple values in the probe same probe sequence with the
same H2. In their own words, this analysis "breaks down" at around `n
= 12`.
Instead we can answer a simpler question, which is, if a probe
sequence examines `k` objects that are not what we are looking for,
what's the number of calls `==`? The expectation is simply `k/128`.
PiperOrigin-RevId: 446518063
Change-Id: Ie879bd4f6c97979822bc9d550b9e2503b1418c78
--
6d4e969ad240d248bfeb644b3b76fffae0f07882 by Christian Blichmann <cblichmann@google.com>:
cmake: Fix description of `ABSL_USE_EXTERNAL_GOOGLETEST` option
There is no `add_subproject()` in CMake.
PiperOrigin-RevId: 443087953
Change-Id: I30c2118638f99ad1389ae197e2c81d1e5f298882
GitOrigin-RevId: 6d4e969ad240d248bfeb644b3b76fffae0f07882
--
6457ad659de86ce4cae1e9f7cb03a701c6c2851e by Abseil Team <absl-team@google.com>:
Introduced ErrnoToStatusCode and ErrnoToStatus to abseil.
PiperOrigin-RevId: 442903450
Change-Id: I9c062b34a3811216f43eef56e631eada3b4e3e84
GitOrigin-RevId: 6457ad659de86ce4cae1e9f7cb03a701c6c2851e
--
3d018c03a34bf273a4b24b3584ed77f0a6d21686 by Abseil Team <absl-team@google.com>:
Fix a spelling typo (s/boundries/boundaries).
PiperOrigin-RevId: 442041877
Change-Id: I608020697d37b85316bb9a0838e4b457659c926c
--
518b8119e51db24ce7fb0fd2fe537ec43825c3e6 by Dino Radakovic <dinor@google.com>:
absl/types/internal/variant: Make include guard uppercase
https://google.github.io/styleguide/cppguide.html#The__define_Guard
PiperOrigin-RevId: 441911692
Change-Id: I9837dd07f20204d8253f20627b0917a34dc21825
--
b91696c38310a7cae8c1ea9e2d479495f5dc3f69 by Greg Falcon <gfalcon@google.com>:
Add an internal-only API to wrap __builtin_prefetch() if available.
This private API is intended for future use by the Abseil implementation. Like any internal-namespaced function, it may be changed or removed at any time.
PiperOrigin-RevId: 441894616
Change-Id: Iaa48bd4680b373f4a0d5afab0cb35e2a1908595f
--
0f01e8b0551a662e02dff60840c54320f987315f by Derek Mauro <dmauro@google.com>:
C++20: Use the standard `constinit` keyword for `ABSL_CONST_INIT` when available
PiperOrigin-RevId: 441778874
Change-Id: I70c616469752ff23b326b1c615437599f42cc6aa
GitOrigin-RevId: 3d018c03a34bf273a4b24b3584ed77f0a6d21686
--
f4c7e510922668c68be4aa79a00867c3d3ca9f95 by Derek Mauro <dmauro@google.com>:
Many improvements to LeakChecker builds
The presence of the LeakChecker is now detected when possible. GCC
users using LeakChecker in standalone mode still need to use
-DLEAK_CHECKER. This is now documented in the header.
The hacky targets used for testing leak checking have been removed in
favor of testing in AddressSanitizer mode on Kokoro.
Fixes#885Fixes#1153
PiperOrigin-RevId: 441203393
Change-Id: Ibe64ef6b104bcaf31839ff7184e558cc86abdd1c
--
5c70a23aa83b8152ab95d2cf21662fc63c80ef7d by Abseil Team <absl-team@google.com>:
Add a benchmark for stacktrace
PiperOrigin-RevId: 441196473
Change-Id: I4c9aa2e797aa2cae09abfaaee3abe5c09eb62fc4
--
50b406052273b9d5bad04a7860a96e4d5d956c02 by Abseil Team <absl-team@google.com>:
Internal change.
PiperOrigin-RevId: 441114481
Change-Id: I667af7a50d5631ca91289dd24c91ba90233e0184
--
568b4eaac120b420bce5290179d407d2b57d5bae by Dino Radakovic <dinor@google.com>:
Internal change
PiperOrigin-RevId: 440894155
Change-Id: Ia587ffc65a8321126585fb363b7c0ca8cc2a0da2
--
d53948eace4f3a10ac5a6c1496dc51b81adc412c by Abseil Team <absl-team@google.com>:
Explicitly give internal linkage to symbols which are not used outside of their
translation units.
PiperOrigin-RevId: 440424519
Change-Id: I531c5e229d443375483b7550a34f48042589a99b
GitOrigin-RevId: f4c7e510922668c68be4aa79a00867c3d3ca9f95
--
0c8848ebedc07470c7ab647a5bb8949481540ce9 by Dino Radakovic <dinor@google.com>:
Define absl::base_internal::invoke using std::invoke when C++ >= 17
PiperOrigin-RevId: 439880834
Change-Id: I3622fcf473501d54c57575118a11d54c19573446
GitOrigin-RevId: 0c8848ebedc07470c7ab647a5bb8949481540ce9
--
ef2bf829c333f378ecc12f3259e3187cdb75a3d5 by Abseil Team <absl-team@google.com>:
debugging: fix the VDSO symbol name used for unwinding on RISC-V Linux
The name listed in `man vdso` is incorrect. Instead, use the name from `linux-5.16/arch/riscv/kernel/vdso/rt_sigreturn.S`
PiperOrigin-RevId: 439654174
Change-Id: Ib39d066f416681720068e806e828a2c76a14a532
--
43dfad824afd36cfc3e5049b4fea71a2bccb066c by Benjamin Barenblat <bbaren@google.com>:
Check printf format strings in str_format_convert_test
Add ABSL_PRINTF_ATTRIBUTE to appropriate functions in
strings/internal/str_format/convert_test. Correct
TypedFormatConvertTest.Char, which was accidentally passing values of
types larger than int to StrPrint.
PiperOrigin-RevId: 439388148
Change-Id: I6cde4e8e0c6455064138192430f07f4c990be0bc
--
f84b4ab2c3b070c8af0c82742ac7a8a4bf443bca by Derek Mauro <dmauro@google.com>:
Use __builtin_memcmp in the absl::string_view implementation
starting with MSVC 16.9, where it first appeared
This enables more constexpr operations
PiperOrigin-RevId: 439317316
Change-Id: Iaf1ce76b60901d4b2d5b96be5900c56572f57b15
GitOrigin-RevId: ef2bf829c333f378ecc12f3259e3187cdb75a3d5
--
b984c7c1cbee4253192c833046bfcfa16dca8ccf by Dino Radakovic <dinor@google.com>:
Restrict visibility of absl/flags:commandlineflag_internal
PiperOrigin-RevId: 438591894
Change-Id: I12a6392b2c7f9f1263c741dfd6c43ae22e903aad
--
81315601aab70bb6ac2d17655a727d7f9ee2ff95 by Justin Lebar <jlebar@google.com>:
Update example in str_join.h to use a lambda.
PiperOrigin-RevId: 438390035
Change-Id: Icc707b972e5a369a71ad774004fdf0a17a9b33a7
--
1c3c7921224e505faca8617b073f657d3737219f by Dino Radakovic <dinor@google.com>:
Internal change
PiperOrigin-RevId: 438386036
Change-Id: I6066da1b5a1ddf5af265944a31ed298297b4f2e1
--
2d6885f78481f04e0e7ee86060aec15b677144f3 by Abseil Team <absl-team@google.com>:
Internal change
PiperOrigin-RevId: 438378389
Change-Id: If70dd9114114eb44e85afccd521e7fb7e1436b88
--
7f8282ddee7fcd032e01cbfe65c11c2d166cceb8 by Derek Mauro <dmauro@google.com>:
Internal change
PiperOrigin-RevId: 438374554
Change-Id: I993367952af1dc83bd5aa0ae19a64c024f457fdd
--
ce65ba28f6031e45db8fa5118a05410f5166fd7a by Abseil Team <absl-team@google.com>:
Spelling gardening: Heterogeneous has an "e" after the "n".
PiperOrigin-RevId: 438374411
Change-Id: If1a9098a5d04338837998883739c7b555efa62b4
GitOrigin-RevId: b984c7c1cbee4253192c833046bfcfa16dca8ccf
uclibc-ng doesn't provide getauxval which results in the following build
failure on arm or ppc with any user of abseil-cpp such as grpc:
/home/buildroot/autobuild/instance-0/output-1/host/opt/ext-toolchain/bin/../lib/gcc/arm-buildroot-linux-uclibcgnueabi/10.3.0/../../../../arm-buildroot-linux-uclibcgnueabi/bin/ld: /home/buildroot/autobuild/instance-0/output-1/host/arm-buildroot-linux-uclibcgnueabi/sysroot/usr/lib/libabsl_random_internal_randen_hwaes.so.2111.0.0: undefined reference to `getauxval'
To fix this build failure, check that __UCLIBC__ is not defined before
using getauxval (as Babel is not able to check function availability)
Fixes:
- http://autobuild.buildroot.org/results/775f3ca3dedebff29e212b29dfa896b7613b7a02
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
--
fb671efb2a70f452f17a884b17cf18817b977a8f by Abseil Team <absl-team@google.com>:
Remove extra semicolon in ABSL_INTERNAL_ASSERT_IS_FULL macro
To fix compilation when empty statement warning is treated as error.
PiperOrigin-RevId: 438342663
Change-Id: I3067fbeffa2691888f37554e88f229f24fb55ecc
--
a58c9396f1d88d11347aed36ef2e1b633071363c by Martijn Vels <mvels@google.com>:
Fix kMaxHeight bounds to kMaxDepth for CordrepBtreeNavigator
Added unit test (confirmed failure mode with old code) and extra assertion in the implementation.
PiperOrigin-RevId: 438327463
Change-Id: I32242c86b0c879b8a42cb9a92075e537d588e09f
--
f348e85dbfc9187ef59085fa2b999374f1670338 by Jorge Gorbe Moya <jgorbe@google.com>:
Make the flags enum in `RefcountAndFlags` a named enum to workaround an lldb
issue (https://github.com/llvm/llvm-project/issues/54602).
PiperOrigin-RevId: 438146097
Change-Id: Ibc2ee26489d99de515a779a903b6458dd0befef7
--
a960a3e9fb2a2e3418f806178e73d8566b78bc85 by Gennadiy Rozental <rogeeff@google.com>:
Introduce support for std::optional<T>/absl::optional<T> flag types.
PiperOrigin-RevId: 438129500
Change-Id: I3d925c0a7f9ce9f857277fac3b0bf664ccd3a95c
GitOrigin-RevId: fb671efb2a70f452f17a884b17cf18817b977a8f
--
291f7ef542f73e4801ab5108014bc02344ef31df by Derek Mauro <dmauro@google.com>:
Internal change
PiperOrigin-RevId: 437835981
Change-Id: I42fd92e74903894533ac9984d7f622e3ba20f468
--
2e8caf1a57c50b518e05b4bca48e4fe1bb19af82 by Andy Getzendanner <durandal@google.com>:
Internal change
PiperOrigin-RevId: 437832673
Change-Id: I61b35089418d01a54cecf161b254b68252bebff3
--
b927482ccc399f7e337b60582988b914d9946e4e by Derek Mauro <dmauro@google.com>:
Simplify endian intrinsics for modern compilers
All modern compilers have either __builtin_bswapN (gcc, clang) or
_byteswap_TYPE (MSVC). The other intrinsic definitions are no longer
necessary.
PiperOrigin-RevId: 437772295
Change-Id: Ifb3d88ba24b9097f87ceb202272b36d2f5e5117f
--
b6782a2247a16d5c14706a74ec577c19963d9f97 by Derek Mauro <dmauro@google.com>:
Internal change
PiperOrigin-RevId: 437373174
Change-Id: I0f77e1780dee90d7a3c32a08d96c4aeb624a57b4
--
a53e0c724e37b0b01515a99bd25394b8e21ffdfc by Derek Mauro <dmauro@google.com>:
Unify detection of SSE2 and SSSE3 instruction sets and
include the proper headers
Fix the intrinsic implementation of FastHexToBufferZeroPad16 in
numbers.h which only relies on SSSE3, not SSE 4.2.
https://godbolt.org/z/Pf5bn1Yv9Closes#639
PiperOrigin-RevId: 437286940
Change-Id: Ic97948399b61b91e9c0bccd09313b795b904d714
--
f173f597cb2a75ef2a989f45a496334b85e6f40d by Abseil Team <absl-team@google.com>:
Change assertion function to enable clearer error messages.
PiperOrigin-RevId: 437227057
Change-Id: If420d2f63b51feef6648762f344d5be012cd9c85
GitOrigin-RevId: 291f7ef542f73e4801ab5108014bc02344ef31df
Abseil Team
Dino Radakovic
Derek Mauro
Benjamin Barenblat
Tom Rybka
Greg Falcon
Andy Getzendanner
Samuel Benzaquen
Evan Brown
Fabrice Fontaine