Also remove the transfer implementations from btree_set.h and flat_hash_set.h, which are equivalent to the default implementations.
Motivation: this will simplify upcoming changes related to trivial relocation.
PiperOrigin-RevId: 477493403
Change-Id: I75babef4c93dec3a8105f86c58af54199bb1ec9c
Certain core libraries in Chrome build with these warnings [1];
btree_map and btree_set cannot be used in those libraries until these
warnings are fixed.
[1] https://crbug.com/1292951
PiperOrigin-RevId: 476908396
Change-Id: I32e9ea1eec911e329d6ff00f04fa2e9cfde8660a
The btree data structure poisons regions of memory it's not using. It leaves
these regions poisoned when it frees memory. This means that a custom memory
allocator that tries to reuse freed memory will trigger an ASAN
use-after-poison error.
The fix is to unpoison each memory region right before freeing it.
PiperOrigin-RevId: 475309671
Change-Id: I29d55c298d3d89a83e1f960deb6e93118891ff83
Addresses failures with the following, in some files:
-Wshorten-64-to-32
-Wimplicit-int-conversion
-Wsign-compare
-Wsign-conversion
-Wtautological-unsigned-zero-compare
(This specific CL focuses on .cc files in */internal/.)
Bug: chromium:1292951
PiperOrigin-RevId: 473868797
Change-Id: Ibe0b76e33f9e001d59862beaac54fb47bacd39b2
Addresses failures with the following, in some files:
-Wshorten-64-to-32
-Wimplicit-int-conversion
-Wsign-compare
-Wsign-conversion
-Wtautological-unsigned-zero-compare
(This specific CL focuses on tests.)
Bug: chromium:1292951
PiperOrigin-RevId: 473055916
Change-Id: I40cdd6c87ba9d0a5fb2db5746cff04f14ee829c2
Addresses failures with the following, in some files:
-Wshorten-64-to-32
-Wimplicit-int-conversion
-Wsign-compare
-Wsign-conversion
-Wtautological-unsigned-zero-compare
(This specific CL focuses on miscellaneous non-test source files.)
Bug: chromium:1292951
PiperOrigin-RevId: 473054605
Change-Id: Ifd7b24966613ca915511a3a607095508068200b8
In some cases we can do a bit better by using
std::min(std::numeric_limits<size_type>::max() / 2, allocator<T>::max_size())
They may help in some cases, particularly on 32-bit platforms.
PiperOrigin-RevId: 471846886
Change-Id: I5bd63de5dd8aec3de6530a33d8904dd6e9bd015e
The problem is that the underlying storage adds redzone after the actual data so from compiler's perspective it looks like a valid memory. In the outlined version the memory is returned in call.i.i with unknown size so the access check can't be removed. The workaround is to always outline the call to InitializeData for ASAN builds.
Outlined version:
%call.i.i = call noundef i32* @absl::FixedArray<int, 4ul, std::__u::allocator<int> >::Storage::InitializeData()(...), !dbg !28
store i32* %call.i.i, i32** %data_.i.i, align 8, !dbg !27
%arrayidx = getelementptr inbounds i32, i32* %call.i.i, i64 5, !dbg !29
%24 = bitcast i32* %arrayidx to i8*, !dbg !29
call void @llvm.asan.check.memaccess(i8* %24, i32 36), !dbg !29
store i32 0, i32* %arrayidx, align 4, !dbg !29
Inlined version:
%arrayidx = getelementptr inbounds %"class.absl::FixedArray", %"class.absl::FixedArray"* %7, i64 0, i32 0, i32 0, i32 1, i64 20, !dbg !40
%27 = bitcast i8* %arrayidx to i32*, !dbg !40
>>>>>>>>>>>>>>> call to @llvm.asan.check.memaccess removed <<<<<<<<<<<<<<
store i32 0, i32* %27, align 4, !dbg !40
Workaround for ASAN stack safety analysis problem with FixedArray container annotations.
PiperOrigin-RevId: 471583635
Change-Id: I0d74eed5782a1cbd340ca4aca1bce71b63b06d43
Addresses failures with the following, in some files:
-Wshorten-64-to-32
-Wimplicit-int-conversion
-Wsign-compare
-Wsign-conversion
-Wtautological-unsigned-zero-compare
(This specific CL focuses on .cc files in */internal/.)
Bug: chromium:1292951
PiperOrigin-RevId: 471561809
Change-Id: I7abd6d83706f5ca135f1ce3458192a498a6280b9
Addresses failures with the following, in some files:
-Wshorten-64-to-32
-Wimplicit-int-conversion
-Wsign-compare
-Wsign-conversion
-Wtautological-unsigned-zero-compare
(This specific CL focuses on .cc files in */internal/.)
Bug: chromium:1292951
PiperOrigin-RevId: 471549854
Change-Id: Id685d0e4666212926f4e001b8ef4930b6a33a4cc
Corrects the computation of max_size(), so that it accounts for the
size of the objects.
PiperOrigin-RevId: 471343778
Change-Id: I68e222cefaa0295b8d8c38d00308a29df4165e81
Addresses failures with the following, in some files:
-Wshorten-64-to-32
-Wimplicit-int-conversion
-Wsign-compare
-Wsign-conversion
-Wtautological-unsigned-zero-compare
(This specific CL focuses on .h and win32 .inc files.)
Bug: chromium:1292951
PiperOrigin-RevId: 463835431
Change-Id: If8e5f7f651d5cd96035e23e4623bdb08a7fedabe
This ensures that emmintrin.h is included with clang-cl. Otherwise, errors like
this occur:
.../absl/container/internal/raw_hash_set.h(536,8): error: unknown type name '__m128i'
inline __m128i _mm_cmpgt_epi8_fixed(__m128i a, __m128i b) {
This aligns the include ifdef guards and the guards that use the provided APIs.
The includes are also reordered, so they appear after the config header which
provides the internal macro values.
PiperOrigin-RevId: 456530271
Change-Id: I86dfd0022fd06fe7aa132138ec4d1bd14a86ba84
This change introduces the symbol
ABSL_INTERNAL_NEED_REDUNDANT_CONSTEXPR_DECL
to guard redundant declarations of static constexpr data
members that are needed prior to C++17.
This change also introduces the symbol
ABSL_INTERNAL_CPLUSPLUS_LANG, which is supposed to be set
to the same value as __cplusplus, except it uses _MSVC_LANG
on MSVC so that the value is correct on MSVC.
Neither of these new symbols should be used outside of Abseil.
Fixes#1191
PiperOrigin-RevId: 453923908
Change-Id: I1316c52c19fa0c168b93cced0c817e4cb7c9c862
Avoid using value move/swap and delete those functions from slot_policy types. There was only one use of params_type::move in `erase`.
PiperOrigin-RevId: 453237739
Change-Id: Ie81c6dba6c4db34e97a067d2c0defcded8044a5a
https://pastebin.com/CmnzwUFN
The key idea is to avoid using 16 byte NEON and use 8 byte NEON which has lower latency for BitMask::Match. Even though 16 byte NEON achieves higher throughput, in SwissMap it's very important to catch these Matches with low latency as probing on average happens at most once.
I also introduced NonIterableMask as ARM has really great cbnz instructions and additional AND on scalar mask had 1 extra latency cycle
PiperOrigin-RevId: 453216147
Change-Id: I842c50d323954f8383ae156491232ced55aacb78
We were using `init_type`s for temp values that we would move into slots, but in this case, we need to have actual slots. We use node handles for managing slots outside of nodes.
Also, in btree::copy_or_move_values_in_order, pass the slots from the iterators rather than references to values. This allows for moving from map keys instead of copying for standard layout types.
In the test, fix a couple of ClangTidy warnings from missing includes and calling `new` instead of `make_unique`.
PiperOrigin-RevId: 452062967
Change-Id: I870e89ae1aa5b3cfa62ae6e75b73ffc3d52e731c
Benchmarks: https://pastebin.com/tZ7dr67W. Works well especially on smaller ranges.
After a week on spending optimizing NEON SIMD where I almost managed to make hash tables work with NEON SIMD without performance hits (still 1 cycle to optimize and I gave up a little), I found an interesting optimization for aarch64 to use cls instruction (count leading sign bits).
The loop has a property that ctrl_ group is not matched against count when the first slot is empty or deleted.
```
void skip_empty_or_deleted() {
while (IsEmptyOrDeleted(*ctrl_)) {
uint32_t shift = Group{ctrl_}.CountLeadingEmptyOrDeleted();
ctrl_ += shift;
slot_ += shift;
}
...
}
```
However, `kEmpty` and `kDeleted` have format of `1xxxxxx0` and `~ctrl & (ctrl >> 7)` always sets the lowest bit to 1.
In naive implementation, it does +1 to start counting zero bits, however, in aarch64 we may start counting one bits immediately. This saves 1 cycle and 5% of iteration performance.
Then it becomes hard to find a supported and sustainable C++ version of it.
`__clsll` is not supported by GCC and was supported only since clang 8, `__builtin_clrsb` is not producing optimal codegen for clang. `__rbit` is not supported by GCC and there is no intrinsic to do that, however, in clang we have `__builtin_bitreverse{32,64}`. For now I decided to enable this only for clang, only if they have appropriate builtins.
PiperOrigin-RevId: 451168570
Change-Id: I7e9256a60aecdc88ced4e6eb15ebc257281b6664
Add a new (internal) feature test macro to detect whether the wrappers are no-ops on a given platform.
Note that one-arg __builtin_prefetch(x) is equivalent to __builtin_prefetch(x, 0, 3), per `man BUILTIN_PREFETCH(3)` and gcc docs.
PiperOrigin-RevId: 449508660
Change-Id: I144e750205eec0c956d8dd62bc72e10bdb87c4f7
The old analysis viewed it as birthday attack, which asks how often
there are multiple values in the probe same probe sequence with the
same H2. In their own words, this analysis "breaks down" at around `n
= 12`.
Instead we can answer a simpler question, which is, if a probe
sequence examines `k` objects that are not what we are looking for,
what's the number of calls `==`? The expectation is simply `k/128`.
PiperOrigin-RevId: 446518063
Change-Id: Ie879bd4f6c97979822bc9d550b9e2503b1418c78
--
3d018c03a34bf273a4b24b3584ed77f0a6d21686 by Abseil Team <absl-team@google.com>:
Fix a spelling typo (s/boundries/boundaries).
PiperOrigin-RevId: 442041877
Change-Id: I608020697d37b85316bb9a0838e4b457659c926c
--
518b8119e51db24ce7fb0fd2fe537ec43825c3e6 by Dino Radakovic <dinor@google.com>:
absl/types/internal/variant: Make include guard uppercase
https://google.github.io/styleguide/cppguide.html#The__define_Guard
PiperOrigin-RevId: 441911692
Change-Id: I9837dd07f20204d8253f20627b0917a34dc21825
--
b91696c38310a7cae8c1ea9e2d479495f5dc3f69 by Greg Falcon <gfalcon@google.com>:
Add an internal-only API to wrap __builtin_prefetch() if available.
This private API is intended for future use by the Abseil implementation. Like any internal-namespaced function, it may be changed or removed at any time.
PiperOrigin-RevId: 441894616
Change-Id: Iaa48bd4680b373f4a0d5afab0cb35e2a1908595f
--
0f01e8b0551a662e02dff60840c54320f987315f by Derek Mauro <dmauro@google.com>:
C++20: Use the standard `constinit` keyword for `ABSL_CONST_INIT` when available
PiperOrigin-RevId: 441778874
Change-Id: I70c616469752ff23b326b1c615437599f42cc6aa
GitOrigin-RevId: 3d018c03a34bf273a4b24b3584ed77f0a6d21686
--
fb671efb2a70f452f17a884b17cf18817b977a8f by Abseil Team <absl-team@google.com>:
Remove extra semicolon in ABSL_INTERNAL_ASSERT_IS_FULL macro
To fix compilation when empty statement warning is treated as error.
PiperOrigin-RevId: 438342663
Change-Id: I3067fbeffa2691888f37554e88f229f24fb55ecc
--
a58c9396f1d88d11347aed36ef2e1b633071363c by Martijn Vels <mvels@google.com>:
Fix kMaxHeight bounds to kMaxDepth for CordrepBtreeNavigator
Added unit test (confirmed failure mode with old code) and extra assertion in the implementation.
PiperOrigin-RevId: 438327463
Change-Id: I32242c86b0c879b8a42cb9a92075e537d588e09f
--
f348e85dbfc9187ef59085fa2b999374f1670338 by Jorge Gorbe Moya <jgorbe@google.com>:
Make the flags enum in `RefcountAndFlags` a named enum to workaround an lldb
issue (https://github.com/llvm/llvm-project/issues/54602).
PiperOrigin-RevId: 438146097
Change-Id: Ibc2ee26489d99de515a779a903b6458dd0befef7
--
a960a3e9fb2a2e3418f806178e73d8566b78bc85 by Gennadiy Rozental <rogeeff@google.com>:
Introduce support for std::optional<T>/absl::optional<T> flag types.
PiperOrigin-RevId: 438129500
Change-Id: I3d925c0a7f9ce9f857277fac3b0bf664ccd3a95c
GitOrigin-RevId: fb671efb2a70f452f17a884b17cf18817b977a8f
--
291f7ef542f73e4801ab5108014bc02344ef31df by Derek Mauro <dmauro@google.com>:
Internal change
PiperOrigin-RevId: 437835981
Change-Id: I42fd92e74903894533ac9984d7f622e3ba20f468
--
2e8caf1a57c50b518e05b4bca48e4fe1bb19af82 by Andy Getzendanner <durandal@google.com>:
Internal change
PiperOrigin-RevId: 437832673
Change-Id: I61b35089418d01a54cecf161b254b68252bebff3
--
b927482ccc399f7e337b60582988b914d9946e4e by Derek Mauro <dmauro@google.com>:
Simplify endian intrinsics for modern compilers
All modern compilers have either __builtin_bswapN (gcc, clang) or
_byteswap_TYPE (MSVC). The other intrinsic definitions are no longer
necessary.
PiperOrigin-RevId: 437772295
Change-Id: Ifb3d88ba24b9097f87ceb202272b36d2f5e5117f
--
b6782a2247a16d5c14706a74ec577c19963d9f97 by Derek Mauro <dmauro@google.com>:
Internal change
PiperOrigin-RevId: 437373174
Change-Id: I0f77e1780dee90d7a3c32a08d96c4aeb624a57b4
--
a53e0c724e37b0b01515a99bd25394b8e21ffdfc by Derek Mauro <dmauro@google.com>:
Unify detection of SSE2 and SSSE3 instruction sets and
include the proper headers
Fix the intrinsic implementation of FastHexToBufferZeroPad16 in
numbers.h which only relies on SSSE3, not SSE 4.2.
https://godbolt.org/z/Pf5bn1Yv9Closes#639
PiperOrigin-RevId: 437286940
Change-Id: Ic97948399b61b91e9c0bccd09313b795b904d714
--
f173f597cb2a75ef2a989f45a496334b85e6f40d by Abseil Team <absl-team@google.com>:
Change assertion function to enable clearer error messages.
PiperOrigin-RevId: 437227057
Change-Id: If420d2f63b51feef6648762f344d5be012cd9c85
GitOrigin-RevId: 291f7ef542f73e4801ab5108014bc02344ef31df
--
8c9dd24a6fbf9ed10ae81f9fa0bc2168558a9700 by Abseil Team <absl-team@google.com>:
Improve WebAssembly detection when using Bazel.
Unfortunately, the --cpu values are not standardized, and both
--cpu=wasm and --cpu=wasm32 are used in the wild. Most notably,
Emscripten's Bazel rules use --cpu=wasm, which was missing.
While there, add support for @platforms//cpu:{wasm32,wasm64}.
This change adds a dependency on @bazel_skylib, which requires
adding the following http_archive() rule to the WORKSPACE file:
http_archive(
name = "bazel_skylib",
urls = ["https://github.com/bazelbuild/bazel-skylib/releases/download/1.2.1/bazel-skylib-1.2.1.tar.gz"],
sha256 = "f7be3474d42aae265405a592bb7da8e171919d74c16f082a5457840f06054728",
)
PiperOrigin-RevId: 436815546
Change-Id: I4e1946070c6964abb12259f25a546f2d24e0992a
--
59514589043d9b0734a01f7aa7bc354f5b495eab by Abseil Team <absl-team@google.com>:
Fix some typos that slipped through.
PiperOrigin-RevId: 436777566
Change-Id: Ibf5c54e2671c749dc87d2bd5d36dcd220ce347d4
GitOrigin-RevId: 8c9dd24a6fbf9ed10ae81f9fa0bc2168558a9700
--
199bdbf9ad253b216ed6c6386bf69bc706330204 by Abseil Team <absl-team@google.com>:
Import of CCTZ from GitHub.
PiperOrigin-RevId: 435425408
Change-Id: I502d21ca5771d14e24fe41b28dc562ea6274996d
--
245a761e96f53857cef9c2359eac9d4bbcfdfd47 by Abseil Team <absl-team@google.com>:
Fix a -Wsign-compare -Wconversion problem in absl::btree discovered while getting cachelib to compile with those flags enabled.
PiperOrigin-RevId: 435419108
Change-Id: I0a9fa53d0163f678dde8960410315a5bc8445c72
GitOrigin-RevId: 199bdbf9ad253b216ed6c6386bf69bc706330204
Evan Brown
Abseil Team
Derek Mauro
Tom Rybka
Jose Renau
Greg Falcon