abseil-cpp

387 Commits

9 Branches

22 Tags

14 MiB

Tag: Branch: Tree: c34c552a1a

Commit Graph

Author	SHA1	Message	Date
Abseil Team	9444b11e0c	absl: fix use-after-free in Mutex/CondVar Both Mutex and CondVar signal PerThreadSem/Waiter after satisfying the wait condition, as the result the waiting thread may return w/o waiting on the PerThreadSem/Waiter at all. If the waiting thread then exits, it currently destroys Waiter object. As the result Waiter::Post can be called on already destroyed object. PerThreadSem/Waiter must be type-stable after creation and must not be destroyed. The futex-based implementation is the only one that is not affected by the bug since there is effectively nothing to destroy (maybe only UBSan/ASan could complain about calling methods on a destroyed object). Here is the problematic sequence of events: 1: void Mutex::Block(PerThreadSynch s) { 2: while (s->state.load(std::memory_order_acquire) == PerThreadSynch::kQueued) { 3: if (!DecrementSynchSem(this, s, s->waitp->timeout)) { 4: PerThreadSynch Mutex::Wakeup(PerThreadSynch *w) { 5: ... 6: w->state.store(PerThreadSynch::kAvailable, std::memory_order_release); 7: IncrementSynchSem(this, w); 8: ... 9: } Consider line 6 is executed, then line 2 observes kAvailable and line 3 is not called. The thread executing Mutex::Block returns from the method, acquires the mutex, releases the mutex, exits and destroys PerThreadSem/Waiter. Now Mutex::Wakeup resumes and executes line 7 on the destroyed object. Boom! CondVar uses a similar pattern. Moreover the semaphore-based Waiter implementation is not even destruction-safe (the Waiter cannot be used to signal own destruction). So even if Mutex/CondVar would always pair Waiter::Post with Waiter::Wait before destroying PerThreadSem/Waiter, it would still be subject to use-after-free bug on the semaphore. PiperOrigin-RevId: 449159939 Change-Id: I497134fa8b6ce1294a422827c5f0de0e897cea31	3 years ago
Abseil Team	0b5af594fc	Export of internal Abseil changes -- 1a5831c2b4b85e0151b7952e47f4b80827937620 by Laramie Leavitt <lar@google.com>: Implement FuzzingBitGen, an adapter which allows existing randomized tests which use absl::BitGenRef to easily integrate with fuzz testing. I found myself implementing a similar option in our tensorstore project to fuzz test a storage layer and figured that it would be more useful as a common tool with defaults that take the non-random path. This is similar to the FuzzedDataProvider mechanism which generates random values from a fuzz string, and is used to generate fuzz test inputs, and internally it uses FuzzedDataProvider. The basic technique used here is to construct mocking lambdas for all of the absl mock distribution configurations, and forwarding the parameters to fuzzing-specific implementations that call into FuzzedDataProvider. The default paths for the distributions are either the bounds or a median value. PiperOrigin-RevId: 358432715 -- e7968538c5ef5cd0b9822dbeac0f659b5e7d49b3 by Derek Mauro <dmauro@google.com>: Give extern C symbols a unique name when the inline namespace is given. This partially addresses #851 PiperOrigin-RevId: 358403842 GitOrigin-RevId: 1a5831c2b4b85e0151b7952e47f4b80827937620 Change-Id: Id5ca0251498e390a8efa7210a17cc2cabb2c7dd8	4 years ago
Abseil Team	768eb2ca28	Export of internal Abseil changes -- f012012ef78234a6a4585321b67d7b7c92ebc266 by Laramie Leavitt <lar@google.com>: Slight restructuring of absl/random/internal randen implementation. Convert round-keys.inc into randen_round_keys.cc file. Consistently use a 128-bit pointer type for internal method parameters. This allows simpler pointer arithmetic in C++ & permits removal of some constants and casts. Remove some redundancy in comments & constexpr variables. Specifically, all references to Randen algorithm parameters use RandenTraits; duplication in RandenSlow removed. PiperOrigin-RevId: 312190313 -- dc8b42e054046741e9ed65335bfdface997c6063 by Abseil Team <absl-team@google.com>: Internal change. PiperOrigin-RevId: 312167304 -- f13d248fafaf206492c1362c3574031aea3abaf7 by Matthew Brown <matthewbr@google.com>: Cleanup StrFormat extensions a little. PiperOrigin-RevId: 312166336 -- 9d9117589667afe2332bb7ad42bc967ca7c54502 by Derek Mauro <dmauro@google.com>: Internal change PiperOrigin-RevId: 312105213 -- 9a12b9b3aa0e59b8ee6cf9408ed0029045543a9b by Abseil Team <absl-team@google.com>: Complete IGNORE_TYPE macro renaming. PiperOrigin-RevId: 311999699 -- 64756f20d61021d999bd0d4c15e9ad3857382f57 by Gennadiy Rozental <rogeeff@google.com>: Switch to fixed bytes specific default value. This fixes the Abseil Flags for big endian platforms. PiperOrigin-RevId: 311844448 -- bdbe6b5b29791dbc3816ada1828458b3010ff1e9 by Laramie Leavitt <lar@google.com>: Change many distribution tests to use pcg_engine as a deterministic source of entropy. It's reasonable to test that the BitGen itself has good entropy, however when testing the cross product of all random distributions x all the architecture variations x all submitted changes results in a large number of tests. In order to account for these failures while still using good entropy requires that our allowed sigma need to account for all of these independent tests. Our current sigma values are too restrictive, and we see a lot of failures, so we have to either relax the sigma values or convert some of the statistical tests to use deterministic values. This changelist does the latter. PiperOrigin-RevId: 311840096 GitOrigin-RevId: f012012ef78234a6a4585321b67d7b7c92ebc266 Change-Id: Ic84886f38ff30d7d72c126e9b63c9a61eb729a1a	5 years ago

Author

SHA1

Message

Date

Abseil Team

9444b11e0c

absl: fix use-after-free in Mutex/CondVar

Both Mutex and CondVar signal PerThreadSem/Waiter after satisfying the wait condition,
as the result the waiting thread may return w/o waiting on the
PerThreadSem/Waiter at all. If the waiting thread then exits, it currently
destroys Waiter object. As the result Waiter::Post can be called on
already destroyed object.
PerThreadSem/Waiter must be type-stable after creation and must not be destroyed.
The futex-based implementation is the only one that is not affected by the bug
since there is effectively nothing to destroy (maybe only UBSan/ASan
could complain about calling methods on a destroyed object).

Here is the problematic sequence of events:

1: void Mutex::Block(PerThreadSynch *s) {
2:   while (s->state.load(std::memory_order_acquire) == PerThreadSynch::kQueued) {
3:     if (!DecrementSynchSem(this, s, s->waitp->timeout)) {

4: PerThreadSynch *Mutex::Wakeup(PerThreadSynch *w) {
5:   ...
6:   w->state.store(PerThreadSynch::kAvailable, std::memory_order_release);
7:   IncrementSynchSem(this, w);
8:   ...
9: }

Consider line 6 is executed, then line 2 observes kAvailable and
line 3 is not called. The thread executing Mutex::Block returns from
the method, acquires the mutex, releases the mutex, exits and destroys
PerThreadSem/Waiter.
Now Mutex::Wakeup resumes and executes line 7 on the destroyed object. Boom!

CondVar uses a similar pattern.

Moreover the semaphore-based Waiter implementation is not even destruction-safe
(the Waiter cannot be used to signal own destruction). So even if Mutex/CondVar
would always pair Waiter::Post with Waiter::Wait before destroying PerThreadSem/Waiter,
it would still be subject to use-after-free bug on the semaphore.

PiperOrigin-RevId: 449159939
Change-Id: I497134fa8b6ce1294a422827c5f0de0e897cea31

3 years ago

Abseil Team

0b5af594fc

Export of internal Abseil changes

--
1a5831c2b4b85e0151b7952e47f4b80827937620 by Laramie Leavitt <lar@google.com>:

Implement FuzzingBitGen, an adapter which allows existing randomized tests which use absl::BitGenRef to easily integrate with fuzz testing.

I found myself implementing a similar option in our tensorstore project to fuzz test a storage layer and figured that it would be more useful as a common tool with defaults that take the non-random path.

This is similar to the FuzzedDataProvider mechanism which generates random values from a fuzz string, and is used to generate fuzz test inputs, and internally it uses FuzzedDataProvider.

The basic technique used here is to construct mocking lambdas for all of the absl mock distribution configurations, and forwarding the parameters to fuzzing-specific implementations that call into FuzzedDataProvider. The default paths for the distributions are either the bounds or a median value.

PiperOrigin-RevId: 358432715

--
e7968538c5ef5cd0b9822dbeac0f659b5e7d49b3 by Derek Mauro <dmauro@google.com>:

Give extern C symbols a unique name when the inline namespace
is given.

This partially addresses #851

PiperOrigin-RevId: 358403842
GitOrigin-RevId: 1a5831c2b4b85e0151b7952e47f4b80827937620
Change-Id: Id5ca0251498e390a8efa7210a17cc2cabb2c7dd8

4 years ago

Abseil Team

768eb2ca28

Export of internal Abseil changes

--
f012012ef78234a6a4585321b67d7b7c92ebc266 by Laramie Leavitt <lar@google.com>:

Slight restructuring of absl/random/internal randen implementation.

Convert round-keys.inc into randen_round_keys.cc file.

Consistently use a 128-bit pointer type for internal method parameters. This allows simpler pointer arithmetic in C++ & permits removal of some constants and casts.

Remove some redundancy in comments & constexpr variables. Specifically, all references to Randen algorithm parameters use RandenTraits; duplication in RandenSlow removed.

PiperOrigin-RevId: 312190313

--
dc8b42e054046741e9ed65335bfdface997c6063 by Abseil Team <absl-team@google.com>:

Internal change.

PiperOrigin-RevId: 312167304

--
f13d248fafaf206492c1362c3574031aea3abaf7 by Matthew Brown <matthewbr@google.com>:

Cleanup StrFormat extensions a little.

PiperOrigin-RevId: 312166336

--
9d9117589667afe2332bb7ad42bc967ca7c54502 by Derek Mauro <dmauro@google.com>:

Internal change

PiperOrigin-RevId: 312105213

--
9a12b9b3aa0e59b8ee6cf9408ed0029045543a9b by Abseil Team <absl-team@google.com>:

Complete IGNORE_TYPE macro renaming.

PiperOrigin-RevId: 311999699

--
64756f20d61021d999bd0d4c15e9ad3857382f57 by Gennadiy Rozental <rogeeff@google.com>:

Switch to fixed bytes specific default value.

This fixes the Abseil Flags for big endian platforms.

PiperOrigin-RevId: 311844448

--
bdbe6b5b29791dbc3816ada1828458b3010ff1e9 by Laramie Leavitt <lar@google.com>:

Change many distribution tests to use pcg_engine as a deterministic source of entropy.

It's reasonable to test that the BitGen itself has good entropy, however when testing the cross product of all random distributions x all the architecture variations x all submitted changes results in a large number of tests. In order to account for these failures while still using good entropy requires that our allowed sigma need to account for all of these independent tests.

Our current sigma values are too restrictive, and we see a lot of failures, so we have to either relax the sigma values or convert some of the statistical tests to use deterministic values.

This changelist does the latter.

PiperOrigin-RevId: 311840096
GitOrigin-RevId: f012012ef78234a6a4585321b67d7b7c92ebc266
Change-Id: Ic84886f38ff30d7d72c126e9b63c9a61eb729a1a

5 years ago

3 Commits (c34c552a1a547a47474f32f1b89de030aa4ff949)