26bc68c4af
Added readability-duplicate-include ( #31608 )
...
* Added readability-duplicate-include
* Fix
* format
2 years ago
54dd7563c2
Fix import ( #31602 )
2 years ago
504d49dc61
A basic set of EventEngine listener tests ( #31540 )
...
* A basic set of EventEngine listener tests
* review comments
* replace expect with asserts
* review comments
2 years ago
90d8754b0e
Sanitizer auto fix ( #31582 )
2 years ago
249645608e
Reland posix event engine tcp client ( #31563 )
...
* Revert "Revert "Posix event engine tcp client (#31074 )" (#31452 )"
This reverts commit d760c1fef7
2 years ago
50fbe51238
Fix IWYU with clang-15 ( #31556 )
...
* Fix iwyu
* iwyu fix
* more fixed
* Build fix
* more fix
* More fixes 3
2 years ago
7eb5fb5a25
Util functions to help with posix event engine listener implementation ( #31435 )
...
* Util functions to help with posix engine listener implementation
* sanity
* update comments in posix_engine_listener_utils.h
* review comments
* iwyu
* revert prev commit
* iwyu
* update build
* regenerate projects
* minor fixes
* comments
2 years ago
2820734f47
Reland: "[EventEngine] Refactor TimerManager to leverage a shared ThreadPool ( #31392 )" ( #31482 )
...
* Revert "Revert "[EventEngine] Refactor TimerManager to leverage a shared ThreadPool (#31392 )" (#31425 )"
This reverts commit 1cbf6d64ae
2 years ago
d760c1fef7
Revert "Posix event engine tcp client ( #31074 )" ( #31452 )
...
This reverts commit 390ae5d412
2 years ago
390ae5d412
Posix event engine tcp client ( #31074 )
...
* removing EndpointConfig::Get method
* Automated change: Fix sanity tests
* cleanup
* fix
* fix review comments
* review comments
* cleanup
* update comments
* fix
* cleanup
* update comments
* Automated change: Fix sanity tests
* fix misleading comments
* bug fixes
* fix
* fix
* start
* revert some changes for bug fixes and allow spurious wakeups for poll based poller
* sanity
* fix
* minor fix
* review comments
* fix
* comment
* remove re-defined function
* fix review comments
* fix windows iocp build issue due to removed function
* change Milliseconds return type
* fix
* remove header
* regenerate projects
* fix sanity
* code
* fix sanity
* Automated change: Fix sanity tests
* Forking iomgr internal_errqueue defines for posix event engine
* fix BUILD file
* start
* update
* sanity
* regenerate_projects
* Automated change: Fix sanity tests
* rename
* review comments
* update
* review comments
* Automated change: Fix sanity tests
* fix BUILD
* add no_windows tag
* update
* update
* remove debug statements
* delete unused file
* update
* fix BUILD
* regenerate projects
* update
* fix BUILD
* update
* fix sanity
* minor additions to configure global tcp user timeout values
* cleanup
* fix build syntax
* start
* update
* fix
* change poller interface for posix event engine
* update event_poller_posix_test
* fix build error
* Add more unit tests
* windows ee changes
* update comment
* remove unused deps
* fix msan issue
* add comment
* adding rcv lowat helper to socket wrapper
* fix sanity
* review comments
* fix build issue
* fix sanity
* sanity
* iwyu sanity
* fix build
* merge conflict
* Automated change: Fix sanity tests
* update
* update
* fix macos issue
* fix msan and ubsan issues
* fix macos build issue
* comment
* review comments
* remove unused dep
* sanity
* merge conflict
* mac build issue
* regenerate projects
* build issue
* fix test issues
* add test
* fix typo and address review comments
* review comments
* Automated change: Fix sanity tests
* update
* merge issue
* add a test to check connect timeouts and cancellation
* regenerate projects
* sanity
* remove unnecessary code
* Add utils to create and prepare socket for tcp client
* some fixes
* fixes
* review comments
* sanity
* fix build issue
* review comments
* add thread annotations
* MacOS fix unused variable
* add some more #endif comments
* fixes
* update
* more fixes
* regenerate projects
* sanity
* some fixes
* more fixes
* rm hdr file
* macos fixes
* typo
* more build issues
* macos crashes
* windows build
* adding Poller::Work to event engine constructor
* update client_test
* iwyu
* iwyu
* regenerate projects
* cleanup
* cleanup
* regenerate projects
* regenerate projects
* fix
* fix
* merge conflict
* comments
* add experiments flag
* fix
* newline
* newline
* iwyu
* fix
* fix
* remove some ifdefs
* fix tsan issue in bm_event_engine_run
* sanity
* sanity
* Add a quiesce to threadpool
* fix
* Automated change: Fix sanity tests
* fixes
* typo
* comments
* fix build
* fixes
* macos fixes
* more macos fixes
* handle shutdown better
* updated poller kicked api contract and fix bug
* iwyu
* fix sanity
Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
Co-authored-by: Craig Tiller <ctiller@google.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
1cbf6d64ae
Revert "[EventEngine] Refactor ThreadManager to leverage a shared ThreadPool ( #31392 )" ( #31425 )
...
* Revert "[EventEngine] Refactor ThreadManager to leverage a shared ThreadPool (#31392 )"
This reverts commit 66df2c646a
2 years ago
5f6c357e74
Move some targets from //: to //src/core: ( #31402 )
...
* temp
* temp
* temp
* top BUILD builds
* everything builds
* buildify
* temp fix_build_deps change
* fix gpr_atm
* additional changes
* fix build files
* nofixdeps
* Generate projects
* json_args
* fix context
* Generate projects
* Sprinkle some avoid_dep tags
* Run fix_build_deps
* Remove additional avoid_dep
* Revert some targets
* grpc_mock_cel
2 years ago
66df2c646a
[EventEngine] Refactor ThreadManager to leverage a shared ThreadPool ( #31392 )
...
* Refactor ThreadManager to leverage a shared ThreadPool
This is a step towards a work-stealing thread pool implementation:
unifying thread pools so that work stealing affects timer callbacks as
well.
A subsequent step will expose the timeout logic so that the timer wakeup
and check can be triggered externally (by pollers, in the common case).
* fix atomic uint64_t type missing on some platforms
* sanitize + platform fixes
* ->quiesce
* shut down the timer manager to release the main thread
* roll back atomics
* use a dedicated thread for timer_manager to prevent local execution (work stealing)
* drain the pools after timer manager tests; sanitize
* iwyu
* reintroduce fork handling
* sanitize
* fix
2 years ago
e8c30b487f
Invoking Poller::Work in posix event engine constructor ( #31128 )
...
* adding Poller::Work to event engine constructor
* iwyu
* iwyu
* cleanup
* regenerate projects
* fix
* comments
* add experiments flag
* fix
* newline
* newline
* iwyu
* remove some ifdefs
* fix tsan issue in bm_event_engine_run
* Add a quiesce to threadpool
* fix
* Automated change: Fix sanity tests
* disable posix_event_engine_test on windows
Co-authored-by: Craig Tiller <ctiller@google.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
b957f45df6
Re-enabled and Fix WindowsEventEngine test suite ( #31393 )
...
This test was disabled (assumed inadvertently) in #30952 , then the test
itself was silently broken in #31265 .
2 years ago
0a8948ca2a
Add a quiesce to threadpool ( #31380 )
...
* Add a quiesce to threadpool
* fix
* Automated change: Fix sanity tests
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
e8b416e654
Add a BUILD file in src/core ( #31294 )
...
* Initial
* Modifying fix_build_deps.py
* Move event_engine_common as a better signal
* fixes
* fixes
* Regenerate projects
* Export headers for event_engine_common
* Regenerate projects
* Fix build files
* Some cleanup
* Reviewer comments
* Remove unnecessary blank line
* Remove unnecessary visibility exception for event_engine_common
2 years ago
f2c4795f4f
core tests: absl::make_unique -> std::make_unique ( #31361 )
...
* core tests: absl::make_unique -> std::make_unique
* iwyu
2 years ago
afa3a6c890
[event_engine] Fix for the EE/iomgr shutdown ordering problem ( #31265 )
...
* Remove ResetDefaultEventEngine
Now that it is a weak_ptr, there's no need to explicitly reset it. When
the tracked shared_ptr is deleted, the weak_ptr will fail to lock, and a
new default EventEngine will be created.
* forget existing engine with FactoryReset
* init/shutdown in event engine for now
* fix
* fix
* fix windows deadlock
* Automated change: Fix sanity tests
* fix
* better windows fix
Co-authored-by: AJ Heller <hork@google.com>
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
2d00d50c59
[event_engine] Improve scaling in threadpool ( #31234 )
...
* fixes
* Automated change: Fix sanity tests
* fix
* fix
* fix
* fix
* fixes
* fixes
* fix
* fix-win
* fix iwyu
* fix
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
cc29a3a1d2
Add testonly attribute to mock_event_engine ( #31235 )
2 years ago
0057598fed
[sleep] Add a test for robustness with errant event engines ( #30622 )
...
* [sleep] Make this robust against poorly implemented event engines
* Automated change: Fix sanity tests
* Reland x2: Make GetDefaultEventEngine return a shared_ptr
* x
* [sleep] Add test for sleep robustness with bad event engines
* Automated change: Fix sanity tests
* fix
* fix
* Automated change: Fix sanity tests
* Update sleep_test.cc
* Update mock_event_engine.h
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
Co-authored-by: AJ Heller <hork@google.com>
2 years ago
7ecca97c05
[flaky] Reduce test size ( #31210 )
2 years ago
6e15936d85
Make event engine client test more robust ( #31187 )
...
* Minor event engine related cleanup
* make client_test more robust
* cleanup
* regenerate_projects
* review comments
* iwyu
* review comments
* typo
2 years ago
e7ad460c7b
Event Engine related minor cleanup ( #31188 )
...
* Minor event engine related cleanup
* regenerate_projects
* fix test
* iwyu
2 years ago
db5fe72da9
[tsan] Threadsafe strerror implementation ( #31167 )
...
* fix
* spread
* fix
* Automated change: Fix sanity tests
* fix
* remove unnecessary .c_strs
* lashes
* fix
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
7655858e93
Reland tcp endpoints for posix event engine ( #31150 )
...
* Revert "Revert "Tcp endpoint implementation for posix event engine. (#30559 )" (#31134 )"
This reverts commit 5fa37322cb
2 years ago
e1e1f6181f
Util to load file contents into grpc_core::Slice ( #31176 )
...
* Util to load file contents into grpc_core::Slice
* Add test
* iwyu
* iwyu again
2 years ago
6cac641c64
EventEngineFactoryReset - remove custom factory and reset default engine ( #30554 )
...
* EventEngineFactoryReset - remove custom factory and reset default engine
* fix
* Automated change: Fix sanity tests
* unused parameter
* fix
* fix
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2 years ago
2ec220a6b1
[sanity] Fix it ( #31163 )
...
* fix sanity
* fix sanity
2 years ago
8174a75079
[iwyu] Add more test directories ( #30952 )
...
* [iwyu] Add more test directories
* fix
* fix
* fix
* fix
* fix
* Automated change: Fix sanity tests
* fix
* iwyu
* Automated change: Fix sanity tests
* fix
* fix
* fix
* fix
* Automated change: Fix sanity tests
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
9c98d67d7a
[c++14] absl::make_unique --> std::make_unique ( #31159 )
...
* [c++14] absl::make_unique --> std::make_unique
* fix
* fix
2 years ago
2ee2c91c92
Reland x2: Make GetDefaultEventEngine return a shared_ptr ( #30619 )
...
* Reland x2: Make GetDefaultEventEngine return a shared_ptr
* remove thread leak from NativeDNSResolver
This is not going to work for resolvers that support cancellation.
* give resolvers bounded lifetimes
Some resolver own EventEngines. EventEngines cannot run off the end of
the process since they have unjoined threads (problematic in a small set
of environments). This gives resolvers bounded lifetimes, and allows
replacement of resolvers without ASAN issues of deleting resolvers in
active use (occurs in tests).
* fix
* fix windows
* fix surface init test
* fix
* sanitize
* use after move
* the test must wait for the callback to be destroyed
* windows fix: delete the resolver on iomgr shutdown, not before
* Make TimerManager threads non-joinable
On gRPC shutdown, any unjoined TimerManager threads will cause TSAN to
detect thread leaks. This fix resolves issues I saw in end2end test
shutdown in another PR, where a single timer manager thread was always
alive after the test ended.
The long-term solution is to integrate the new ThreadPool here, but this
unblocks me for now.
* backport fix
* fix
* shared_ptr<EventEngine> in EventEngine benchmarks
2 years ago
298ace8cbf
small test cleanup ( #31137 )
2 years ago
5fa37322cb
Revert "Tcp endpoint implementation for posix event engine. ( #30559 )" ( #31134 )
...
This reverts commit a81391d931
2 years ago
a81391d931
Tcp endpoint implementation for posix event engine. ( #30559 )
...
* convert tcp code to use EndpointConfig and successfully compile
* regenerate projects
* copying config options used in tcp code to create map backed endpoint configs
* fix minor build issues
* fix includes in tests
* fix some build and test issues
* modifying endpoint config definition from an abstract interface into a concrete implementation
* minor fix
* add unreachable code check
* fix some windows build issues
* fix more build issues
* adding a grpc_tcp_generic_options struct to hold data extracted from EndpointConfig
* code formatting
* simplify
* fix sanity checks
* fix windows issues
* some posix fixes
* disabling copy constructor and copy assignment operator for endpoint config
* fix sanity checks
* fix syntax error
* fix weird formatting
* fix build issue
* fix review comments
* reverting un-necessary change
* remove channel args completely from windows impl since it is unused
* remove unused headers and fix usage in endpoint_pair_windows.cc
* cleanup
* cleanup
* fix some initialization issues
* re-generate projects
* removing some files
* fix ifdef for windows
* fixing windows build issue due to improper header file include
* Automated change: Fix sanity tests
* start
* regenerate-projects
* delete some files
* Automated change: Fix sanity tests
* review comments
* add comment
* Migrating posix event pollers to use new event poller interface
* add inline attributes
* Automated change: Fix sanity tests
* remove ref/unref from iomgr engine closure ad add custom closure types internal to the pollers
* updating time util usage
* use unique_ptrs
* update comments
* Automated change: Fix sanity tests
* review comments
* rename GetPointer to GetVoidPointer
* cleanup
* Automated change: Fix sanity tests
* removing EndpointConfig::Get method
* Automated change: Fix sanity tests
* cleanup
* fix
* fix review comments
* review comments
* cleanup
* update comments
* fix
* cleanup
* update comments
* Automated change: Fix sanity tests
* fix misleading comments
* bug fixes
* fix
* fix
* start
* revert some changes for bug fixes and allow spurious wakeups for poll based poller
* sanity
* fix
* minor fix
* review comments
* fix
* comment
* remove re-defined function
* fix review comments
* fix windows iocp build issue due to removed function
* change Milliseconds return type
* fix
* remove header
* regenerate projects
* fix sanity
* code
* fix sanity
* Automated change: Fix sanity tests
* Forking iomgr internal_errqueue defines for posix event engine
* fix BUILD file
* start
* update
* sanity
* regenerate_projects
* Automated change: Fix sanity tests
* rename
* review comments
* update
* review comments
* Automated change: Fix sanity tests
* fix BUILD
* add no_windows tag
* update
* update
* remove debug statements
* delete unused file
* update
* fix BUILD
* regenerate projects
* update
* fix BUILD
* update
* fix sanity
* minor additions to configure global tcp user timeout values
* cleanup
* fix build syntax
* change poller interface for posix event engine
* update event_poller_posix_test
* fix build error
* Add more unit tests
* windows ee changes
* update comment
* remove unused deps
* fix msan issue
* add comment
* adding rcv lowat helper to socket wrapper
* fix sanity
* review comments
* fix build issue
* fix sanity
* sanity
* iwyu sanity
* fix build
* merge conflict
* Automated change: Fix sanity tests
* update
* update
* fix macos issue
* fix msan and ubsan issues
* fix macos build issue
* comment
* review comments
* remove unused dep
* sanity
* merge conflict
* mac build issue
* regenerate projects
* build issue
* fix typo and address review comments
* review comments
* Automated change: Fix sanity tests
* update
* merge issue
* review comments
* sanity
* fix build issue
* review comments
* add thread annotations
* MacOS fix unused variable
* add some more #endif comments
* missing semi-colon
* Automated change: Fix sanity tests
* sanity
* sanity again
* sanity try again
* sanity again
* try again with no_include pragma
Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2 years ago
fb14fdf0e0
Performant thread-safe Work Queue ( #30821 )
...
* WorkQueue
* weaken the large obj stress test for Windows; documentation
* update comment
* Add WorkQueue microbenchmark. Results below ...
------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1 297 ns 297 ns 2343500 items_per_second=3.3679M/s
BM_WorkQueueIntptrPopFront/8 7022 ns 7020 ns 99356 items_per_second=1.13956M/s
BM_WorkQueueIntptrPopFront/64 59606 ns 59590 ns 11770 items_per_second=1074k/s
BM_WorkQueueIntptrPopFront/512 477867 ns 477748 ns 1469 items_per_second=1071.7k/s
BM_WorkQueueIntptrPopFront/4096 3815786 ns 3814925 ns 184 items_per_second=1073.68k/s
I0902 19:05:22.138022069 12 test_config.cc:194] TestEnvironment ends
================================================================================
* use int64_t for times. 0 performance change
------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
------------------------------------------------------------------------------------------
BM_WorkQueueIntptrPopFront/1 277 ns 277 ns 2450292 items_per_second=3.60967M/s
BM_WorkQueueIntptrPopFront/8 6718 ns 6716 ns 105497 items_per_second=1.19126M/s
BM_WorkQueueIntptrPopFront/64 56428 ns 56401 ns 12268 items_per_second=1.13474M/s
BM_WorkQueueIntptrPopFront/512 458953 ns 458817 ns 1550 items_per_second=1.11591M/s
BM_WorkQueueIntptrPopFront/4096 3686357 ns 3685120 ns 191 items_per_second=1.1115M/s
I0902 19:25:31.549382949 12 test_config.cc:194] TestEnvironment ends
================================================================================
* add PopBack tests: same performance profile exactly
* use Mutex instead of Spinlock
It's safer, and so far equally performant in benchmarks of opt builds
* add deque test for comparison. It is faster on all tests.
* Add sparsely-populated multi-threaded benchmarks.
* fix
* fix
* refactor to help thread safety analysis
* Specialize WorkQueue for Closure*s and AnyInvocables
* remove unused callback storage
* add single-threaded benchmark for closure vs invocable
* sanitize
* missing include
* move bm_work_queue to microbenchmarks so it isn't exported
* s/workqueue/work_queue/g
* use nullptr instead of optionals for popped closures
* reviewer test suggestion
* private things are private
* add a work_queue fuzzer
Ran for 10 minutes @ 42 jobs @ 42 workers. Zero failures.
Checked in a selection of 100 good seeds after merging the thousands of
results.
* fix
* fix header guards
* nuke the corpora
* feedback
* sanitize
* Timestamp::Now
* fix
* fuzzers do not work on windows
* windows does not like multithreaded benchmark tests
2 years ago
c0e5e35c7a
Replace EventEngine::Promise with grpc_core::Notification ( #31027 )
...
* Replace EventEngine::Promise with grpc_core::Notification
* remove promise.h
* Automated change: Fix sanity tests
* fix windows
* fix iocp
* fix client_test
Co-authored-by: drfloob <drfloob@users.noreply.github.com>
2 years ago
1ad4bbe78e
Revert "fix ( #31018 )" ( #31021 )
...
This reverts commit 2e6f268dfc
2 years ago
4420d11ee0
[event-engine] Throttle thread starting in thread pool ( #31075 )
...
* [event_engine] Thread pool thread start throttling
* add test
* review feedback
* fix
2 years ago
2e6f268dfc
fix ( #31018 )
2 years ago
6b95573959
Reland: [event_engine] Thread pool that can handle deletion in a callback ( #30996 )
...
* Revert "Revert "Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback"" (#30973 )" (#30995 )"
This reverts commit dffdd8c083
2 years ago
96f5cddb5f
[time] Introduce time sources ( #30815 )
...
* [time] Introduce time sources
* make import trivial
* Automated change: Fix sanity tests
* fix
* Automated change: Fix sanity tests
* fix
* Automated change: Fix sanity tests
* Automated change: Fix sanity tests
* review feedback
* fix
* Automated change: Fix sanity tests
* ios fix
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
4e7f0e1eac
Update to the event engine poller interface ( #30828 )
...
* change poller interface for posix event engine
* update event_poller_posix_test
* windows ee changes
* update comment
* remove unused deps
* review comments
* fix build issue
* fix sanity
* fix build
* Automated change: Fix sanity tests
* review comments
* remove unused dep
* sanity
* fix typo and address review comments
* review comments
* Automated change: Fix sanity tests
* fix typo in comment
Co-authored-by: Vignesh2208 <Vignesh2208@users.noreply.github.com>
2 years ago
dffdd8c083
Revert "Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback"" ( #30973 )" ( #30995 )
...
This reverts commit fed749d100
2 years ago
888c64adf8
Fix the build script ( #30985 )
...
* Fix the build script
* Set -ex
* fix unsigned int comparisons and ignored return values
Co-authored-by: AJ Heller <hork@google.com>
2 years ago
fed749d100
Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback"" ( #30973 )
...
* Revert "Revert "[event_engine] Thread pool that can handle deletion in a callback (#30763 )" (#30972 )"
This reverts commit ccc787a020
2 years ago
14c3839749
[event_engine] More robust ee::Promise<> implementation ( #30977 )
...
* fix promise
* iocp
* iwyu
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
ccc787a020
Revert "[event_engine] Thread pool that can handle deletion in a callback ( #30763 )" ( #30972 )
...
This reverts commit f6e1cf1dc0
2 years ago
f6e1cf1dc0
[event_engine] Thread pool that can handle deletion in a callback ( #30763 )
...
* [event_engine] Thread pool that can handle deletion in a callback
* missed file
* Automated change: Fix sanity tests
* simplify memory model
* review feedback
* Automated change: Fix sanity tests
* detect-stuckness
* fix
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
Esun Kim
Yash Tibrewal
Vignesh Babu
Cheng-Yu Chung
AJ Heller
Craig Tiller
Mark D. Roth
Richard Belleville