175ccc3a90
Reland global config changes ( #32661 )
...
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
0c1797cd9f
Revert "[config] Move global config alongside core configuration" ( #32659 )
...
Reverts grpc/grpc#30788
(it breaks grpc_objc_bazel_test (see
https://github.com/grpc/grpc/pull/30788#issuecomment-1476372187 ) and
also seems to be breaking some other internal stuff).
2 years ago
b7a83305e6
[config] Move global config alongside core configuration ( #30788 )
...
This is a big rewrite of global config.
It does a few things, all somewhat intertwined:
1. centralize the list of configuration we have to a yaml file that can
be parsed, and code generated from it
2. add an initialization and a reset stage so that config vars can be
centrally accessed very quickly without the need for caching them
3. makes the syntax more C++ like (less macros!)
4. (optionally) adds absl flags to the OSS build
This first round of changes is intended to keep the system where it is
without major changes. We pick up absl flags to match internal code and
remove one point of deviation - but importantly continue to read from
the environment variables. In doing so we don't force absl flags on our
customers - it's possible to configure grpc without the flags - but
instead allow users that do use absl flags to configure grpc using that
mechanism. Importantly this lets internal customers configure grpc the
same everywhere.
Future changes along this path will be two-fold:
1. Move documentation generation into the code generation step, so that
within the source of truth yaml file we can find all documentation and
data about a configuration knob - eliminating the chance of forgetting
to document something in all the right places.
2. Provide fuzzing over configurations. Currently most config variables
get stashed in static constants across the codebase. To fuzz over these
we'd need a way to reset those cached values between fuzzing rounds,
something that is terrifically difficult right now, but with these
changes should simply be a reset on `ConfigVars`.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
---------
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
febed5121a
[EventEngine] Skip pathological iomgr test on Windows experiments ( #32569 )
...
With iomgr, this test is effectively rate limited by ExecCtx and the
single thread running pollset_work, which results in thousands of tiny
writes happening before every read. A small set of _synchronous_ 8k
reads then dominate the read-side of the test. This is an efficient
balance.
With the Windows EventEngine, the fully asynchronous, multi-threaded
reads and writes end up alternating roughly 1:1, meaning that a read
callback is executed for every tiny handful of bytes, tens of thousands
of times. Compared to the Posix EventEngine, without things like TCP_INQ
and/or recvmsg's timeout, I don't know of any great signal for how much
data can safely be received in a batch (e.g., we don't want to wait for
data that will never come, and we don't want to run callbacks for 2
bytes over and over again if we have KB in the pipe).
I believe the Windows EventEngine is WAI. I can significantly improve
this test performance by artificially slowing the reader down (adding a
>= 1ms sleep), but I believe that improves this use case to the
detriment of all others.
2 years ago
fa5a6c42a6
[EventEngine] Modify iomgr to allow creation of posix event engine listeners and server side endpoints ( #31928 )
...
* [WIP] EventEngine iomgr endpoint shims
* [WIP] EventEngine::Endpoint iomgr shims for the PosixEventEngine
* Util functions to help with posix engine listener implementation
* sanity
* update comments in posix_engine_listener_utils.h
* review comments
* iwyu
* revert prev commit
* iwyu
* update build
* update
* regenerate projects
* regenerate projects
* minor fixes
* update BUILD
* sanity
* update build
* regenerate projects
* fix unused parameter
* sanity
* update
* sanity
* regenerate_projects
* remove unused variable
* start
* update
* regenerate_projects
* sanity
* update
* fixes
* update
* regenerate_projects
* update
* fix sanity and msan failure
* more fixes
* build failure
* update
* fix
* sanity
* fixes
* update
* regenerate projects
* fix sanity
* review comments
* An EventEngine subclass to be implemented by all posix based event engines
* sanity
* comments
* update
* review comments
* re-word
* fix
* update
* review comments
* regenerate projects
* syntax fix
* add lock free event benchmark
* releasable mutex lock
* fix build isue
* update
* start
* regenerate projects
* update
* fix
* windows build
* update
* windows portability issue
* update
* update
* update
* update
* format
* update
* update
* start
* Update tcp server interface to move on_accept_cb to create method
* update
* start
* update
* update
* update
* update
* update
* update
* update
* update
* sanity
* update
* update
* update
* windows build
* fix msan
* fix sanity
* regenerate projects
* update
* iwyu
* Fix resolved address length related bugs in tcp_socket_utils and listener_utils
* iwyu
* cleanup
* cleanup src/core/lib/event_engine/tcp_socket_utils.cc
* iwyu
* fix
* regenerate projects
* fix sanity
* re-write endpoint shim
* more re-write
* cleanup
* update
* regenerate projects
* review comments
* build issue
* more build issue fixes plus adding event_engine_trace
* even more build issue fixes
* iwyu
* add static_cast
* fix sanity
* update
* update
* sanity
* fix
* Fix
* Review comments
* fix
* iwyu
* fix build issue
Co-authored-by: AJ Heller <hork@google.com>
2 years ago
3980ed706a
[EventEngine] Refactoring the EventEngine Test Suite: Part 1 ( #32127 )
...
* Add back EventEngine test suite tools (echo binary)
This reverts commit d5c12195a2
2 years ago
8cf04e9a54
[EventEngine] Modify iomgr to allow creation of event engine clients and client side endpoints ( #31661 )
...
* [WIP] EventEngine iomgr endpoint shims
* [WIP] EventEngine::Endpoint iomgr shims for the PosixEventEngine
* Util functions to help with posix engine listener implementation
* sanity
* update comments in posix_engine_listener_utils.h
* review comments
* iwyu
* revert prev commit
* iwyu
* update build
* update
* regenerate projects
* regenerate projects
* minor fixes
* update BUILD
* sanity
* update build
* regenerate projects
* fix unused parameter
* sanity
* update
* sanity
* regenerate_projects
* remove unused variable
* start
* update
* regenerate_projects
* sanity
* update
* fixes
* update
* regenerate_projects
* update
* fix sanity and msan failure
* more fixes
* build failure
* update
* fix
* sanity
* fixes
* update
* regenerate projects
* fix sanity
* review comments
* An EventEngine subclass to be implemented by all posix based event engines
* sanity
* comments
* update
* review comments
* re-word
* fix
* update
* review comments
* regenerate projects
* syntax fix
* add lock free event benchmark
* releasable mutex lock
* fix build isue
* update
* start
* regenerate projects
* update
* fix
* windows build
* update
* windows portability issue
* update
* update
* update
* update
* format
* update
* update
* update
* update
* update
* fix sanity
* regenerate projects
* update
* iwyu
* Fix resolved address length related bugs in tcp_socket_utils and listener_utils
* iwyu
* cleanup src/core/lib/event_engine/tcp_socket_utils.cc
* iwyu
* fix
* regenerate projects
* fix sanity
* re-write endpoint shim
* more re-write
* cleanup
* update
* review comments
* build issue
* more build issue fixes plus adding event_engine_trace
* even more build issue fixes
* iwyu
* add static_cast
* update
* remove redundant code
* update
* deduplicate
* iwyu
* Fix review comments and regenerate_projects
* sanity
* review comments
* fix include guards
Co-authored-by: AJ Heller <hork@google.com>
2 years ago
67f364e23e
[cleanup] Eliminate usage of GRPC_ASSERT(false...); ( #31757 )
...
* crash function
* progress
* fix
* fix
* Automated change: Fix sanity tests
* fix
* fix
* fix
* fixes
* Automated change: Fix sanity tests
* fix
* Automated change: Fix sanity tests
* fix
* fix
* use cpp attr
* Automated change: Fix sanity tests
* fix
* fix
* fix
* fix
* fix
* fix
* add exclusion
* fix
* typo
* fix
* fmt
* Update tcp_socket_utils.cc
* Automated change: Fix sanity tests
* fix
* revert php changes
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
2904ee8fd3
Add support for systemd socket activation ( #31667 )
...
* Revert "Revert "Add support for systemd socket activation (#30485 )" (#31617 )"
This reverts commit 867dc6cae2
2 years ago
f99b8b5bc4
Convert c-style comments to C++-style comments ( #31923 )
...
* baseline
* fix clang-tidy
* manually revert these files
* manually fixup at eof
* revert 2 more files
* change check_deprecated_grpc++.py
* change end2end_defs.include template
* fix check_include_guards.py
* untrack tools/distrib/python/convert_cstyle_comments_to_cpp.py
not yet ready to be submitted
* fix
yapf check_include_guards.py
remove a space...
* fix version.cc.template
* fix version_info.h.template
2 years ago
531287b2ee
Update tcp server interface to move on_accept_cb to the create method ( #31912 )
...
* Update tcp server interface to move on_accept_cb to create method
* update
* update
2 years ago
71f7bbd86c
Make tcp client posix test more robust ( #31804 )
...
* Make tcp client posix test more robust
* review comments
* fix
* update
* review comments
* update
* fix sanity
* fix clang tidy
* readability
* minor update
2 years ago
9a7e77318b
[cleanups] Remove `include/grpc/impl/codegen/grpc_types.h` ( #31835 )
...
* Remove `include/grpc/impl/codegen/grpc_types.h`
* Automated change: Fix sanity tests
* Fix typo
* Automated change: Fix sanity tests
2 years ago
9d09a8ff2f
Remove `include/grpc/impl/codegen/sync.h` ( #31777 )
2 years ago
f5d21653fc
Revert "[event_engine] Move combiner executor usage to event engine ( #31713 )" ( #31753 )
...
This reverts commit b04aa1cf92
2 years ago
77b3972f95
Clean up the TODO item in build target `grpc_base` part `channel_args` ( #31642 )
...
* Clean up the TODO item in build target `grpc_base` part `channel_args`
* Update
* Update
* Update
* Automated change: Fix sanity tests
* update
* Automated change: Fix sanity tests
2 years ago
b04aa1cf92
[event_engine] Move combiner executor usage to event engine ( #31713 )
...
* [event_engine] Move combiner executor usage to event engine
* fix
* review feedback
* fix
* x
* fix
* fix
2 years ago
864f08ff0c
Fix broken clang-tidy ( #31666 )
2 years ago
3ef817b3bf
Clean up the TODO item in build target `grpc_base` -- part `slice` ( #31003 )
...
* Clean up the TODO item in build target `grpc_base` part `slice`
* Automated change: Fix sanity tests
2 years ago
1e2af690f2
Add a check to flush stale TracedBuffer entries to prevent them growing too long. ( #31588 )
...
* Add a check to flush stale TracedBuffer entries to prevent them growing too long
* update
* fix tsan and sanity
* sanity
* review comments
2 years ago
250f9fc026
Revived google-readability-casting ( #31607 )
...
* Added google-readability-casting
* Fix
* Use brace init for constants
* more
* Fix
* Fix 2
* More fix
* Fixf
2 years ago
867dc6cae2
Revert "Add support for systemd socket activation ( #30485 )" ( #31617 )
...
This reverts commit a638c407bb
2 years ago
a638c407bb
Add support for systemd socket activation ( #30485 )
...
* Add socket-activated functionality
* Fix GRPC_ERROR_NONE in tcp_server_posix_test and sanity checks
* Fix epoll1 error, sanity checks
* Use AF_INET6 in tcp_server_posix_test
Co-authored-by: Vignesh Babu <vigneshbabu@google.com>
2 years ago
40ec6850d7
C++ize TracedBuffer Interface and eliminate lock contention during getsockopt operation ( #31565 )
...
* C++ize TracedBuffer Interface and fix bug to eliminate lock contention during getsockopt operation
* fix
* fix sanity
* fix sanity
* review comments
2 years ago
85794d4094
Deprecated grpc_error_std_string ( #31267 )
2 years ago
ae471418e4
Removed legacy macros completely ( #31248 )
...
* Removed legacy macros
* Removed GRPC_ERROR_NONE;
* Fix CFStreamClientTests.mm
2 years ago
3c96517fc0
Simplified GRPC_ERROR_CREATE macros ( #31222 )
...
* Simplified GRPC_ERROR_CREATE macros
* Code migration
2 years ago
347bf1a1f4
Removed GRPC_ERROR_[INT|STR] macros ( #31195 )
...
* Clean-up which
* Reformat
2 years ago
db5fe72da9
[tsan] Threadsafe strerror implementation ( #31167 )
...
* fix
* spread
* fix
* Automated change: Fix sanity tests
* fix
* remove unnecessary .c_strs
* lashes
* fix
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
9c98d67d7a
[c++14] absl::make_unique --> std::make_unique ( #31159 )
...
* [c++14] absl::make_unique --> std::make_unique
* fix
* fix
2 years ago
2ee2c91c92
Reland x2: Make GetDefaultEventEngine return a shared_ptr ( #30619 )
...
* Reland x2: Make GetDefaultEventEngine return a shared_ptr
* remove thread leak from NativeDNSResolver
This is not going to work for resolvers that support cancellation.
* give resolvers bounded lifetimes
Some resolver own EventEngines. EventEngines cannot run off the end of
the process since they have unjoined threads (problematic in a small set
of environments). This gives resolvers bounded lifetimes, and allows
replacement of resolvers without ASAN issues of deleting resolvers in
active use (occurs in tests).
* fix
* fix windows
* fix surface init test
* fix
* sanitize
* use after move
* the test must wait for the callback to be destroyed
* windows fix: delete the resolver on iomgr shutdown, not before
* Make TimerManager threads non-joinable
On gRPC shutdown, any unjoined TimerManager threads will cause TSAN to
detect thread leaks. This fix resolves issues I saw in end2end test
shutdown in another PR, where a single timer manager thread was always
alive after the test ended.
The long-term solution is to integrate the new ThreadPool here, but this
unblocks me for now.
* backport fix
* fix
* shared_ptr<EventEngine> in EventEngine benchmarks
2 years ago
d43511f4af
Removed GRPC_ERROR_NONE ( #31131 )
2 years ago
ba8af0157b
Removing GRPC_ERROR_[REF|UNREF|IS_NONE] ( #31089 )
...
* Removing GRPC_ERROR_[REF|UNREF|IS_NONE]
* Clean up unnecessary error
* Fix
* Fix2
* Remove unnecessary else
2 years ago
ebc4f236b6
[slice] Remove _internal variants of APIs ( #30953 )
...
* [slice] Remove _internal variants of APIs
* Automated change: Fix sanity tests
* fix
* reduce bloat
* fixes
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
f15ba1ffc7
[tls] Remove support for pthread tls ( #31040 )
...
* [tls] Remove support for pthread tls
* fix
* fix
* fix
* lalala
* fix
* fix
* Clean up deployment target changes
* additional clean up of deployment target
* regen podspec for updated file list
* remove destination change
* deployment target override for ios cpp test
* fix?
* fix run test script & update test destination
* [tls] Remove support for pthread tls
* fix
* fix
* fix
* lalala
* fix
* fix
* Clean up deployment target changes
* additional clean up of deployment target
* regen podspec for updated file list
* remove destination change
* deployment target override for ios cpp test
* fix?
* fix run test script & update test destination
* merge
* fix
* final script fix for proper destination & target
* more deployment target fix
* objc ios test script fix
Co-authored-by: Denny C. Dai <dennycd@me.com>
Co-authored-by: dennycd <dennycd@google.com>
2 years ago
bd6fab6ab7
[iomgr] Reduce the size of combiner_test ( #31085 )
2 years ago
1ad4bbe78e
Revert "fix ( #31018 )" ( #31021 )
...
This reverts commit 2e6f268dfc
2 years ago
2e6f268dfc
fix ( #31018 )
2 years ago
96f5cddb5f
[time] Introduce time sources ( #30815 )
...
* [time] Introduce time sources
* make import trivial
* Automated change: Fix sanity tests
* fix
* Automated change: Fix sanity tests
* fix
* Automated change: Fix sanity tests
* Automated change: Fix sanity tests
* review feedback
* fix
* Automated change: Fix sanity tests
* ios fix
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
888c64adf8
Fix the build script ( #30985 )
...
* Fix the build script
* Set -ex
* fix unsigned int comparisons and ignored return values
Co-authored-by: AJ Heller <hork@google.com>
2 years ago
c2ab8c99bb
Revert "Revert "[c++] Move environment functions to C++ ( #30937 )" ( #30986 )" ( #30988 )
...
This reverts commit 96264e07b8
2 years ago
96264e07b8
Revert "[c++] Move environment functions to C++ ( #30937 )" ( #30986 )
...
This reverts commit 74c0d6fe3f
2 years ago
74c0d6fe3f
[c++] Move environment functions to C++ ( #30937 )
...
* [gprpp] Move env to C++
* move headers/impl
* Automated change: Fix sanity tests
* fix
* fix
* Automated change: Fix sanity tests
* Update http_proxy.cc
* fix
* fix
* rename
* fix merge
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
b463cabeec
Update flaky tests ( #30873 )
...
* Update flaky tests list
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
ca8b437613
[experiments] Single experiments flag ( #30822 )
...
* single experiments flag
* add missing files
* fix
* empty-string-fix
* fix
* Automated change: Fix sanity tests
* ensure cold path is out of line
* add log for enabled/disabled experiments
* remove bad sentence
* Automated change: Fix sanity tests
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
70dd34b438
Revert "Revert "Modifying iomgr tcp code to use event engine EndpointConfig instead of channel_args"" ( #30509 )
...
* Revert "Revert "Modifying iomgr tcp code to use event engine EndpointConfig instead of channel_args (#30028 )" (#30495 )"
This reverts commit 0b1a8c984b
2 years ago
c580d0d9a7
[experiments] Add experiment framework ( #30775 )
...
* [experiments] Add experiment framework
* auto-ci-config
* fix
* fix
* Automated change: Fix sanity tests
* support different configs
* Automated change: Fix sanity tests
* cleaner generated code
* Automated change: Fix sanity tests
* enforce expiry
* make sure expiry isnt far in the future
* fix
* Automated change: Fix sanity tests
* remove specified testing
* cleaner
* fix
* fix
* Automated change: Fix sanity tests
* fix
* fix
* docstring
* clean up code
* Automated change: Fix sanity tests
* review feedback
* build fix
* Automated change: Fix sanity tests
* ownership
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
535168431a
[lb_policy] Move to core configuration system ( #30483 )
...
* [load_balancing] Move interfaces to src/core/lib/load_balancing
* Automated change: Fix sanity tests
* progress
* progress
* fix
* missed file
* Automated change: Fix sanity tests
* update tests
* Automated change: Fix sanity tests
* Automated change: Fix sanity tests
* fix
* fix
* Automated change: Fix sanity tests
* fix
* fixes
* fix
* fix
* fix
* fix
* fix
Co-authored-by: ctiller <ctiller@users.noreply.github.com>
2 years ago
406d518742
Revert "4 cpus is the max ( #30696 )" ( #30697 )
...
This reverts commit 75c780039b
2 years ago
75c780039b
4 cpus is the max ( #30696 )
2 years ago
Craig Tiller
Jan Tattermusch
AJ Heller
Vignesh Babu
Andres Beltran
Yijie Ma
Cheng-Yu Chung
Esun Kim