First: ugh.
Second: allow tracer global variables to be registered and then parsed out of a configuration environment variable.
Expose TSI trace config variable directly to ease this a little.
This is a multipoller based on epoll rather than poll.
Note that this implementation is aimed at correctness rather than
performance, although it should immediately have better scalability to
large numbers of FDs, both due to epoll's O(1) sized API and due to not
needing to wake up polling threads to do interest set changes.
One notable difference here is that we directly attach a wakeup fd
rather than using the freelisting kick mechanism that the poll() based
implementations use, because modifying the epoll set to use a different
kick fd each time isn't free.
This introduces the wakeup fd interface, corresponding approximately to
the existing Google version, complete with a ported giant detailed usage
comment.
The implementation has two layers, "specialized" and "fallback". The
specialized layer is intended to be a suitable platform specific
implementation like eventfd, whereas "fallback" is probably pipe, with
runtime detection of whether the specialized version works on this
system (currently stubbed out).
This change pulls out a separate pollset_kick module, which currently
uses a freelist of pipes dynamically assigned to pollsets when they
enter polling rather than the previous racy sharding mechanism.
We ultimately may wish to eliminate the dynamic assignment for multipoll
sets, but this should be sufficient for the moment.
Checked that the opensource version still builds.
Just addressing core for now. We will do c++ later.
Change on 2015/01/12 by jboeuf <jboeuf@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83804014
Fixed any exposed bugs across the stack.
Add a poll() based implementation. Heavily leverages pollset infrastructure to allow small polls to be the norm. Exposes a mechanism to plug in epoll/kqueue for platforms where we have them.
Simplify iomgr callbacks to return one bit of success or failure (instead of the multi valued result that was mostly unused previously). This will ease the burden on new implementations, and the previous system provided no real value anyway.
Removed timeouts on endpoint read/write routines. This simplifies porting burden by providing a more orthogonal interface, and the functionality can always be replicated when desired by using an alarm combined with endpoint_shutdown. I'm fairly certain we ended up with this interface because it was convenient to do from libevent.
Things that need attention still:
- adding an fd to a pollset is O(n^2) - but this is probably ok given that we'll not
use this for multipolling once platform specific implementations are added.
- we rely on the backup poller too often - especially for SSL handshakes and for client
connection establishment we should have a better mechanism ([] []
- Linux needs to use epoll for multiple fds, FreeBSD variants (including
Darwin) need to use kqueue. ([] []
- Linux needs to use eventfd for poll kicking. ([]
Change on 2015/01/07 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83461069
Fixed any exposed bugs across the stack.
Add a poll() based implementation. Heavily leverages pollset infrastructure to allow small polls to be the norm. Exposes a mechanism to plug in epoll/kqueue for platforms where we have them.
Simplify iomgr callbacks to return one bit of success or failure (instead of the multi valued result that was mostly unused previously). This will ease the burden on new implementations, and the previous system provided no real value anyway.
Removed timeouts on endpoint read/write routines. This simplifies porting burden by providing a more orthogonal interface, and the functionality can always be replicated when desired by using an alarm combined with endpoint_shutdown. I'm fairly certain we ended up with this interface because it was convenient to do from libevent.
Things that need attention still:
- adding an fd to a pollset is O(n^2) - but this is probably ok given that we'll not
use this for multipolling once platform specific implementations are added.
- we rely on the backup poller too often - especially for SSL handshakes and for client
connection establishment we should have a better mechanism ([] []
- Linux needs to use epoll for multiple fds, FreeBSD variants (including
Darwin) need to use kqueue. ([] []
- Linux needs to use eventfd for poll kicking. ([]
Change on 2015/01/07 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=83461069
-) Properly linking built-in OpenSSL into the grpc library.
-) grpc now properly depends on gpr when linking shared code.
-) Properly naming the shared library with all their aliases.
-) Properly installing the shared library aliases on the system.
-) Potentially supporting Darwin and MINGW32 targets for shared libraries.
Caveat: if using shared libraries, some tests will not compile anymore if they want to use OpenSSL functions, as they are no longer publically available externally. Which is the feature we were seeking.
The Makefile currently does it properly, by linking the tests statically. This only applies when using external code and Makefiles, if said external code improperly assumes SSL is available through grpc.
Change on 2014/12/22 by nnoble <nnoble@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82656438
This CL introduces the public side of this interface. There will need to be an
iomgr-private API also, but this will be a per-implementation API and so is not
covered here.
I've taken care of wiring the interface through the codebase in the manner that
I expect it will be used.
Change on 2014/12/17 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82376987
Fold endpoint interface into iomgr, move secure_endpoint into security/.
This will make it easier for endpoint to rely on some iomgr defined types (like
pollset).
Change on 2014/12/17 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82338036
This is a trivial C89-ification of the []2 implementation of the same idea.
Indeed the implementation files and tests have been branched from the [] versions.
Change on 2014/12/15 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82168822
a disaster and does not support url_safe which we need for the JWT.
Change on 2014/12/12 by jboeuf <jboeuf@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82020601
Change on 2014/12/12 by jtattermusch <jtattermusch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=82017498
-) Introducing gpr_vlog so to spare a few vsprintf later (at least one for now)
-) Renaming statistics/log.* to statistics/census_log.* to avoid collisions.
Change on 2014/12/12 by nnoble <nnoble@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81995756
Safely managing disconnection and goaways needs reference counting at the child channel level, which was near impossible to provide with the previous embedding of the child channel in the client channel.
This is a (hopefully) no-op refactoring to provide that split. The next CL in this series will actually get disconnection and goaway somewhat right.
Change on 2014/12/12 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81985590
Move eventmanager and platform dependent endpoint functionality into a single
library called 'iomgr'.
This is primarily to prepare for a Windows port - where posix socket semantics
lead to poor quality code.
Mostly this is a code movement CL, with some small changes to help prepare the
way for porting:
- em style fd objects can only be held internally in iomgr, and own their memory
- added grpc_iomgr_create_endpoint_pair() to accomodate the common pattern of
creating a tcp endpoint from the output of socketpair - this will help keep
our tests portable
- separated em alarm interface into a separate file, as this part of event
manager is needed higher up the stack
- made the eventmanager bits a true singleton, simplifying API's across the
stack as there's no longer a reason to carry a pointer there.
Initial design document is here:
https://docs.google.com/document/d/1VmafcHvvrP5kwtQkz84R5yXF7u7fW-9Pn0bkSUQHDt8/edit?disco=AAAAARNByxg
Change on 2014/12/09 by ctiller <ctiller@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81716456
Change on 2014/12/09 by jtattermusch <jtattermusch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81704538
Current state for Windows platform:
-- gpr and grpc_test_util can be compiled
-- gpr_log_test and gpr_cmdline_test pass
Change on 2014/12/08 by jtattermusch <jtattermusch@google.com>
-------------
Created by MOE: http://code.google.com/p/moe-java
MOE_MIGRATED_REVID=81615574
Craig Tiller
Nicolas "Pixel" Noble
Craig Tiller
Jan Tattermusch
David Klempner
Nicolas Noble
Nicolas "Pixel" Noble
jboeuf
hongyu