Commit Graph

468 Commits (d5c4333b9e00520fb95bcbb5b7433288090706d3)

Author SHA1 Message Date
David Wilson d5c4333b9e debug: functions for triggering EINTR 8 years ago
David Wilson 6377f2d69c issue #257: split pool shutdown and join. 8 years ago
David Wilson e35694acd5 ansible: flake8 fixes. 8 years ago
David Wilson 3909cb11f6 service: recreate the pool after fork. 8 years ago
David Wilson 05e0b134f9 service: simplify CALL_SERVICE stub and fix race.
If PushService.store_and_forward() loses the race to arrive at a brand
new context first, and the context's main thread is already executing a
CALL_FUNCTION that is blocked on the result of PushService, deadlock
could occur in the old scheme.

Instead (for now) simply spam a thread for each incoming message, and
use the get_or_create_pool() lock to ensure things work out in the end.
This could potentially generate a huge number of threads given the wrong
app, but we'll fix that problem when it appears.
8 years ago
David Wilson 92ecf29559 core: check in the hacks that let Ansible work just now. 8 years ago
David Wilson f6d9b074ff master: reduce module verbosity somewhat. 8 years ago
David Wilson 526590027a issue #186: PushFileService improvements.
New method to send all modules and files in one roundtrip.
8 years ago
David Wilson 9e78c20eba core/parent: add Context.call_no_reply(). 8 years ago
David Wilson b3a5fa70b0 core: copy debug setting to child's Router too.
core.Router doesn't pay attention to this attribute, but after
upgrade_router() has been called, the new parent.Router will.
8 years ago
David Wilson 23b2a545cf fork: avoid another logging deadlock at startup.
The very first task /must/ be clearing out logging locks, since
_at_fork() functions call LOG.debug() via Side.close(). Additionally,
the root logger is not included in loggerDict, so we must specify it
explicitly.
8 years ago
David Wilson 785df88fa4 issue #186: core: remove long-forgotten hack.
This is likely to break something, it was definitely needed at some
point, but I never put much effort into figuring out why. Meanwhile,
Python appears to make find_module('ansible.module_utils.facts.')
requests in some circumstances, which causes us to indicate the module
exists while this hack exists.

So remove it, and let's see what breaks.
8 years ago
David Wilson 76beea6554 issue #186: move target._get_file into mitogen.service
For lack of a better place to keep the client function, make it a
classmethod of FileService itself for now.

The old _get_file() is removed in a subsequent commit.
8 years ago
David Wilson a3b747af1b issue #186: add PushFileService
This is like FileService but blocks until the file is pushed by a parent
context, with deduplicating behaviour at each level in the hierarchy. It
does not stream large files, so it is only suitable for small files like
Python modules.

Additionally add SerializedInvoker for use with PushFileService, which
ensures all method calls to a single service occur in sequence.
8 years ago
David Wilson 2e8c027322 issue #213: avoid service.Pool construction race
Ensure concurrent calls to service.Pool do not result in a duplicate
pool being constructed in a child.
8 years ago
David Wilson 34daec4a7a core: prevent warning when CALL_FUNCTION used without reply_to
Such as when the stub CALL_SERVICE handler is used.
8 years ago
David Wilson d9087c510b ansible: move FileService into mitogen.service. 8 years ago
David Wilson b577a11f86 master: fix IdAllocator log messages. 8 years ago
David Wilson f7d2eace08 tests: importer fixes 8 years ago
David Wilson f7b368b1fb master: implement ModuleResponder.forward_module(). 8 years ago
David Wilson 9492dbc4d7 parent: split out minify.py and add stub where master can install it.
This needs a cleaner mechanism to install it, at least this one is
documented.
8 years ago
David Wilson 325d13538f issue #196: debug: don't statically import master. 8 years ago
dw c24d29d367
Merge pull request #255 from dw/dmw
Dmw
8 years ago
David Wilson 3b0addcfb0 service: v2. Closes #213 8 years ago
David Wilson 469bde63c2 parent: fix log message ordering 8 years ago
David Wilson a4ddef25a1 core: move reader/writer debug prints
They stop working with kqueue/epoll poller in the old location. Also
comment them out again, should never have been checked in uncommented.
8 years ago
David Wilson 3f595bbc7e issue #213: use import_module() in parent.py.
This dynamic import crap really needs to be ripped out of parent.py
again. Static imports work much better for the module loader too.
8 years ago
David Wilson e118963b30 issue #254: fork: take care not to rely on FD numbers. 8 years ago
David Wilson fc59f57ba2 issue #213: core: split out import_module() for use in services.py. 8 years ago
David Wilson 49fb25ee1c issue #213: core: fix shutdown crash due to member variable rename 8 years ago
David Wilson 40c6c6426f issue #213: core: fix test breakage due to log message change 8 years ago
David Wilson 2310497d55 issue #213: core: have Message.reply() log msg for zero reply_to
It's easy to call msg.reply() by accident on a message that never had
reply_to set, resulting in a "invalid handle" error message coming from
router. Instead log a more accurate message on the stack that actualy
caused the problem.
8 years ago
David Wilson d2714752ee docs: tidy ups 8 years ago
David Wilson 61365236ad docs/select: fix up more references, fix headings. 8 years ago
David Wilson 4bf3d01104 select: add missing get(block=..) parameter. 8 years ago
David Wilson ddf28987a0 master: split Select() into new module to reduce wire size.
service.py currently imports master.py(+parent.py) just to get Select().
8 years ago
David Wilson 7a592d1c34 core: better Poller.__repr__ 8 years ago
David Wilson b0ce6eecd7 fork: support on_start= argument. 8 years ago
David Wilson 00edf0d66d core: have ExternalContext accept a config dict rather than kwargs.
The parameter lists had gotten out of control.
8 years ago
David Wilson 7d0209d8de issue #249: have upgrade_router() upgrade the poller too.
Now when a child becomes a parent, it gets a new poller suitable for
many more children than was possible using select().
8 years ago
David Wilson 55fff54774 core: make try/catch logic a little clearer in Latch.get() 8 years ago
David Wilson 05a5f2b6e5 core: if Poller.poll() fails, TimeoutError would be raised.
We must check whether poller threw an exception both in the case that we
weren't woken and the case that we were.
8 years ago
David Wilson 5bdc1719c5 issue #249: epoll() raises IOError for EINTR, not select.error. 8 years ago
David Wilson 6d18ce81d8 issue #249: restore duplex behaviour for epoll
With epoll() there is only one kernel-side object per file descriptor,
which is why _control() is such a pain. Since we merge receive/transmit
watching into that single object, we must always test the mask for both
conditions when reading results.

Kqueue isn't/doesn't appear to be like this. The identity of a Kqueue
event is keyed on (fd, filter), and we register a separate event for
both transmit and receive, so the 'elif' in KqueuePoller.poll() does not
appear to need to change.

Previously, a FD marked for read+write would not indicate writeability
until it was no longer readable.
8 years ago
David Wilson 6b98818046 issue #249: epoll distinguishes between hangup and disconnect
..typical Linux, for certain descriptor types only. So our receive mask
must match both, and normalize it into a read event like every other
poller.
8 years ago
David Wilson 07056b0dd1 issue #249: fix ordering bug masked by previous implementation 8 years ago
David Wilson 36a1024861 issue #249: port Latch to poller too.
This is probably going to suck for perf :/
8 years ago
David Wilson 1070dfae72 issue #249: fix iter_read() regression. 8 years ago
David Wilson dcf0aa351e issue #249: whoops, fix new poller timeouts. 8 years ago
David Wilson aa8f786413 issue #249: fix Poller.from_existing() for v2 API 8 years ago