Commit Graph

3001 Commits (8925f8a9d796bb005bcbf335cd1fb732482a16b2)
 

Author SHA1 Message Date
David Wilson fdbd954113 ansible: preload built-in modules in ModuleDepScanner.
For "ansible -m setup" over a 25ms link, avoids 65 roundtrips and
reduces runtime from 5.7s to 4.1s (-28%).

For "ansible -m setup" over a simulated 250 ms link, reduces runtime
from m27.015s to 0m8.254s (-69%).
7 years ago
David Wilson b577a11f86 master: fix IdAllocator log messages. 7 years ago
David Wilson f7d2eace08 tests: importer fixes 7 years ago
David Wilson 8d45e609ee ansible: preload always-requested modules.
Avoid 9 roundtrips during setup. In combination with previous change,
reduces 'ansible -m stat' execution over 25ms link from 3.7s to 3.07s.

1,3d0
< _on_get_module('ansible')
< _on_get_module('ansible.module_utils')
< _on_get_module('ansible.module_utils.basic')
69,74d65
< _on_get_module('ansible.module_utils.json_utils')
< _on_get_module('ansible.release')
< _on_get_module('ansible_mitogen')
< _on_get_module('ansible_mitogen.runner')
< _on_get_module('ansible_mitogen.target')
< _on_get_module('mitogen.fork')
7 years ago
David Wilson f7b368b1fb master: implement ModuleResponder.forward_module(). 7 years ago
David Wilson a578250bfb ansible: remove indirect master.py imports.
Avoids sending 10 modules:

77d76
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'ansible_mitogen.module_finder')
79d77
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'ansible_mitogen.services')
81,84d78
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.compat')
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.compat.collections')
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.compat.functools')
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.compat.tokenize')
86,87d79
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.master')
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.minify')
89,90d80
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.select')
< _send_load_module(mitogen.ssh.Stream(u'ssh.localhost'), 'mitogen.service')
7 years ago
David Wilson 9492dbc4d7 parent: split out minify.py and add stub where master can install it.
This needs a cleaner mechanism to install it, at least this one is
documented.
7 years ago
David Wilson 325d13538f issue #196: debug: don't statically import master. 7 years ago
David Wilson cee76ee7b9 tests: gcloud setup fixes. 7 years ago
David Wilson aba6cb302a docs: add example sudoers rule
hat tip @seuf :)
7 years ago
dw c24d29d367
Merge pull request #255 from dw/dmw
Dmw
7 years ago
David Wilson 3b0addcfb0 service: v2. Closes #213 7 years ago
David Wilson 469bde63c2 parent: fix log message ordering 7 years ago
David Wilson a4ddef25a1 core: move reader/writer debug prints
They stop working with kqueue/epoll poller in the old location. Also
comment them out again, should never have been checked in uncommented.
7 years ago
David Wilson 3f595bbc7e issue #213: use import_module() in parent.py.
This dynamic import crap really needs to be ripped out of parent.py
again. Static imports work much better for the module loader too.
7 years ago
David Wilson e118963b30 issue #254: fork: take care not to rely on FD numbers. 7 years ago
David Wilson fc59f57ba2 issue #213: core: split out import_module() for use in services.py. 7 years ago
David Wilson 49fb25ee1c issue #213: core: fix shutdown crash due to member variable rename 7 years ago
David Wilson 40c6c6426f issue #213: core: fix test breakage due to log message change 7 years ago
David Wilson 2310497d55 issue #213: core: have Message.reply() log msg for zero reply_to
It's easy to call msg.reply() by accident on a message that never had
reply_to set, resulting in a "invalid handle" error message coming from
router. Instead log a more accurate message on the stack that actualy
caused the problem.
7 years ago
David Wilson 633585524f tests: don't need separate module for id_allocation_test any more
This used to be because everything would explode while importing
__main__ under py.test, but that was fixed months ago.
7 years ago
David Wilson dd48c41332 Ignore another annoying flake8 message. 7 years ago
David Wilson d2714752ee docs: tidy ups 7 years ago
David Wilson 61365236ad docs/select: fix up more references, fix headings. 7 years ago
David Wilson 4bf3d01104 select: add missing get(block=..) parameter. 7 years ago
David Wilson ddf28987a0 master: split Select() into new module to reduce wire size.
service.py currently imports master.py(+parent.py) just to get Select().
7 years ago
David Wilson 7a592d1c34 core: better Poller.__repr__ 7 years ago
David Wilson b0ce6eecd7 fork: support on_start= argument. 7 years ago
David Wilson 00edf0d66d core: have ExternalContext accept a config dict rather than kwargs.
The parameter lists had gotten out of control.
7 years ago
David Wilson eac4cc7afe tests: speed things up 7 years ago
David Wilson 863a95e860 docs: update contributors 7 years ago
David Wilson b0aa413173 tests: import benchmark reproduction setup playbook
Incomplete, also needs to mess around with Gcloud routing and handle box
setup/teardown, because that's another "well engineered" waste of time.
7 years ago
David Wilson a99c7a12f9 tests: output split groups in gcloud.py 7 years ago
David Wilson 1a0e630a80 tests: add debops to requirements 7 years ago
David Wilson dbcee4041a tests: refactor gcloud.py to be dynamic inventory. 7 years ago
David Wilson a3995f8e5f ansible: remove hard-coded dw username. 7 years ago
David Wilson 1eb5c20f57 ansible: add dummy init.pys so setup.py find_packages() DTRT. 7 years ago
David Wilson 2c58591129 Bump Ansible version in dev_requirements.txt 7 years ago
David Wilson 7d0209d8de issue #249: have upgrade_router() upgrade the poller too.
Now when a child becomes a parent, it gets a new poller suitable for
many more children than was possible using select().
7 years ago
David Wilson 55fff54774 core: make try/catch logic a little clearer in Latch.get() 7 years ago
David Wilson 05a5f2b6e5 core: if Poller.poll() fails, TimeoutError would be raised.
We must check whether poller threw an exception both in the case that we
weren't woken and the case that we were.
7 years ago
David Wilson 5bdc1719c5 issue #249: epoll() raises IOError for EINTR, not select.error. 7 years ago
David Wilson 70376d861a issue #217: docs: remove limitation 7 years ago
David Wilson 6d18ce81d8 issue #249: restore duplex behaviour for epoll
With epoll() there is only one kernel-side object per file descriptor,
which is why _control() is such a pain. Since we merge receive/transmit
watching into that single object, we must always test the mask for both
conditions when reading results.

Kqueue isn't/doesn't appear to be like this. The identity of a Kqueue
event is keyed on (fd, filter), and we register a separate event for
both transmit and receive, so the 'elif' in KqueuePoller.poll() does not
appear to need to change.

Previously, a FD marked for read+write would not indicate writeability
until it was no longer readable.
7 years ago
David Wilson 6b98818046 issue #249: epoll distinguishes between hangup and disconnect
..typical Linux, for certain descriptor types only. So our receive mask
must match both, and normalize it into a read event like every other
poller.
7 years ago
David Wilson 07056b0dd1 issue #249: fix ordering bug masked by previous implementation 7 years ago
David Wilson 4cd9e09130 issue #249: docs: remove limitation 7 years ago
David Wilson 36a1024861 issue #249: port Latch to poller too.
This is probably going to suck for perf :/
7 years ago
David Wilson 1070dfae72 issue #249: fix iter_read() regression. 7 years ago
David Wilson dcf0aa351e issue #249: whoops, fix new poller timeouts. 7 years ago