Commit Graph

329 Commits (a075cc02426a068f08dad5b357f4dfe048f49393)
 

Author SHA1 Message Date
David Wilson a075cc0242 ansible: document Strategy's implementation 7 years ago
David Wilson 18eaf14dca ansible: migrate logging variables into utils. 7 years ago
David Wilson 5d8cb0f5fb ansible: document the connection class. 7 years ago
David Wilson b7f563a6f0 ansible: remove old action subdirectory. 7 years ago
David Wilson 5f45c2d49a ansible: clean up, structure and document strategy module. 7 years ago
David Wilson 10c2de2845 examples: rename playbooks for clarity. 7 years ago
David Wilson ea126bb7e8 docs: add stats tracking 7 years ago
David Wilson 020036f807 core: add a nasty hack for Ansible modules. 7 years ago
David Wilson 875ee29d4f docs: initial Ansible extension docs. 7 years ago
David Wilson 84e9d42e71 ansible: handle local connections and synchronize module too. 7 years ago
David Wilson fa804c7c80 parent: simplify route() call
Doesn't need to go via defer() since it's always running on the broker
thread.
7 years ago
David Wilson 08b80b543f Some gitignore extras 7 years ago
David Wilson bdd8648bae ssh: enable compression by default
Using the same test as in 7af97c0365,
transmitted wire bytes drops from 135,531 to 133,071 (-1.81%), while
received drops from 21,073 to 14,775 (-30%).

Combined, both changes shave 13,914 bytes (-8.6%) off aggregate
bandwidth usage.

Make it configurable as compression hurts in some scenarios.
7 years ago
David Wilson ccd9c62399 tests: stubs for future whitelist/blacklist tests. 7 years ago
David Wilson 74468abf0a ansible: fix syntax error 7 years ago
David Wilson 1fb848b5e6 examples: add test case for local Ansible connections 7 years ago
David Wilson 2a786ce146 docs: fix preamble_size.py after whitelist changes 7 years ago
David Wilson 4d940f08ae importer: drop redundant prefix from pkg_present
For the 52 submodules of ansible.modules.system, this produced a 1602
byte pkg_present list. After stripping it becomes 406 bytes, and the
entire LOAD_MODULE size drops from 1988 bytes to 792 bytes (-60%).

For the 68 submodules of ansible.module_utils, 1902 bytes pkg_present
becomes 474 bytes (-75%), and LOAD_MODULE size drops from 2867 bytes to
1439 bytes (-49%).

In a simple test running Ansible's "setup" module followed by its "apt"
module, wire bytes sent drops from 140,357 to 135,531 (-3.4%).
7 years ago
David Wilson 6e7bb4fd13 ansible: support local connections too 7 years ago
David Wilson 71a6b9e32a master: Select.all() sugar 7 years ago
David Wilson 4aacf2da4c ansible: we're gonna need more playbook steps 7 years ago
David Wilson 57894b18e0 ansible: support class_only=True in wrapped loader 7 years ago
David Wilson 54a3777ff3 ansible: blacklist everything except our own namespaces
Farewell, pointless roundtrips, we hardly knew ye.
7 years ago
David Wilson b543b84e80 importer: share blacklist logic between master/parent 7 years ago
David Wilson 8ec6ae1da0 importer: module whitelist/blacklist support
Hoped to avoid it, but it's the obvious solution for Ansible.
7 years ago
David Wilson 43ba1c76dc core: wrap selects in EINTR handlers
This isn't nearly enough, but it catches the most common victim of
EINTR.
7 years ago
David Wilson aafe458a13 core: #39: don't call logging framework when logging is disabled
It looks ugly as sin, but this nets about a 20% drop in user CPU time,
and close to 15% increase in throughput.

The average log call is around 10 opcodes, prefixing with '_v and' costs
an extra 2, but both are simple operations, and the remaining 10 are
skipped entirely when _v or _vv are False.
7 years ago
David Wilson 331f77ee2b ansible: generalized action module wrapping. 7 years ago
David Wilson 2a097dfaf8 examples: import client/server.py test scripts as examples 7 years ago
David Wilson ac4a42a43e docs: fix Select() example for new style API. 7 years ago
David Wilson e59cc89059 examples: fix mitop example. 7 years ago
David Wilson 2ce71b338b unix: add small explainer note until real docs exist 7 years ago
Alex Willmer a69d9650ad Fix indentation of Router.route anchor 7 years ago
Alex Willmer f4405c6bbc Split IterReadTest & WriteAllTest into test_parent
This matches their respective functions under test, which now reside in
mitogen.parent.
7 years ago
Alex Willmer 4fb1b928b7 Adjust unittest for master -> parent split
Tests should now match changes in
4d31300dd0
7 years ago
Alex Willmer 9ccff0f5d1 Fix TypeError while initiliazing ImporterMixin
Since f44356af32 mitogen.core.Importer()
takes a Router instance.
7 years ago
Alex Willmer eb1ad66469 docs: Fix out of date get_data() snippets
As of adc8fe3aed Receiver objects do not
have a get_data() method and Receiver.get() does not unpickle the
message.
7 years ago
Alex Willmer 3261c561dd Fix AttributeError in mitogen.core.Context.send_await()
As of adc8fe3aed Receiver objects do not
have a get_data() method and Receiver.get() does not unpickle the
message.
7 years ago
Alex Willmer fa0f390d4d FIx syntax error in FindRelatedImportsTest.test_django_db_models 7 years ago
David Wilson 6905dc4e8d master: use queue-like Latch in Select() too. 7 years ago
David Wilson d13ab9a022 ansible: remove hard-wired UNIX socket path. 7 years ago
David Wilson 20afa5b90c Latch v2: combined queue + one self-pipe-per-thread
Turns out it is far too easy to burn through available file descriptors,
so try something else: self-pipes are per thread, and only temporarily
associated with a Lack that wishes to sleep.

Reduce pointless locking by giving Latch its own queue, and removing
Queue.Queue() use in some places.

Temporarily undo merging of of Waker and Latch, let's do this one step
at a time.
7 years ago
David Wilson e6a107c5aa core: replace Queue with Latch
On Python 2.x, operations on pthread objects with a timeout set actually
cause internal polling. When polling fails to yield a positive result,
it quickly backs off to a 50ms loop, which results in a huge amount of
latency throughout.

Instead, give up using Queue.Queue.get(timeout=...) and replace it with
the UNIX self-pipe trick. Knocks another 45% off my.yml in the Ansible
examples directory against a local VM.

This has the potential to burn a *lot* of file descriptors, but hell,
it's not the 1940s any more, RAM is all but infinite. I can live with
that.

This gets things down to around 75ms per playbook step, still hunting
for additional sources of latency.
7 years ago
David Wilson 27f1eddbd4 ansible: delete old needless host_vars directory 7 years ago
David Wilson beb16d0db6 master: helper functions to force disconnect everything 7 years ago
David Wilson 9b5d0a9b97 service: initial version 7 years ago
David Wilson 6059e0c1db ansible: support become:True and become_user 7 years ago
David Wilson 8204dc69e7 ansible: add become test case 7 years ago
David Wilson ed09076bdd ansible: automatically configure connection plug-in. 7 years ago
David Wilson a35fcf44cc ansible: restructure to avoid intermediate imports 7 years ago