Commit Graph

286 Commits (d36a320e7f2afaff410ef8a6bd947eb1aa3347f8)

Author SHA1 Message Date
David Wilson 90c2ed03d0 ansible: fix synchronize module
Broken by recent connection delegation fixes.
6 years ago
David Wilson 7458dfae85 ansible: avoid roundtrip for small file transfers.
Calls to connect.put_file() where the file is sufficiently small enough
to fit in a single RPC proceed without waiting for an RPC response. If
the write fails the target context will log an exception, and any
subsequent step depending on the written file will fail.

I verified every built-in action plugin for file transfer calls, and
they all depend on the transferred file in the following step, so this
should be safe.

Reduces template/copy actions to 2-RTT, loop-20-templates.yml runtime
reduced from 30 seconds to 10 seconds over a 250ms link compared to
v0.2.2, and from 123 seconds compared to vanilla with pipelining
enabled.
6 years ago
David Wilson 084c0ac065 ansible: avoid roundtrip in copy action due to fixup_perms2().
On top of existing temporary files work, this reduces the number of
roundtrips required for "copy" and "template" actions from 6 to 3.
6 years ago
David Wilson e18396d54d ansible: enable profiling by default!
Thankfully this never made it into a release
6 years ago
David Wilson bce4f59138 issue #345: disable IdentitiesOnly by default. 6 years ago
David Wilson 2e3d04bbb8 issue #342: forward _create_control_path() to SSH plugin.
network_cli connection type loads the "ssh" (mitogen_ssh) plugin and
expects a private method to exist.
6 years ago
David Wilson ac9b84d237 issue #321: 2.4+ compatibility fixes, disable test on Vanilla. 6 years ago
David Wilson 4afe2fdc55 issue #321: fix probable threading issue. 6 years ago
David Wilson f24f02ba06 issue #321: take remote_tmp and system_tmpdirs into account.
Can't simply ignore these settings as some users may have weird noexec
filesystems.
6 years ago
David Wilson a2686b1a2c issue #321: simplify temp directory handling. 6 years ago
David Wilson 3d588323ff issue #340: use expanded delegate_to hostname, not template.
PlayContext.delegate_to is the unexpanded template, Ansible doesn't keep
a copy of it around anywhere convenient. We either need to re-expand it
or take the expanded version that was stored on the Task, which is what
is done here.
6 years ago
David Wilson a6995a5288 issue #338: refactor env handling into class and fix tests. 6 years ago
David Wilson 916e46621b issue #340: add connection delegation tests. 6 years ago
David Wilson 6dcd5f8998 issue #340: split up Connection._connect()
The logic was getting too busy.
6 years ago
David Wilson a1e653978b issue #340: connection delegation used wrong variable name.
When inventory name did not match remote_addr, it would attempt to SSH
to the inventory name.
6 years ago
David Wilson 370b98f960 ansible: tidy up connection.py.
- more docstrings.
- _wrap_or_none -> optional_secret()
6 years ago
David Wilson 9e572a7939 ansible: fix duplicate MuxProcess socket write.
The while: loop was necessary due to some cutpaste further on down the
file.
6 years ago
David Wilson b521f215fd ansible: handle >2.6 magic exceptions + sys.excepthook damage
Closes #332.
6 years ago
David Wilson 053c594d65 ansible: prevent logs spamming user console on exit.
Closes #331.
6 years ago
David Wilson 1f21a30e7f issue #251: ansible: watch for delegate_to during connection delegation.
This needs more work -- pretty certain that python_path and suchlike are
coming from the wrong place. Possibly we need another config_from_..()
specialized for delegate_to.
6 years ago
David Wilson 6c4b01642c ansible: don't crash when adhoc tries to run a missing module.
ansible-playbook prints a separate error during parsing stage, adhoc
performs no such check.
6 years ago
David Wilson 34a9f67115 issue #339: whoops, actually wire up new connection method. 6 years ago
David Wilson 1473f49505 ansible: emulate /etc/environment reloading behaviour of vanilla.
This change is relatively incomplete -- ideally we could snapshot
os.environ and /etc/environment at startup and respect key deletions
too, but that's a lot more work. Wait for a bug report instead.

Closes #338.
6 years ago
David Wilson 4077182fb2 ansible: plugins were missing absolute_import. 6 years ago
David Wilson 5c573f7fcb ansible: insert short sleep when MITOGEN_PROFILING active.
Hacky, but works fine.
6 years ago
David Wilson d62e6e2a7f ansible: serialize calls to ModuleDepService.
Concurrent calls to ModuleDepService would cause significant wasted
work, as potentially all pool threads run the same uncached module dep
scan.

Without:
         3243581 function calls (3233009 primitive calls) in 4770.672 seconds

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     2523    0.011    0.000   39.849    0.016 services.py:409(scan)

With:
         2801561 function calls (2800042 primitive calls) in 5166.843 seconds

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     2506    0.009    0.000    1.967    0.001 services.py:411(scan)

Ignore timing variance due to problems with the test job.
6 years ago
David Wilson 3e0de9790c issue #324: fix Python 3 fallout for custom module_utils.
Also enable at last one of its tests.
6 years ago
David Wilson f4a66194e4 ansible: Py3.x fixes for Ansible PR #41749 workaround. 6 years ago
David Wilson 6c03b83748 issue #291: don't attempt mitogen import until sys.path modified.
Given an extracted download of mitogen-2.2.tar.gz, with strategy_plugins
pointing into it, if an old version of the package was pip-installed,
then the old pip-installed package would be imported and override
whatever came from the tarball.

Instead, modify sys.path before attempting any import. This still isn't
perfect, but it's better.
6 years ago
David Wilson f8b3441431 ansible: work around Ansible PR #41749 6 years ago
David Wilson bfe9f81d0b ansible: fix RPC time logging.
Rendering call arguemtns was broken for non-positional arguments.
6 years ago
David Wilson 71b4294888 issue #291: 2.6 compat fix. 6 years ago
David Wilson b44b823c4a ansible: make _remote_expand_user() pay attention to sudoable=.. 6 years ago
David Wilson a8e4dcc98d issue #301: correct remote_tmp evaluation context.
Vanilla Ansible expands remote_tmp variables in the context of the login
account, not any become_user account.
6 years ago
David Wilson de0d526487 issue #291: restore behaviour of invoking binaries via /bin/sh
This ensures failed task output matches vanilla Ansible exactly (as it
did before starting #291).
6 years ago
David Wilson c5ea7c45a1 comments/docs: correct mitogen.master.Context -> mitogen.parent.Context. 6 years ago
David Wilson 2c74eac19a issue #291: more Ansible-compatible script invocation
When running any kind of script, rewrite the hashbang like Ansible does,
but subsequently ignore it and explicitly use a fragment of shell from
the ansible_*_interpreter variable to call the interpreter, just like
Ansible does.

This fixes hashbangs containing '/usr/bin/env A=1 bash' on Linux, where
putting that into a hashbang line results in an infinite loop.
6 years ago
David Wilson 17dda781c0 issue #317: ansible: fix log filtering in several cases
* mitogen/ansible_mitogen should only generate ERROR-level logs in
  log_path unless -vvv is enabled.
* Targets were accidentally configured to always have DEBUG set, causing
  many log messages to be sent on the wire even though they would be
  filtered in the master.

Closes #317.
6 years ago
David Wilson b6d6468c92 issue #301: support expandvars() for remote_tmp only.
Vanilla Ansible support expandvars-like expansions widely in a variety
of places. Prefer to whitelist those we need, rather than sprinkling
hellish semantics everywhere.
6 years ago
David Wilson 417e02bd6d ansible: copy Ansible's method for finding the display. 6 years ago
David Wilson e39c602fd3 issue #291: support UNIX hashbang syntax for ansible_*_interpreter.
Closes #291.
6 years ago
David Wilson 15d68b3c32 issue #309: fix environment cleanup regression.
Closes #309.
6 years ago
David Wilson 3f3c9cd001 Merge commit 'refs/pull/303/head' of github.com:dw/mitogen into dmw
issue #303
6 years ago
David Wilson b9c88d344b issue #299: ansible: fix PluginLoader.get() monkey-patch
This prototype is broken for network_cli connections.
6 years ago
napkindrawing 745d72bb1d core: support for "doas" become_method 6 years ago
David Wilson 6dd9e8984d issue #297: fix viewkeys use on 3.x. 6 years ago
David Wilson 124d8023a2 issue #297: don't bother trying to restore old CWD. 6 years ago
David Wilson d8e0c9e12c issue #297: local commands must execute with WorkerProcess environment. 6 years ago
David Wilson 012745efea issue #297: local actions must execute with fixed directory.
Local actions must execute in the the parent directory of the playbook
that defines the action.
6 years ago
David Wilson 5b03e06457 issue #294: ansible: fix mixed vanilla/Mitogen runs.
Don't bother trying to understand what damage PluginLoader has done to
ansible.plugins.* namespace, just ask it for the base class instead.
6 years ago