Commit Graph

93 Commits (9177764c3bbd7e16c3459b5dfd2a338cbe93890e)

Author SHA1 Message Date
David Wilson 50241a922f ansible: call on_fork() on broker shutdown; closes #420. 6 years ago
David Wilson 35092c5d35 tests: Unicode/bytes fixes for integration/connection/exec_command.yml 6 years ago
David Wilson 6c71c5bfef issue #369: disable reset_connection on Ansible<2.5.6
https://github.com/ansible/ansible/issues/27520
6 years ago
David Wilson c4aec22a33 issue #369: fix one more _reset() reference. 6 years ago
David Wilson 6107ebdc0d issue #396: fix compatibility with Connection._reset(). 6 years ago
David Wilson 54445470e2 issue #409: add missing path config variables to severa plugins
So every method can be redirected to a stub implementation.
6 years ago
David Wilson 18af1dfb51 ansible: kubectl_path argument appears in wrong connection method
Closes #409.
6 years ago
David Wilson 96f000c5ea ansible: tilde-expand SSH key before passing to SSH; closes #334. 6 years ago
David Wilson 536690760d issue #369: teach CallChain to reset the connection. 6 years ago
David Wilson 33412927f5 issue #369: refactor Connection to support reset()
Now the tests pass.
6 years ago
David Wilson 4c81eba599 Merge commit 'refs/pull/377/head' of github.com:dw/mitogen into dmw
(Pull #377)

Changes:
- additional_parameters -> extra_args
- Merge with kubectl changes from dmw branch
- Update docs
- Remove unused username class member
- Avoid mutable kubectl_args class member
- Use six.iteritems
6 years ago
David Wilson 5521945bd2 ansible: temporary files take 5. 6 years ago
Yannig Perré 17548d1e49 [Enhancement] handle kubectl vars from Ansible connector.
This change allows the kubectl connector to support the same options as
Ansible's original connector.

The playbook sample comes with an example of a pod containing two containers
and checking that moving from one container to another, the version of Python
changes as expected.
6 years ago
Yannig Perré 6828926a36 Kubernetes connection support for mitogen. 6 years ago
David Wilson 638b196a45 ansible: fix put_file() for large temporary files.
Reverts 49736b3a, large file copies can't avoid the RTT.

The parent stack must be blocked while FileService progresses, as unlike
the small file path, it does not make a snapshot of the (possibly
temporary) file passed by the action plug-in. So we need to keep that
file alive while the service runs.

Add a new integration test and a new soak test to cover both.
6 years ago
David Wilson 90f89f95fb ansible: fix exec_command() regression. 6 years ago
David Wilson 49736b3ae8 ansible: fix FileService call, and remove another roundtrip. 6 years ago
David Wilson e241081cae ansible: stop sharing target temp_dir in runner.
This cannot work with delegate_to, since delegate_to permits multiple
concurrent tasks to be executing on the same target.
6 years ago
David Wilson 43d9815f6d ansible: use CallChain everywhere.
This replaces the 'dump to logger' behaviour of pipelined calls from
before with a call chain that returns any exception on next synchronized
call.
6 years ago
David Wilson b254eb3399 ansible: fix non-action connection instantiation.
e.g. by synchronize module.
6 years ago
David Wilson 705d77a9be ansible: remove a bunch more aliasing from connection.py. 6 years ago
David Wilson 66142e7d75 ansible: fork isolated tasks from correct parent.
Closes #355.
6 years ago
David Wilson da8c6b45b0 ansible: remove task_vars aliasing from connection.py.
Crazy spam creep.
6 years ago
David Wilson 86942b6bf9 ansible: add explanatory exception
If disconnection occurs during a Connection.call(), return
AnsibleConnectionFailure.
6 years ago
David Wilson 9792b8b54f ansible: use template-expanded delegate_to hostname in one more location. 6 years ago
David Wilson 90c2ed03d0 ansible: fix synchronize module
Broken by recent connection delegation fixes.
6 years ago
David Wilson 7458dfae85 ansible: avoid roundtrip for small file transfers.
Calls to connect.put_file() where the file is sufficiently small enough
to fit in a single RPC proceed without waiting for an RPC response. If
the write fails the target context will log an exception, and any
subsequent step depending on the written file will fail.

I verified every built-in action plugin for file transfer calls, and
they all depend on the transferred file in the following step, so this
should be safe.

Reduces template/copy actions to 2-RTT, loop-20-templates.yml runtime
reduced from 30 seconds to 10 seconds over a 250ms link compared to
v0.2.2, and from 123 seconds compared to vanilla with pipelining
enabled.
6 years ago
David Wilson bce4f59138 issue #345: disable IdentitiesOnly by default. 6 years ago
David Wilson a2686b1a2c issue #321: simplify temp directory handling. 6 years ago
David Wilson 3d588323ff issue #340: use expanded delegate_to hostname, not template.
PlayContext.delegate_to is the unexpanded template, Ansible doesn't keep
a copy of it around anywhere convenient. We either need to re-expand it
or take the expanded version that was stored on the Task, which is what
is done here.
6 years ago
David Wilson 916e46621b issue #340: add connection delegation tests. 6 years ago
David Wilson 6dcd5f8998 issue #340: split up Connection._connect()
The logic was getting too busy.
6 years ago
David Wilson a1e653978b issue #340: connection delegation used wrong variable name.
When inventory name did not match remote_addr, it would attempt to SSH
to the inventory name.
6 years ago
David Wilson 370b98f960 ansible: tidy up connection.py.
- more docstrings.
- _wrap_or_none -> optional_secret()
6 years ago
David Wilson 1f21a30e7f issue #251: ansible: watch for delegate_to during connection delegation.
This needs more work -- pretty certain that python_path and suchlike are
coming from the wrong place. Possibly we need another config_from_..()
specialized for delegate_to.
6 years ago
David Wilson 34a9f67115 issue #339: whoops, actually wire up new connection method. 6 years ago
David Wilson bfe9f81d0b ansible: fix RPC time logging.
Rendering call arguemtns was broken for non-positional arguments.
6 years ago
David Wilson 71b4294888 issue #291: 2.6 compat fix. 6 years ago
David Wilson a8e4dcc98d issue #301: correct remote_tmp evaluation context.
Vanilla Ansible expands remote_tmp variables in the context of the login
account, not any become_user account.
6 years ago
David Wilson c5ea7c45a1 comments/docs: correct mitogen.master.Context -> mitogen.parent.Context. 6 years ago
David Wilson 2c74eac19a issue #291: more Ansible-compatible script invocation
When running any kind of script, rewrite the hashbang like Ansible does,
but subsequently ignore it and explicitly use a fragment of shell from
the ansible_*_interpreter variable to call the interpreter, just like
Ansible does.

This fixes hashbangs containing '/usr/bin/env A=1 bash' on Linux, where
putting that into a hashbang line results in an infinite loop.
6 years ago
David Wilson e39c602fd3 issue #291: support UNIX hashbang syntax for ansible_*_interpreter.
Closes #291.
6 years ago
napkindrawing 745d72bb1d core: support for "doas" become_method 6 years ago
David Wilson d8e0c9e12c issue #297: local commands must execute with WorkerProcess environment. 6 years ago
David Wilson 012745efea issue #297: local actions must execute with fixed directory.
Local actions must execute in the the parent directory of the playbook
that defines the action.
6 years ago
David Wilson 410016ff47 Initial Python 3.x port work.
* ansible: use unicode_literals everywhere since it only needs to be
  compatible back to 2.6.
* compat/collections.py: delete this entirely and rip out the parts of
  functools that require it.
* Introduce serializable Kwargs dict subclass that translates keys to
  Unicode on instantiation.
* enable_debug_logging() must set _v/_vv globals.
* cStringIO does not exist in 3.x.
* Treat IOLogger and LogForwarder input as latin-1.
* Avoid ResourceWarnings in first stage by explicitly closing fps.
* Fix preamble_size.py syntax errors.
6 years ago
David Wilson 04b65020ac issue #278: ansible: support mitogen_ssh_debug_level variable. 6 years ago
David Wilson 2fbe1f1b54 Get integration tests running under 2.6.
Closes #270
Closes #273
7 years ago
David Wilson e35694acd5 ansible: flake8 fixes. 7 years ago
David Wilson caffaa79f7 issue #186: rework async/forked tasks again.
The controller must know the ID of the forked child in order to
propagate dependencies to it, so forking+starting the module run cannot
happen entirely on the target, without some additional mechanism to
wait-and-repropagate the deps as they arrive on the target.

Rework things so that init_child() also handles starting the fork parent,
and returns it along with the context's home directory in a single round
trip.

Now master knows the identity of the fork parent, it can directly create
fork children and call run_module_async() in them. This necessitates 2
roundtrips to start an asynchronous task.

This whole thing sucks and entirely needs simplified, but for now things
almost work, so keeping it.

connection.py:
  * Expect ContextService to return the entire dict return value of
    init_child(). Store the fork_contxt from the return value.

planner.py:
  * Rework Planner to store the invocation as an instance attribute, to
    simplify method calls.
  * Add Planner.get_push_files() and Planner.get_module_deps().
  * Add _propagate_deps() which takes a Planner and ensures the deps it
    describes are sent to a (non forked or forked) context.
  * Move async task logic out of target.py and into invoke() /
    _invoke_*().

process.py:
  * Services no longer need references to each other. planner.py handles
    sending module deps with one extra RPC.

services.py:
  * Return "init_child_result" key instead of simple "home_dir" key.
  * Get rid of dep propagation from ModuleDepService, it lives in
    planner.py now.

target.py:
  * Get rid of async task start logic, lives in planner.py now.
7 years ago