Commit Graph

196 Commits (48535cc66177a73a2d5d418838b15b4ddf4cda69)

Author SHA1 Message Date
David Wilson 2ad0d0521d ansible: reload /etc/resolv.conf for each task.
The OpenShift installer modifies /etc/resolv.conf then tests the new
resolver configuration, however, there was no mechanism to reload
resolv.conf in our reuseable interpreter.

https://github.com/openshift/openshift-ansible/blob/release-3.9/roles/openshift_web_console/tasks/install.yml#L137

This inserts an explicit call to res_init() for every new style
invocation, with an approximate cost of ~1usec on Linux since glibc
verifies resolv.conf has changed before reloading it.

There is little to be done for users of the thread-safe resolver APIs,
their state is hidden from us. If bugs like that manifest, whack-a-mole
style 'del sys.modules[thatmod]' patches may suffice.
7 years ago
David Wilson 49eae23f92 issue #218: ansibe: use Secret and Blob types. 7 years ago
David Wilson 69f58875c8 tests: more ansible fixes from recent brakage. 7 years ago
David Wilson ad1f624750 ansible: document and rearrange Runner params.
Move emulate_tty to where it's used.
7 years ago
David Wilson 1186b927f9 ansible: remove seemingly unused raw_params
Traced git log all the way back to beginning of time, and checked
Ansible versions starting Jan 2016. Zero clue where this came from, but
the convention suggests it came from Ansible at some point.
7 years ago
David Wilson f737ff5276 ansible: stop passing through remote_tmp variable
Ansiballz does not use remote_tmp so neither should we, per #239.
7 years ago
David Wilson f9e1905ec6 issue #199: ansible: stop writing temp files for new style modules
While adding support for non-new style module types, NewStyleRunner
began writing modules to a temporary file, and sys.argv was patched to
actually include the script filename. The argv change was never required
to fix any particular bug, and a search of the standard modules reveals
no argv users. Update argv[0] to be '', like an interactive interpreter
would have.

While fixing #210, new style runner began setting __file__ to the
temporary file path in order to allow apt.py to discover the Ansiballz
temporary directory. 5 out of 1,516 standard modules follow this
pattern, but in each case, none actually attempt to access __file__,
they just call dirname on it. Therefore do not write the contents of
file, simply set it to the path as it would exist, within a real
temporary directory.

Finally move temporary directory creation out of runner and into target.
Now a single directory exists for the duration of a run, and is emptied
by runner.py as necessary after each task invocation.

This could be further extended to stop rewriting non-new-style modules
in a with_items loop, but that's another step.

Finally the last bullet point in the documentation almost isn't a lie
again.
7 years ago
David Wilson 1bc08323bf ansible: more compatible module script naming. 7 years ago
David Wilson 8bd34e1e28 ansible: gracefully report connection timeouts as StreamError. 7 years ago
David Wilson 4d1c6d2101 issue #127: ssh: reasonable solution to host key checking.
Ideally it would be possible to specify a callback function, but this is
not possible for proxied connections. So simply provide the 3 most
useful modes, defaulting to the most secure.

Closes #127. Closes #134.
7 years ago
David Wilson 9cc8b21a07 ansible: missing mitogen_su module. 7 years ago
David Wilson 78c401ba4d ansible: support su become method. 7 years ago
David Wilson 3203846708 issue #239: ansible: ignore remote_tmp in new style runner. 7 years ago
David Wilson 94e048a2e5 ansible: ensure FileService uses exact CHUNK_SIZE multiple
9.8% throughput increase with sudo.
7 years ago
David Wilson dafe12b315 ansible: fix AnsibleUnicode crash when processing "~username". 7 years ago
David Wilson e1a3cea2f9 ansible: FileService: don't send empty last chunk 7 years ago
David Wilson 2a56c672ca ansible: FileService docstring updates. 7 years ago
David Wilson f5d22a3ca1 core: support deleting handlers, make Receiver.close() unregister 7 years ago
David Wilson 69e5902e61 issue #212: support explicit acknowledgements in FileService. 7 years ago
David Wilson b0309b539c ansible: disable interpreter recycling for connections.
Must explicitly specify enable_lru=True in ContextService.get() to
trigger recycling.
7 years ago
David Wilson 65e6a44fe7 docs: add links. 7 years ago
David Wilson 90f7b4baea ansible: stub plugin documentation. 7 years ago
David Wilson c85a5b6446 ansible: make call timing more readable 7 years ago
David Wilson bba2a42e44 ansible: add mitogen_sudo method, split out connection subclasses.
Slowly moving towards real implementations in those files.
7 years ago
David Wilson 0a9126c510 ansible: connection plugins missing from previous commit. 7 years ago
David Wilson 7c5bbc5168 setns: support changing user.
To match existing third party plugin.
7 years ago
David Wilson 003f30b5a9 issue #226: test fixes. 7 years ago
David Wilson 6edb3f165d ansible: avoid a race during shutdown. 7 years ago
David Wilson 780b63520f issue #226: don't attempt to fchmod() a pathname 7 years ago
David Wilson 219a202a82 issue #226: ansible: file transfer improvements
* put_data() supports setting mode and times.
* put_file() refuses to copy non-regular files (sockets, FIFOs).
* put_file() saves one RTT for <32KiB files by using put_data() and
  embedding file content in argument list.
* FileService returns dict with size/mode/owner/group/mtime/atime.
* FileService refuses to copy non-regular files.
* transfer_file() preserves file mode.
* transfer_file() preserves atime/mtime.
* transfer_file() optionally preserves ownership.
* transfer_file() optionally calls fsync().
* transfer_file() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* transfer_file() ensures temporary file is deleted on any error.
* write_path() writes to a temporary file and deletes it on failure.
* write_path() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* write_path() supports setting symbolic owner/group.
* write_path() optionally calls fsync().
* write_path() supports setting symbolic mode/mtime/atime.

Closes #226, #227, #229
7 years ago
David Wilson b5be0fd65b ansible: log _get_file() timings. 7 years ago
David Wilson 95039eea11 ansible: make key_from_kwargs() 10x faster
It was half the cost of the service call
7 years ago
David Wilson 098995539d ansible: implement FreeBSD jail support. 7 years ago
David Wilson b1563cd8c1 ansible: hijack lxd connections too. 7 years ago
David Wilson 131bc768c7 ansible: implement LXC support. 7 years ago
David Wilson 6fb3a76e68 master: annotate LogForwarder messages.
mitogen/master.py:
    Annotate forwarded log entries with their original source, logger
    name, and message.

ansible:
    mark stderr in red with -vvv

    Tempting to make this appaer 100% of the time, but some crappy
    bashrcs may cause lots of junk to be printed.
7 years ago
David Wilson 4893889a88 ansible: remove vestiges of old/wrong sudo_exe source. 7 years ago
David Wilson 3fab8a3af5 ansible: connection delegation v1
This implements the first edition of Connection Delegation, where
delegating connection establishment is initially single-threaded.

ansible_mitogen/strategy.py:
ansible_mitogen/plugins/connection/*:

  Begin splitting connection.Connection into subclasses, exposing them
  directly as "mitogen_ssh", "mitogen_local", etc. connection types.

  This is far from removing strategy.py, but it's a tiny start.

ansible_mitogen/connection.py:

  * config_from_play_context() and config_from_host_vars() build up a
    huge dictionary containing either more or less PlayContext contents,
    or our best attempt at reconstructing a host's connection config
    from its hostvars, where that config is not the current
    WorkerProcess target.

    They both produce the same format with the same keys, allowing
    remaining code to have a single input format.

    These dicts contain fields named after how Ansible refers to them,
    e.g. "sudo_exe".

  * _config_from_via() parses a basic connection specification like
    "username@inventory_name" into one of the aforementioned dicts.

  * _stack_from_config() produces a list of dicts describing the order
    in which (Mitogen) connections should be established, such that each
    element is proxied via= the previous element. The dicts produced by
    this function use Mitogen keyword arguments, the former di.

    These dicts contain fields named after how Mitogen refers to them,
    e.g. "sudo_path".

  * Pass the stack to ContextService, which is responsible for actual
    setup of the full chain.

ansible_mitogen/services.py:

  Teach get() to walk the supplied stack, establishing each connection
  in turn, creating refounts for it before continuing.

  TODO: refcounting is broken in a variety of cases.
7 years ago
David Wilson 7c6ce726aa ansible: rename variable to reflect correct time unit 7 years ago
David Wilson 2f1df7f82d ansible: FileService wasn't sleeping properly.
"_schedule_pending" is a function, "_pending_by_stream" is the map we
want to test.
7 years ago
David Wilson 86c9978e09 Ensure mod.__file__ is set; closes #210. 7 years ago
David Wilson b2abe74ab6 issue #210: run DebOps under v2.5.1 too. 7 years ago
David Wilson 89fc842ca8 ansible: typo. 7 years ago
David Wilson 21082cec40 ansible: fix ugly formatting. 7 years ago
David Wilson 376fc85000 ansible: FileService docstrings. 7 years ago
David Wilson cf30e88a3e ansible: implement missing FileService.on_shutdown() 7 years ago
David Wilson 5913be64d7 docs: remove last remaining major risk :D 7 years ago
David Wilson cb73c44084 ansible: implement streaming in Connection.put_file().
This is the function the copy module uses.
7 years ago
David Wilson 29087018c7 ansible: implement streaming in FileService.
This commit only uses it for the target.get_file() helper, which is only
used for transferring modules. The next commit wires it into the
Connection.transfer_file() API, which is the method the copy module
uses.
7 years ago
David Wilson 8c3b1fcf15 ansible: disable script interpreter processing for new-style
Re: #199
7 years ago