Commit Graph

3534 Commits (5ab872f28930a70e03253d0fb97acbbee2a47430)
 

Author SHA1 Message Date
David Wilson 780b63520f issue #226: don't attempt to fchmod() a pathname 7 years ago
David Wilson 58d8f60f57 docs: better connection type docs 7 years ago
David Wilson 219a202a82 issue #226: ansible: file transfer improvements
* put_data() supports setting mode and times.
* put_file() refuses to copy non-regular files (sockets, FIFOs).
* put_file() saves one RTT for <32KiB files by using put_data() and
  embedding file content in argument list.
* FileService returns dict with size/mode/owner/group/mtime/atime.
* FileService refuses to copy non-regular files.
* transfer_file() preserves file mode.
* transfer_file() preserves atime/mtime.
* transfer_file() optionally preserves ownership.
* transfer_file() optionally calls fsync().
* transfer_file() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* transfer_file() ensures temporary file is deleted on any error.
* write_path() writes to a temporary file and deletes it on failure.
* write_path() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* write_path() supports setting symbolic owner/group.
* write_path() optionally calls fsync().
* write_path() supports setting symbolic mode/mtime/atime.

Closes #226, #227, #229
7 years ago
dw 8beb6d339b
Merge pull request #225 from willmerae/setup-import-os
setup: Add missing os import
7 years ago
dw 956e39fd66
Merge pull request #228 from dw/dmw
setns method
7 years ago
David Wilson e8b4c4e683 issue #223: implement setns connection type
machinectl does not support any sensible form of pipe to the child
process, so it is necessary to bypass it when talking to a systemd
container (see systemd/systemd#8850).

This can also form the basis for issue #223, where the post-fork
namespace switching dance required to connect to the Pythonless
container will be the same.
7 years ago
Alex Willmer fbcb789f3f setup: Add missing os import 7 years ago
David Wilson b5be0fd65b ansible: log _get_file() timings. 7 years ago
David Wilson 95039eea11 ansible: make key_from_kwargs() 10x faster
It was half the cost of the service call
7 years ago
dw 0ee4ddcec1
Merge pull request #224 from dw/dmw
docs: links to Ansible docs
7 years ago
David Wilson 79c2d6c289 docs: links to Ansible docs 7 years ago
dw ae7ad88c93
Merge pull request #222 from dw/dmw
FreeBSD Jail support
7 years ago
David Wilson 098995539d ansible: implement FreeBSD jail support. 7 years ago
David Wilson 3196b6e7f7 Add FreeBSD jail support. 7 years ago
dw be7db7918f
Merge pull request #221 from dw/dmw
docs: typo
7 years ago
David Wilson d8e71799e7 docs: typo 7 years ago
dw 475b160fd7
Merge pull request #220 from dw/dmw
Docker fixes, logging improvements, LXC support
7 years ago
David Wilson b1563cd8c1 ansible: hijack lxd connections too. 7 years ago
David Wilson 131bc768c7 ansible: implement LXC support. 7 years ago
David Wilson b3d352c601 Add lxc container support. 7 years ago
David Wilson 1be03eb458 parent: change create_child interface.
To allow for additional arguments.
7 years ago
David Wilson 1fc7df5be5 Move canonical library version to __init__.py. 7 years ago
David Wilson 6fb3a76e68 master: annotate LogForwarder messages.
mitogen/master.py:
    Annotate forwarded log entries with their original source, logger
    name, and message.

ansible:
    mark stderr in red with -vvv

    Tempting to make this appaer 100% of the time, but some crappy
    bashrcs may cause lots of junk to be printed.
7 years ago
David Wilson 9d0949eb99 docker: fixes & add username parameter. 7 years ago
David Wilson 823dea4bd0 docs: link ChangeLog. 7 years ago
dw 09473da3e0
Merge pull request #219 from dw/dmw
Doc updates, connection delegation v1.
7 years ago
David Wilson 4893889a88 ansible: remove vestiges of old/wrong sudo_exe source. 7 years ago
David Wilson 02ce332b26 docs: show become_user example for connection delegation. 7 years ago
David Wilson 39f5aa76ae docs: add initial ChangeLog. 7 years ago
David Wilson 3ce6b36932 docs: major Ansible page update. 7 years ago
David Wilson fa4746f656 ansible: add connection delegation test targets.
For command line use only, no integration tests yet.
7 years ago
David Wilson 3fab8a3af5 ansible: connection delegation v1
This implements the first edition of Connection Delegation, where
delegating connection establishment is initially single-threaded.

ansible_mitogen/strategy.py:
ansible_mitogen/plugins/connection/*:

  Begin splitting connection.Connection into subclasses, exposing them
  directly as "mitogen_ssh", "mitogen_local", etc. connection types.

  This is far from removing strategy.py, but it's a tiny start.

ansible_mitogen/connection.py:

  * config_from_play_context() and config_from_host_vars() build up a
    huge dictionary containing either more or less PlayContext contents,
    or our best attempt at reconstructing a host's connection config
    from its hostvars, where that config is not the current
    WorkerProcess target.

    They both produce the same format with the same keys, allowing
    remaining code to have a single input format.

    These dicts contain fields named after how Ansible refers to them,
    e.g. "sudo_exe".

  * _config_from_via() parses a basic connection specification like
    "username@inventory_name" into one of the aforementioned dicts.

  * _stack_from_config() produces a list of dicts describing the order
    in which (Mitogen) connections should be established, such that each
    element is proxied via= the previous element. The dicts produced by
    this function use Mitogen keyword arguments, the former di.

    These dicts contain fields named after how Mitogen refers to them,
    e.g. "sudo_path".

  * Pass the stack to ContextService, which is responsible for actual
    setup of the full chain.

ansible_mitogen/services.py:

  Teach get() to walk the supplied stack, establishing each connection
  in turn, creating refounts for it before continuing.

  TODO: refcounting is broken in a variety of cases.
7 years ago
David Wilson 3a07ed3d65 Add mitogen.service to preamble_size.py 7 years ago
David Wilson 962ba862e9 tests: use test-targets group, not all group 7 years ago
David Wilson 7c6ce726aa ansible: rename variable to reflect correct time unit 7 years ago
David Wilson 2f1df7f82d ansible: FileService wasn't sleeping properly.
"_schedule_pending" is a function, "_pending_by_stream" is the map we
want to test.
7 years ago
dw 3978e4e165
Merge pull request #211 from dw/dmw
Docstring fixes, Ansible 2.5.1 fix & CI
7 years ago
David Wilson 86c9978e09 Ensure mod.__file__ is set; closes #210. 7 years ago
David Wilson b2abe74ab6 issue #210: run DebOps under v2.5.1 too. 7 years ago
David Wilson e63ae4768e core: support Receiver.get(thread_dead=False)
For tests.
7 years ago
David Wilson 89fc842ca8 ansible: typo. 7 years ago
David Wilson 21082cec40 ansible: fix ugly formatting. 7 years ago
David Wilson 376fc85000 ansible: FileService docstrings. 7 years ago
David Wilson cf30e88a3e ansible: implement missing FileService.on_shutdown() 7 years ago
David Wilson 6541779dd6 tests: import Ansible file transfer benchmark 7 years ago
dw 9ec20086c2
Merge pull request #209 from dw/dmw
Streaming file transfer :D
7 years ago
David Wilson d2345d11e9 docs: remove warning labels. 7 years ago
David Wilson 5913be64d7 docs: remove last remaining major risk :D 7 years ago
David Wilson cb73c44084 ansible: implement streaming in Connection.put_file().
This is the function the copy module uses.
7 years ago
David Wilson 29087018c7 ansible: implement streaming in FileService.
This commit only uses it for the target.get_file() helper, which is only
used for transferring modules. The next commit wires it into the
Connection.transfer_file() API, which is the method the copy module
uses.
7 years ago