Commit Graph

996 Commits (ff7fb00569155bc53b00b65682c3812680c46ff9)
 

Author SHA1 Message Date
David Wilson ff7fb00569 parent: return latch to wait() caller to allow graceful timeout 7 years ago
David Wilson b0309b539c ansible: disable interpreter recycling for connections.
Must explicitly specify enable_lru=True in ContextService.get() to
trigger recycling.
7 years ago
David Wilson ec22fee7a7 docs: more ansible updates 7 years ago
David Wilson c0ced6d04a core: fix monster fork FD leak
_sockets only refers to the idle sockets list, it doesn't refer to every
socket currently in use by a Latch, for example, the 2*16 used by e.g.
Ansible's sleeping service pool.
7 years ago
David Wilson 7316c08237 core: fix _tls_init() race.
The GIL could be lost between the check for an empty list and popping a
socket off the list. Previously _tls_init (per its name) used per-thread
storage, hence the bug.
7 years ago
David Wilson 79fd35092b docs: more links 7 years ago
David Wilson fdac4ecce8 setns: typos 7 years ago
David Wilson 65e6a44fe7 docs: add links. 7 years ago
David Wilson e93ac2f3a7 debug: implement some basic helpers to debugger. 7 years ago
David Wilson baa4e75526 tests: tidy up thread_pileup for use as a toy benchmark. 7 years ago
David Wilson 90f7b4baea ansible: stub plugin documentation. 7 years ago
David Wilson c85a5b6446 ansible: make call timing more readable 7 years ago
David Wilson 29f6c46e97 docs: add file transfer safety section. 7 years ago
David Wilson bba2a42e44 ansible: add mitogen_sudo method, split out connection subclasses.
Slowly moving towards real implementations in those files.
7 years ago
David Wilson 0a9126c510 ansible: connection plugins missing from previous commit. 7 years ago
David Wilson b61c291faf examples: import mitogen-fuse.py. 7 years ago
David Wilson 7c5bbc5168 setns: support changing user.
To match existing third party plugin.
7 years ago
David Wilson 947d35649c parent: note exception machine's hostname.
For dumb situations where user (i.e. me) is trying to fix a problem in
the wrong place.
7 years ago
David Wilson 003f30b5a9 issue #226: test fixes. 7 years ago
David Wilson 6edb3f165d ansible: avoid a race during shutdown. 7 years ago
David Wilson 780b63520f issue #226: don't attempt to fchmod() a pathname 7 years ago
David Wilson 58d8f60f57 docs: better connection type docs 7 years ago
David Wilson 219a202a82 issue #226: ansible: file transfer improvements
* put_data() supports setting mode and times.
* put_file() refuses to copy non-regular files (sockets, FIFOs).
* put_file() saves one RTT for <32KiB files by using put_data() and
  embedding file content in argument list.
* FileService returns dict with size/mode/owner/group/mtime/atime.
* FileService refuses to copy non-regular files.
* transfer_file() preserves file mode.
* transfer_file() preserves atime/mtime.
* transfer_file() optionally preserves ownership.
* transfer_file() optionally calls fsync().
* transfer_file() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* transfer_file() ensures temporary file is deleted on any error.
* write_path() writes to a temporary file and deletes it on failure.
* write_path() uses unique temporary file name to avoid conflicting
  with parallel transfers.
* write_path() supports setting symbolic owner/group.
* write_path() optionally calls fsync().
* write_path() supports setting symbolic mode/mtime/atime.

Closes #226, #227, #229
7 years ago
David Wilson e8b4c4e683 issue #223: implement setns connection type
machinectl does not support any sensible form of pipe to the child
process, so it is necessary to bypass it when talking to a systemd
container (see systemd/systemd#8850).

This can also form the basis for issue #223, where the post-fork
namespace switching dance required to connect to the Pythonless
container will be the same.
7 years ago
David Wilson b5be0fd65b ansible: log _get_file() timings. 7 years ago
David Wilson 95039eea11 ansible: make key_from_kwargs() 10x faster
It was half the cost of the service call
7 years ago
David Wilson 79c2d6c289 docs: links to Ansible docs 7 years ago
David Wilson 098995539d ansible: implement FreeBSD jail support. 7 years ago
David Wilson 3196b6e7f7 Add FreeBSD jail support. 7 years ago
David Wilson d8e71799e7 docs: typo 7 years ago
David Wilson b1563cd8c1 ansible: hijack lxd connections too. 7 years ago
David Wilson 131bc768c7 ansible: implement LXC support. 7 years ago
David Wilson b3d352c601 Add lxc container support. 7 years ago
David Wilson 1be03eb458 parent: change create_child interface.
To allow for additional arguments.
7 years ago
David Wilson 1fc7df5be5 Move canonical library version to __init__.py. 7 years ago
David Wilson 6fb3a76e68 master: annotate LogForwarder messages.
mitogen/master.py:
    Annotate forwarded log entries with their original source, logger
    name, and message.

ansible:
    mark stderr in red with -vvv

    Tempting to make this appaer 100% of the time, but some crappy
    bashrcs may cause lots of junk to be printed.
7 years ago
David Wilson 9d0949eb99 docker: fixes & add username parameter. 7 years ago
David Wilson 823dea4bd0 docs: link ChangeLog. 7 years ago
David Wilson 4893889a88 ansible: remove vestiges of old/wrong sudo_exe source. 7 years ago
David Wilson 02ce332b26 docs: show become_user example for connection delegation. 7 years ago
David Wilson 39f5aa76ae docs: add initial ChangeLog. 7 years ago
David Wilson 3ce6b36932 docs: major Ansible page update. 7 years ago
David Wilson fa4746f656 ansible: add connection delegation test targets.
For command line use only, no integration tests yet.
7 years ago
David Wilson 3fab8a3af5 ansible: connection delegation v1
This implements the first edition of Connection Delegation, where
delegating connection establishment is initially single-threaded.

ansible_mitogen/strategy.py:
ansible_mitogen/plugins/connection/*:

  Begin splitting connection.Connection into subclasses, exposing them
  directly as "mitogen_ssh", "mitogen_local", etc. connection types.

  This is far from removing strategy.py, but it's a tiny start.

ansible_mitogen/connection.py:

  * config_from_play_context() and config_from_host_vars() build up a
    huge dictionary containing either more or less PlayContext contents,
    or our best attempt at reconstructing a host's connection config
    from its hostvars, where that config is not the current
    WorkerProcess target.

    They both produce the same format with the same keys, allowing
    remaining code to have a single input format.

    These dicts contain fields named after how Ansible refers to them,
    e.g. "sudo_exe".

  * _config_from_via() parses a basic connection specification like
    "username@inventory_name" into one of the aforementioned dicts.

  * _stack_from_config() produces a list of dicts describing the order
    in which (Mitogen) connections should be established, such that each
    element is proxied via= the previous element. The dicts produced by
    this function use Mitogen keyword arguments, the former di.

    These dicts contain fields named after how Mitogen refers to them,
    e.g. "sudo_path".

  * Pass the stack to ContextService, which is responsible for actual
    setup of the full chain.

ansible_mitogen/services.py:

  Teach get() to walk the supplied stack, establishing each connection
  in turn, creating refounts for it before continuing.

  TODO: refcounting is broken in a variety of cases.
7 years ago
David Wilson 3a07ed3d65 Add mitogen.service to preamble_size.py 7 years ago
David Wilson 962ba862e9 tests: use test-targets group, not all group 7 years ago
David Wilson 7c6ce726aa ansible: rename variable to reflect correct time unit 7 years ago
David Wilson 2f1df7f82d ansible: FileService wasn't sleeping properly.
"_schedule_pending" is a function, "_pending_by_stream" is the map we
want to test.
7 years ago
dw 3978e4e165
Merge pull request #211 from dw/dmw
Docstring fixes, Ansible 2.5.1 fix & CI
7 years ago
David Wilson 86c9978e09 Ensure mod.__file__ is set; closes #210. 7 years ago