This implements the first edition of Connection Delegation, where
delegating connection establishment is initially single-threaded.
ansible_mitogen/strategy.py:
ansible_mitogen/plugins/connection/*:
Begin splitting connection.Connection into subclasses, exposing them
directly as "mitogen_ssh", "mitogen_local", etc. connection types.
This is far from removing strategy.py, but it's a tiny start.
ansible_mitogen/connection.py:
* config_from_play_context() and config_from_host_vars() build up a
huge dictionary containing either more or less PlayContext contents,
or our best attempt at reconstructing a host's connection config
from its hostvars, where that config is not the current
WorkerProcess target.
They both produce the same format with the same keys, allowing
remaining code to have a single input format.
These dicts contain fields named after how Ansible refers to them,
e.g. "sudo_exe".
* _config_from_via() parses a basic connection specification like
"username@inventory_name" into one of the aforementioned dicts.
* _stack_from_config() produces a list of dicts describing the order
in which (Mitogen) connections should be established, such that each
element is proxied via= the previous element. The dicts produced by
this function use Mitogen keyword arguments, the former di.
These dicts contain fields named after how Mitogen refers to them,
e.g. "sudo_path".
* Pass the stack to ContextService, which is responsible for actual
setup of the full chain.
ansible_mitogen/services.py:
Teach get() to walk the supplied stack, establishing each connection
in turn, creating refounts for it before continuing.
TODO: refcounting is broken in a variety of cases.
This commit only uses it for the target.get_file() helper, which is only
used for transferring modules. The next commit wires it into the
Connection.transfer_file() API, which is the method the copy module
uses.
This change blocks off 2 common scenarios where a race condition is
upgraded to a hang, when the library could internally do better.
* Since we don't know whether the receiver of a `reply_to` is expecting
a raw or pickled message, and since in the case of a raw reply, there
is no way to signal "dead" to the receiver, override the reply_to
field to explicitly mark a message as dead using a special handle.
This replaces the serialized _DEAD sentinel value with a slightly
neater interface, in the form of the reserved IS_DEAD handle, and
enables an important subsequent change: when a context cannot route a
message, it can send a generic 'dead' reply back towards the message
source, ensuring any sleeping thread is woken with ChannelError.
The use of this field could potentially be extended later on if
additional flags are needed, but for now this seems to suffice.
* Teach Router._invoke() to reply with a dead message when it receives a
message for an invalid local handle.
* Teach Router._async_route() to reply with a dead message when it
receives an unroutable message.
The module name comes from YAML via Jinja2.. it's always Unicode. Mixing
it into a temporary directory name produces a Unicode tempdir name,
which ends up in sys.argv via TemporaryArgv.
This is a partial fix, there are still at least 2 cases needing covered:
- In-progress connections must have CallError or similar sent to any
waiters
- Once connection delegation exists, it is possible for other worker
processes to be active (and in any step in the process), trying to
communicate with a context that we know can no longer be communicated
with. The solution to that isn't clear yet.
Additionally ensure root has /bin/bash shell in both Docker images.
And by "compatible" I mean "terrible". This does not implement async job
timeouts, but I'm not going to bother, upstream async implementation is
so buggy and inconsistent it resists even having its behaviour captured
in tests.
There is no guarantee on the ordering select() returns file descriptors.
So if, e.g. in the case of sudo_nonexistent.yml, sudo prints an error
to a single FD before exitting, there was previously no gurantee
iter_read() would read off the error before failing due to detecting
disconnect on any FD.
Now instead we keep reading while any non-disconnected FD exists.
Presently there is still no mechanism to add :attr:`tty_stream` to the
multiplexer after connection is successful, but for now it's not
expected that anything will be logged to it anyway.
Closes#148.
David Wilson