archive
/
mitogen
mirror of https://github.com/mitogen-hq/mitogen.git
1
0
Fork 0
Code Releases Activity

issue #337: ssh: support disabling PTY allocation

Browse Source 
`.ssh(batch_mode=True)`
pull/372/head
David Wilson 6 years ago
parent c4c6ae88a4
commit 5004207705
4 changed files with 327 additions and 169 deletions
Show Stats Download Patch File Download Diff File

315
docs/api.rst
Unescape Escape View File

@ -35,9 +35,9 @@ mitogen.core
Decorator that marks a function or class method to automatically receive a Decorator that marks a function or class method to automatically receive a
kwarg named `econtext`, referencing the kwarg named `econtext`, referencing the
:py:class:`mitogen.core.ExternalContext` active in the context in which the :class:`mitogen.core.ExternalContext` active in the context in which the
function is being invoked in. The decorator is only meaningful when the function is being invoked in. The decorator is only meaningful when the
function is invoked via :py:data:`CALL_FUNCTION function is invoked via :data:`CALL_FUNCTION
<mitogen.core.CALL_FUNCTION>`. <mitogen.core.CALL_FUNCTION>`.
When the function is invoked directly, `econtext` must still be passed to When the function is invoked directly, `econtext` must still be passed to
@ -47,10 +47,10 @@ mitogen.core
.. decorator:: takes_router .. decorator:: takes_router
Decorator that marks a function or class method to automatically receive a Decorator that marks a function or class method to automatically receive a
kwarg named `router`, referencing the :py:class:`mitogen.core.Router` kwarg named `router`, referencing the :class:`mitogen.core.Router`
active in the context in which the function is being invoked in. The active in the context in which the function is being invoked in. The
decorator is only meaningful when the function is invoked via decorator is only meaningful when the function is invoked via
:py:data:`CALL_FUNCTION <mitogen.core.CALL_FUNCTION>`. :data:`CALL_FUNCTION <mitogen.core.CALL_FUNCTION>`.
When the function is invoked directly, `router` must still be passed to it When the function is invoked directly, `router` must still be passed to it
explicitly. explicitly.
@ -94,18 +94,18 @@ Message Class
.. attribute:: router .. attribute:: router
The :py:class:`mitogen.core.Router` responsible for routing the The :class:`mitogen.core.Router` responsible for routing the
message. This is :py:data:`None` for locally originated messages. message. This is :data:`None` for locally originated messages.
.. attribute:: receiver .. attribute:: receiver
The :py:class:`mitogen.core.Receiver` over which the message was last The :class:`mitogen.core.Receiver` over which the message was last
received. Part of the :py:class:`mitogen.select.Select` interface. received. Part of the :class:`mitogen.select.Select` interface.
Defaults to :py:data:`None`. Defaults to :data:`None`.
.. attribute:: dst_id .. attribute:: dst_id
Integer target context ID. :py:class:`mitogen.core.Router` delivers Integer target context ID. :class:`mitogen.core.Router` delivers
messages locally when their :attr:`dst_id` matches messages locally when their :attr:`dst_id` matches
:data:`mitogen.context_id`, otherwise they are routed up or downstream. :data:`mitogen.context_id`, otherwise they are routed up or downstream.
@ -117,12 +117,12 @@ Message Class
.. attribute:: auth_id .. attribute:: auth_id
The context ID under whose authority the message is acting. See The context ID under whose authority the message is acting. See
:py:ref:`source-verification`. :ref:`source-verification`.
.. attribute:: handle .. attribute:: handle
Integer target handle in the destination context. This is one of the Integer target handle in the destination context. This is one of the
:py:ref:`standard-handles`, or a dynamically generated handle used to :ref:`standard-handles`, or a dynamically generated handle used to
receive a one-time reply, such as the return value of a function call. receive a one-time reply, such as the return value of a function call.
.. attribute:: reply_to .. attribute:: reply_to
@ -143,12 +143,12 @@ Message Class
.. py:method:: __init__ (\**kwargs) .. py:method:: __init__ (\**kwargs)
Construct a message from from the supplied `kwargs`. :py:attr:`src_id` Construct a message from from the supplied `kwargs`. :attr:`src_id`
and :py:attr:`auth_id` are always set to :py:data:`mitogen.context_id`. and :attr:`auth_id` are always set to :data:`mitogen.context_id`.
.. py:classmethod:: pickled (obj, \**kwargs) .. py:classmethod:: pickled (obj, \**kwargs)
Construct a pickled message, setting :py:attr:`data` to the Construct a pickled message, setting :attr:`data` to the
serialization of `obj`, and setting remaining fields using `kwargs`. serialization of `obj`, and setting remaining fields using `kwargs`.
:returns: :returns:
@ -156,10 +156,10 @@ Message Class
.. method:: unpickle (throw=True) .. method:: unpickle (throw=True)
Unpickle :py:attr:`data`, optionally raising any exceptions present. Unpickle :attr:`data`, optionally raising any exceptions present.
:param bool throw: :param bool throw:
If :py:data:`True`, raise exceptions, otherwise it is the caller's If :data:`True`, raise exceptions, otherwise it is the caller's
responsibility. responsibility.
:raises mitogen.core.CallError: :raises mitogen.core.CallError:
@ -169,8 +169,8 @@ Message Class
.. method:: reply (obj, router=None, \**kwargs) .. method:: reply (obj, router=None, \**kwargs)
Compose a reply to this message and send it using :py:attr:`router`, or Compose a reply to this message and send it using :attr:`router`, or
`router` is :py:attr:`router` is :data:`None`. `router` is :attr:`router` is :data:`None`.
:param obj: :param obj:
Either a :class:`Message`, or an object to be serialized in order Either a :class:`Message`, or an object to be serialized in order
@ -190,8 +190,8 @@ Router Class
.. class:: Router .. class:: Router
Route messages between parent and child contexts, and invoke handlers Route messages between parent and child contexts, and invoke handlers
defined on our parent context. :py:meth:`Router.route() <route>` straddles defined on our parent context. :meth:`Router.route() <route>` straddles
the :py:class:`Broker <mitogen.core.Broker>` and user threads, it is safe the :class:`Broker <mitogen.core.Broker>` and user threads, it is safe
to call anywhere. to call anywhere.
**Note:** This is the somewhat limited core version of the Router class **Note:** This is the somewhat limited core version of the Router class
@ -217,7 +217,7 @@ Router Class
.. method:: stream_by_id (dst_id) .. method:: stream_by_id (dst_id)
Return the :py:class:`mitogen.core.Stream` that should be used to Return the :class:`mitogen.core.Stream` that should be used to
communicate with `dst_id`. If a specific route for `dst_id` is not communicate with `dst_id`. If a specific route for `dst_id` is not
known, a reference to the parent context's stream is returned. known, a reference to the parent context's stream is returned.
@ -260,24 +260,24 @@ Router Class
:param function policy: :param function policy:
Function invoked as `policy(msg, stream)` where `msg` is a Function invoked as `policy(msg, stream)` where `msg` is a
:py:class:`mitogen.core.Message` about to be delivered, and :class:`mitogen.core.Message` about to be delivered, and
`stream` is the :py:class:`mitogen.core.Stream` on which it was `stream` is the :class:`mitogen.core.Stream` on which it was
received. The function must return :py:data:`True`, otherwise an received. The function must return :data:`True`, otherwise an
error is logged and delivery is refused. error is logged and delivery is refused.
Two built-in policy functions exist: Two built-in policy functions exist:
* :py:func:`mitogen.core.has_parent_authority`: requires the * :func:`mitogen.core.has_parent_authority`: requires the
message arrived from a parent context, or a context acting with a message arrived from a parent context, or a context acting with a
parent context's authority (``auth_id``). parent context's authority (``auth_id``).
* :py:func:`mitogen.parent.is_immediate_child`: requires the * :func:`mitogen.parent.is_immediate_child`: requires the
message arrived from an immediately connected child, for use in message arrived from an immediately connected child, for use in
messaging patterns where either something becomes buggy or messaging patterns where either something becomes buggy or
insecure by permitting indirect upstream communication. insecure by permitting indirect upstream communication.
In case of refusal, and the message's ``reply_to`` field is In case of refusal, and the message's ``reply_to`` field is
nonzero, a :py:class:`mitogen.core.CallError` is delivered to the nonzero, a :class:`mitogen.core.CallError` is delivered to the
sender indicating refusal occurred. sender indicating refusal occurred.
:return: :return:
@ -297,7 +297,7 @@ Router Class
destination is the local context, then arrange for it to be dispatched destination is the local context, then arrange for it to be dispatched
using the local handlers. using the local handlers.
This is a lower overhead version of :py:meth:`route` that may only be This is a lower overhead version of :meth:`route` that may only be
called from the I/O multiplexer thread. called from the I/O multiplexer thread.
:param mitogen.core.Stream stream: :param mitogen.core.Stream stream:
@ -308,11 +308,11 @@ Router Class
.. method:: route(msg) .. method:: route(msg)
Arrange for the :py:class:`Message` `msg` to be delivered to its Arrange for the :class:`Message` `msg` to be delivered to its
destination using any relevant downstream context, or if none is found, destination using any relevant downstream context, or if none is found,
by forwarding the message upstream towards the master context. If `msg` by forwarding the message upstream towards the master context. If `msg`
is destined for the local context, it is dispatched using the handles is destined for the local context, it is dispatched using the handles
registered with :py:meth:`add_handler`. registered with :meth:`add_handler`.
This may be called from any thread. This may be called from any thread.
@ -321,7 +321,7 @@ Router Class
.. class:: Router (broker=None) .. class:: Router (broker=None)
Extend :py:class:`mitogen.core.Router` with functionality useful to Extend :class:`mitogen.core.Router` with functionality useful to
masters, and child contexts who later become masters. Currently when this masters, and child contexts who later become masters. Currently when this
class is required, the target context's router is upgraded at runtime. class is required, the target context's router is upgraded at runtime.
@ -334,16 +334,16 @@ Router Class
customers or projects. customers or projects.
:param mitogen.master.Broker broker: :param mitogen.master.Broker broker:
:py:class:`Broker` instance to use. If not specified, a private :class:`Broker` instance to use. If not specified, a private
:py:class:`Broker` is created. :class:`Broker` is created.
.. attribute:: profiling .. attribute:: profiling
When :data:`True`, cause the broker thread and any subsequent broker When :data:`True`, cause the broker thread and any subsequent broker
and main threads existing in any child to write and main threads existing in any child to write
``/tmp/mitogen.stats.<pid>.<thread_name>.log`` containing a ``/tmp/mitogen.stats.<pid>.<thread_name>.log`` containing a
:py:mod:`cProfile` dump on graceful exit. Must be set prior to :mod:`cProfile` dump on graceful exit. Must be set prior to
construction of any :py:class:`Broker`, e.g. via: construction of any :class:`Broker`, e.g. via:
.. code:: .. code::
@ -378,7 +378,7 @@ Router Class
and router, and responds to function calls identically to children and router, and responds to function calls identically to children
created using other methods. created using other methods.
For long-lived processes, :py:meth:`local` is always better as it For long-lived processes, :meth:`local` is always better as it
guarantees a pristine interpreter state that inherited little from the guarantees a pristine interpreter state that inherited little from the
parent. Forking should only be used in performance-sensitive scenarios parent. Forking should only be used in performance-sensitive scenarios
where short-lived children must be spawned to isolate potentially buggy where short-lived children must be spawned to isolate potentially buggy
@ -420,10 +420,10 @@ Router Class
immediate copy-on-write to large portions of the process heap. immediate copy-on-write to large portions of the process heap.
* Locks held in the parent causing random deadlocks in the child, such * Locks held in the parent causing random deadlocks in the child, such
as when another thread emits a log entry via the :py:mod:`logging` as when another thread emits a log entry via the :mod:`logging`
package concurrent to another thread calling :py:meth:`fork`. package concurrent to another thread calling :meth:`fork`.
* Objects existing in Thread-Local Storage of every non-:py:meth:`fork` * Objects existing in Thread-Local Storage of every non-:meth:`fork`
thread becoming permanently inaccessible, and never having their thread becoming permanently inaccessible, and never having their
object destructors called, including TLS usage by native extension object destructors called, including TLS usage by native extension
code, triggering many new variants of all the issues above. code, triggering many new variants of all the issues above.
@ -434,16 +434,16 @@ Router Class
case, children continually reuse the same state due to repeatedly case, children continually reuse the same state due to repeatedly
forking from a static parent. forking from a static parent.
:py:meth:`fork` cleans up Mitogen-internal objects, in addition to :meth:`fork` cleans up Mitogen-internal objects, in addition to
locks held by the :py:mod:`logging` package, reseeds locks held by the :mod:`logging` package, reseeds
:py:func:`random.random`, and the OpenSSL PRNG via :func:`random.random`, and the OpenSSL PRNG via
:py:func:`ssl.RAND_add`, but only if the :py:mod:`ssl` module is :func:`ssl.RAND_add`, but only if the :mod:`ssl` module is
already loaded. You must arrange for your program's state, including already loaded. You must arrange for your program's state, including
any third party packages in use, to be cleaned up by specifying an any third party packages in use, to be cleaned up by specifying an
`on_fork` function. `on_fork` function.
The associated stream implementation is The associated stream implementation is
:py:class:`mitogen.fork.Stream`. :class:`mitogen.fork.Stream`.
:param function on_fork: :param function on_fork:
Function invoked as `on_fork()` from within the child process. This Function invoked as `on_fork()` from within the child process. This
@ -459,19 +459,19 @@ Router Class
serialization. serialization.
:param Context via: :param Context via:
Same as the `via` parameter for :py:meth:`local`. Same as the `via` parameter for :meth:`local`.
:param bool debug: :param bool debug:
Same as the `debug` parameter for :py:meth:`local`. Same as the `debug` parameter for :meth:`local`.
:param bool profiling: :param bool profiling:
Same as the `profiling` parameter for :py:meth:`local`. Same as the `profiling` parameter for :meth:`local`.
.. method:: local (remote_name=None, python_path=None, debug=False, connect_timeout=None, profiling=False, via=None) .. method:: local (remote_name=None, python_path=None, debug=False, connect_timeout=None, profiling=False, via=None)
Construct a context on the local machine as a subprocess of the current Construct a context on the local machine as a subprocess of the current
process. The associated stream implementation is process. The associated stream implementation is
:py:class:`mitogen.master.Stream`. :class:`mitogen.master.Stream`.
:param str remote_name: :param str remote_name:
The ``argv[0]`` suffix for the new process. If `remote_name` is The ``argv[0]`` suffix for the new process. If `remote_name` is
@ -493,9 +493,9 @@ Router Class
another tool, such as ``["/usr/bin/env", "python"]``. another tool, such as ``["/usr/bin/env", "python"]``.
:param bool debug: :param bool debug:
If :data:`True`, arrange for debug logging (:py:meth:`enable_debug`) to If :data:`True`, arrange for debug logging (:meth:`enable_debug`) to
be enabled in the new context. Automatically :data:`True` when be enabled in the new context. Automatically :data:`True` when
:py:meth:`enable_debug` has been called, but may be used :meth:`enable_debug` has been called, but may be used
selectively otherwise. selectively otherwise.
:param bool unidirectional: :param bool unidirectional:
@ -510,14 +510,14 @@ Router Class
healthy. Defaults to 30 seconds. healthy. Defaults to 30 seconds.
:param bool profiling: :param bool profiling:
If :data:`True`, arrange for profiling (:py:data:`profiling`) to be If :data:`True`, arrange for profiling (:data:`profiling`) to be
enabled in the new context. Automatically :data:`True` when enabled in the new context. Automatically :data:`True` when
:py:data:`profiling` is :data:`True`, but may be used selectively :data:`profiling` is :data:`True`, but may be used selectively
otherwise. otherwise.
:param mitogen.core.Context via: :param mitogen.core.Context via:
If not :data:`None`, arrange for construction to occur via RPCs If not :data:`None`, arrange for construction to occur via RPCs
made to the context `via`, and for :py:data:`ADD_ROUTE made to the context `via`, and for :data:`ADD_ROUTE
<mitogen.core.ADD_ROUTE>` messages to be generated as appropriate. <mitogen.core.ADD_ROUTE>` messages to be generated as appropriate.
.. code-block:: python .. code-block:: python
@ -534,7 +534,7 @@ Router Class
The ``doas`` process is started in a newly allocated pseudo-terminal, The ``doas`` process is started in a newly allocated pseudo-terminal,
and supports typing interactive passwords. and supports typing interactive passwords.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str username: :param str username:
Username to use, defaults to ``root``. Username to use, defaults to ``root``.
@ -559,7 +559,7 @@ Router Class
temporary new Docker container using the ``docker`` program. One of temporary new Docker container using the ``docker`` program. One of
`container` or `image` must be specified. `container` or `image` must be specified.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str container: :param str container:
Existing container to connect to. Defaults to :data:`None`. Existing container to connect to. Defaults to :data:`None`.
@ -578,7 +578,7 @@ Router Class
Construct a context on the local machine within a FreeBSD jail using Construct a context on the local machine within a FreeBSD jail using
the ``jexec`` program. the ``jexec`` program.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str container: :param str container:
Existing container to connect to. Defaults to :data:`None`. Existing container to connect to. Defaults to :data:`None`.
@ -594,7 +594,7 @@ Router Class
Construct a context on the local machine within an LXC classic Construct a context on the local machine within an LXC classic
container using the ``lxc-attach`` program. container using the ``lxc-attach`` program.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str container: :param str container:
Existing container to connect to. Defaults to :data:`None`. Existing container to connect to. Defaults to :data:`None`.
@ -608,7 +608,7 @@ Router Class
Construct a context on the local machine within a LXD container using Construct a context on the local machine within a LXD container using
the ``lxc`` program. the ``lxc`` program.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str container: :param str container:
Existing container to connect to. Defaults to :data:`None`. Existing container to connect to. Defaults to :data:`None`.
@ -656,7 +656,7 @@ Router Class
``su`` process is started in a newly allocated pseudo-terminal, and ``su`` process is started in a newly allocated pseudo-terminal, and
supports typing interactive passwords. supports typing interactive passwords.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str username: :param str username:
Username to pass to ``su``, defaults to ``root``. Username to pass to ``su``, defaults to ``root``.
@ -683,7 +683,7 @@ Router Class
The ``sudo`` process is started in a newly allocated pseudo-terminal, The ``sudo`` process is started in a newly allocated pseudo-terminal,
and supports typing interactive passwords. and supports typing interactive passwords.
Accepts all parameters accepted by :py:meth:`local`, in addition to: Accepts all parameters accepted by :meth:`local`, in addition to:
:param str username: :param str username:
Username to pass to sudo as the ``-u`` parameter, defaults to Username to pass to sudo as the ``-u`` parameter, defaults to
@ -694,28 +694,32 @@ Router Class
:param str password: :param str password:
The password to use if/when sudo requests it. Depending on the sudo The password to use if/when sudo requests it. Depending on the sudo
configuration, this is either the current account password or the configuration, this is either the current account password or the
target account password. :py:class:`mitogen.sudo.PasswordError` target account password. :class:`mitogen.sudo.PasswordError`
will be raised if sudo requests a password but none is provided. will be raised if sudo requests a password but none is provided.
:param bool set_home: :param bool set_home:
If :py:data:`True`, request ``sudo`` set the ``HOME`` environment If :data:`True`, request ``sudo`` set the ``HOME`` environment
variable to match the target UNIX account. variable to match the target UNIX account.
:param bool preserve_env: :param bool preserve_env:
If :py:data:`True`, request ``sudo`` to preserve the environment of If :data:`True`, request ``sudo`` to preserve the environment of
the parent process. the parent process.
:param list sudo_args: :param list sudo_args:
Arguments in the style of :py:data:`sys.argv` that would normally Arguments in the style of :data:`sys.argv` that would normally
be passed to ``sudo``. The arguments are parsed in-process to set be passed to ``sudo``. The arguments are parsed in-process to set
equivalent parameters. Re-parsing ensures unsupported options cause equivalent parameters. Re-parsing ensures unsupported options cause
:py:class:`mitogen.core.StreamError` to be raised, and that :class:`mitogen.core.StreamError` to be raised, and that
attributes of the stream match the actual behaviour of ``sudo``. attributes of the stream match the actual behaviour of ``sudo``.
.. method:: ssh (hostname, username=None, ssh_path=None, port=None, check_host_keys='enforce', password=None, identity_file=None, identities_only=True, compression=True, \**kwargs) .. method:: ssh (hostname, username=None, ssh_path=None, port=None, check_host_keys='enforce', password=None, identity_file=None, identities_only=True, compression=True, \**kwargs)
Construct a remote context over a ``ssh`` invocation. The ``ssh`` Construct a remote context over an OpenSSH ``ssh`` invocation.
process is started in a newly allocated pseudo-terminal, and supports
typing interactive passwords.
Accepts all parameters accepted by :py:meth:`local`, in addition to: By default, the ``ssh`` process is started in a newly allocated
pseudo-terminal to support typing interactive passwords, however when
making many connections, this may be disabled by specifying
`batch_mode=True`, as most operating systems have a conservative upper
limit on the number of pseudo-terminals that may exist.
Accepts all parameters accepted by :meth:`local`, in addition to:
:param str username: :param str username:
The SSH username; default is unspecified, which causes SSH to pick The SSH username; default is unspecified, which causes SSH to pick
@ -737,7 +741,7 @@ Router Class
unknown hosts cause a connection failure. unknown hosts cause a connection failure.
:param str password: :param str password:
Password to type if/when ``ssh`` requests it. If not specified and Password to type if/when ``ssh`` requests it. If not specified and
a password is requested, :py:class:`mitogen.ssh.PasswordError` is a password is requested, :class:`mitogen.ssh.PasswordError` is
raised. raised.
:param str identity_file: :param str identity_file:
Path to an SSH private key file to use for authentication. Default Path to an SSH private key file to use for authentication. Default
@ -755,12 +759,17 @@ Router Class
present in ``~/.ssh``. This ensures authentication attempts only present in ``~/.ssh``. This ensures authentication attempts only
occur using the supplied password or SSH key. occur using the supplied password or SSH key.
:param bool compression: :param bool compression:
If :py:data:`True`, enable ``ssh`` compression support. Compression If :data:`True`, enable ``ssh`` compression support. Compression
has a minimal effect on the size of modules transmitted, as they has a minimal effect on the size of modules transmitted, as they
are already compressed, however it has a large effect on every are already compressed, however it has a large effect on every
remaining message in the otherwise uncompressed stream protocol, remaining message in the otherwise uncompressed stream protocol,
such as function call arguments and return values. such as function call arguments and return values.
:parama int ssh_debug_level: :param bool batch_mode:
If :data:`True`, disable pseudo-terminal allocation. When
:data:`True`, the `password=` parameter may not be used, since no
PTY exists to enter the password, and the `check_host_keys=`
parameter may not be set to `accept`.
:param int ssh_debug_level:
Optional integer `0..3` indicating the SSH client debug level. Optional integer `0..3` indicating the SSH client debug level.
:raises mitogen.ssh.PasswordError: :raises mitogen.ssh.PasswordError:
A password was requested but none was specified, or the specified A password was requested but none was specified, or the specified
@ -808,12 +817,12 @@ Context Class
The message. The message.
:returns: :returns:
:py:class:`mitogen.core.Receiver` configured to receive any replies :class:`mitogen.core.Receiver` configured to receive any replies
sent to the message's `reply_to` handle. sent to the message's `reply_to` handle.
.. method:: send_await (msg, deadline=None) .. method:: send_await (msg, deadline=None)
As with :py:meth:`send_async`, but expect a single reply As with :meth:`send_async`, but expect a single reply
(`persist=False`) delivered within `deadline` seconds. (`persist=False`) delivered within `deadline` seconds.
:param mitogen.core.Message msg: :param mitogen.core.Message msg:
@ -830,7 +839,7 @@ Context Class
.. class:: Context .. class:: Context
Extend :py:class:`mitogen.core.Router` with functionality useful to Extend :class:`mitogen.core.Router` with functionality useful to
masters, and child contexts who later become parents. Currently when this masters, and child contexts who later become parents. Currently when this
class is required, the target context's router is upgraded at runtime. class is required, the target context's router is upgraded at runtime.
@ -843,7 +852,7 @@ Context Class
terminate a hung context using this method. This will be fixed shortly. terminate a hung context using this method. This will be fixed shortly.
:param bool wait: :param bool wait:
If :py:data:`True`, block the calling thread until the context has If :data:`True`, block the calling thread until the context has
completely terminated. completely terminated.
:returns: :returns:
If `wait` is :data:`False`, returns a :class:`mitogen.core.Latch` If `wait` is :data:`False`, returns a :class:`mitogen.core.Latch`
@ -888,7 +897,7 @@ Context Class
Function keyword arguments, if any. See :ref:`serialization-rules` Function keyword arguments, if any. See :ref:`serialization-rules`
for permitted types. for permitted types.
:returns: :returns:
:py:class:`mitogen.core.Receiver` configured to receive the result :class:`mitogen.core.Receiver` configured to receive the result
of the invocation: of the invocation:
.. code-block:: python .. code-block:: python
@ -903,11 +912,11 @@ Context Class
Asynchronous calls may be dispatched in parallel to multiple Asynchronous calls may be dispatched in parallel to multiple
contexts and consumed as they complete using contexts and consumed as they complete using
:py:class:`mitogen.select.Select`. :class:`mitogen.select.Select`.
.. method:: call (fn, \*args, \*\*kwargs) .. method:: call (fn, \*args, \*\*kwargs)
Equivalent to :py:meth:`call_async(fn, \*args, \**kwargs).get().unpickle() Equivalent to :meth:`call_async(fn, \*args, \**kwargs).get().unpickle()
<call_async>`. <call_async>`.
:returns: :returns:
@ -935,7 +944,7 @@ Receiver Class
Receivers are used to wait for pickled responses from another context to be Receivers are used to wait for pickled responses from another context to be
sent to a handle registered in this context. A receiver may be single-use sent to a handle registered in this context. A receiver may be single-use
(as in the case of :py:meth:`mitogen.parent.Context.call_async`) or (as in the case of :meth:`mitogen.parent.Context.call_async`) or
multiple use. multiple use.
:param mitogen.core.Router router: :param mitogen.core.Router router:
@ -959,12 +968,12 @@ Receiver Class
If not :data:`None`, a reference to a function invoked as If not :data:`None`, a reference to a function invoked as
`notify(receiver)` when a new message is delivered to this receiver. `notify(receiver)` when a new message is delivered to this receiver.
Used by :py:class:`mitogen.select.Select` to implement waiting on Used by :class:`mitogen.select.Select` to implement waiting on
multiple receivers. multiple receivers.
.. py:method:: to_sender () .. py:method:: to_sender ()
Return a :py:class:`mitogen.core.Sender` configured to deliver messages Return a :class:`mitogen.core.Sender` configured to deliver messages
to this receiver. Since a Sender can be serialized, this makes it to this receiver. Since a Sender can be serialized, this makes it
convenient to pass `(context_id, handle)` pairs around:: convenient to pass `(context_id, handle)` pairs around::
@ -981,15 +990,15 @@ Receiver Class
.. py:method:: empty () .. py:method:: empty ()
Return :data:`True` if calling :py:meth:`get` would block. Return :data:`True` if calling :meth:`get` would block.
As with :py:class:`Queue.Queue`, :data:`True` may be returned even As with :class:`Queue.Queue`, :data:`True` may be returned even
though a subsequent call to :py:meth:`get` will succeed, since a though a subsequent call to :meth:`get` will succeed, since a
message may be posted at any moment between :py:meth:`empty` and message may be posted at any moment between :meth:`empty` and
:py:meth:`get`. :meth:`get`.
:py:meth:`empty` is only useful to avoid a race while installing :meth:`empty` is only useful to avoid a race while installing
:py:attr:`notify`: :attr:`notify`:
.. code-block:: python .. code-block:: python
@ -1003,8 +1012,8 @@ Receiver Class
.. py:method:: close () .. py:method:: close ()
Cause :py:class:`mitogen.core.ChannelError` to be raised in any thread Cause :class:`mitogen.core.ChannelError` to be raised in any thread
waiting in :py:meth:`get` on this receiver. waiting in :meth:`get` on this receiver.
.. py:method:: get (timeout=None) .. py:method:: get (timeout=None)
@ -1022,17 +1031,17 @@ Receiver Class
:returns: :returns:
`(msg, data)` tuple, where `msg` is the `(msg, data)` tuple, where `msg` is the
:py:class:`mitogen.core.Message` that was received, and `data` is :class:`mitogen.core.Message` that was received, and `data` is
its unpickled data part. its unpickled data part.
.. py:method:: get_data (timeout=None) .. py:method:: get_data (timeout=None)
Like :py:meth:`get`, except only return the data part. Like :meth:`get`, except only return the data part.
.. py:method:: __iter__ () .. py:method:: __iter__ ()
Block and yield `(msg, data)` pairs delivered to this receiver until Block and yield `(msg, data)` pairs delivered to this receiver until
:py:class:`mitogen.core.ChannelError` is raised. :class:`mitogen.core.ChannelError` is raised.
Sender Class Sender Class
@ -1043,10 +1052,10 @@ Sender Class
.. class:: Sender (context, dst_handle) .. class:: Sender (context, dst_handle)
Senders are used to send pickled messages to a handle in another context, Senders are used to send pickled messages to a handle in another context,
it is the inverse of :py:class:`mitogen.core.Sender`. it is the inverse of :class:`mitogen.core.Sender`.
Senders may be serialized, making them convenient to wire up data flows. Senders may be serialized, making them convenient to wire up data flows.
See :py:meth:`mitogen.core.Receiver.to_sender` for more information. See :meth:`mitogen.core.Receiver.to_sender` for more information.
:param mitogen.core.Context context: :param mitogen.core.Context context:
Context to send messages to. Context to send messages to.
@ -1055,7 +1064,7 @@ Sender Class
.. py:method:: close () .. py:method:: close ()
Send a dead message to the remote end, causing :py:meth:`ChannelError` Send a dead message to the remote end, causing :meth:`ChannelError`
to be raised in any waiting thread. to be raised in any waiting thread.
.. py:method:: send (data) .. py:method:: send (data)
@ -1074,11 +1083,11 @@ Select Class
Support scatter/gather asynchronous calls and waiting on multiple Support scatter/gather asynchronous calls and waiting on multiple
receivers, channels, and sub-Selects. Accepts a sequence of receivers, channels, and sub-Selects. Accepts a sequence of
:py:class:`mitogen.core.Receiver` or :py:class:`mitogen.select.Select` :class:`mitogen.core.Receiver` or :class:`mitogen.select.Select`
instances and returns the first value posted to any receiver or select. instances and returns the first value posted to any receiver or select.
If `oneshot` is :data:`True`, then remove each receiver as it yields a If `oneshot` is :data:`True`, then remove each receiver as it yields a
result; since :py:meth:`__iter__` terminates once the final receiver is result; since :meth:`__iter__` terminates once the final receiver is
removed, this makes it convenient to respond to calls made in parallel: removed, this makes it convenient to respond to calls made in parallel:
.. code-block:: python .. code-block:: python
@ -1093,7 +1102,7 @@ Select Class
# Iteration ends when last Receiver yields a result. # Iteration ends when last Receiver yields a result.
print('Received total %s from %s receivers' % (total, len(recvs))) print('Received total %s from %s receivers' % (total, len(recvs)))
:py:class:`Select` may drive a long-running scheduler: :class:`Select` may drive a long-running scheduler:
.. code-block:: python .. code-block:: python
@ -1104,7 +1113,7 @@ Select Class
for context, workfunc in get_new_work(): for context, workfunc in get_new_work():
select.add(context.call_async(workfunc)) select.add(context.call_async(workfunc))
:py:class:`Select` may be nested: :class:`Select` may be nested:
.. code-block:: python .. code-block:: python
@ -1122,11 +1131,11 @@ Select Class
.. py:classmethod:: all (it) .. py:classmethod:: all (it)
Take an iterable of receivers and retrieve a :py:class:`Message` from Take an iterable of receivers and retrieve a :class:`Message` from
each, returning the result of calling `msg.unpickle()` on each in turn. each, returning the result of calling `msg.unpickle()` on each in turn.
Results are returned in the order they arrived. Results are returned in the order they arrived.
This is sugar for handling batch :py:class:`Context.call_async` This is sugar for handling batch :class:`Context.call_async`
invocations: invocations:
.. code-block:: python .. code-block:: python
@ -1146,28 +1155,28 @@ Select Class
for context in contexts) for context in contexts)
Result processing happens concurrently to new results arriving, so Result processing happens concurrently to new results arriving, so
:py:meth:`all` should always be faster. :meth:`all` should always be faster.
.. py:method:: get (timeout=None, block=True) .. py:method:: get (timeout=None, block=True)
Fetch the next available value from any receiver, or raise Fetch the next available value from any receiver, or raise
:py:class:`mitogen.core.TimeoutError` if no value is available within :class:`mitogen.core.TimeoutError` if no value is available within
`timeout` seconds. `timeout` seconds.
On success, the message's :py:attr:`receiver On success, the message's :attr:`receiver
<mitogen.core.Message.receiver>` attribute is set to the receiver. <mitogen.core.Message.receiver>` attribute is set to the receiver.
:param float timeout: :param float timeout:
Timeout in seconds. Timeout in seconds.
:param bool block: :param bool block:
If :py:data:`False`, immediately raise If :data:`False`, immediately raise
:py:class:`mitogen.core.TimeoutError` if the select is empty. :class:`mitogen.core.TimeoutError` if the select is empty.
:return: :return:
:py:class:`mitogen.core.Message` :class:`mitogen.core.Message`
:raises mitogen.core.TimeoutError: :raises mitogen.core.TimeoutError:
Timeout was reached. Timeout was reached.
:raises mitogen.core.LatchError: :raises mitogen.core.LatchError:
:py:meth:`close` has been called, and the underlying latch is no :meth:`close` has been called, and the underlying latch is no
longer valid. longer valid.
.. py:method:: __bool__ () .. py:method:: __bool__ ()
@ -1178,8 +1187,8 @@ Select Class
Remove the select's notifier function from each registered receiver, Remove the select's notifier function from each registered receiver,
mark the associated latch as closed, and cause any thread currently mark the associated latch as closed, and cause any thread currently
sleeping in :py:meth:`get` to be woken with sleeping in :meth:`get` to be woken with
:py:class:`mitogen.core.LatchError`. :class:`mitogen.core.LatchError`.
This is necessary to prevent memory leaks in long-running receivers. It This is necessary to prevent memory leaks in long-running receivers. It
is called automatically when the Python :keyword:`with` statement is is called automatically when the Python :keyword:`with` statement is
@ -1187,35 +1196,35 @@ Select Class
.. py:method:: empty () .. py:method:: empty ()
Return :data:`True` if calling :py:meth:`get` would block. Return :data:`True` if calling :meth:`get` would block.
As with :py:class:`Queue.Queue`, :data:`True` may be returned even As with :class:`Queue.Queue`, :data:`True` may be returned even
though a subsequent call to :py:meth:`get` will succeed, since a though a subsequent call to :meth:`get` will succeed, since a
message may be posted at any moment between :py:meth:`empty` and message may be posted at any moment between :meth:`empty` and
:py:meth:`get`. :meth:`get`.
:py:meth:`empty` may return :data:`False` even when :py:meth:`get` :meth:`empty` may return :data:`False` even when :meth:`get`
would block if another thread has drained a receiver added to this would block if another thread has drained a receiver added to this
select. This can be avoided by only consuming each receiver from a select. This can be avoided by only consuming each receiver from a
single thread. single thread.
.. py:method:: __iter__ (self) .. py:method:: __iter__ (self)
Yield the result of :py:meth:`get` until no receivers remain in the Yield the result of :meth:`get` until no receivers remain in the
select, either because `oneshot` is :data:`True`, or each receiver was select, either because `oneshot` is :data:`True`, or each receiver was
explicitly removed via :py:meth:`remove`. explicitly removed via :meth:`remove`.
.. py:method:: add (recv) .. py:method:: add (recv)
Add the :py:class:`mitogen.core.Receiver` or Add the :class:`mitogen.core.Receiver` or
:py:class:`mitogen.core.Channel` `recv` to the select. :class:`mitogen.core.Channel` `recv` to the select.
.. py:method:: remove (recv) .. py:method:: remove (recv)
Remove the :py:class:`mitogen.core.Receiver` or Remove the :class:`mitogen.core.Receiver` or
:py:class:`mitogen.core.Channel` `recv` from the select. Note that if :class:`mitogen.core.Channel` `recv` from the select. Note that if
the receiver has notified prior to :py:meth:`remove`, then it will the receiver has notified prior to :meth:`remove`, then it will
still be returned by a subsequent :py:meth:`get`. This may change in a still be returned by a subsequent :meth:`get`. This may change in a
future version. future version.
@ -1226,7 +1235,7 @@ Channel Class
.. class:: Channel (router, context, dst_handle, handle=None) .. class:: Channel (router, context, dst_handle, handle=None)
A channel inherits from :py:class:`mitogen.core.Sender` and A channel inherits from :class:`mitogen.core.Sender` and
`mitogen.core.Receiver` to provide bidirectional functionality. `mitogen.core.Receiver` to provide bidirectional functionality.
Since all handles aren't known until after both ends are constructed, for Since all handles aren't known until after both ends are constructed, for
@ -1248,8 +1257,8 @@ Broker Class
.. attribute:: shutdown_timeout = 3.0 .. attribute:: shutdown_timeout = 3.0
Seconds grace to allow :py:class:`streams <Stream>` to shutdown Seconds grace to allow :class:`streams <Stream>` to shutdown
gracefully before force-disconnecting them during :py:meth:`shutdown`. gracefully before force-disconnecting them during :meth:`shutdown`.
.. method:: defer (func, \*args, \*kwargs) .. method:: defer (func, \*args, \*kwargs)
@ -1259,26 +1268,26 @@ Broker Class
.. method:: start_receive (stream) .. method:: start_receive (stream)
Mark the :py:attr:`receive_side <Stream.receive_side>` on `stream` as Mark the :attr:`receive_side <Stream.receive_side>` on `stream` as
ready for reading. Safe to call from any thread. When the associated ready for reading. Safe to call from any thread. When the associated
file descriptor becomes ready for reading, file descriptor becomes ready for reading,
:py:meth:`BasicStream.on_receive` will be called. :meth:`BasicStream.on_receive` will be called.
.. method:: stop_receive (stream) .. method:: stop_receive (stream)
Mark the :py:attr:`receive_side <Stream.receive_side>` on `stream` as Mark the :attr:`receive_side <Stream.receive_side>` on `stream` as
not ready for reading. Safe to call from any thread. not ready for reading. Safe to call from any thread.
.. method:: _start_transmit (stream) .. method:: _start_transmit (stream)
Mark the :py:attr:`transmit_side <Stream.transmit_side>` on `stream` as Mark the :attr:`transmit_side <Stream.transmit_side>` on `stream` as
ready for writing. Must only be called from the Broker thread. When the ready for writing. Must only be called from the Broker thread. When the
associated file descriptor becomes ready for writing, associated file descriptor becomes ready for writing,
:py:meth:`BasicStream.on_transmit` will be called. :meth:`BasicStream.on_transmit` will be called.
.. method:: stop_receive (stream) .. method:: stop_receive (stream)
Mark the :py:attr:`transmit_side <Stream.receive_side>` on `stream` as Mark the :attr:`transmit_side <Stream.receive_side>` on `stream` as
not ready for writing. Safe to call from any thread. not ready for writing. Safe to call from any thread.
.. method:: shutdown .. method:: shutdown
@ -1288,12 +1297,12 @@ Broker Class
.. method:: join .. method:: join
Wait for the broker to stop, expected to be called after Wait for the broker to stop, expected to be called after
:py:meth:`shutdown`. :meth:`shutdown`.
.. method:: keep_alive .. method:: keep_alive
Return :data:`True` if any reader's :py:attr:`Side.keep_alive` Return :data:`True` if any reader's :attr:`Side.keep_alive`
attribute is :data:`True`, or any :py:class:`Context` is still attribute is :data:`True`, or any :class:`Context` is still
registered that is not the master. Used to delay shutdown while some registered that is not the master. Used to delay shutdown while some
important work is in progress (e.g. log draining). important work is in progress (e.g. log draining).
@ -1301,11 +1310,11 @@ Broker Class
.. method:: _broker_main .. method:: _broker_main
Handle events until :py:meth:`shutdown`. On shutdown, invoke Handle events until :meth:`shutdown`. On shutdown, invoke
:py:meth:`Stream.on_shutdown` for every active stream, then allow up to :meth:`Stream.on_shutdown` for every active stream, then allow up to
:py:attr:`shutdown_timeout` seconds for the streams to unregister :attr:`shutdown_timeout` seconds for the streams to unregister
themselves before forcefully calling themselves before forcefully calling
:py:meth:`Stream.on_disconnect`. :meth:`Stream.on_disconnect`.
.. currentmodule:: mitogen.master .. currentmodule:: mitogen.master
@ -1321,7 +1330,7 @@ Broker Class
:param bool install_watcher: :param bool install_watcher:
If :data:`True`, an additional thread is started to monitor the If :data:`True`, an additional thread is started to monitor the
lifetime of the main thread, triggering :py:meth:`shutdown` lifetime of the main thread, triggering :meth:`shutdown`
automatically in case the user forgets to call it, or their code automatically in case the user forgets to call it, or their code
crashed. crashed.
@ -1332,8 +1341,8 @@ Broker Class
.. attribute:: shutdown_timeout = 5.0 .. attribute:: shutdown_timeout = 5.0
Seconds grace to allow :py:class:`streams <Stream>` to shutdown Seconds grace to allow :class:`streams <Stream>` to shutdown
gracefully before force-disconnecting them during :py:meth:`shutdown`. gracefully before force-disconnecting them during :meth:`shutdown`.
Utility Functions Utility Functions
@ -1349,7 +1358,7 @@ A random assortment of utility functions useful on masters and children.
Many tools love to subclass built-in types in order to implement useful Many tools love to subclass built-in types in order to implement useful
functionality, such as annotating the safety of a Unicode string, or adding functionality, such as annotating the safety of a Unicode string, or adding
additional methods to a dict. However, cPickle loves to preserve those additional methods to a dict. However, cPickle loves to preserve those
subtypes during serialization, resulting in CallError during :py:meth:`call subtypes during serialization, resulting in CallError during :meth:`call
<mitogen.parent.Context.call>` in the target when it tries to deserialize <mitogen.parent.Context.call>` in the target when it tries to deserialize
the data. the data.
@ -1369,12 +1378,12 @@ A random assortment of utility functions useful on masters and children.
Remove all entries mentioning ``site-packages`` or ``Extras`` from the Remove all entries mentioning ``site-packages`` or ``Extras`` from the
system path. Used primarily for testing on OS X within a virtualenv, where system path. Used primarily for testing on OS X within a virtualenv, where
OS X bundles some ancient version of the :py:mod:`six` module. OS X bundles some ancient version of the :mod:`six` module.
.. currentmodule:: mitogen.utils .. currentmodule:: mitogen.utils
.. function:: log_to_file (path=None, io=False, level='INFO') .. function:: log_to_file (path=None, io=False, level='INFO')
Install a new :py:class:`logging.Handler` writing applications logs to the Install a new :class:`logging.Handler` writing applications logs to the
filesystem. Useful when debugging slave IO problems. filesystem. Useful when debugging slave IO problems.
Parameters to this function may be overridden at runtime using environment Parameters to this function may be overridden at runtime using environment
@ -1382,14 +1391,14 @@ A random assortment of utility functions useful on masters and children.
:param str path: :param str path:
If not :data:`None`, a filesystem path to write logs to. Otherwise, If not :data:`None`, a filesystem path to write logs to. Otherwise,
logs are written to :py:data:`sys.stderr`. logs are written to :data:`sys.stderr`.
:param bool io: :param bool io:
If :data:`True`, include extremely verbose IO logs in the output. If :data:`True`, include extremely verbose IO logs in the output.
Useful for debugging hangs, less useful for debugging application code. Useful for debugging hangs, less useful for debugging application code.
:param str level: :param str level:
Name of the :py:mod:`logging` package constant that is the minimum Name of the :mod:`logging` package constant that is the minimum
level to log at. Useful levels are ``DEBUG``, ``INFO``, ``WARNING``, level to log at. Useful levels are ``DEBUG``, ``INFO``, ``WARNING``,
and ``ERROR``. and ``ERROR``.
@ -1397,7 +1406,7 @@ A random assortment of utility functions useful on masters and children.
.. function:: run_with_router(func, \*args, \**kwargs) .. function:: run_with_router(func, \*args, \**kwargs)
Arrange for `func(router, \*args, \**kwargs)` to run with a temporary Arrange for `func(router, \*args, \**kwargs)` to run with a temporary
:py:class:`mitogen.master.Router`, ensuring the Router and Broker are :class:`mitogen.master.Router`, ensuring the Router and Broker are
correctly shut down during normal or exceptional return. correctly shut down during normal or exceptional return.
:returns: :returns:
@ -1406,7 +1415,7 @@ A random assortment of utility functions useful on masters and children.
.. currentmodule:: mitogen.utils .. currentmodule:: mitogen.utils
.. decorator:: with_router .. decorator:: with_router
Decorator version of :py:func:`run_with_router`. Example: Decorator version of :func:`run_with_router`. Example:
.. code-block:: python .. code-block:: python

55
mitogen/ssh.py
Unescape Escape View File

@ -111,7 +111,6 @@ class HostKeyError(mitogen.core.StreamError):
class Stream(mitogen.parent.Stream): class Stream(mitogen.parent.Stream):
create_child = staticmethod(mitogen.parent.hybrid_tty_create_child)
child_is_immediate_subprocess = False child_is_immediate_subprocess = False
#: Default to whatever is available as 'python' on the remote machine, #: Default to whatever is available as 'python' on the remote machine,
@ -121,8 +120,8 @@ class Stream(mitogen.parent.Stream):
#: Number of -v invocations to pass on command line. #: Number of -v invocations to pass on command line.
ssh_debug_level = 0 ssh_debug_level = 0
#: Once connected, points to the corresponding TtyLogStream, allowing it to #: If batch_mode=False, points to the corresponding TtyLogStream, allowing
#: be disconnected at the same time this stream is being torn down. #: it to be disconnected at the same time this stream is being torn down.
tty_stream = None tty_stream = None
#: The path to the SSH binary. #: The path to the SSH binary.
@ -137,15 +136,27 @@ class Stream(mitogen.parent.Stream):
ssh_args = None ssh_args = None
check_host_keys_msg = 'check_host_keys= must be set to accept, enforce or ignore' check_host_keys_msg = 'check_host_keys= must be set to accept, enforce or ignore'
batch_mode_check_host_keys_msg = (
'check_host_keys cannot be set to "accept" when batch mode is '
'enabled, since batch mode disables PTY allocation.'
)
batch_mode_password_msg = (
'A password cannot be set when batch mode is enabled, '
'since batch mode disables PTY allocation.'
)
def construct(self, hostname, username=None, ssh_path=None, port=None, def construct(self, hostname, username=None, ssh_path=None, port=None,
check_host_keys='enforce', password=None, identity_file=None, check_host_keys='enforce', password=None, identity_file=None,
compression=True, ssh_args=None, keepalive_enabled=True, compression=True, ssh_args=None, keepalive_enabled=True,
keepalive_count=3, keepalive_interval=15, keepalive_count=3, keepalive_interval=15, batch_mode=False,
identities_only=True, ssh_debug_level=None, **kwargs): identities_only=True, ssh_debug_level=None, **kwargs):
super(Stream, self).construct(**kwargs) super(Stream, self).construct(**kwargs)
if check_host_keys not in ('accept', 'enforce', 'ignore'): if check_host_keys not in ('accept', 'enforce', 'ignore'):
raise ValueError(self.check_host_keys_msg) raise ValueError(self.check_host_keys_msg)
if check_host_keys == 'accept' and batch_mode:
raise ValueError(self.batch_mode_check_host_keys_msg)
if password is not None and batch_mode:
raise ValueError(self.batch_mode_password_msg)
self.hostname = hostname self.hostname = hostname
self.username = username self.username = username
@ -158,6 +169,14 @@ class Stream(mitogen.parent.Stream):
self.keepalive_enabled = keepalive_enabled self.keepalive_enabled = keepalive_enabled
self.keepalive_count = keepalive_count self.keepalive_count = keepalive_count
self.keepalive_interval = keepalive_interval self.keepalive_interval = keepalive_interval
self.batch_mode = batch_mode
if self.batch_mode:
self.create_child = mitogen.parent.create_child
self.create_child_args = {
'merge_stdio': True,
}
else:
self.create_child = mitogen.parent.hybrid_tty_create_child
if ssh_path: if ssh_path:
self.ssh_path = ssh_path self.ssh_path = ssh_path
if ssh_args: if ssh_args:
@ -166,7 +185,8 @@ class Stream(mitogen.parent.Stream):
self.ssh_debug_level = ssh_debug_level self.ssh_debug_level = ssh_debug_level
def on_disconnect(self, broker): def on_disconnect(self, broker):
self.tty_stream.on_disconnect(broker) if self.tty_stream is not None:
self.tty_stream.on_disconnect(broker)
super(Stream, self).on_disconnect(broker) super(Stream, self).on_disconnect(broker)
def get_boot_command(self): def get_boot_command(self):
@ -193,10 +213,15 @@ class Stream(mitogen.parent.Stream):
'-o', 'ServerAliveInterval %s' % (self.keepalive_interval,), '-o', 'ServerAliveInterval %s' % (self.keepalive_interval,),
'-o', 'ServerAliveCountMax %s' % (self.keepalive_count,), '-o', 'ServerAliveCountMax %s' % (self.keepalive_count,),
] ]
if self.batch_mode:
bits += ['-o', 'BatchMode yes']
if self.check_host_keys == 'enforce': if self.check_host_keys == 'enforce':
bits += ['-o', 'StrictHostKeyChecking yes'] bits += ['-o', 'StrictHostKeyChecking yes']
if self.check_host_keys == 'accept': if self.check_host_keys == 'accept':
bits += ['-o', 'StrictHostKeyChecking ask'] if self.batch_mode:
bits += ['-o', 'StrictHostKeyChecking no']
else:
bits += ['-o', 'StrictHostKeyChecking ask']
elif self.check_host_keys == 'ignore': elif self.check_host_keys == 'ignore':
bits += [ bits += [
'-o', 'StrictHostKeyChecking no', '-o', 'StrictHostKeyChecking no',
@ -240,19 +265,23 @@ class Stream(mitogen.parent.Stream):
# with ours. # with ours.
raise HostKeyError(self.hostkey_config_msg) raise HostKeyError(self.hostkey_config_msg)
def _ec0_received(self):
if self.tty_stream is not None:
self._router.broker.start_receive(self.tty_stream)
return super(Stream, self)._ec0_received()
def _connect_bootstrap(self, extra_fd): def _connect_bootstrap(self, extra_fd):
self.tty_stream = mitogen.parent.TtyLogStream(extra_fd, self) fds = [self.receive_side.fd]
if extra_fd is not None:
self.tty_stream = mitogen.parent.TtyLogStream(extra_fd, self)
fds.append(extra_fd)
password_sent = False it = mitogen.parent.iter_read(fds=fds, deadline=self.connect_deadline)
it = mitogen.parent.iter_read(
fds=[self.receive_side.fd, extra_fd],
deadline=self.connect_deadline
)
password_sent = False
for buf, partial in filter_debug(self, it): for buf, partial in filter_debug(self, it):
LOG.debug('%r: received %r', self, buf) LOG.debug('%r: received %r', self, buf)
if buf.endswith(self.EC0_MARKER): if buf.endswith(self.EC0_MARKER):
self._router.broker.start_receive(self.tty_stream)
self._ec0_received() self._ec0_received()
return return
elif HOSTKEY_REQ_PROMPT in buf.lower(): elif HOSTKEY_REQ_PROMPT in buf.lower():

39
tests/data/fakessh.py
Unescape Escape View File

@ -6,6 +6,45 @@ import shlex
import subprocess import subprocess
import sys import sys
HOST_KEY_ASK_MSG = """
The authenticity of host '[91.121.165.123]:9122 ([91.121.165.123]:9122)' can't be established.
ECDSA key fingerprint is SHA256:JvfPvazZzQ9/CUdKN7tiYlNZtDRdEgDsYVIzOgPrsR4.
Are you sure you want to continue connecting (yes/no)?
""".strip('\n')
HOST_KEY_STRICT_MSG = """Host key verification failed.\n"""
def tty(msg):
fp = open('/dev/tty', 'w', 0)
fp.write(msg)
fp.close()
def stderr(msg):
fp = open('/dev/stderr', 'w', 0)
fp.write(msg)
fp.close()
def confirm(msg):
tty(msg)
fp = open('/dev/tty', 'r', 0)
try:
return fp.readline()
finally:
fp.close()
if os.getenv('FAKESSH_MODE') == 'ask':
assert 'y\n' == confirm(HOST_KEY_ASK_MSG)
if os.getenv('FAKESSH_MODE') == 'strict':
stderr(HOST_KEY_STRICT_MSG)
sys.exit(255)
parser = optparse.OptionParser() parser = optparse.OptionParser()
parser.add_option('--user', '-l', action='store') parser.add_option('--user', '-l', action='store')
parser.add_option('-o', dest='options', action='append') parser.add_option('-o', dest='options', action='append')

87
tests/ssh_test.py
Unescape Escape View File

@ -1,3 +1,4 @@
import os
import sys import sys
import mitogen import mitogen
@ -13,9 +14,9 @@ import plain_old_module
class FakeSshTest(testlib.RouterMixin, unittest2.TestCase): class FakeSshTest(testlib.RouterMixin, unittest2.TestCase):
def test_okay(self): def test_okay(self):
context = self.router.ssh( context = self.router.ssh(
hostname='hostname', hostname='hostname',
username='mitogen__has_sudo', username='mitogen__has_sudo',
ssh_path=testlib.data_path('fakessh.py'), ssh_path=testlib.data_path('fakessh.py'),
) )
#context.call(mitogen.utils.log_to_file, '/tmp/log') #context.call(mitogen.utils.log_to_file, '/tmp/log')
#context.call(mitogen.utils.disable_site_packages) #context.call(mitogen.utils.disable_site_packages)
@ -123,6 +124,86 @@ class BannerTest(testlib.DockerMixin, unittest2.TestCase):
self.assertEquals(name, context.name) self.assertEquals(name, context.name)
class BatchModeTest(testlib.DockerMixin, testlib.TestCase):
stream_class = mitogen.ssh.Stream
#
# Test that:
#
# - batch_mode=false, host_key_checking=accept
# - batch_mode=false, host_key_checking=enforce
# - batch_mode=false, host_key_checking=ignore
#
# - batch_mode=true, host_key_checking=accept
# - batch_mode=true, host_key_checking=enforce
# - batch_mode=true, host_key_checking=ignore
# - batch_mode=true, password is not None
#
def fake_ssh(self, FAKESSH_MODE=None, **kwargs):
os.environ['FAKESSH_MODE'] = str(FAKESSH_MODE)
try:
return self.router.ssh(
hostname='hostname',
username='mitogen__has_sudo',
ssh_path=testlib.data_path('fakessh.py'),
**kwargs
)
finally:
del os.environ['FAKESSH_MODE']
def test_false_accept(self):
# Should succeed.
self.fake_ssh(FAKESSH_MODE='ask', check_host_keys='accept')
def test_false_enforce(self):
# Should succeed.
self.fake_ssh(check_host_keys='enforce')
def test_false_ignore(self):
# Should succeed.
self.fake_ssh(check_host_keys='ignore')
def test_false_password(self):
# Should succeed.
self.docker_ssh(username='mitogen__has_sudo_nopw',
password='has_sudo_nopw_password')
def test_true_accept(self):
e = self.assertRaises(ValueError,
lambda: self.fake_ssh(check_host_keys='accept', batch_mode=True)
)
self.assertEquals(e.args[0],
self.stream_class.batch_mode_check_host_keys_msg)
def test_true_enforce(self):
e = self.assertRaises(mitogen.ssh.HostKeyError,
lambda: self.docker_ssh(
batch_mode=True,
check_host_keys='enforce',
ssh_args=['-o', 'UserKnownHostsFile /dev/null'],
)
)
self.assertEquals(e.args[0], self.stream_class.hostkey_failed_msg)
def test_true_ignore(self):
e = self.assertRaises(mitogen.ssh.HostKeyError,
lambda: self.fake_ssh(
FAKESSH_MODE='strict',
batch_mode=True,
check_host_keys='ignore',
)
)
self.assertEquals(e.args[0], self.stream_class.hostkey_failed_msg)
def test_true_password(self):
e = self.assertRaises(ValueError,
lambda: self.fake_ssh(
password='nope',
batch_mode=True,
)
)
self.assertEquals(e.args[0], self.stream_class.batch_mode_password_msg)
if __name__ == '__main__': if __name__ == '__main__':
unittest2.main() unittest2.main()

Write Preview
Loading…
Cancel
Save