.. _changelog:

Release Notes
=============


.. raw:: html

    <style>
        div#release-notes h2 {
            border-bottom: 1px dotted #c0c0c0;
            margin-top: 50px !important;
        }
    </style>


v0.2.8 (unreleased)
-------------------

To avail of fixes in an unreleased version, please download a ZIP file
`directly from GitHub <https://github.com/dw/mitogen/>`_.

Enhancements
~~~~~~~~~~~~

* :gh:issue:`556`,
  :gh:issue:`587`: Ansible 2.8 is partially
  supported. `Become plugins
  <https://docs.ansible.com/ansible/latest/plugins/become.html>`_ and
  `interpreter discovery
  <https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html>`_
  are not yet handled.

* :gh:issue:`419`, :gh:issue:`470`, file descriptor usage during large runs is
  halved, as it is no longer necessary to manage read and write sides
  distinctly in order to work around a design problem.

* :gh:issue:`419`: almost all connection setup happens on one thread, reducing
  contention and context switching early in a run.

* :gh:issue:`419`: Connection setup is better pipelined, eliminating some
  network round-trips. Most infrastructure is in place to support future
  removal of the final round-trips between a target fully booting and receiving
  function calls.

* :gh:pull:`595`: the :meth:`~mitogen.parent.Router.buildah` connection method
  is available to manipulate `Buildah <https://buildah.io/>`_ containers, and
  is exposed to Ansible as the :ans:conn:`buildah`.

* :gh:issue:`615`: the ``mitogen_fetch``
  action is included, and the standard Ansible :ans:mod:`fetch` is redirected
  to it. This implements streaming file transfer in every case, including when
  ``become`` is active, preventing excessive CPU usage and memory spikes, and
  significantly improving throughput. A copy of 2 files of 512 MiB each drops
  from 47 seconds to just under 7 seconds, with peak memory usage dropping from
  10.7 GiB to 64.8 MiB.

* `Operon <https://networkgenomics.com/operon/>`_ no longer requires a custom
  installation, both Operon and Ansible are supported by a unified release.

* The ``MITOGEN_CPU_COUNT`` environment variable shards the connection
  multiplexer into per-CPU workers. This may improve throughput for runs
  involving large file transfers, and is required for future in-process SSH
  support. One multiplexer starts by default, to match existing behaviour.

* :gh:commit:`d6faff06`,
  :gh:commit:`807cbef9`,
  :gh:commit:`e93762b3`,
  :gh:commit:`50bfe4c7`: locking is
  avoided on hot paths, and some locks are released earlier, before waking a
  thread that must immediately take the same lock.


Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~

* :gh:issue:`363`: fix an obscure race matching *Permission denied* errors from
  some versions of ``su`` running on heavily loaded machines.

* :gh:issue:`410`: Uses of :linux:man7:`unix` sockets are replaced with
  traditional :linux:man7:`pipe` pairs when SELinux is detected, to work around
  a broken heuristic in popular SELinux policies that prevents inheriting
  :linux:man7:`unix` sockets across privilege domains.

* `#467 <httpe://github.com/dw/mitogen/issues/467>`_: an incompatibility
  running Mitogen under Molecule was resolved.

* :gh:issue:`547`, :gh:issue:`598`: fix a serious deadlock
  possible while initializing the service pool of any child, such as during
  connection, ``async`` tasks, tasks using custom :mod:`module_utils`,
  ``mitogen_task_isolation: fork`` modules, and those present on an internal
  blacklist of misbehaving modules.

  This deadlock is relatively easy hit, has been present since 0.2.0, and is
  likely to have impacted many users. For new connections it could manifest as
  a *Connection timed out* error, for forked tasks it could manifest as a
  timeout or an apparent hang.

* :gh:issue:`549`: the open file descriptor limit for the Ansible process is
  increased to the available hard limit. It is common for distributions to ship
  with a much higher hard limit than their default soft limit, allowing *"too
  many open files"* errors to be avoided more often in large runs without user
  configuration.

* :gh:issue:`558`, :gh:issue:`582`: on Ansible 2.3 a remote directory was
  unconditionally deleted after the first module belonging to an action plug-in
  had executed, causing the :ans:mod:`unarchive` module to fail.

* :gh:issue:`578`: the extension could crash while rendering an error message,
  due to an incorrect format string.

* :gh:issue:`590`: the importer can handle modules that replace themselves in
  :data:`sys.modules` during import.

* :gh:issue:`591`: the target's current working directory is restored to a
  known-existent directory between tasks to ensure :func:`os.getcwd` will not
  fail when called, in the same way that :class:`AnsibleModule` restores it
  during initialization. However this restore happens before the module ever
  executes, ensuring any code that calls :func:`os.getcwd` prior to
  :class:`AnsibleModule` initialization, such as the Ansible 2.7
  :ans:mod:`pip`, cannot fail due to the behavior of a prior task.

* :gh:issue:`593`: the SSH connection method exposes
  ``mitogen_ssh_keepalive_interval`` and ``mitogen_ssh_keepalive_count``
  variables, and the default timeout for an SSH server has been increased from
  `15*3` seconds to `30*10` seconds.

* :gh:issue:`600`: functionality to reflect changes to ``/etc/environment`` did
  not account for Unicode file contents. The file may now use any single byte
  encoding.

* :gh:issue:`602`: connection configuration is more accurately inferred for
  `meta: reset_connection`, the `synchronize` module, and for any action
  plug-ins that establish additional connections.

* :gh:issue:`598`, :gh:issue:`605`: fix a deadlock managing a shared counter
  used for load balancing.

* :gh:issue:`615`: streaming file transfer is implemented for ``fetch`` and
  other actions that transfer files from the target to the controller.
  Previously the file was sent in one message, requiring it to fit in RAM and
  be smaller than the internal message size limit.

* :gh:commit:`7ae926b3`: the Ansible :ans:mod:`lineinfile` began leaking
  writable temporary file descriptors since Ansible 2.7.0. When
  :ans:mod:`~lineinfile` was used to create or modify a script, and that script
  was later executed, the execution could fail with "*text file busy*" due to
  the leaked descriptor. Temporary descriptors are now tracked and cleaned up
  on exit for all modules.


Core Library
~~~~~~~~~~~~

* Log readability is improving, and many :func:`repr` strings are more
  descriptive. The old pseudo-function-call format is slowly migrating to
  human-readable output where possible. For example,
  *"Stream(ssh:123).connect()"* might be written *"connecting to ssh:123"*.

* :func:`bytearray` was removed from the list of supported serialization types.
  It was never portable between Python versions, unused, and never made much
  sense to support.

* :gh:issue:`170`: to improve subprocess
  management and asynchronous connect, a :class:`~mitogen.parent.TimerList`
  interface is available, accessible as :attr:`Broker.timers` in an
  asynchronous context.

* :gh:issue:`419`: the internal
  :class:`~mitogen.core.Stream` has been refactored into 7 new classes,
  modularizing protocol behaviour, output buffering, line-oriented input
  parsing, option handling and connection management. Connection setup is
  internally asynchronous, laying almost all the groundwork needed for fully
  asynchronous connect, proxied Ansible become plug-ins, and integrating
  `libssh <https://www.libssh.org/>`_.

* :gh:issue:`169`,
  :gh:issue:`419`: zombie subprocess reaping
  has vastly improved, by using timers to efficiently poll for a slow child to
  finish exiting, and delaying broker shutdown while any subprocess remains.
  Polling avoids relying on process-global configuration such as a `SIGCHLD`
  handler, or :func:`signal.set_wakeup_fd` available in modern Python.

* :gh:issue:`256`,
  :gh:issue:`419`: most :func:`os.dup` use
  was eliminated, along with almost all manual file descriptor management.
  Descriptors are trapped in :func:`os.fdopen` objects at creation, ensuring a
  leaked object will close itself, and ensuring every descriptor is fused to a
  `closed` flag, preventing historical bugs where a double close could destroy
  descriptors belonging to unrelated streams.

* :gh:issue:`533`: routing accounts for
  a race between a parent (or cousin) sending a message to a child via an
  intermediary, where the child had recently disconnected, and
  :data:`~mitogen.core.DEL_ROUTE` propagating from the intermediary
  to the sender, informing it that the child no longer exists. This condition
  is detected at the intermediary and a :ref:`dead message <IS_DEAD>` is
  returned to the sender.

  Previously since the intermediary had already removed its route for the
  child, the *route messages upwards* rule would be triggered, causing the
  message (with a privileged :ref:`src_id/auth_id <stream-protocol>`) to be
  sent upstream, resulting in a ``bad auth_id`` error logged at the first
  upstream parent, and a possible hang due to a request message being dropped.

* :gh:issue:`586`: fix import of
  :mod:`__main__` on later versions of Python 3 when running from the
  interactive console.

* :gh:issue:`606`: fix example code on the
  documentation front page.

* :gh:issue:`612`: fix various errors
  introduced by stream refactoring.

* :gh:issue:`615`: when routing fails to
  deliver a message for some reason other than the sender cannot or should not
  reach the recipient, and no reply-to address is present on the message,
  instead send a :ref:`dead message <IS_DEAD>` to the original recipient. This
  ensures a descriptive messages is delivered to a thread sleeping on the reply
  to a function call, where the reply might be dropped due to exceeding the
  maximum configured message size.

* :gh:issue:`624`: the number of threads used for a child's auto-started thread
  pool has been reduced from 16 to 2. This may drop to 1 in future, and become
  configurable via a :class:`Router` option.

* :gh:commit:`a5536c35`: avoid quadratic
  buffer management when logging lines received from a child's redirected
  standard IO.

* :gh:commit:`49a6446a`: the
  :meth:`empty` methods of :class:`~mitogen.core.Latch`,
  :class:`~mitogen.core.Receiver` and :class:`~mitogen.select.Select` are
  obsoleted by a more general :meth:`size` method. :meth:`empty` will be
  removed in 0.3

* :gh:commit:`ecc570cb`: previously
  :meth:`mitogen.select.Select.add` would enqueue one wake event when adding an
  existing receiver, latch or subselect that contained multiple buffered items,
  causing :meth:`get` calls to block or fail even though data existed to return.

* :gh:commit:`5924af15`: *[security]*
  unidirectional routing, where contexts may optionally only communicate with
  parents and never siblings (so that air-gapped networks cannot be
  unintentionally bridged) was not inherited when a child was initiated
  directly from an another child. This did not effect Ansible, since the
  controller initiates any new child used for routing, only forked tasks are
  initiated by children.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Andreas Hubert <https://github.com/peshay>`_.
`Anton Markelov <https://github.com/strangeman>`_,
`Dan <https://github.com/dsgnr>`_,
`Dave Cottlehuber <https://github.com/dch>`_,
`Denis Krienbühl <https://github.com/href>`_,
`El Mehdi CHAOUKI <https://github.com/elmchaouki>`_,
`Florent Dutheil <https://github.com/fdutheil>`_,
`James Hogarth <https://github.com/hogarthj>`_,
`Jordan Webb <https://github.com/jordemort>`_,
`Marc Hartmayer <https://github.com/marc1006>`_,
`Nigel Metheringham <https://github.com/nigelm>`_,
`Orion Poplawski <https://github.com/opoplawski>`_,
`Pieter Voet <https://github.com/pietervoet/>`_,
`Stefane Fermigier <https://github.com/sfermigier>`_,
`Szabó Dániel Ernő <https://github.com/r3ap3rpy>`_,
`Ulrich Schreiner <https://github.com/ulrichSchreiner>`_,
`yen <https://github.com/antigenius0910>`_,
`Yuki Nishida <https://github.com/yuki-nishida-exa>`_,
`@alexhexabeam <https://github.com/alexhexabeam>`_,
`@DavidVentura <https://github.com/DavidVentura>`_,
`@dbiegunski <https://github.com/dbiegunski>`_,
`@ghp-rr <https://github.com/ghp-rr>`_,
`@migalsp <https://github.com/migalsp>`_,
`@rizzly <https://github.com/rizzly>`_,
`@SQGE <https://github.com/SQGE>`_, and
`@tho86 <https://github.com/tho86>`_.


v0.2.7 (2019-05-19)
-------------------

This release primarily exists to add a descriptive error message when running
on Ansible 2.8, which is not yet supported.

Fixes
~~~~~

* :gh:issue:`557`: fix a crash when running
  on machines with high CPU counts.

* :gh:issue:`570`: the :ans:mod:`firewalld` internally caches a dbus name that
  changes across :ans:mod:`~firewalld` restarts, causing a failure if the
  service is restarted between :ans:mod:`~firewalld` module invocations.

* :gh:issue:`575`: fix a crash when
  rendering an error message to indicate no usable temporary directories could
  be found.

* :gh:issue:`576`: fix a crash during
  startup on SuSE Linux 11, due to an incorrect version compatibility check in
  the Mitogen code.

* :gh:issue:`581`: a
  ``mitogen_mask_remote_name`` Ansible variable is exposed, to allow masking
  the username, hostname and process ID of ``ansible-playbook`` running on the
  controller machine.

* :gh:issue:`587`: display a friendly
  message when running on an unsupported version of Ansible, to cope with
  potential influx of 2.8-related bug reports.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Orion Poplawski <https://github.com/opoplawski>`_,
`Thibaut Barrère <https://github.com/thbar>`_,
`@Moumoutaru <https://github.com/Moumoutaru>`_, and
`@polski-g <https://github.com/polski-g>`_.


v0.2.6 (2019-03-06)
-------------------

Fixes
~~~~~

* :gh:issue:`542`: some versions of OS X
  ship a default Python that does not support :func:`select.poll`. Restore the
  0.2.3 behaviour of defaulting to Kqueue in this case, but still prefer
  :func:`select.poll` if it is available.

* :gh:issue:`545`: an optimization
  introduced in :gh:issue:`493` caused a
  64-bit integer to be assigned to a 32-bit field on ARM 32-bit targets,
  causing runs to fail.

* :gh:issue:`548`: `mitogen_via=` could fail
  when the selected transport was set to ``smart``.

* :gh:issue:`550`: avoid some broken
  TTY-related `ioctl()` calls on Windows Subsystem for Linux 2016 Anniversary
  Update.

* :gh:issue:`554`: third party Ansible
  action plug-ins that invoked :func:`_make_tmp_path` repeatedly could trigger
  an assertion failure.

* :gh:issue:`555`: work around an old idiom
  that reloaded :mod:`sys` in order to change the interpreter's default encoding.

* :gh:commit:`ffae0355`: needless
  information was removed from the documentation and installation procedure.


Core Library
~~~~~~~~~~~~

* :gh:issue:`535`: to support function calls
  on a service pool from another thread, :class:`mitogen.select.Select`
  additionally permits waiting on :class:`mitogen.core.Latch`.

* :gh:issue:`535`:
  :class:`mitogen.service.Pool.defer` allows any function to be enqueued for
  the thread pool from another thread.

* :gh:issue:`535`: a new
  :mod:`mitogen.os_fork` module provides a :func:`os.fork` wrapper that pauses
  thread activity during fork. On Python<2.6, :class:`mitogen.core.Broker` and
  :class:`mitogen.service.Pool` automatically record their existence so that a
  :func:`os.fork` monkey-patch can automatically pause them for any attempt to
  start a subprocess.

* :gh:commit:`ca63c26e`:
  :meth:`mitogen.core.Latch.put`'s `obj` argument was made optional.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Fabian Arrotin <https://github.com/arrfab>`_,
`Giles Westwood <https://github.com/gilesw>`_,
`Matt Layman <https://github.com/mblayman>`_,
`Percy Grunwald <https://github.com/percygrunwald>`_,
`Petr Enkov <https://github.com/enkov>`_,
`Tony Finch <https://github.com/fanf2>`_,
`@elbunda <https://github.com/elbunda>`_, and
`@zyphermonkey <https://github.com/zyphermonkey>`_.


v0.2.5 (2019-02-14)
-------------------

Fixes
~~~~~

* :gh:issue:`511`,
  :gh:issue:`536`: changes in 0.2.4 to
  repair ``delegate_to`` handling broke default ``ansible_python_interpreter``
  handling. Test coverage was added.

* :gh:issue:`532`: fix a race in the service
  used to propagate Ansible modules, that could easily manifest when starting
  asynchronous tasks in a loop.

* :gh:issue:`536`: changes in 0.2.4 to
  support Python 2.4 interacted poorly with modules that imported
  ``simplejson`` from a controller that also loaded an incompatible newer
  version of ``simplejson``.

* :gh:issue:`537`: a swapped operator in the
  CPU affinity logic meant 2 cores were reserved on 1<n<4 core machines, rather
  than 1 core as desired. Test coverage was added.

* :gh:issue:`538`: the source distribution
  includes a ``LICENSE`` file.

* :gh:issue:`539`: log output is no longer
  duplicated when the Ansible ``log_path`` setting is enabled.

* :gh:issue:`540`: the ``stderr`` stream of
  async module invocations was previously discarded.

* :gh:issue:`541`: Python error logs
  originating from the ``boto`` package are quiesced, and only appear in
  ``-vvv`` output. This is since EC2 modules may trigger errors during normal
  operation, when retrying transiently failing requests.

* :gh:commit:`748f5f67`,
  :gh:commit:`21ad299d`,
  :gh:commit:`8ae6ca1d`,
  :gh:commit:`7fd0d349`:
  the ``ansible_ssh_host``, ``ansible_ssh_user``, ``ansible_user``,
  ``ansible_become_method``, and ``ansible_ssh_port`` variables more correctly
  match typical behaviour when ``mitogen_via=`` is active.

* :gh:commit:`2a8567b4`: fix a race
  initializing a child's service thread pool on Python 3.4+, due to a change in
  locking scheme used by the Python import mechanism.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Carl George <https://github.com/carlwgeorge>`_,
`Guy Knights <https://github.com/knightsg>`_, and
`Josh Smift <https://github.com/jbscare>`_.


v0.2.4 (2019-02-10)
-------------------

Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~

This release includes a huge variety of important fixes and new optimizations.
It is 35% faster than 0.2.3 on a synthetic 64 target run that places heavy load
on the connection multiplexer.

Enhancements
^^^^^^^^^^^^

* :gh:issue:`76`,
  :gh:issue:`351`,
  :gh:issue:`352`: disconnect propagation
  has improved, allowing Ansible to cancel waits for responses from abruptly
  disconnected targets. This ensures a task will reliably fail rather than
  hang, for example on network failure or EC2 instance maintenance.

* :gh:issue:`369`,
  :gh:issue:`407`: :meth:`Connection.reset`
  is implemented, allowing :ans:mod:`meta: reset_connection <meta>` to shut
  down the remote interpreter as documented, and improving support for the
  :ans:mod:`reboot`.

* :gh:commit:`09aa27a6`: the
  ``mitogen_host_pinned`` strategy wraps the ``host_pinned`` strategy
  introduced in Ansible 2.7.

* :gh:issue:`477`: Python 2.4 is fully
  supported by the core library and tested automatically, in any parent/child
  combination of 2.4, 2.6, 2.7 and 3.6 interpreters.

* :gh:issue:`477`: Ansible 2.3 is fully
  supported and tested automatically. In combination with the core library
  Python 2.4 support, this allows Red Hat Enterprise Linux 5 targets to be
  managed with Mitogen. The ``simplejson`` package need not be installed on
  such targets, as is usually required by Ansible.

* :gh:issue:`412`: to simplify diagnosing
  connection configuration problems, Mitogen ships a ``mitogen_get_stack``
  action that is automatically added to the action plug-in path. See
  :ref:`mitogen-get-stack` for more information.

* :gh:commit:`152effc2`,
  :gh:commit:`bd4b04ae`: a CPU affinity
  policy was added for Linux controllers, reducing latency and SMP overhead on
  hot paths exercised for every task. This yielded a 19% speedup in a 64-target
  job composed of many short tasks, and should easily be visible as a runtime
  improvement in many-host runs.

* :gh:commit:`2b44d598`: work around a
  defective caching mechanism by pre-heating it before spawning workers. This
  saves 40% runtime on a synthetic repetitive task.

* :gh:commit:`0979422a`: an expensive
  dependency scanning step was redundantly invoked for every task,
  bottlenecking the connection multiplexer.

* :gh:commit:`eaa990a97`: a new
  ``mitogen_ssh_compression`` variable is supported, allowing Mitogen's default
  SSH compression to be disabled. SSH compression is a large contributor to CPU
  usage in many-target runs, and severely limits file transfer. On a `"shell:
  hostname"` task repeated 500 times, Mitogen requires around 800 bytes per
  task with compression, rising to 3 KiB without. File transfer throughput
  rises from ~25MiB/s when enabled to ~200MiB/s when disabled.

* :gh:issue:`260`,
  :gh:commit:`a18a083c`: brokers no
  longer wait for readiness indication to transmit, and instead assume
  transmission will succeed. As this is usually true, one loop iteration and
  two poller reconfigurations are avoided, yielding a significant reduction in
  interprocess round-trip latency.

* :gh:issue:`415`, :gh:issue:`491`, :gh:issue:`493`: the interface employed
  for in-process queues changed from :freebsd:man2:`kqueue` /
  :linux:man7:`epoll` to :linux:man2:`poll`, which requires no setup or
  teardown, yielding a 38% latency reduction for inter-thread communication.


Fixes
^^^^^

* :gh:issue:`251`,
  :gh:issue:`359`,
  :gh:issue:`396`,
  :gh:issue:`401`,
  :gh:issue:`404`,
  :gh:issue:`412`,
  :gh:issue:`434`,
  :gh:issue:`436`,
  :gh:issue:`465`: connection delegation and
  ``delegate_to:`` handling suffered a major regression in 0.2.3. The 0.2.2
  behaviour has been restored, and further work has been made to improve the
  compatibility of connection delegation's configuration building methods.

* :gh:issue:`323`,
  :gh:issue:`333`: work around a Windows
  Subsystem for Linux bug that caused tracebacks to appear during shutdown.

* :gh:issue:`334`: the SSH method
  tilde-expands private key paths using Ansible's logic. Previously the path
  was passed unmodified to SSH, which expanded it using :func:`pwd.getpwnam`.
  This differs from :func:`os.path.expanduser`, which uses the ``HOME``
  environment variable if it is set, causing behaviour to diverge when Ansible
  was invoked across user accounts via ``sudo``.

* :gh:issue:`364`: file transfers from
  controllers running Python 2.7.2 or earlier could be interrupted due to a
  forking bug in the :mod:`tempfile` module.

* :gh:issue:`370`: the Ansible :ans:mod:`reboot` is supported.

* :gh:issue:`373`: the LXC and LXD methods print a useful hint on failure, as
  no useful error is normally logged to the console by these tools.

* :gh:issue:`374`,
  :gh:issue:`391`: file transfer and module
  execution from 2.x controllers to 3.x targets was broken due to a regression
  caused by refactoring, and compounded by :gh:issue:`426`.

* :gh:issue:`400`: work around a threading
  bug in the AWX display callback when running with high verbosity setting.

* :gh:issue:`409`: the setns method was
  silently broken due to missing tests. Basic coverage was added to prevent a
  recurrence.

* :gh:issue:`409`: the LXC and LXD methods
  support ``mitogen_lxc_path`` and ``mitogen_lxc_attach_path`` variables to
  control the location of third pary utilities.

* :gh:issue:`410`: the sudo method supports
  the SELinux ``--type`` and ``--role`` options.

* :gh:issue:`420`: if a :class:`Connection`
  was constructed in the Ansible top-level process, for example while executing
  ``meta: reset_connection``, resources could become undesirably shared in
  subsequent children.

* :gh:issue:`426`: an oversight while
  porting to Python 3 meant no automated 2->3 tests were running. A significant
  number of 2->3 bugs were fixed, mostly in the form of Unicode/bytes
  mismatches.

* :gh:issue:`429`: the ``sudo`` method can
  now recognize internationalized password prompts.

* :gh:issue:`362`,
  :gh:issue:`435`: the previous fix for slow
  Python 2.x subprocess creation on Red Hat caused newly spawned children to
  have a reduced open files limit. A more intrusive fix has been added to
  directly address the problem without modifying the subprocess environment.

* :gh:issue:`397`,
  :gh:issue:`454`: the previous approach to
  handling modern Ansible temporary file cleanup was too aggressive, and could
  trigger early finalization of Cython-based extension modules, leading to
  segmentation faults.

* :gh:issue:`499`: the ``allow_same_user``
  Ansible configuration setting is respected.

* :gh:issue:`527`: crashes in modules are
  trapped and reported in a manner that matches Ansible. In particular, a
  module crash no longer leads to an exception that may crash the corresponding
  action plug-in.

* :gh:commit:`dc1d4251`: the :ans:mod:`synchronize` could fail with the Docker
  transport due to a missing attribute.

* :gh:commit:`599da068`: fix a race
  when starting async tasks, where it was possible for the controller to
  observe no status file on disk before the task had a chance to write one.

* :gh:commit:`2c7af9f04`: Ansible
  modules were repeatedly re-transferred. The bug was hidden by the previously
  mandatorily enabled SSH compression.


Core Library
~~~~~~~~~~~~

* :gh:issue:`76`: routing records the
  destination context IDs ever received on each stream, and when disconnection
  occurs, propagates :data:`mitogen.core.DEL_ROUTE` messages towards every
  stream that ever communicated with the disappearing peer, rather than simply
  towards parents. Conversations between nodes anywhere in the tree receive
  :data:`mitogen.core.DEL_ROUTE` when either participant disconnects, allowing
  receivers to wake with :class:`mitogen.core.ChannelError`, even when one
  participant is not a parent of the other.

* :gh:issue:`109`,
  :gh:commit:`57504ba6`: newer Python 3
  releases explicitly populate :data:`sys.meta_path` with importer internals,
  causing Mitogen to install itself at the end of the importer chain rather
  than the front.

* :gh:issue:`310`: support has returned for
  trying to figure out the real source of non-module objects installed in
  :data:`sys.modules`, so they can be imported. This is needed to handle syntax
  sugar used by packages like :mod:`plumbum`.

* :gh:issue:`349`: an incorrect format
  string could cause large stack traces when attempting to import built-in
  modules on Python 3.

* :gh:issue:`387`,
  :gh:issue:`413`: dead messages include an
  optional reason in their body. This is used to cause
  :class:`mitogen.core.ChannelError` to report far more useful diagnostics at
  the point the error occurs that previously would have been buried in debug
  log output from an unrelated context.

* :gh:issue:`408`: a variety of fixes were
  made to restore Python 2.4 compatibility.

* :gh:issue:`399`,
  :gh:issue:`437`: ignore a
  :class:`DeprecationWarning` to avoid failure of the ``su`` method on Python
  3.7.

* :gh:issue:`405`: if an oversized message
  is rejected, and it has a ``reply_to`` set, a dead message is returned to the
  sender. This ensures function calls exceeding the configured maximum size
  crash rather than hang.

* :gh:issue:`406`:
  :class:`mitogen.core.Broker` did not call :meth:`mitogen.core.Poller.close`
  during shutdown, leaking the underlying poller FD in masters and parents.

* :gh:issue:`406`: connections could leak
  FDs when a child process failed to start.

* :gh:issue:`288`,
  :gh:issue:`406`,
  :gh:issue:`417`: connections could leave
  FD wrapper objects that had not been closed lying around to be closed during
  garbage collection, causing reused FD numbers to be closed at random moments.

* :gh:issue:`411`: the SSH method typed
  "``y``" rather than the requisite "``yes``" when `check_host_keys="accept"`
  was configured. This would lead to connection timeouts due to the hung
  response.

* :gh:issue:`414`,
  :gh:issue:`425`: avoid deadlock of forked
  children by reinitializing the :mod:`mitogen.service` pool lock.

* :gh:issue:`416`: around 1.4KiB of memory
  was leaked on every RPC, due to a list of strong references keeping alive any
  handler ever registered for disconnect notification.

* :gh:issue:`418`: the
  :func:`mitogen.parent.iter_read` helper would leak poller FDs, because
  execution of its :keyword:`finally` block was delayed on Python 3. Now
  callers explicitly close the generator when finished.

* :gh:issue:`422`: the fork method could
  fail to start if :data:`sys.stdout` was opened in block buffered mode, and
  buffered data was pending in the parent prior to fork.

* :gh:issue:`438`: a descriptive error is
  logged when stream corruption is detected.

* :gh:issue:`439`: descriptive errors are
  raised when attempting to invoke unsupported function types.

* :gh:issue:`444`: messages regarding
  unforwardable extension module are no longer logged as errors.

* :gh:issue:`445`: service pools unregister
  the :data:`mitogen.core.CALL_SERVICE` handle at shutdown, ensuring any
  outstanding messages are either processed by the pool as it shuts down, or
  have dead messages sent in reply to them, preventing peer contexts from
  hanging due to a forgotten buffered message.

* :gh:issue:`446`: given thread A calling
  :meth:`mitogen.core.Receiver.close`, and thread B, C, and D sleeping in
  :meth:`mitogen.core.Receiver.get`, previously only one sleeping thread would
  be woken with :class:`mitogen.core.ChannelError` when the receiver was
  closed. Now all threads are woken per the docstring.

* :gh:issue:`447`: duplicate attempts to
  invoke :meth:`mitogen.core.Router.add_handler` cause an error to be raised,
  ensuring accidental re-registration of service pools are reported correctly.

* :gh:issue:`448`: the import hook
  implementation now raises :class:`ModuleNotFoundError` instead of
  :class:`ImportError` in Python 3.6 and above, to cope with an upcoming
  version of the :mod:`subprocess` module requiring this new subclass to be
  raised.

* :gh:issue:`453`: the loggers used in
  children for standard IO redirection have propagation disabled, preventing
  accidental reconfiguration of the :mod:`logging` package in a child from
  setting up a feedback loop.

* :gh:issue:`456`: a descriptive error is
  logged when :meth:`mitogen.core.Broker.defer` is called after the broker has
  shut down, preventing new messages being enqueued that will never be sent,
  and subsequently producing a program hang.

* :gh:issue:`459`: the beginnings of a
  :meth:`mitogen.master.Router.get_stats` call has been added. The initial
  statistics cover the module loader only.

* :gh:issue:`462`: Mitogen could fail to
  open a PTY on broken Linux systems due to a bad interaction between the glibc
  :func:`grantpt` function and an incorrectly mounted ``/dev/pts`` filesystem.
  Since correct group ownership is not required in most scenarios, when this
  problem is detected, the PTY is allocated and opened directly by the library.

* :gh:issue:`479`: Mitogen could fail to
  import :mod:`__main__` on Python 3.4 and newer due to a breaking change in
  the :mod:`pkgutil` API. The program's main script is now handled specially.

* :gh:issue:`481`: the version of `sudo`
  that shipped with CentOS 5 replaced itself with the program to be executed,
  and therefore did not hold any child PTY open on our behalf. The child
  context is updated to preserve any PTY FD in order to avoid the kernel
  sending `SIGHUP` early during startup.

* :gh:issue:`523`: the test suite didn't
  generate a code coverage report if any test failed.

* :gh:issue:`524`: Python 3.6+ emitted a
  :class:`DeprecationWarning` for :func:`mitogen.utils.run_with_router`.

* :gh:issue:`529`: Code coverage of the
  test suite was not measured across all Python versions.

* :gh:commit:`16ca111e`: handle OpenSSH
  7.5 permission denied prompts when ``~/.ssh/config`` rewrites are present.

* :gh:commit:`9ec360c2`: a new
  :meth:`mitogen.core.Broker.defer_sync` utility function is provided.

* :gh:commit:`f20e0bba`:
  :meth:`mitogen.service.FileService.register_prefix` permits granting
  unprivileged access to whole filesystem subtrees, rather than single files at
  a time.

* :gh:commit:`8f85ee03`:
  :meth:`mitogen.core.Router.myself` returns a :class:`mitogen.core.Context`
  referring to the current process.

* :gh:commit:`824c7931`: exceptions
  raised by the import hook were updated to include probable reasons for
  a failure.

* :gh:commit:`57b652ed`: a stray import
  meant an extra roundtrip and ~4KiB of data was wasted for any context that
  imported :mod:`mitogen.parent`.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Alex Willmer <https://github.com/moreati>`_,
`Andreas Krüger <https://github.com/woopstar>`_,
`Anton Stroganov <https://github.com/Aeon>`_,
`Berend De Schouwer <https://github.com/berenddeschouwer>`_,
`Brian Candler <https://github.com/candlerb>`_,
`dsgnr <https://github.com/dsgnr>`_,
`Duane Zamrok <https://github.com/dewthefifth>`_,
`Eric Chang <https://github.com/changchichung>`_,
`Gerben Meijer <https://github.com/infernix>`_,
`Guy Knights <https://github.com/knightsg>`_,
`Jesse London <https://github.com/jesteria>`_,
`Jiří Vávra <https://github.com/Houbovo>`_,
`Johan Beisser <https://github.com/jbeisser>`_,
`Jonathan Rosser <https://github.com/jrosser>`_,
`Josh Smift <https://github.com/jbscare>`_,
`Kevin Carter <https://github.com/cloudnull>`_,
`Mehdi <https://github.com/mehdisat7>`_,
`Michael DeHaan <https://github.com/mpdehaan>`_,
`Michal Medvecky <https://github.com/michalmedvecky>`_,
`Mohammed Naser <https://github.com/mnaser/>`_,
`Peter V. Saveliev <https://github.com/svinota/>`_,
`Pieter Avonts <https://github.com/pieteravonts/>`_,
`Ross Williams <https://github.com/overhacked/>`_,
`Sergey <https://github.com/LuckySB/>`_,
`Stéphane <https://github.com/sboisson/>`_,
`Strahinja Kustudic <https://github.com/kustodian>`_,
`Tom Parker-Shemilt <https://github.com/palfrey/>`_,
`Younès HAFRI <https://github.com/yhafri>`_,
`@killua-eu <https://github.com/killua-eu>`_,
`@myssa91 <https://github.com/myssa91>`_,
`@ohmer1 <https://github.com/ohmer1>`_,
`@s3c70r <https://github.com/s3c70r/>`_,
`@syntonym <https://github.com/syntonym/>`_,
`@trim777 <https://github.com/trim777/>`_,
`@whky <https://github.com/whky/>`_, and
`@yodatak <https://github.com/yodatak/>`_.


v0.2.3 (2018-10-23)
-------------------

Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~

Enhancements
^^^^^^^^^^^^

* :gh:pull:`315`,
  :gh:issue:`392`: Ansible 2.6 and 2.7 are
  supported.

* :gh:issue:`321`, :gh:issue:`336`: temporary file handling was simplified,
  undoing earlier damage caused by compatibility fixes, improving 2.6
  compatibility, and avoiding two network roundtrips for every related action
  (:ans:mod:`~assemble`, :ans:mod:`~aws_s3`, :ans:mod:`~copy`,
  :ans:mod:`~patch`, :ans:mod:`~script`, :ans:mod:`~template`,
  :ans:mod:`~unarchive`, :ans:mod:`~uri`). See :ref:`ansible_tempfiles` for a
  complete description.

* :gh:pull:`376`, :gh:pull:`377`: the ``kubectl`` connection type is now
  supported. Contributed by Yannig Perré.

* :gh:commit:`084c0ac0`: avoid a roundtrip in :ans:mod:`~copy` and
  :ans:mod:`~template` due to an unfortunate default.

* :gh:commit:`7458dfae`: avoid a
  roundtrip when transferring files smaller than 124KiB. Copy and template
  actions are now 2-RTT, reducing runtime for a 20-iteration template loop over
  a 250 ms link from 30 seconds to 10 seconds compared to v0.2.2, down from 120
  seconds compared to vanilla.

* :gh:issue:`337`: To avoid a scaling
  limitation, a PTY is no longer allocated for an SSH connection unless the
  configuration specifies a password.

* :gh:commit:`d62e6e2a`: many-target
  runs executed the dependency scanner redundantly due to missing
  synchronization, wasting significant runtime in the connection multiplexer.
  In one case work was reduced by 95%, which may manifest as faster runs.

* :gh:commit:`5189408e`: threads are
  cooperatively scheduled, minimizing `GIL
  <https://en.wikipedia.org/wiki/Global_interpreter_lock>`_ contention, and
  reducing context switching by around 90%. This manifests as an overall
  improvement, but is easily noticeable on short many-target runs, where
  startup overhead dominates runtime.

* The `faulthandler <https://faulthandler.readthedocs.io/>`_ module is
  automatically activated if it is installed, simplifying debugging of hangs.
  See :ref:`diagnosing-hangs` for details.

* The ``MITOGEN_DUMP_THREAD_STACKS`` environment variable's value now indicates
  the number of seconds between stack dumps. See :ref:`diagnosing-hangs` for
  details.


Fixes
^^^^^

* :gh:issue:`251`,
  :gh:issue:`340`: Connection Delegation
  could establish connections to the wrong target when ``delegate_to:`` is
  present.

* :gh:issue:`291`: when Mitogen had
  previously been installed using ``pip`` or ``setuptools``, the globally
  installed version could conflict with a newer version bundled with an
  extension that had been installed using the documented steps. Now the bundled
  library always overrides over any system-installed copy.

* :gh:issue:`324`: plays with a
  `custom module_utils <https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-module-utils-path>`_
  would fail due to fallout from the Python 3 port and related tests being
  disabled.

* :gh:issue:`331`: the connection
  multiplexer subprocess always exits before the main Ansible process, ensuring
  logs generated by it do not overwrite the user's prompt when ``-vvv`` is
  enabled.

* :gh:issue:`332`: support a new
  :func:`sys.excepthook`-based module exit mechanism added in Ansible 2.6.

* :gh:issue:`338`: compatibility: changes to
  ``/etc/environment`` and ``~/.pam_environment`` made by a task are reflected
  in the runtime environment of subsequent tasks. See
  :ref:`ansible_process_env` for a complete description.

* :gh:issue:`343`: the sudo ``--login``
  option is supported.

* :gh:issue:`344`: connections no longer
  fail when the controller's login username contains slashes.

* :gh:issue:`345`: the ``IdentitiesOnly
  yes`` option is no longer supplied to OpenSSH by default, better matching
  Ansible's behaviour.

* :gh:issue:`355`: tasks configured to run
  in an isolated forked subprocess were forked from the wrong parent context.
  This meant built-in modules overridden via a custom ``module_utils`` search
  path may not have had any effect.

* :gh:issue:`362`: to work around a slow
  algorithm in the :mod:`subprocess` module, the maximum number of open files
  in processes running on the target is capped to 512, reducing the work
  required to start a subprocess by >2000x in default CentOS configurations.

* :gh:issue:`397`: recent Mitogen master
  versions could fail to clean up temporary directories in a number of
  circumstances, and newer Ansibles moved to using :mod:`atexit` to effect
  temporary directory cleanup in some circumstances.

* :gh:commit:`b9112a9c`,
  :gh:commit:`2c287801`: OpenSSH 7.5
  permission denied prompts are now recognized. Contributed by Alex Willmer.

* A missing check caused an exception traceback to appear when using the
  ``ansible`` command-line tool with a missing or misspelled module name.

* Ansible since >=2.7 began importing :mod:`__main__` from
  :mod:`ansible.module_utils.basic`, causing an error during execution, due to
  the controller being configured to refuse network imports outside the
  ``ansible.*`` namespace. Update the target implementation to construct a stub
  :mod:`__main__` module to satisfy the otherwise seemingly vestigial import.


Core Library
~~~~~~~~~~~~

* A new :class:`mitogen.parent.CallChain` class abstracts safe pipelining of
  related function calls to a target context, cancelling the chain if an
  exception occurs.

* :gh:issue:`305`: fix a long-standing minor
  race relating to the logging framework, where *no route for Message..*
  would frequently appear during startup.

* :gh:issue:`313`:
  :meth:`mitogen.parent.Context.call` was documented as capable of accepting
  static methods. While possible on Python 2.x the result is ugly, and in every
  case it should be trivial to replace with a classmethod. The documentation
  was fixed.

* :gh:issue:`337`: to avoid a scaling
  limitation, a PTY is no longer allocated for each OpenSSH client if it can be
  avoided. PTYs are only allocated if a password is supplied, or when
  `host_key_checking=accept`. This is since Linux has a default of 4096 PTYs
  (``kernel.pty.max``), while OS X has a default of 127 and an absolute maximum
  of 999 (``kern.tty.ptmx_max``).

* :gh:issue:`339`: the LXD connection method
  was erroneously executing LXC Classic commands.

* :gh:issue:`345`: the SSH connection method
  allows optionally disabling ``IdentitiesOnly yes``.

* :gh:issue:`356`: if the master Python
  process does not have :data:`sys.executable` set, the default Python
  interpreter used for new children on the local machine defaults to
  ``"/usr/bin/python"``.

* :gh:issue:`366`,
  :gh:issue:`380`: attempts by children to
  import :mod:`__main__` where the main program module lacks an execution guard
  are refused, and an error is logged. This prevents a common and highly
  confusing error when prototyping new scripts.

* :gh:pull:`371`: the LXC connection method
  uses a more compatible method to establish an non-interactive session.
  Contributed by Brian Candler.

* :gh:commit:`af2ded66`: add
  :func:`mitogen.fork.on_fork` to allow non-Mitogen managed process forks to
  clean up Mitogen resources in the child.

* :gh:commit:`d6784242`: the setns method
  always resets ``HOME``, ``SHELL``, ``LOGNAME`` and ``USER`` environment
  variables to an account in the target container, defaulting to ``root``.

* :gh:commit:`830966bf`: the UNIX
  listener no longer crashes if the peer process disappears in the middle of
  connection setup.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Alex Russu <https://github.com/alexrussu>`_,
`Alex Willmer <https://github.com/moreati>`_,
`atoom <https://github.com/atoom>`_,
`Berend De Schouwer <https://github.com/berenddeschouwer>`_,
`Brian Candler <https://github.com/candlerb>`_,
`Dan Quackenbush <https://github.com/danquack>`_,
`dsgnr <https://github.com/dsgnr>`_,
`Jesse London <https://github.com/jesteria>`_,
`John McGrath <https://github.com/jmcgrath207>`_,
`Jonathan Rosser <https://github.com/jrosser>`_,
`Josh Smift <https://github.com/jbscare>`_,
`Luca Nunzi <https://github.com/0xlc>`_,
`Orion Poplawski <https://github.com/opoplawski>`_,
`Peter V. Saveliev <https://github.com/svinota/>`_,
`Pierre-Henry Muller <https://github.com/pierrehenrymuller>`_,
`Pierre-Louis Bonicoli <https://github.com/jesteria>`_,
`Prateek Jain <https://github.com/prateekj201>`_,
`RedheatWei <https://github.com/RedheatWei>`_,
`Rick Box <https://github.com/boxrick>`_,
`nikitakazantsev12 <https://github.com/nikitakazantsev12>`_,
`Tawana Musewe <https://github.com/tbtmuse>`_,
`Timo Beckers <https://github.com/ti-mo>`_, and
`Yannig Perré <https://github.com/yannig>`_.


v0.2.2 (2018-07-26)
-------------------

Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~

* :gh:issue:`291`: ``ansible_*_interpreter``
  variables are parsed using a restrictive shell-like syntax, supporting a
  common idiom where ``ansible_python_interpreter`` is set to ``/usr/bin/env
  python``.

* :gh:issue:`299`: fix the ``network_cli``
  connection type when the Mitogen strategy is active. Mitogen cannot help
  network device connections, however it should still be possible to use device
  connections while Mitogen is active.

* :gh:pull:`301`: variables like ``$HOME`` in
  the ``remote_tmp`` setting are evaluated correctly.

* :gh:pull:`303`: the :ref:`doas` become method
  is supported. Contributed by `Mike Walker
  <https://github.com/napkindrawing>`_.

* :gh:issue:`309`: fix a regression to
  process environment cleanup, caused by the change in v0.2.1 to run local
  tasks with the correct environment.

* :gh:issue:`317`: respect the verbosity
  setting when writing to Ansible's ``log_path``, if it is enabled. Child log
  filtering was also incorrect, causing the master to needlessly wake many
  times. This nets a 3.5% runtime improvement running against the local
  machine.

* The ``mitogen_ssh_debug_level`` variable is supported, permitting SSH debug
  output to be included in Mitogen's ``-vvv`` output when both are specified.


Core Library
~~~~~~~~~~~~

* :gh:issue:`291`: the ``python_path``
  parameter may specify an argument vector prefix rather than a string program
  path.

* :gh:issue:`300`: the broker could crash on
  OS X during shutdown due to scheduled `kqueue
  <https://www.freebsd.org/cgi/man.cgi?query=kqueue>`_ filter changes for
  descriptors that were closed before the IO loop resumes. As a temporary
  workaround, kqueue's bulk change feature is not used.

* :gh:pull:`303`: the :ref:`doas` become method
  is now supported. Contributed by `Mike Walker
  <https://github.com/napkindrawing>`_.

* :gh:issue:`307`: SSH login banner output
  containing the word 'password' is no longer confused for a password prompt.

* :gh:issue:`319`: SSH connections would
  fail immediately on Windows Subsystem for Linux, due to use of `TCSAFLUSH`
  with :func:`termios.tcsetattr`. The flag is omitted if WSL is detected.

* :gh:issue:`320`: The OS X poller
  could spuriously wake up due to ignoring an error bit set on events returned
  by the kernel, manifesting as a failure to read from an unrelated descriptor.

* :gh:issue:`342`: The ``network_cli``
  connection type would fail due to a missing internal SSH plugin method.

* Standard IO forwarding accidentally configured the replacement ``stdout`` and
  ``stderr`` write descriptors as non-blocking, causing subprocesses that
  generate more output than kernel buffer space existed to throw errors. The
  write ends are now configured as blocking.

* When :func:`mitogen.core.enable_profiling` is active, :mod:`mitogen.service`
  threads are profiled just like other threads.

* The ``ssh_debug_level`` parameter is supported, permitting SSH debug output
  to be redirected to a Mitogen logger when specified.

* Debug logs containing command lines are printed with the minimal quoting and
  escaping required.


Thanks!
~~~~~~~

Mitogen would not be possible without the support of users. A huge thanks for
the bug reports and pull requests in this release contributed by
`Alex Russu <https://github.com/alexrussu>`_,
`Andy Freeland <https://github.com/rouge8>`_,
`Ayaz Ahmed Khan <https://github.com/ayaz>`_,
`Colin McCarthy <https://github.com/colin-mccarthy>`_,
`Dan Quackenbush <https://github.com/danquack>`_,
`Duane Zamrok <https://github.com/dewthefifth>`_,
`Gonzalo Servat <https://github.com/gservat>`_,
`Guy Knights <https://github.com/knightsg>`_,
`Josh Smift <https://github.com/jbscare>`_,
`Mark Janssen <https://github.com/sigio>`_,
`Mike Walker <https://github.com/napkindrawing>`_,
`Orion Poplawski <https://github.com/opoplawski>`_,
`falbanese <https://github.com/falbanese>`_,
`Tawana Musewe <https://github.com/tbtmuse>`_, and
`Zach Swanson <https://github.com/zswanson>`_.


v0.2.1 (2018-07-10)
-------------------

Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~

* :gh:issue:`297`: compatibility: local
  actions set their working directory to that of their defining playbook, and
  inherit a process environment as if they were executed as a subprocess of the
  forked task worker.


v0.2.0 (2018-07-09)
-------------------

Mitogen 0.2.x is the inaugural feature-frozen branch eligible for fixes only,
except for problem areas listed as in-scope below. While stable from a
development perspective, it should still be considered "beta" at least for the
initial releases.

**In Scope**

* Python 3.x performance improvements
* Subprocess reaping improvements
* Major documentation improvements
* PyPI/packaging improvements
* Test suite improvements
* Replacement CI system to handle every supported OS
* Minor deviations from vanilla Ansible behaviour
* Ansible ``raw`` action support

The goal is a *tick/tock* model where even-numbered series are a maturation of
the previous unstable series, and unstable series are released on PyPI with
``--pre`` enabled. The API and user visible behaviour should remain unchanged
within a stable series.


Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~

* Support for Ansible 2.3 - 2.7.x and any mixture of Python 2.6, 2.7 or 3.6 on
  controller and target nodes.

* Drop-in support for many Ansible connection types.

* Preview of Connection Delegation feature.

* Built-in file transfer compatible with connection delegation.


Core Library
~~~~~~~~~~~~

* Synchronous connection establishment via OpenSSH, sudo, su, Docker, LXC and
  FreeBSD Jails, local subprocesses and :func:`os.fork`. Parallel connection
  setup is possible using multiple threads. Connections may be used from one or
  many threads after establishment.

* UNIX masters and children, with Linux, MacOS, FreeBSD, NetBSD, OpenBSD and
  Windows Subsystem for Linux explicitly supported.

* Automatic tests covering Python 2.6, 2.7 and 3.6 on Linux only.