.. _changelog:
Release Notes
=============
.. raw:: html
v0.2.8 (unreleased)
-------------------
To avail of fixes in an unreleased version, please download a ZIP file
`directly from GitHub `_.
Enhancements
^^^^^^^^^^^^
* `#587 `_: Ansible 2.8 is partially
supported. `Become plugins
`_ and
`interpreter discovery
`_
are not yet handled.
* The ``MITOGEN_CPU_COUNT`` environment variable shards the connection
multiplexer into per-CPU workers. This improves throughput for large runs
especially involving file transfer, and is a prerequisite for future
in-process SSH support. One multiplexer starts by default, to match existing
behaviour.
* `#419 `_,
`#470 `_, file descriptor usage
during large runs is halved, as it is no longer necessary to manage read and
write sides distinctly in order to work around a design limitation.
* `#419 `_: almost all connection
setup happens on one thread, reducing GIL contention and context switching
early in a run.
* `#419 `_: 2 network round-trips
were removed from early connection setup.
* `? `_,
`? `_,
`? `_,
`? `_: locking is avoided in
some hot paths, and locks that must be taken are held for less time.
Mitogen for Ansible
^^^^^^^^^^^^^^^^^^^
* `#363 `_: fix an obscure race
matching *Permission denied* errors from some versions of ``su`` running on
heavily loaded machines.
* `#549 `_: the open file descriptor
limit for the Ansible process is increased to the available hard limit. It is
common for distributions to ship with a much higher hard limit than their
default soft limit, allowing *"too many open files"* errors to be avoided
more often in large runs without user configuration.
* `#578 `_: the extension could crash
while rendering an error message, due to an incorrect format string.
* `#590 `_: the importer can handle
modules that replace themselves in :mod:`sys.modules` during import.
* `#591 `_: the target's current
working directory is restored to a known-existent directory between tasks to
ensure :func:`os.getcwd` will not fail when called, in the same way that
:class:`AnsibleModule` restores it during initialization. However this
restore happens before the module ever executes, ensuring any code that calls
:func:`os.getcwd` prior to :class:`AnsibleModule` initialization, such as the
Ansible 2.7 ``pip`` module, cannot fail due to the behavior of a prior task.
* `#593 `_: the SSH connection method
exposes ``mitogen_ssh_keepalive_interval`` and
``mitogen_ssh_keepalive_count`` variables, and the default timeout for an SSH
server has been increased from `15*3` seconds to `30*10` seconds.
* `7ae926b3 `_: the
``lineinfile`` module began leaking writable temporary file descriptors since
Ansible 2.7.0. When ``lineinfile`` was used to create or modify a script, and
that script was later executed, the execution could fail with "*text file
busy*" due to the leaked descriptor. Temporary descriptors are now tracked
and cleaned up on exit for all modules.
Core Library
~~~~~~~~~~~~
* Log readability is improving, and many :func:`repr` strings are more
descriptive. The old pseudo-function-call format is slowly migrating to
human-readable output where possible. For example,
*"Stream(ssh:123).connect()"* might be written *"connecting to ssh:123"*.
* :func:`bytearray` was removed from the list of supported serialization types.
It was never portable between Python versions, unused, and never made much
sense to support.
* `#170 `_: to improve subprocess
management and asynchronous connect, a :class:`mitogen.parent.TimerList`
interface is available, accessible as :attr:`Broker.timers` in an
asynchronous context.
* `#419 `_: the internal
:class:`mitogen.core.Stream` has been refactored into 7 new classes,
modularizing protocol behaviour, output buffering, line-oriented input
parsing, option handling and connection management. Connection setup is
internally asynchronous, laying almost all the groundwork needed for fully
asynchronous connect, proxied Ansible become plug-ins, and integrating
`libssh `_.
* `#169 `_,
`#419 `_: zombie child reaping has
vastly improved, by using timers to efficiently poll for a slow child to
finish exiting. Polling avoids relying on process-global configuration such
as a `SIGCHLD` handler, or :func:`signal.set_wakeup_fd` available in modern
Python.
* `#256 `_,
`#419 `_: most :func:`os.dup` use
was eliminated, along with almost all manual file descriptor management.
Descriptors are trapped in :func:`os.fdopen` objects at creation, ensuring a
leaked object will close itself, and ensuring every descriptor is fused to a
`closed` flag, preventing historical bugs where a double close could destroy
descriptors belonging to unrelated streams.
* `#606 `_: fix example code on the
documentation front page.
* `a5536c35 `_: avoid quadratic
buffer management when logging lines received from a child's redirected
standard IO.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Andreas Hubert `_.
`Anton Markelov `_,
`Nigel Metheringham `_,
`Orion Poplawski `_,
`Pieter Voet `_,
`Stefane Fermigier `_,
`Szabó Dániel Ernő `_,
`Ulrich Schreiner `_,
`Yuki Nishida `_,
`@ghp-rr `_, and
`@rizzly `_.
v0.2.7 (2019-05-19)
-------------------
This release primarily exists to add a descriptive error message when running
on Ansible 2.8, which is not yet supported.
Fixes
~~~~~
* `#557 `_: fix a crash when running
on machines with high CPU counts.
* `#570 `_: the ``firewalld`` module
internally caches a dbus name that changes across ``firewalld`` restarts,
causing a failure if the service is restarted between ``firewalld`` module invocations.
* `#575 `_: fix a crash when
rendering an error message to indicate no usable temporary directories could
be found.
* `#576 `_: fix a crash during
startup on SuSE Linux 11, due to an incorrect version compatibility check in
the Mitogen code.
* `#581 `_: a
``mitogen_mask_remote_name`` Ansible variable is exposed, to allow masking
the username, hostname and process ID of ``ansible-playbook`` running on the
controller machine.
* `#587 `_: display a friendly
message when running on an unsupported version of Ansible, to cope with
potential influx of 2.8-related bug reports.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Orion Poplawski `_,
`Thibaut Barrère `_,
`@Moumoutaru `_, and
`@polski-g `_.
v0.2.6 (2019-03-06)
-------------------
Fixes
~~~~~
* `#542 `_: some versions of OS X
ship a default Python that does not support :func:`select.poll`. Restore the
0.2.3 behaviour of defaulting to Kqueue in this case, but still prefer
:func:`select.poll` if it is available.
* `#545 `_: an optimization
introduced in `#493 `_ caused a
64-bit integer to be assigned to a 32-bit field on ARM 32-bit targets,
causing runs to fail.
* `#548 `_: `mitogen_via=` could fail
when the selected transport was set to ``smart``.
* `#550 `_: avoid some broken
TTY-related `ioctl()` calls on Windows Subsystem for Linux 2016 Anniversary
Update.
* `#554 `_: third party Ansible
action plug-ins that invoked :func:`_make_tmp_path` repeatedly could trigger
an assertion failure.
* `#555 `_: work around an old idiom
that reloaded :mod:`sys` in order to change the interpreter's default encoding.
* `ffae0355 `_: needless
information was removed from the documentation and installation procedure.
Core Library
~~~~~~~~~~~~
* `#535 `_: to support function calls
on a service pool from another thread, :class:`mitogen.select.Select`
additionally permits waiting on :class:`mitogen.core.Latch`.
* `#535 `_:
:class:`mitogen.service.Pool.defer` allows any function to be enqueued for
the thread pool from another thread.
* `#535 `_: a new
:mod:`mitogen.os_fork` module provides a :func:`os.fork` wrapper that pauses
thread activity during fork. On Python<2.6, :class:`mitogen.core.Broker` and
:class:`mitogen.service.Pool` automatically record their existence so that a
:func:`os.fork` monkey-patch can automatically pause them for any attempt to
start a subprocess.
* `ca63c26e `_:
:meth:`mitogen.core.Latch.put`'s `obj` argument was made optional.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Fabian Arrotin `_,
`Giles Westwood `_,
`Matt Layman `_,
`Percy Grunwald `_,
`Petr Enkov `_,
`Tony Finch `_,
`@elbunda `_, and
`@zyphermonkey `_.
v0.2.5 (2019-02-14)
-------------------
Fixes
~~~~~
* `#511 `_,
`#536 `_: changes in 0.2.4 to
repair ``delegate_to`` handling broke default ``ansible_python_interpreter``
handling. Test coverage was added.
* `#532 `_: fix a race in the service
used to propagate Ansible modules, that could easily manifest when starting
asynchronous tasks in a loop.
* `#536 `_: changes in 0.2.4 to
support Python 2.4 interacted poorly with modules that imported
``simplejson`` from a controller that also loaded an incompatible newer
version of ``simplejson``.
* `#537 `_: a swapped operator in the
CPU affinity logic meant 2 cores were reserved on 1`_: the source distribution
includes a ``LICENSE`` file.
* `#539 `_: log output is no longer
duplicated when the Ansible ``log_path`` setting is enabled.
* `#540 `_: the ``stderr`` stream of
async module invocations was previously discarded.
* `#541 `_: Python error logs
originating from the ``boto`` package are quiesced, and only appear in
``-vvv`` output. This is since EC2 modules may trigger errors during normal
operation, when retrying transiently failing requests.
* `748f5f67 `_,
`21ad299d `_,
`8ae6ca1d `_,
`7fd0d349 `_:
the ``ansible_ssh_host``, ``ansible_ssh_user``, ``ansible_user``,
``ansible_become_method``, and ``ansible_ssh_port`` variables more correctly
match typical behaviour when ``mitogen_via=`` is active.
* `2a8567b4 `_: fix a race
initializing a child's service thread pool on Python 3.4+, due to a change in
locking scheme used by the Python import mechanism.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Carl George `_,
`Guy Knights `_, and
`Josh Smift `_.
v0.2.4 (2019-02-10)
-------------------
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
This release includes a huge variety of important fixes and new optimizations.
It is 35% faster than 0.2.3 on a synthetic 64 target run that places heavy load
on the connection multiplexer.
Enhancements
^^^^^^^^^^^^
* `#76 `_,
`#351 `_,
`#352 `_: disconnect propagation
has improved, allowing Ansible to cancel waits for responses from abruptly
disconnected targets. This ensures a task will reliably fail rather than
hang, for example on network failure or EC2 instance maintenance.
* `#369 `_,
`#407 `_: :meth:`Connection.reset`
is implemented, allowing `meta: reset_connection
`_ to shut
down the remote interpreter as documented, and improving support for the
`reboot
`_
module.
* `09aa27a6 `_: the
``mitogen_host_pinned`` strategy wraps the ``host_pinned`` strategy
introduced in Ansible 2.7.
* `#477 `_: Python 2.4 is fully
supported by the core library and tested automatically, in any parent/child
combination of 2.4, 2.6, 2.7 and 3.6 interpreters.
* `#477 `_: Ansible 2.3 is fully
supported and tested automatically. In combination with the core library
Python 2.4 support, this allows Red Hat Enterprise Linux 5 targets to be
managed with Mitogen. The ``simplejson`` package need not be installed on
such targets, as is usually required by Ansible.
* `#412 `_: to simplify diagnosing
connection configuration problems, Mitogen ships a ``mitogen_get_stack``
action that is automatically added to the action plug-in path. See
:ref:`mitogen-get-stack` for more information.
* `152effc2 `_,
`bd4b04ae `_: a CPU affinity
policy was added for Linux controllers, reducing latency and SMP overhead on
hot paths exercised for every task. This yielded a 19% speedup in a 64-target
job composed of many short tasks, and should easily be visible as a runtime
improvement in many-host runs.
* `2b44d598 `_: work around a
defective caching mechanism by pre-heating it before spawning workers. This
saves 40% runtime on a synthetic repetitive task.
* `0979422a `_: an expensive
dependency scanning step was redundantly invoked for every task,
bottlenecking the connection multiplexer.
* `eaa990a97 `_: a new
``mitogen_ssh_compression`` variable is supported, allowing Mitogen's default
SSH compression to be disabled. SSH compression is a large contributor to CPU
usage in many-target runs, and severely limits file transfer. On a `"shell:
hostname"` task repeated 500 times, Mitogen requires around 800 bytes per
task with compression, rising to 3 KiB without. File transfer throughput
rises from ~25MiB/s when enabled to ~200MiB/s when disabled.
* `#260 `_,
`a18a083c `_: brokers no
longer wait for readiness indication to transmit, and instead assume
transmission will succeed. As this is usually true, one loop iteration and
two poller reconfigurations are avoided, yielding a significant reduction in
interprocess round-trip latency.
* `#415 `_,
`#491 `_,
`#493 `_: the interface employed
for in-process queues changed from `kqueue
`_ / `epoll
`_ to `poll()
`_, which requires no setup
or teardown, yielding a 38% latency reduction for inter-thread communication.
Fixes
^^^^^
* `#251 `_,
`#359 `_,
`#396 `_,
`#401 `_,
`#404 `_,
`#412 `_,
`#434 `_,
`#436 `_,
`#465 `_: connection delegation and
``delegate_to:`` handling suffered a major regression in 0.2.3. The 0.2.2
behaviour has been restored, and further work has been made to improve the
compatibility of connection delegation's configuration building methods.
* `#323 `_,
`#333 `_: work around a Windows
Subsystem for Linux bug that caused tracebacks to appear during shutdown.
* `#334 `_: the SSH method
tilde-expands private key paths using Ansible's logic. Previously the path
was passed unmodified to SSH, which expanded it using :func:`pwd.getpwnam`.
This differs from :func:`os.path.expanduser`, which uses the ``HOME``
environment variable if it is set, causing behaviour to diverge when Ansible
was invoked across user accounts via ``sudo``.
* `#364 `_: file transfers from
controllers running Python 2.7.2 or earlier could be interrupted due to a
forking bug in the :mod:`tempfile` module.
* `#370 `_: the Ansible
`reboot `_
module is supported.
* `#373 `_: the LXC and LXD methods
print a useful hint on failure, as no useful error is normally logged to the
console by these tools.
* `#374 `_,
`#391 `_: file transfer and module
execution from 2.x controllers to 3.x targets was broken due to a regression
caused by refactoring, and compounded by `#426
`_.
* `#400 `_: work around a threading
bug in the AWX display callback when running with high verbosity setting.
* `#409 `_: the setns method was
silently broken due to missing tests. Basic coverage was added to prevent a
recurrence.
* `#409 `_: the LXC and LXD methods
support ``mitogen_lxc_path`` and ``mitogen_lxc_attach_path`` variables to
control the location of third pary utilities.
* `#410 `_: the sudo method supports
the SELinux ``--type`` and ``--role`` options.
* `#420 `_: if a :class:`Connection`
was constructed in the Ansible top-level process, for example while executing
``meta: reset_connection``, resources could become undesirably shared in
subsequent children.
* `#426 `_: an oversight while
porting to Python 3 meant no automated 2->3 tests were running. A significant
number of 2->3 bugs were fixed, mostly in the form of Unicode/bytes
mismatches.
* `#429 `_: the ``sudo`` method can
now recognize internationalized password prompts.
* `#362 `_,
`#435 `_: the previous fix for slow
Python 2.x subprocess creation on Red Hat caused newly spawned children to
have a reduced open files limit. A more intrusive fix has been added to
directly address the problem without modifying the subprocess environment.
* `#397 `_,
`#454 `_: the previous approach to
handling modern Ansible temporary file cleanup was too aggressive, and could
trigger early finalization of Cython-based extension modules, leading to
segmentation faults.
* `#499 `_: the ``allow_same_user``
Ansible configuration setting is respected.
* `#527 `_: crashes in modules are
trapped and reported in a manner that matches Ansible. In particular, a
module crash no longer leads to an exception that may crash the corresponding
action plug-in.
* `dc1d4251 `_: the
``synchronize`` module could fail with the Docker transport due to a missing
attribute.
* `599da068 `_: fix a race
when starting async tasks, where it was possible for the controller to
observe no status file on disk before the task had a chance to write one.
* `2c7af9f04 `_: Ansible
modules were repeatedly re-transferred. The bug was hidden by the previously
mandatorily enabled SSH compression.
Core Library
~~~~~~~~~~~~
* `#76 `_: routing records the
destination context IDs ever received on each stream, and when disconnection
occurs, propagates :data:`mitogen.core.DEL_ROUTE` messages towards every
stream that ever communicated with the disappearing peer, rather than simply
towards parents. Conversations between nodes anywhere in the tree receive
:data:`mitogen.core.DEL_ROUTE` when either participant disconnects, allowing
receivers to wake with :class:`mitogen.core.ChannelError`, even when one
participant is not a parent of the other.
* `#109 `_,
`57504ba6 `_: newer Python 3
releases explicitly populate :data:`sys.meta_path` with importer internals,
causing Mitogen to install itself at the end of the importer chain rather
than the front.
* `#310 `_: support has returned for
trying to figure out the real source of non-module objects installed in
:data:`sys.modules`, so they can be imported. This is needed to handle syntax
sugar used by packages like :mod:`plumbum`.
* `#349 `_: an incorrect format
string could cause large stack traces when attempting to import built-in
modules on Python 3.
* `#387 `_,
`#413 `_: dead messages include an
optional reason in their body. This is used to cause
:class:`mitogen.core.ChannelError` to report far more useful diagnostics at
the point the error occurs that previously would have been buried in debug
log output from an unrelated context.
* `#408 `_: a variety of fixes were
made to restore Python 2.4 compatibility.
* `#399 `_,
`#437 `_: ignore a
:class:`DeprecationWarning` to avoid failure of the ``su`` method on Python
3.7.
* `#405 `_: if an oversized message
is rejected, and it has a ``reply_to`` set, a dead message is returned to the
sender. This ensures function calls exceeding the configured maximum size
crash rather than hang.
* `#406 `_:
:class:`mitogen.core.Broker` did not call :meth:`mitogen.core.Poller.close`
during shutdown, leaking the underlying poller FD in masters and parents.
* `#406 `_: connections could leak
FDs when a child process failed to start.
* `#288 `_,
`#406 `_,
`#417 `_: connections could leave
FD wrapper objects that had not been closed lying around to be closed during
garbage collection, causing reused FD numbers to be closed at random moments.
* `#411 `_: the SSH method typed
"``y``" rather than the requisite "``yes``" when `check_host_keys="accept"`
was configured. This would lead to connection timeouts due to the hung
response.
* `#414 `_,
`#425 `_: avoid deadlock of forked
children by reinitializing the :mod:`mitogen.service` pool lock.
* `#416 `_: around 1.4KiB of memory
was leaked on every RPC, due to a list of strong references keeping alive any
handler ever registered for disconnect notification.
* `#418 `_: the
:func:`mitogen.parent.iter_read` helper would leak poller FDs, because
execution of its :keyword:`finally` block was delayed on Python 3. Now
callers explicitly close the generator when finished.
* `#422 `_: the fork method could
fail to start if :data:`sys.stdout` was opened in block buffered mode, and
buffered data was pending in the parent prior to fork.
* `#438 `_: a descriptive error is
logged when stream corruption is detected.
* `#439 `_: descriptive errors are
raised when attempting to invoke unsupported function types.
* `#444 `_: messages regarding
unforwardable extension module are no longer logged as errors.
* `#445 `_: service pools unregister
the :data:`mitogen.core.CALL_SERVICE` handle at shutdown, ensuring any
outstanding messages are either processed by the pool as it shuts down, or
have dead messages sent in reply to them, preventing peer contexts from
hanging due to a forgotten buffered message.
* `#446 `_: given thread A calling
:meth:`mitogen.core.Receiver.close`, and thread B, C, and D sleeping in
:meth:`mitogen.core.Receiver.get`, previously only one sleeping thread would
be woken with :class:`mitogen.core.ChannelError` when the receiver was
closed. Now all threads are woken per the docstring.
* `#447 `_: duplicate attempts to
invoke :meth:`mitogen.core.Router.add_handler` cause an error to be raised,
ensuring accidental re-registration of service pools are reported correctly.
* `#448 `_: the import hook
implementation now raises :class:`ModuleNotFoundError` instead of
:class:`ImportError` in Python 3.6 and above, to cope with an upcoming
version of the :mod:`subprocess` module requiring this new subclass to be
raised.
* `#453 `_: the loggers used in
children for standard IO redirection have propagation disabled, preventing
accidental reconfiguration of the :mod:`logging` package in a child from
setting up a feedback loop.
* `#456 `_: a descriptive error is
logged when :meth:`mitogen.core.Broker.defer` is called after the broker has
shut down, preventing new messages being enqueued that will never be sent,
and subsequently producing a program hang.
* `#459 `_: the beginnings of a
:meth:`mitogen.master.Router.get_stats` call has been added. The initial
statistics cover the module loader only.
* `#462 `_: Mitogen could fail to
open a PTY on broken Linux systems due to a bad interaction between the glibc
:func:`grantpt` function and an incorrectly mounted ``/dev/pts`` filesystem.
Since correct group ownership is not required in most scenarios, when this
problem is detected, the PTY is allocated and opened directly by the library.
* `#479 `_: Mitogen could fail to
import :mod:`__main__` on Python 3.4 and newer due to a breaking change in
the :mod:`pkgutil` API. The program's main script is now handled specially.
* `#481 `_: the version of `sudo`
that shipped with CentOS 5 replaced itself with the program to be executed,
and therefore did not hold any child PTY open on our behalf. The child
context is updated to preserve any PTY FD in order to avoid the kernel
sending `SIGHUP` early during startup.
* `#523 `_: the test suite didn't
generate a code coverage report if any test failed.
* `#524 `_: Python 3.6+ emitted a
:class:`DeprecationWarning` for :func:`mitogen.utils.run_with_router`.
* `#529 `_: Code coverage of the
test suite was not measured across all Python versions.
* `16ca111e `_: handle OpenSSH
7.5 permission denied prompts when ``~/.ssh/config`` rewrites are present.
* `9ec360c2 `_: a new
:meth:`mitogen.core.Broker.defer_sync` utility function is provided.
* `f20e0bba `_:
:meth:`mitogen.service.FileService.register_prefix` permits granting
unprivileged access to whole filesystem subtrees, rather than single files at
a time.
* `8f85ee03 `_:
:meth:`mitogen.core.Router.myself` returns a :class:`mitogen.core.Context`
referring to the current process.
* `824c7931 `_: exceptions
raised by the import hook were updated to include probable reasons for
a failure.
* `57b652ed `_: a stray import
meant an extra roundtrip and ~4KiB of data was wasted for any context that
imported :mod:`mitogen.parent`.
Thanks!
~~~~~~~
Mitogen would not be possible without the support of users. A huge thanks for
bug reports, testing, features and fixes in this release contributed by
`Alex Willmer `_,
`Andreas Krüger `_,
`Anton Stroganov `_,
`Berend De Schouwer `_,
`Brian Candler `_,
`dsgnr `_,
`Duane Zamrok `_,
`Eric Chang `_,
`Gerben Meijer `_,
`Guy Knights `_,
`Jesse London `_,
`Jiří Vávra `_,
`Johan Beisser `_,
`Jonathan Rosser `_,
`Josh Smift `_,
`Kevin Carter `_,
`Mehdi `_,
`Michael DeHaan `_,
`Michal Medvecky `_,
`Mohammed Naser `_,
`Peter V. Saveliev `_,
`Pieter Avonts `_,
`Ross Williams `_,
`Sergey `_,
`Stéphane `_,
`Strahinja Kustudic `_,
`Tom Parker-Shemilt `_,
`Younès HAFRI `_,
`@killua-eu `_,
`@myssa91 `_,
`@ohmer1 `_,
`@s3c70r `_,
`@syntonym `_,
`@trim777 `_,
`@whky `_, and
`@yodatak `_.
v0.2.3 (2018-10-23)
-------------------
Mitogen for Ansible
~~~~~~~~~~~~~~~~~~~
Enhancements
^^^^^^^^^^^^
* `#315 `_,
`#392 `_: Ansible 2.6 and 2.7 are
supported.
* `#321 `_,
`#336 `_: temporary file handling
was simplified, undoing earlier damage caused by compatibility fixes,
improving 2.6 compatibility, and avoiding two network roundtrips for every
related action
(`assemble `_,
`aws_s3 `_,
`copy `_,
`patch `_,
`script `_,
`template `_,
`unarchive `_,
`uri `_). See
:ref:`ansible_tempfiles` for a complete description.
* `#376 `_,
`#377 `_: the ``kubectl`` connection
type is now supported. Contributed by Yannig Perré.
* `084c0ac0 `_: avoid a
roundtrip in
`copy `_ and
`template `_
due to an unfortunate default.
* `7458dfae `_: avoid a
roundtrip when transferring files smaller than 124KiB. Copy and template
actions are now 2-RTT, reducing runtime for a 20-iteration template loop over
a 250 ms link from 30 seconds to 10 seconds compared to v0.2.2, down from 120
seconds compared to vanilla.
* `#337 `_: To avoid a scaling
limitation, a PTY is no longer allocated for an SSH connection unless the
configuration specifies a password.
* `d62e6e2a `_: many-target
runs executed the dependency scanner redundantly due to missing
synchronization, wasting significant runtime in the connection multiplexer.
In one case work was reduced by 95%, which may manifest as faster runs.
* `5189408e `_: threads are
cooperatively scheduled, minimizing `GIL
`_ contention, and
reducing context switching by around 90%. This manifests as an overall
improvement, but is easily noticeable on short many-target runs, where
startup overhead dominates runtime.
* The `faulthandler `_ module is
automatically activated if it is installed, simplifying debugging of hangs.
See :ref:`diagnosing-hangs` for details.
* The ``MITOGEN_DUMP_THREAD_STACKS`` environment variable's value now indicates
the number of seconds between stack dumps. See :ref:`diagnosing-hangs` for
details.
Fixes
^^^^^
* `#251 `_,
`#340 `_: Connection Delegation
could establish connections to the wrong target when ``delegate_to:`` is
present.
* `#291 `_: when Mitogen had
previously been installed using ``pip`` or ``setuptools``, the globally
installed version could conflict with a newer version bundled with an
extension that had been installed using the documented steps. Now the bundled
library always overrides over any system-installed copy.
* `#324 `_: plays with a
`custom module_utils `_
would fail due to fallout from the Python 3 port and related tests being
disabled.
* `#331 `_: the connection
multiplexer subprocess always exits before the main Ansible process, ensuring
logs generated by it do not overwrite the user's prompt when ``-vvv`` is
enabled.
* `#332 `_: support a new
:func:`sys.excepthook`-based module exit mechanism added in Ansible 2.6.
* `#338 `_: compatibility: changes to
``/etc/environment`` and ``~/.pam_environment`` made by a task are reflected
in the runtime environment of subsequent tasks. See
:ref:`ansible_process_env` for a complete description.
* `#343 `_: the sudo ``--login``
option is supported.
* `#344 `_: connections no longer
fail when the controller's login username contains slashes.
* `#345 `_: the ``IdentitiesOnly
yes`` option is no longer supplied to OpenSSH by default, better matching
Ansible's behaviour.
* `#355 `_: tasks configured to run
in an isolated forked subprocess were forked from the wrong parent context.
This meant built-in modules overridden via a custom ``module_utils`` search
path may not have had any effect.
* `#362 `_: to work around a slow
algorithm in the :mod:`subprocess` module, the maximum number of open files
in processes running on the target is capped to 512, reducing the work
required to start a subprocess by >2000x in default CentOS configurations.
* `#397 `_: recent Mitogen master
versions could fail to clean up temporary directories in a number of
circumstances, and newer Ansibles moved to using :mod:`atexit` to effect
temporary directory cleanup in some circumstances.
* `b9112a9c `_,
`2c287801 `_: OpenSSH 7.5
permission denied prompts are now recognized. Contributed by Alex Willmer.
* A missing check caused an exception traceback to appear when using the
``ansible`` command-line tool with a missing or misspelled module name.
* Ansible since >=2.7 began importing :mod:`__main__` from
:mod:`ansible.module_utils.basic`, causing an error during execution, due to
the controller being configured to refuse network imports outside the
``ansible.*`` namespace. Update the target implementation to construct a stub
:mod:`__main__` module to satisfy the otherwise seemingly vestigial import.
Core Library
~~~~~~~~~~~~
* A new :class:`mitogen.parent.CallChain` class abstracts safe pipelining of
related function calls to a target context, cancelling the chain if an
exception occurs.
* `#305 `_: fix a long-standing minor
race relating to the logging framework, where *no route for Message..*
would frequently appear during startup.
* `#313 `_:
:meth:`mitogen.parent.Context.call` was documented as capable of accepting
static methods. While possible on Python 2.x the result is ugly, and in every
case it should be trivial to replace with a classmethod. The documentation
was fixed.
* `#337 `_: to avoid a scaling
limitation, a PTY is no longer allocated for each OpenSSH client if it can be
avoided. PTYs are only allocated if a password is supplied, or when
`host_key_checking=accept`. This is since Linux has a default of 4096 PTYs
(``kernel.pty.max``), while OS X has a default of 127 and an absolute maximum
of 999 (``kern.tty.ptmx_max``).
* `#339 `_: the LXD connection method
was erroneously executing LXC Classic commands.
* `#345 `_: the SSH connection method
allows optionally disabling ``IdentitiesOnly yes``.
* `#356