From dfc67b89fdb055010c8a05987eb5d2be142e1775 Mon Sep 17 00:00:00 2001 From: David Wilson Date: Mon, 10 Sep 2018 18:32:23 +0100 Subject: [PATCH] docs: some more cleanups - add faulthandler/thread stacks to changelog. - various api.rst cleanups. - docs: explain chain_id in howitworks. --- docs/ansible.rst | 9 ++++++--- docs/api.rst | 26 +++++++++++++------------- docs/changelog.rst | 8 ++++++++ docs/howitworks.rst | 7 +++++++ mitogen/parent.py | 9 +++++---- 5 files changed, 39 insertions(+), 20 deletions(-) diff --git a/docs/ansible.rst b/docs/ansible.rst index 7a06c120..485c24dc 100644 --- a/docs/ansible.rst +++ b/docs/ansible.rst @@ -313,9 +313,10 @@ Performance ^^^^^^^^^^^ One roundtrip initiates a transfer larger than 124 KiB, while smaller transfers -are embedded in a 0-roundtrip remote call. For tools operating via SSH -multiplexing, 4 roundtrips are required to configure the IO channel, in -addition to the time to start the local and remote processes. +are embedded in a 0-roundtrip pipelined call. For tools operating via SSH +multiplexing, 4 roundtrips are required to configure the IO channel, followed +by 6 roundtrips to transfer the file in the case of ``sftp``, in addition to +the time to start the local and remote processes. An invocation of ``scp`` with an empty ``.profile`` over a 30 ms link takes ~140 ms, wasting 110 ms per invocation, rising to ~2,000 ms over a 400 ms @@ -848,6 +849,8 @@ logging is necessary. File-based logging can be enabled by setting enabled, one file per context will be created on the local machine and every target machine, as ``/tmp/mitogen..log``. +.. _diagnosing-hangs: + Diagnosing Hangs ~~~~~~~~~~~~~~~~ diff --git a/docs/api.rst b/docs/api.rst index cb980b55..c74193e3 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -792,8 +792,8 @@ Context Class .. method:: send (msg) - Arrange for `msg` to be delivered to this context. Updates the - message's `dst_id` prior to routing it via the associated router. + Arrange for `msg` to be delivered to this context. + :attr:`dst_id ` is set to the target context ID. :param mitogen.core.Message msg: The message. @@ -801,9 +801,9 @@ Context Class .. method:: send_async (msg, persist=False) Arrange for `msg` to be delivered to this context, with replies - delivered to a newly constructed Receiver. Updates the message's - `dst_id` prior to routing it via the associated router and registers a - handle which is placed in the message's `reply_to`. + directed to a newly constructed receiver. :attr:`dst_id + ` is set to the target context ID, and :attr:`reply_to + ` is set to the newly constructed receiver's handle. :param bool persist: If :data:`False`, the handler will be unregistered after a single @@ -818,15 +818,15 @@ Context Class .. method:: send_await (msg, deadline=None) - As with :meth:`send_async`, but expect a single reply - (`persist=False`) delivered within `deadline` seconds. + Like :meth:`send_async`, but expect a single reply (`persist=False`) + delivered within `deadline` seconds. :param mitogen.core.Message msg: The message. - :param float deadline: If not :data:`None`, seconds before timing out waiting for a reply. - + :returns: + The deserialized reply. :raises mitogen.core.TimeoutError: No message was received and `deadline` passed. @@ -838,9 +838,9 @@ Context Class .. class:: Context - Extend :class:`mitogen.core.Router` with functionality useful to - masters, and child contexts who later become parents. Currently when this - class is required, the target context's router is upgraded at runtime. + Extend :class:`mitogen.core.Context` with functionality useful to masters, + and child contexts who later become parents. Currently when this class is + required, the target context's router is upgraded at runtime. .. attribute:: default_call_chain @@ -1196,7 +1196,7 @@ Broker Class Responsible for handling I/O multiplexing in a private thread. **Note:** This is the somewhat limited core version of the Broker class - used by child contexts. The master subclass is documented below this one. + used by child contexts. The master subclass is documented below. .. attribute:: shutdown_timeout = 3.0 diff --git a/docs/changelog.rst b/docs/changelog.rst index 7beb043c..f4c46976 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -74,6 +74,14 @@ Enhancements improvement, but is easily noticeable on short many-target runs, where startup overhead dominates runtime. +* The `faulthandler `_ module is + automatically activated if it is installed, simplifying debugging of hangs. + See :ref:`diagnosing-hangs` for more information. + +* The ``MITOGEN_DUMP_THREAD_STACKS`` environment variable's value now indicates + the number of seconds between stack dumps. See :ref:`diagnosing-hangs` for + more information. + Fixes ^^^^^ diff --git a/docs/howitworks.rst b/docs/howitworks.rst index 1e3d2768..a3b08eac 100644 --- a/docs/howitworks.rst +++ b/docs/howitworks.rst @@ -377,6 +377,13 @@ Children listen on the following handles: 6-tuples from :class:`mitogen.parent.CallChain`, imports ``mod_name``, then attempts to execute `class_name.func_name(\*args, \**kwargs)`. + * `chain_id`: if not :data:`None`, an identifier unique to the originating + :class:`mitogen.parent.CallChain`. When set, if an exception occurs + during a call, future calls with the same ID automatically fail with the + same exception without ever executing, and failed calls with no + `reply_to` set are not dumped to the logging framework as they otherwise + would. This is used to implement pipelining. + When this channel is closed (by way of receiving a dead message), the child's main thread begins graceful shutdown of its own :py:class:`Broker` and :py:class:`Router`. diff --git a/mitogen/parent.py b/mitogen/parent.py index 8e9f53a5..bb2b5d1e 100644 --- a/mitogen/parent.py +++ b/mitogen/parent.py @@ -1149,14 +1149,15 @@ class CallChain(object): calls execute until :meth:`reset` is invoked. No exception is logged for calls made with :meth:`call_no_reply`, instead - it is saved and reported as the result of subsequent :meth:`call` or - :meth:`call_async` calls. + the exception is saved and reported as the result of subsequent + :meth:`call` or :meth:`call_async` calls. Sequences of asynchronous calls can be made without wasting network round-trips to discover if prior calls succeed, and chains originating from multiple unrelated source contexts may overlap concurrently at a target - context without interference. In this example, 4 calls complete in one - round-trip:: + context without interference. + + In this example, 4 calls complete in one round-trip:: chain = mitogen.parent.CallChain(context, pipelined=True) chain.call_no_reply(os.mkdir, '/tmp/foo')