diff --git a/docs/getting_started.rst b/docs/getting_started.rst index 264fab78..a205d203 100644 --- a/docs/getting_started.rst +++ b/docs/getting_started.rst @@ -2,7 +2,98 @@ Getting Started =============== -xxx +.. warning:: + + This section is incomplete. + + +Liability Waiver +---------------- + +.. image:: images/radiation.png + :align: right + +Before proceeding, it is crucial you understand what you're involving yourself +and possibly your team with: + +* Constructing the most fundamental class, :py:class:`Broker + `, causes a new thread to be spawned, exposing a huge + class of difficult to analyse behaviours that Python software generally does + not suffer from. + + While every effort is made to hide this complexity, you should expect + threading-related encounters during development. See :ref:`troubleshooting` + for more information. + +* While high-level abstractions are provided, you must understand how Mitogen + works before depending on it. Mitogen interacts with many aspects of the + operating system, network, SSH, sudo, sockets, TTYs, Python runtime, and + timing and ordering uncertainty introduced through interaction with the + network and OS scheduling. + + Knowledge of this domain is typically gained through painful years of ugly + attempts hacking system-level programs, and learning through continual + suffering how to debug the messes left behind. If you feel you lack resources + to diagnose problems independently, Mitogen is not appropriate, prefer a + higher level solution instead. Bug reports failing this expectation risk + unfavourable treatment. + + +Broker And Router +----------------- + +.. image:: images/layout.png +.. currentmodule:: mitogen.master + +Execution starts when your program constructs a :py:class:`Broker` and +associated :py:class:`Router`. The broker is responsible for multiplexing IO to +children from a private thread, while in children, it is additionally +responsible for ensuring robust destruction if communication with the master +is lost. + +:py:class:`Router` is responsible for receiving messages and either dispatching +them to a callback from the broker thread (registered by +:py:meth:`add_handler() `), or forwarding them +to a :py:class:`Stream `. See :ref:`routing` for an +in-depth description. :py:class:`Router` also doubles as the entry point to +Mitogen's public API. + +.. code-block:: python + + broker = mitogen.master.Broker() + router = mitogen.master.Router(broker) + + try: + # Your code here. + finally: + broker.shutdown() + +As your program will not exit if threads are still running when the main thread +exits, it is crucial :py:meth:`Broker.shutdown` is called reliably at exit. +Helpers are provided by :py:mod:`mitogen.utils` to ensure :py:class:`Broker` is +reliably destroyed: + +.. code-block:: python + + def do_mitogen_stuff(router): + # Your code here. + + mitogen.utils.run_with_router(do_mitogen_stuff) + +If your program cannot live beneath :py:func:`mitogen.utils.run_with_router` on +the stack, you must must arrange for :py:meth:`Broker.shutdown` to be called +anywhere exit of the main thread may be triggered. + + +Creating A Context +------------------ + +Contexts simply refer to external Python programs over which your program has +control. They can be created as subprocesses on the local machine, in another +user account via ``sudo``, on a remote machine via ``ssh``, and in any +recursive combination of the above. + +Now a :py:class:`Router` exists, our first :py:class:`Context` can be created. .. _serialization-rules: @@ -28,4 +119,25 @@ User-defined types may not be used, except for: * :py:class:`mitogen.core.CallError` * :py:class:`mitogen.core.Context` -* :py:class:`mitogen.core._Dead` +* :py:class:`mitogen.core._DEAD` + + +.. _troubleshooting: + +Troubleshooting +--------------- + +.. warning:: + + This section is incomplete. + +A typical example is a hang due to your application's main thread exitting +perhaps due to an unhandled exception, without first arranging for any +:py:class:`Broker ` to be shut down gracefully. + +Another example would be your main thread hanging indefinitely because a bug +in Mitogen fails to notice an event (such as RPC completion) your thread is +waiting for will never complete. Solving this kind of hang is a work in +progress. + +router.enable_debug() diff --git a/docs/howitworks.rst b/docs/howitworks.rst index 5b9a4dc0..e624af37 100644 --- a/docs/howitworks.rst +++ b/docs/howitworks.rst @@ -398,6 +398,9 @@ presence of many machines, and would require manually splitting up the parts of Twisted that we would like to use. + +.. _routing: + Message Routing --------------- diff --git a/docs/images/layout.graphml b/docs/images/layout.graphml new file mode 100644 index 00000000..4aa1f95e --- /dev/null +++ b/docs/images/layout.graphml @@ -0,0 +1,665 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + Master + + + + + + + + + + Folder 1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Context + + + + + + + + + call(func, *args) +send(msg) + + + + + + + + + + + + + Broker + + + + + + + + + start_transmit(strm) +stop_transmit(strm) + + + + + + + + + + + + Broker Thread + + + + + + + + + + + + + + + + + + User Threads + + + + + + + + + + + + + + + + + + + Router + + + + + + + + handlers[] + ssh(hostname=...) +sudo(username=...) +route(msg) + + + + + + + + + + + + + Stream + + + + + + + + + send() +on_transmit() + + + + + + + + + + + + + Channel + + + + + + + + + put() + + + + + + + + + + + + + + + + + Child + + + + + + + + + + Folder 2 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Broker + + + + + + + + + start_receive(strm) +stop_receive(strm) + + + + + + + + + + + + Broker Thread + + + + + + + + + + + + + + + + + + Main Thread + + + + + + + + + + + + + + + + + + + Router + + + + + + + + handlers[] + route(msg) + + + + + + + + + + + + + Stream + + + + + + + + + send() +on_receive() + + + + + + + + + + + + + ExternalContext + + + + + + + + + _dispatch_calls() + + + + + + + + + + + + + Channel + + + + + + + + CALL_FUNCTION + get() + + + + + + + + + + + + + + Network + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/layout.png b/docs/images/layout.png new file mode 100644 index 00000000..2c4d9524 Binary files /dev/null and b/docs/images/layout.png differ diff --git a/docs/images/radiation.png b/docs/images/radiation.png new file mode 100644 index 00000000..3181a369 Binary files /dev/null and b/docs/images/radiation.png differ