archive
/
mitogen
mirror of https://github.com/mitogen-hq/mitogen.git
1
0
Fork 0
Code Releases Activity

docs: add domainrefs plugin to make link aliases everywhere \o/

Browse Source 
PATENT PENDING
new-serialization
David Wilson 5 years ago
parent 6b180a4091
commit d75c9cffc3
6 changed files with 439 additions and 413 deletions
Show Stats Download Patch File Download Diff File

122
docs/ansible_detailed.rst
Unescape Escape View File

@ -175,16 +175,10 @@ Noteworthy Differences
your_ssh_username = (ALL) NOPASSWD:/usr/bin/python -c*
* The `buildah <https://docs.ansible.com/ansible/latest/plugins/connection/buildah.html>`_,
`docker <https://docs.ansible.com/ansible/2.6/plugins/connection/docker.html>`_,
`jail <https://docs.ansible.com/ansible/2.6/plugins/connection/jail.html>`_,
`kubectl <https://docs.ansible.com/ansible/2.6/plugins/connection/kubectl.html>`_,
`local <https://docs.ansible.com/ansible/2.6/plugins/connection/local.html>`_,
`lxc <https://docs.ansible.com/ansible/2.6/plugins/connection/lxc.html>`_,
`lxd <https://docs.ansible.com/ansible/2.6/plugins/connection/lxd.html>`_,
and `ssh <https://docs.ansible.com/ansible/2.6/plugins/connection/ssh.html>`_
built-in connection types are supported, along with Mitogen-specific
:ref:`machinectl <machinectl>`, :ref:`mitogen_doas <doas>`,
* The :ans:conn:`~buildah`, :ans:conn:`~docker`, :ans:conn:`~jail`,
:ans:conn:`~kubectl`, :ans:conn:`~local`, :ans:conn:`~lxd`, and
:ans:conn:`~ssh` built-in connection types are supported, along with
Mitogen-specific :ref:`machinectl <machinectl>`, :ref:`mitogen_doas <doas>`,
:ref:`mitogen_su <su>`, :ref:`mitogen_sudo <sudo>`, and :ref:`setns <setns>`
types. File bugs to register interest in others.
@ -199,16 +193,14 @@ Noteworthy Differences
artificial serialization, causing slowdown equivalent to `task_duration *
num_targets`. This will be addressed soon.
* The Ansible 2.7 `reboot
<https://docs.ansible.com/ansible/latest/modules/reboot_module.html>`_ module
may require a ``pre_reboot_delay`` on systemd hosts, as insufficient time
exists for the reboot command's exit status to be reported before necessary
processes are torn down.
* The Ansible 2.7 :ans:mod:`reboot` may require a ``pre_reboot_delay`` on
systemd hosts, as insufficient time exists for the reboot command's exit
status to be reported before necessary processes are torn down.
* On OS X when a SSH password is specified and the default connection type of
``smart`` is used, Ansible may select the Paramiko plug-in rather than
Mitogen. If you specify a password on OS X, ensure ``connection: ssh``
appears in your playbook, ``ansible.cfg``, or as ``-c ssh`` on the
:ans:conn:`~smart` is used, Ansible may select the :ans:conn:`paramiko_ssh`
rather than Mitogen. If you specify a password on OS X, ensure ``connection:
ssh`` appears in your playbook, ``ansible.cfg``, or as ``-c ssh`` on the
command-line.
* Ansible permits up to ``forks`` connections to be setup in parallel, whereas
@ -345,19 +337,12 @@ command line, or as host and group variables.
File Transfer
~~~~~~~~~~~~~
Normally `sftp(1)`_ or `scp(1)`_ are used to copy files by the
`assemble <http://docs.ansible.com/ansible/latest/modules/assemble_module.html>`_,
`copy <http://docs.ansible.com/ansible/latest/modules/copy_module.html>`_,
`patch <http://docs.ansible.com/ansible/latest/modules/patch_module.html>`_,
`script <http://docs.ansible.com/ansible/latest/modules/script_module.html>`_,
`template <http://docs.ansible.com/ansible/latest/modules/template_module.html>`_, and
`unarchive <http://docs.ansible.com/ansible/latest/modules/unarchive_module.html>`_
actions, or when uploading modules with pipelining disabled. With Mitogen
copies are implemented natively using the same interpreters, connection tree,
and routed message bus that carries RPCs.
.. _scp(1): https://linux.die.net/man/1/scp
.. _sftp(1): https://linux.die.net/man/1/sftp
Normally :linux:man1:`sftp` or :linux:man1:`scp` are used to copy files by the
:ans:mod:`~assemble`, :ans:mod:`~aws_s3`, :ans:mod:`~copy`, :ans:mod:`~patch`,
:ans:mod:`~script`, :ans:mod:`~template`, :ans:mod:`~unarchive`, and
:ans:mod:`~uri` actions, or when uploading modules with pipelining disabled.
With Mitogen copies are implemented natively using the same interpreters,
connection tree, and routed message bus that carries RPCs.
This permits direct streaming between endpoints regardless of execution
environment, without necessitating temporary copies in intermediary accounts or
@ -373,15 +358,15 @@ Safety
^^^^^^
Transfers proceed to a hidden file in the destination directory, with content
and metadata synced using `fsync(2) <https://linux.die.net/man/2/fsync>`_ prior
to rename over any existing file. This ensures the file remains consistent at
all times, in the event of a crash, or when overlapping `ansible-playbook` runs
deploy differing file contents.
and metadata synced using :linux:man2:`fsync` prior to rename over any existing
file. This ensures the file remains consistent at all times, in the event of a
crash, or when overlapping `ansible-playbook` runs deploy differing file
contents.
The `sftp(1)`_ and `scp(1)`_ tools may cause undetected data corruption
in the form of truncated files, or files containing intermingled data segments
from overlapping runs. As part of normal operation, both tools expose a window
where readers may observe inconsistent file contents.
The :linux:man1:`sftp` and :linux:man1:`scp` tools may cause undetected data
corruption in the form of truncated files, or files containing intermingled
data segments from overlapping runs. As part of normal operation, both tools
expose a window where readers may observe inconsistent file contents.
Performance
@ -499,11 +484,11 @@ Ansible may:
* Create a directory owned by the SSH user either under ``remote_tmp``, or a
system-default directory,
* Upload action dependencies such as non-new style modules or rendered
templates to that directory via `sftp(1)`_ or `scp(1)`_.
templates to that directory via :linux:man1:`sftp` or :linux:man1:`scp`.
* Attempt to modify the directory's access control list to grant access to the
target user using `setfacl(1) <https://linux.die.net/man/1/setfacl>`_,
requiring that tool to be installed and a supported filesystem to be in use,
or for the ``allow_world_readable_tmpfiles`` setting to be :data:`True`.
target user using :linux:man1:`setfacl`, requiring that tool to be installed
and a supported filesystem to be in use, or for the
``allow_world_readable_tmpfiles`` setting to be :data:`True`.
* Create a directory owned by the target user either under ``remote_tmp``, or
a system-default directory, if a new-style module needs a temporary directory
and one was not previously created for a supporting file earlier in the
@ -569,9 +554,9 @@ in regular Ansible:
operations relating to modifying the directory to support cross-account
access are avoided.
* An explicit work-around is included to avoid the `copy` and `template`
actions needlessly triggering a round-trip to set their temporary file as
executable.
* An explicit work-around is included to avoid the :ans:mod:`~copy` and
:ans:mod:`~template` actions needlessly triggering a round-trip to set their
temporary file as executable.
* During task shutdown, it is not necessary to wait to learn if the target has
succeeded in deleting a temporary directory, since any error that may occur
@ -601,10 +586,10 @@ DNS Resolution
^^^^^^^^^^^^^^
Modifications to ``/etc/resolv.conf`` cause the glibc resolver configuration to
be reloaded via `res_init(3) <https://linux.die.net/man/3/res_init>`_. This
isn't necessary on some Linux distributions carrying glibc patches to
automatically check ``/etc/resolv.conf`` periodically, however it is necessary
on at least Debian and BSD derivatives.
be reloaded via :linux:man3:`res_init`. This isn't necessary on some Linux
distributions carrying glibc patches to automatically check
``/etc/resolv.conf`` periodically, however it is necessary on at least Debian
and BSD derivatives.
``/etc/environment``
@ -728,9 +713,7 @@ configuration of each task.
Buildah
~~~~~~~
Like `buildah
<https://docs.ansible.com/ansible/2.6/plugins/connection/buildah.html>`_ except
connection delegation is supported.
Like the :ans:conn:`buildah` except connection delegation is supported.
* ``ansible_host``: Name of Buildah container (default: inventory hostname).
* ``ansible_user``: Name of user within the container to execute as.
@ -771,9 +754,7 @@ When used as the ``mitogen_doas`` connection method:
Docker
~~~~~~
Like `docker
<https://docs.ansible.com/ansible/2.6/plugins/connection/docker.html>`_ except
connection delegation is supported.
Like the :ans:conn:`docker` except connection delegation is supported.
* ``ansible_host``: Name of Docker container (default: inventory hostname).
* ``ansible_user``: Name of user within the container to execute as.
@ -789,9 +770,7 @@ connection delegation is supported.
FreeBSD Jail
~~~~~~~~~~~~
Like `jail
<https://docs.ansible.com/ansible/2.6/plugins/connection/jail.html>`_ except
connection delegation is supported.
Like the :ans:conn:`jail` except connection delegation is supported.
* ``ansible_host``: Name of jail (default: inventory hostname).
* ``ansible_user``: Name of user within the jail to execute as.
@ -807,9 +786,7 @@ connection delegation is supported.
Kubernetes Pod
~~~~~~~~~~~~~~
Like `kubectl
<https://docs.ansible.com/ansible/2.6/plugins/connection/kubectl.html>`_ except
connection delegation is supported.
Like the :ans:conn:`kubectl` except connection delegation is supported.
* ``ansible_host``: Name of pod (default: inventory hostname).
* ``ansible_user``: Name of user to authenticate to API as.
@ -823,9 +800,7 @@ connection delegation is supported.
Local
~~~~~
Like `local
<https://docs.ansible.com/ansible/2.6/plugins/connection/local.html>`_ except
connection delegation is supported.
Like the :ans:conn:`local` except connection delegation is supported.
* ``ansible_python_interpreter``
@ -852,10 +827,9 @@ additional differences exist that may break existing playbooks.
LXC
~~~
Connect to classic LXC containers, like `lxc
<https://docs.ansible.com/ansible/2.6/plugins/connection/lxc.html>`_ except
connection delegation is supported, and ``lxc-attach`` is always used rather
than the LXC Python bindings, as is usual with ``lxc``.
Connect to classic LXC containers, like the :ans:conn:`lxc` except connection
delegation is supported, and ``lxc-attach`` is always used rather than the LXC
Python bindings, as is usual with ``lxc``.
* ``ansible_python_interpreter``
* ``ansible_host``: Name of LXC container (default: inventory hostname).
@ -873,10 +847,9 @@ than the LXC Python bindings, as is usual with ``lxc``.
LXD
~~~
Connect to modern LXD containers, like `lxd
<https://docs.ansible.com/ansible/2.6/plugins/connection/lxd.html>`_ except
connection delegation is supported. The ``lxc`` command must be available on
the host machine.
Connect to modern LXD containers, like the :ans:conn:`lxd` except connection
delegation is supported. The ``lxc`` command must be available on the host
machine.
* ``ansible_python_interpreter``
* ``ansible_host``: Name of LXC container (default: inventory hostname).
@ -1001,8 +974,7 @@ When used as the ``mitogen_sudo`` connection method:
SSH
~~~
Like `ssh <https://docs.ansible.com/ansible/2.6/plugins/connection/ssh.html>`_
except connection delegation is supported.
Like the :ans:conn:`ssh` except connection delegation is supported.
* ``ansible_ssh_timeout``
* ``ansible_host``, ``ansible_ssh_host``

633
docs/changelog.rst
View File

File diff suppressed because it is too large Load Diff

46
docs/conf.py
Unescape Escape View File

@ -2,13 +2,14 @@ import os
import sys
sys.path.append('..')
sys.path.append('.')
import mitogen
VERSION = '%s.%s.%s' % mitogen.__version__
author = u'Network Genomics'
copyright = u'2019, Network Genomics'
exclude_patterns = ['_build', '.venv']
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinxcontrib.programoutput']
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinxcontrib.programoutput', 'domainrefs']
html_show_copyright = False
html_show_sourcelink = False
html_show_sphinx = False
@ -36,6 +37,49 @@ templates_path = ['_templates']
todo_include_todos = False
version = VERSION
domainrefs = {
'gh:commit': {
'text': '%s',
'url': 'https://github.com/dw/mitogen/commit/%s',
},
'gh:issue': {
'text': '#%s',
'url': 'https://github.com/dw/mitogen/issues/%s',
},
'gh:pull': {
'text': '#%s',
'url': 'https://github.com/dw/mitogen/pull/%s',
},
'ans:mod': {
'text': '%s Module',
'url': 'https://docs.ansible.com/ansible/latest/modules/%s_module.html',
},
'ans:conn': {
'text': '%s Connection Plug-in',
'url': 'https://docs.ansible.com/ansible/latest/plugins/connection/%s.html',
},
'freebsd:man2': {
'text': '%s(2)',
'url': 'https://www.freebsd.org/cgi/man.cgi?query=%s',
},
'linux:man1': {
'text': '%s(1)',
'url': 'http://man7.org/linux/man-pages/man1/%s.1.html',
},
'linux:man2': {
'text': '%s(2)',
'url': 'http://man7.org/linux/man-pages/man2/%s.2.html',
},
'linux:man3': {
'text': '%s(3)',
'url': 'http://man7.org/linux/man-pages/man3/%s.3.html',
},
'linux:man7': {
'text': '%s(7)',
'url': 'http://man7.org/linux/man-pages/man7/%s.7.html',
},
}
rst_epilog = """
.. |mitogen_version| replace:: %(VERSION)s

41
docs/domainrefs.py
Unescape Escape View File

@ -0,0 +1,41 @@
import functools
import re
import docutils.nodes
import docutils.utils
CUSTOM_RE = re.compile('(.*) <(.*)>')
def role(config, role, rawtext, text, lineno, inliner, options={}, content=[]):
template = 'https://docs.ansible.com/ansible/latest/modules/%s_module.html'
match = CUSTOM_RE.match(text)
if match: # "custom text <real link>"
title = match.group(1)
text = match.group(2)
elif text.startswith('~'): # brief
text = text[1:]
title = config.get('brief', '%s') % (
docutils.utils.unescape(text),
)
else:
title = config.get('text', '%s') % (
docutils.utils.unescape(text),
)
node = docutils.nodes.reference(
rawsource=rawtext,
text=title,
refuri=config['url'] % (text,),
**options
)
return [node], []
def setup(app):
for name, info in app.config._raw_config['domainrefs'].items():
app.add_role(name, functools.partial(role, info))

6
docs/index.rst
Unescape Escape View File

@ -27,8 +27,8 @@ and efficient low-level API on which tools like `Salt`_, `Ansible`_, or
`Fabric`_, ultimately it is not intended for direct use by consumer software.
.. _Salt: https://docs.saltstack.com/en/latest/
.. _Ansible: http://docs.ansible.com/
.. _Fabric: http://www.fabfile.org/
.. _Ansible: https://docs.ansible.com/
.. _Fabric: https://www.fabfile.org/
The focus is to centralize and perfect the intricate dance required to run
Python code safely and efficiently on a remote machine, while **avoiding
@ -132,7 +132,7 @@ any tool such as `py2exe`_ that correctly implement the protocols in PEP-302,
allowing truly single file applications to run across multiple machines without
further effort.
.. _py2exe: http://www.py2exe.org/
.. _py2exe: https://www.py2exe.org/
Common sources of import latency and bandwidth consumption are mitigated:

4
mitogen/fork.py
Unescape Escape View File

@ -73,8 +73,8 @@ def reset_logging_framework():
threads in the parent may have been using the logging package at the moment
of fork.
It is not possible to solve this problem in general; see
https://github.com/dw/mitogen/issues/150 for a full discussion.
It is not possible to solve this problem in general; see :gh:issue:`150`
for a full discussion.
"""
logging._lock = threading.RLock()

Write Preview
Loading…
Cancel
Save