Merge branch 'dmw'
David Wilson
6 years ago
commit
232aaf5c95
@ -1,226 +0,0 @@
|
||||
|
||||
Mitogen Compared To
|
||||
-------------------
|
||||
|
||||
This provides a little free-text summary of conceptual differences between
|
||||
Mitogen and other tools, along with some basic perceptual metrics (project
|
||||
maturity/age, quality of tests, function matrix)
|
||||
|
||||
|
||||
Ansible
|
||||
#######
|
||||
|
||||
Ansible_ is a complete provisioning system, Mitogen is a small component of such a system.
|
||||
|
||||
You should use Ansible if ...
|
||||
|
||||
You should not use Ansible if ...
|
||||
|
||||
|
||||
.. _Ansible: https://docs.ansible.com/ansible/latest/index.html
|
||||
.. _ansible.src: https://github.com/ansible/ansible/
|
||||
|
||||
Baker
|
||||
#####
|
||||
|
||||
Baker_ lets you easily add a command line interface to your Python
|
||||
functions using a simple decorator, to create scripts with "sub-commands",
|
||||
similar to Django's ``manage.py``, ``svn``, ``hg``, etc.
|
||||
|
||||
- Unmaintained since 2015
|
||||
- No obvious remote execution functionality
|
||||
|
||||
.. _Baker: https://bitbucket.org/mchaput/baker
|
||||
|
||||
Chopsticks
|
||||
##########
|
||||
|
||||
Chopsticks_ also supports recursion! but the recursively executed instance has no special knowledge of its identity in a tree structure, and little support for functions running in the master to directly invoke functions in a recursive context.. effectively each recursion produces a new master, from which function calls must be made.
|
||||
|
||||
executing functions from __main__ entails picking just that function and deps
|
||||
out of the main module, not transferring the module intact. that approach works
|
||||
but it's much messier than just arranging for __main__ to be imported and
|
||||
executed through the import mechanism.
|
||||
|
||||
supports sudo but no support for require_tty or typing a sudo password. also supports SSH and Docker.
|
||||
|
||||
good set of tests
|
||||
|
||||
real PEP-302 module loader, but doesn't try to cope with master also relying on
|
||||
a PEP-302 module loader (e.g. py2exe).
|
||||
|
||||
Based on the tox configuration Python 2.7, and 3.3 to 3.6 are supported.
|
||||
|
||||
I/O multiplexer in the master, but not in children.
|
||||
|
||||
As with Execnet it includes its own serialization - pencode_ supports
|
||||
|
||||
- most Python primitive types (``bytes``/``str``/``unicode``, ``list``, ``tuple`` ...)
|
||||
- identity references
|
||||
- self referencing (recursive) data srtuctures
|
||||
|
||||
pencode lacks support for arbitrary classes. Byte strings require special
|
||||
treatment if they contain non-ascii characters. Some primitive types
|
||||
(e.g. ``complex``) are not handled. This would be straightforwar to address.
|
||||
Values are length-prefixed with a 32 bit unsigned integer, meaning values
|
||||
are limited to 4 billion bytes or items in length.
|
||||
|
||||
design is reminiscent of Mitogen in places (Tunnel is practically identical to
|
||||
Mitogen's Stream), and closer to Execnet elsewhere (lack of uniformity,
|
||||
tendency to prefer logic expressed in if/else special case soup rather than the
|
||||
type system, though some of that is due to supporting Python 3, so not judging
|
||||
too harshly!)
|
||||
|
||||
Chopsticks has its own `Chopsticks vs`_ comparisons.
|
||||
|
||||
You should use Chopsticks if you need Python 3 support.
|
||||
|
||||
.. _Chopsticks: https://chopsticks.readthedocs.io/en/stable/
|
||||
.. _Chopsticks.src: https://github.com/lordmauve/chopsticks/
|
||||
.. _Chopsticks vs: https://chopsticks.readthedocs.io/en/stable/intro.html#chopsticks-vs
|
||||
.. _pencode: https://github.com/lordmauve/chopsticks/blob/master/doc/pencode.rst
|
||||
.. _pencode.src: https://github.com/lordmauve/chopsticks/blob/master/chopsticks/pencode.py
|
||||
|
||||
Disco
|
||||
#####
|
||||
|
||||
Disco_ is a lightweight, open-source framework for distributed computing
|
||||
based on the MapReduce paradigm.
|
||||
|
||||
- An Erlang core, with Python bindings
|
||||
- Wire format is pickle, according to `Execnet vs NLTK for distributed NLTK`_
|
||||
|
||||
.. _Disco: http://discoproject.org/
|
||||
.. _Execnet vs NLTK for distributed NLTK: https://streamhacker.com/2009/12/14/execnet-disco-distributed-nltk/
|
||||
|
||||
Execnet
|
||||
#######
|
||||
|
||||
Execnet_
|
||||
|
||||
- Parent and children may use threads, gevent, or eventlet, Mitogen only supports threads.
|
||||
- No recursion
|
||||
- Similar Channel abstraction but better developed.. includes waiting for remote to close its end
|
||||
- Heavier emphasis on passing chunks of Python source code around, modules are loaded one-at-a-time with no dependency resolution mechanism
|
||||
- Built-in unidirectional rsync-alike, compared to Mitogen's SSH emulation which allows use of real rsync in any supported mode
|
||||
- no support for sudo, but supports connecting to vagrant
|
||||
- works with read-only filesystem
|
||||
- includes its own serialization_ independent of the standard library
|
||||
|
||||
The obj and all contained objects must be of a builtin python type
|
||||
(so nested dicts, sets, etc. are all ok but not user-level instances).
|
||||
|
||||
- Known uses include `pytest-xdist`_, and `Distributed NLTK`_
|
||||
|
||||
You should use Execnet if you value code maturity more than featureset.
|
||||
|
||||
.. _Execnet: https://codespeak.net/execnet/
|
||||
.. _serialization: https://codespeak.net/execnet/basics.html#dumps-loads
|
||||
.. _pytest-xdist: https://pypi.python.org/pypi/pytest-xdist
|
||||
.. _Distributed NLTK: https://streamhacker.com/2009/12/14/execnet-disco-distributed-nltk/
|
||||
|
||||
Fabric
|
||||
######
|
||||
|
||||
Fabric_ allows execution of shell snippets on remote machines, Python functions run
|
||||
locally, any remote interaction is fundamentally done via shell, with all the
|
||||
limitations that entails. prefers to depend on SSH features (e.g. tunnelling)
|
||||
than reinvent them
|
||||
|
||||
You should use Fabric if you enjoy being woken at 4am to pages about broken
|
||||
shell snippets.
|
||||
|
||||
.. _fabric: http://www.fabfile.org/
|
||||
|
||||
Invoke
|
||||
######
|
||||
|
||||
Invoke_
|
||||
|
||||
Python 2.6+, 3.3+
|
||||
|
||||
Basically a Fabric-alike
|
||||
|
||||
.. _invoke: http://www.pyinvoke.org/
|
||||
|
||||
Multiprocessing
|
||||
###############
|
||||
|
||||
multiprocessing_ was added to the stdlib in Python 2.6.
|
||||
|
||||
multiprocessing is a package that supports spawning processes using an
|
||||
API similar to the threading module. The multiprocessing package offers
|
||||
both local and remote concurrency
|
||||
|
||||
There is a backport_ for Python 2.4 & 2.5, but it is not pure Python.
|
||||
pymultiprocessing_ appears to be a pure Python implementation.
|
||||
An ecosystem_ of packages has built up around multiprocessing.
|
||||
|
||||
The `programming guidelines`_ section notes
|
||||
|
||||
- Arguments to proxies must be picklable. On Windows this also applies to
|
||||
``multiprocessing.Process.__init__()`` arguments.
|
||||
- Callers should beware replacing ``sys.stdin``, because
|
||||
``multiprocessing.Process._bootstrap()``
|
||||
will close it and open /dev/null instead
|
||||
|
||||
.. _programming guidelines: https://docs.python.org/2/library/multiprocessing.html#programming-guidelines
|
||||
.. _backport: https://pypi.python.org/pypi/multiprocessing
|
||||
.. _pymultiprocessing: https://pypi.python.org/pypi/pymultiprocessing
|
||||
.. _ecosystem: https://pypi.python.org/pypi?%3Aaction=search&term=multiprocessing&submit=search
|
||||
|
||||
Paver
|
||||
#####
|
||||
|
||||
Paver_
|
||||
|
||||
More or less another task execution framework / make-alike, doesn't really deal
|
||||
with remote execution at all.
|
||||
|
||||
.. _Paver: https://github.com/paver/paver/
|
||||
|
||||
Plumbum
|
||||
#######
|
||||
|
||||
Plumbum_
|
||||
|
||||
Shell-only
|
||||
|
||||
Basically syntax sugar for running shell commands. Nicer than raw shell
|
||||
(depending on your opinions of operating overloading), but it's still shell.
|
||||
|
||||
.. _Plumbum: https://pypi.python.org/pypi/plumbum
|
||||
|
||||
Pyro4
|
||||
#####
|
||||
|
||||
Pyro4_
|
||||
...
|
||||
|
||||
.. _Pyro4: https://pythonhosted.org/Pyro4/
|
||||
|
||||
RPyC
|
||||
####
|
||||
|
||||
RPyC_
|
||||
|
||||
- supports transparent object proxies similar to Pyro (with all the pain and suffering hidden network IO entails)
|
||||
- significantly more 'frameworkey' feel
|
||||
- runs multiplexer in a thread too?
|
||||
- bootstrap over SSH only, no recursion and no sudo
|
||||
- requires a writable filesystem
|
||||
|
||||
.. _RPyC: https://rpyc.readthedocs.io/en/latest/
|
||||
|
||||
Salt
|
||||
####
|
||||
|
||||
Salt_
|
||||
|
||||
- no crappy deps
|
||||
|
||||
You should use Salt if you enjoy firefighting endless implementation bugs,
|
||||
otherwise you should prefer Ansible.
|
||||
|
||||
.. _Salt: https://docs.saltstack.com/en/latest/topics/
|
||||
.. _Salt.src: https://github.com/saltstack/salt
|
||||
@ -1,6 +1,6 @@
|
||||
|
||||
- import_playbook: from_config_path.yml
|
||||
- import_playbook: from_config_path_pkg.yml
|
||||
- import_playbook: adjacent_to_playbook.yml
|
||||
#- import_playbook: from_config_path.yml
|
||||
#- import_playbook: from_config_path_pkg.yml
|
||||
#- import_playbook: adjacent_to_playbook.yml
|
||||
- import_playbook: adjacent_to_role.yml
|
||||
- import_playbook: overrides_builtin.yml
|
||||
#- import_playbook: overrides_builtin.yml
|
||||
|
||||
Loading…
Reference in New Issue