Mitogen Compared To
-------------------
This provides a little free-text summary of conceptual differences between
Mitogen and other tools, along with some basic perceptual metrics (project
maturity/age, quality of tests, function matrix)
Ansible
#######
Ansible_ is a complete provisioning system, Mitogen is a small component of such a system.
You should use Ansible if ...
You should not use Ansible if ...
.. _Ansible: https://docs.ansible.com/ansible/latest/index.html
.. _ansible.src: https://github.com/ansible/ansible/
Baker
#####
Baker_ lets you easily add a command line interface to your Python
functions using a simple decorator, to create scripts with "sub-commands",
similar to Django's `` manage.py `` , `` svn `` , `` hg `` , etc.
- Unmaintained since 2015
- No obvious remote execution functionality
.. _Baker: https://bitbucket.org/mchaput/baker
Chopsticks
##########
Chopsticks_ also supports recursion! but the recursively executed instance has no special knowledge of its identity in a tree structure, and little support for functions running in the master to directly invoke functions in a recursive context.. effectively each recursion produces a new master, from which function calls must be made.
executing functions from __main__ entails picking just that function and deps
out of the main module, not transferring the module intact. that approach works
but it's much messier than just arranging for __main__ to be imported and
executed through the import mechanism.
supports sudo but no support for require_tty or typing a sudo password. also supports SSH and Docker.
good set of tests
real PEP-302 module loader, but doesn't try to cope with master also relying on
a PEP-302 module loader (e.g. py2exe).
Based on the tox configuration Python 2.7, and 3.3 to 3.6 are supported.
I/O multiplexer in the master, but not in children.
As with Execnet it includes its own serialization - pencode_ supports
- most Python primitive types (`` bytes `` /`` str `` /`` unicode `` , `` list `` , `` tuple `` ...)
- identity references
- self referencing (recursive) data srtuctures
pencode lacks support for arbitrary classes. Byte strings require special
treatment if they contain non-ascii characters. Some primitive types
(e.g. `` complex `` ), but adding support would be trivial.
design is reminiscent of Mitogen in places (Tunnel is practically identical to
Mitogen's Stream), and closer to Execnet elsewhere (lack of uniformity,
tendency to prefer logic expressed in if/else special case soup rather than the
type system, though some of that is due to supporting Python 3, so not judging
too harshly!)
Chopsticks has its own `Chopsticks vs`_ comparisons.
You should use Chopsticks if you need Python 3 support.
.. _Chopsticks: https://chopsticks.readthedocs.io/en/stable/
.. _Chopsticks.src: https://github.com/lordmauve/chopsticks/
.. _Chopsticks vs: https://chopsticks.readthedocs.io/en/stable/intro.html#chopsticks-vs
.. _pencode: https://github.com/lordmauve/chopsticks/blob/master/doc/pencode.rst
.. _pencode.src: https://github.com/lordmauve/chopsticks/blob/master/chopsticks/pencode.py
Disco
#####
Disco_ is a lightweight, open-source framework for distributed computing
based on the MapReduce paradigm.
- An Erlang core, with Python bindings
- Wire format is pickle, according to `Execnet vs NLTK for distributed NLTK`_
.. _Disco: http://discoproject.org/
.. _Execnet vs NLTK for distributed NLTK: https://streamhacker.com/2009/12/14/execnet-disco-distributed-nltk/
Execnet
#######
Execnet_
- Parent and children may use threads, gevent, or eventlet, Mitogen only supports threads.
- No recursion
- Similar Channel abstraction but better developed.. includes waiting for remote to close its end
- Heavier emphasis on passing chunks of Python source code around, modules are loaded one-at-a-time with no dependency resolution mechanism
- Built-in unidirectional rsync-alike, compared to Mitogen's SSH emulation which allows use of real rsync in any supported mode
- no support for sudo, but supports connecting to vagrant
- works with read-only filesystem
- includes its own serialization_ independent of the standard library
The obj and all contained objects must be of a builtin python type
(so nested dicts, sets, etc. are all ok but not user-level instances).
- Known uses include `pytest-xdist`_ , and `Distributed NLTK`_
You should use Execnet if you value code maturity more than featureset.
.. _Execnet: https://codespeak.net/execnet/
.. _serialization: https://codespeak.net/execnet/basics.html#dumps-loads
.. _pytest-xdist: https://pypi.python.org/pypi/pytest-xdist
.. _Distributed NLTK: https://streamhacker.com/2009/12/14/execnet-disco-distributed-nltk/
Fabric
######
Fabric_ allows execution of shell snippets on remote machines, Python functions run
locally, any remote interaction is fundamentally done via shell, with all the
limitations that entails. prefers to depend on SSH features (e.g. tunnelling)
than reinvent them
You should use Fabric if you enjoy being woken at 4am to pages about broken
shell snippets.
.. _fabric: http://www.fabfile.org/
Invoke
######
Invoke_
Python 2.6+, 3.3+
Basically a Fabric-alike
.. _invoke: http://www.pyinvoke.org/
Multiprocessing
###############
multiprocessing_ was added to the stdlib in Python 2.6.
multiprocessing is a package that supports spawning processes using an
API similar to the threading module. The multiprocessing package offers
both local and remote concurrency
There is a backport_ for Python 2.4 & 2.5, but it is not pure Python.
pymultiprocessing_ appears to be a pure Python implementation.
An ecosystem_ of packages has built up around multiprocessing.
The `programming guidelines`_ section notes
- Arguments to proxies must be picklable. On Windows this also applies to
`` multiprocessing.Process.__init__() `` arguments.
- Callers should beware replacing `` sys.stdin `` , because
`` multiprocessing.Process._bootstrap() ``
will close it and open /dev/null instead
.. _programming guidelines: https://docs.python.org/2/library/multiprocessing.html#programming-guidelines
.. _backport: https://pypi.python.org/pypi/multiprocessing
.. _pymultiprocessing: https://pypi.python.org/pypi/pymultiprocessing
.. _ecosystem: https://pypi.python.org/pypi?%3Aaction=search&term=multiprocessing&submit=search
Paver
#####
Paver_
More or less another task execution framework / make-alike, doesn't really deal
with remote execution at all.
.. _Paver: https://github.com/paver/paver/
Plumbum
#######
Plumbum_
Shell-only
Basically syntax sugar for running shell commands. Nicer than raw shell
(depending on your opinions of operating overloading), but it's still shell.
.. _Plumbum: https://pypi.python.org/pypi/plumbum
Pyro4
#####
Pyro4_
...
.. _Pyro4: https://pythonhosted.org/Pyro4/
RPyC
####
RPyC_
- supports transparent object proxies similar to Pyro (with all the pain and suffering hidden network IO entails)
- significantly more 'frameworkey' feel
- runs multiplexer in a thread too?
- bootstrap over SSH only, no recursion and no sudo
- requires a writable filesystem
.. _RPyC: https://rpyc.readthedocs.io/en/latest/
Salt
####
Salt_
- no crappy deps
You should use Salt if you enjoy firefighting endless implementation bugs,
otherwise you should prefer Ansible.
.. _Salt: https://docs.saltstack.com/en/latest/topics/
.. _Salt.src: https://github.com/saltstack/salt
