diff --git a/docs/api.rst b/docs/api.rst index 16baf822..5d98b8ce 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -55,10 +55,10 @@ Helper Functions .. autofunction:: econtext.master.minimize_source -Context Class -------------- +Broker Class +------------ -.. autoclass:: econtext.master.Context +.. autoclass:: econtext.master.Broker :members: @@ -68,7 +68,7 @@ Stream Classes .. autoclass:: econtext.master.LocalStream :members: -.. autoclass:: econtext.master.SSHStream +.. autoclass:: econtext.master.SshStream :members: diff --git a/docs/history.rst b/docs/history.rst index cd467750..f19c014d 100644 --- a/docs/history.rst +++ b/docs/history.rst @@ -3,6 +3,43 @@ History And Future ================== +History +####### + +The first version of econtext was written in late 2006 for use in an +infrastructure management program, however at the time I lacked the pragmatism +necessary for pushing my little design from concept to a working +implementation. I tired of it when I couldn't find a way to combine every +communication style (*blocking execute function, asynchronous execute function, +proxy slave-of-slave context*) into one neat abstraction. That unification +still has not happened, but I'm no longer as burdened by such simple problems. + +Every few years I would pick through the source code, especially after periods +of working commercially with some contemporary management systems, none of +which in the meantime had anything close to as neat an approach to running +Python code on remote machines, and suffered from shockingly beginner-level +bugs such as failing to even report SSH diagnostic messages. + +And every few years I'd put that code down again, especially since moving to an +OS X laptop where :py:func:`select.poll` was not available, the struggle to get +back on top of the project seemed more hassle than it was worth. + +That changed in 2016 during a quiet evening at home with a clear head and +nothing better to do, after a full day of exposure to Ansible's intensely +unbearable tendency to make running a 50 line Python script across a 1Gbit/sec +LAN feel like I were configuring a host on Mars. Poking through what Ansible +was doing, I was shocked to discover it writing temporary files everywhere, and +uploading a 56KiB zip file apparently for every playbook step. + +Searching around for something to play with, I came across my forgotten +``src/econtext`` directory and somehow in a few hours managed to squash most of +the race conditions and logic bugs that were preventing reliable operation, +write the IO and log forwarders, rewrite the module importer, move from +:py:func:`select.poll` to :py:func:`select.select`, and even refactor the +special cases out of the main loop. + +So there you have it. As of writing :py:mod:`econtext.core` consists of 550 +source lines, and those 550 lines have taken me almost a decade to write. Future @@ -12,3 +49,6 @@ Future * Python 3 support. * Predictive import: reduce roundtrips by pipelining modules observed to probably be requested in future. +* Provide a means for waiting on multiple + :py:class:`Channels `. +* Comprehensive integration tests. diff --git a/docs/index.rst b/docs/index.rst index 4b833b6a..738a3b58 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,10 +2,10 @@ Python Execution Contexts ========================= -**4.98KiB of sugar and no fat!** +**5KiB of sugar and no fat!** .. toctree:: - :maxdepth: 2 + :maxdepth: 1 self howitworks @@ -52,7 +52,7 @@ connection. $ python preamble_size.py SSH command size: 411 - Preamble size: 5098 (4.98KiB) + Preamble size: 5085 (4.96KiB) econtext.master size: 2403 (2.35KiB) Once bootstrapped, the remote process is configured with a customizable @@ -171,6 +171,49 @@ and timeouts can be configured to ensure failed calls do not block progress of the parent. +Support For Single File Programs +################################ + +Programs that are self-contained within a single Python script are supported. +External contexts are configured such that any attempt to execute a function +from the main Python script will correctly cause that script to be imported as +usual into the slave process. + +.. code-block:: python + + #!/usr/bin/env python + """ + Install our application on a remote machine. + + Usage: + install_app.py + + Where: + Hostname to install to. + """ + import os + import sys + + import econtext + + + def install_app(): + os.system('tar zxvf my_app.tar.gz') + + + def main(broker): + if len(sys.argv) != 2: + print __doc__ + sys.exit(1) + + context = broker.get_remote(sys.argv[1]) + context.call(install_app) + + if __name__ == '__main__' and not econtext.slave: + import econtext.utils + econtext.utils.run_with_broker(main) + + Event-driven IO ############### diff --git a/econtext/master.py b/econtext/master.py index 82616241..5f22a31a 100644 --- a/econtext/master.py +++ b/econtext/master.py @@ -219,7 +219,7 @@ class LocalStream(econtext.core.Stream): raise econtext.core.StreamError('Bootstrap failed; stdout: %r', s) -class SSHStream(LocalStream): +class SshStream(LocalStream): #: The path to the SSH binary. ssh_path = 'ssh' @@ -228,7 +228,7 @@ class SSHStream(LocalStream): if self._context.username: bits += ['-l', self._context.username] bits.append(self._context.hostname) - base = super(SSHStream, self).get_boot_command() + base = super(SshStream, self).get_boot_command() return bits + map(commands.mkarg, base) @@ -237,7 +237,7 @@ class Broker(econtext.core.Broker): graceful_count = 1 def create_listener(self, address=None, backlog=30): - """Listen on `address `for connections from newly spawned contexts.""" + """Listen on `address` for connections from newly spawned contexts.""" self._listener = Listener(self, address, backlog) def get_local(self, name='default', python_path=None): @@ -256,7 +256,7 @@ class Broker(econtext.core.Broker): name = hostname context = Context(self, name, hostname, username) - context.stream = SSHStream(context) + context.stream = SshStream(context) if python_path: context.stream.python_path = python_path context.stream.connect()