mitogen/docs/history.rst


History And Future
==================


History
#######

The first version of econtext was written in late 2006 for use in an
infrastructure management program, however at the time I lacked the pragmatism
necessary for pushing my little design from concept to finished implementation.
I tired of it when no way could be found to unify every communication style
(*blocking execute function, asynchronous execute function, proxy
slave-of-slave context*) into one neat abstraction. That unification never
happened, but I'm no longer worried by it.

Every few years I would pick through the source code, especially after periods
of commercial work involving some contemporary infrastructure management
systems, none of which had nearly as neat an approach to running Python code
remotely, and suffered from shockingly beginner-level bugs such as failing to
report SSH diagnostic messages.

And every few years I'd put that code down again, especially since moving to an
OS X laptop where :py:func:`select.poll` was not available, the struggle to get
back on top seemed more hassle than it was worth.

That changed in 2016 during a quiet evening at home with a clear head and
nothing better to do, after a full day of exposure to Ansible's intensely
unbearable tendency to make running a 50 line Python script across a 1Gbit/sec
LAN feel like I were configuring a host on Mars. Poking through Ansible, I was
shocked to discover it writing temporary files everywhere, and uploading a
56KiB zip file apparently for every playbook step.

.. figure:: _static/wtf.gif

    All contemporary Devops tooling

Searching around for something to play with, I came across my forgotten
``src/econtext`` directory and somehow in a few hours managed to squash most of
the race conditions and logic bugs that were preventing reliable operation,
write the IO and log forwarders, rewrite the module importer, move from
:py:func:`select.poll` to :py:func:`select.select`, and even refactor the
special cases out of the main loop.

So there you have it. As of writing :py:mod:`econtext.core` consists of 681
source lines, and those 681 lines have taken over a decade to write. I have
long had a preference for avoiding infrastructure work commercially, not least
for the inescapable depression induced by considering the wasted effort across
the world caused by universally horrific tooling. This is my small contribution
to a solution, I hope you find it useful.


Future
######

`View the issue list on GitHub`_

.. _View the issue list on GitHub: https://github.com/dw/econtext/issues?q=is%3Aopen%20is%3Aissue%20-label%3Abug
Add history.rst. 8 years ago
			`History And Future`
			`==================`


Write history section. 8 years ago			`History`
			`#######`

			`The first version of econtext was written in late 2006 for use in an`
			`infrastructure management program, however at the time I lacked the pragmatism`
Many updates: graceful shutdown, docs, threading * Start splitting docs up into internals.rst / api.rst * Docs for lots more of econtext.core. * Get rid of _update_stream() and has_output(), replace with individual functions called as state changes. * Add Broker.on_thread() and remove Stream._lock: simply call on_thread() to ensure buffer management is linearized. * Rename read_side/write_side to receive_side/transmit_side like event handler names. * Clean up some more repr / debug logs. * Move handle cleanup to Context.on_shutdown where it belongs. * Make wake() a noop when called from broker thread. * Replace graceful_count crap with Side.graceful attribute, add Broker.keep_alive() to check whether any registered readers want to be kept alive for graceful shutdown() or any child contexts with a connected stream exist. * Make master.Broker timeout slightly longer than slave broker. * Add generic on_thread() to allow running code on the IO thread. 8 years ago			`necessary for pushing my little design from concept to finished implementation.`
			`I tired of it when no way could be found to unify every communication style`
			`(*blocking execute function, asynchronous execute function, proxy`
			`slave-of-slave context*) into one neat abstraction. That unification never`
			`happened, but I'm no longer worried by it.`
Write history section. 8 years ago
			`Every few years I would pick through the source code, especially after periods`
Many updates: graceful shutdown, docs, threading * Start splitting docs up into internals.rst / api.rst * Docs for lots more of econtext.core. * Get rid of _update_stream() and has_output(), replace with individual functions called as state changes. * Add Broker.on_thread() and remove Stream._lock: simply call on_thread() to ensure buffer management is linearized. * Rename read_side/write_side to receive_side/transmit_side like event handler names. * Clean up some more repr / debug logs. * Move handle cleanup to Context.on_shutdown where it belongs. * Make wake() a noop when called from broker thread. * Replace graceful_count crap with Side.graceful attribute, add Broker.keep_alive() to check whether any registered readers want to be kept alive for graceful shutdown() or any child contexts with a connected stream exist. * Make master.Broker timeout slightly longer than slave broker. * Add generic on_thread() to allow running code on the IO thread. 8 years ago			`of commercial work involving some contemporary infrastructure management`
			`systems, none of which had nearly as neat an approach to running Python code`
			`remotely, and suffered from shockingly beginner-level bugs such as failing to`
			`report SSH diagnostic messages.`
Write history section. 8 years ago
			`And every few years I'd put that code down again, especially since moving to an`
			OS X laptop where :py:func:`select.poll` was not available, the struggle to get
More doc updates. 8 years ago			`back on top seemed more hassle than it was worth.`
Write history section. 8 years ago
			`That changed in 2016 during a quiet evening at home with a clear head and`
			`nothing better to do, after a full day of exposure to Ansible's intensely`
			`unbearable tendency to make running a 50 line Python script across a 1Gbit/sec`
More doc updates. 8 years ago			`LAN feel like I were configuring a host on Mars. Poking through Ansible, I was`
			`shocked to discover it writing temporary files everywhere, and uploading a`
			`56KiB zip file apparently for every playbook step.`
Write history section. 8 years ago
More docs. 8 years ago			`.. figure:: _static/wtf.gif`

			`All contemporary Devops tooling`
Add wtf.gif. 8 years ago
Write history section. 8 years ago			`Searching around for something to play with, I came across my forgotten`
			``src/econtext`` directory and somehow in a few hours managed to squash most of
			`the race conditions and logic bugs that were preventing reliable operation,`
			`write the IO and log forwarders, rewrite the module importer, move from`
			:py:func:`select.poll` to :py:func:`select.select`, and even refactor the
			`special cases out of the main loop.`

Update numbers. 7 years ago			So there you have it. As of writing :py:mod:`econtext.core` consists of 681
			`source lines, and those 681 lines have taken over a decade to write. I have`
			`long had a preference for avoiding infrastructure work commercially, not least`
			`for the inescapable depression induced by considering the wasted effort across`
			`the world caused by universally horrific tooling. This is my small contribution`
			`to a solution, I hope you find it useful.`
Add history.rst. 8 years ago

			`Future`
			`######`

Move issue lists into GitHub. 7 years ago			`View the issue list on GitHub`_

Fix feature list link 7 years ago			`.. _View the issue list on GitHub: https://github.com/dw/econtext/issues?q=is%3Aopen%20is%3Aissue%20-label%3Abug`