Commit Graph

479 Commits (50241a922fec63153723ef7cd5935883898e6b86)

Author SHA1 Message Date
David Wilson 20f5d89dfa issue #156: fix several more races
* Don't need to sleep if queue>sleepers, can just pop the right queue
  element and return it.

* If queue>sleeping and waking==sleeping, no mechanism existed to ensure
  a thread newly added to sleeping would ever be woken. Above change
  fixes that.

* Cannot trust select() return value, scheduler might sleep us
  indefinitely while put() writes a byte.

* Sleeping threads didn't pop FIFO, they popped in whatever order
  scheduler woke them up. Must recover index and use it to pick the pop
  index.
7 years ago
David Wilson 526b0a514b issue #156: prevent Latch.close() triggering spurious wakeups 7 years ago
David Wilson 18e2977baf docs: annoying phrasing 7 years ago
David Wilson 2c22c41819 issue #156: don't decrement `waking` if we timed out rather than being woken. 7 years ago
David Wilson 07a8994ff5 issue #156: waking thread result dictionary with an integer. 7 years ago
David Wilson b5398bd17f issue #156: docs typo 7 years ago
David Wilson 512ff77a46 issue #156: prevent non-sleeping threads from starving sleeping threads.
See new docs
7 years ago
David Wilson 148ce1d703 issue #155: increase context ID width to 32 bits
Needed to make large range allocations (1000 per ALLOCATE_ID roundtrip)
feasible.
7 years ago
David Wilson 071d9fbfb3 docs: tidy ansible docs. 7 years ago
David Wilson f241eac5ce parent: allow Python to determine its install prefix from argv[0]
Fixes support for virtualenv. Closes #152.
7 years ago
David Wilson dec3af375a issue #144: ansible: increase default pool size to 16. 7 years ago
David Wilson 4f361be7e7 issue #144: teach Select() to close its latch
Causes all threads sleeping on the select to wake.
7 years ago
David Wilson 8aada2646c core: support throwing LatchError in every sleeping thread
This is to allow Select() to be used as a generic queueing primitive
that supports graceful shutdown.
7 years ago
David Wilson 7a74bb0a39 docs: update ansible risks/differences. 7 years ago
David Wilson bcc15987fc docs: extra ansible paragraph. 7 years ago
David Wilson 7a394dc73e ansible: allow establishment of duplicate SSH connections 7 years ago
David Wilson 4691ce0b95 issue #150: ansible: add basic Docker support. 7 years ago
David Wilson 67ff762ba5 issue #139: docs: remove note about bad buffering 7 years ago
David Wilson d1888f1908 docs: reorder sections 7 years ago
David Wilson 014247ce66 docs: another crazy Ansible success story 7 years ago
Alex Willmer 7063d172e9 tests: Add Tox config for Python 2.6 and 2.7
I could not get Python 2.5 or earlier to work. Too many packages
(critically docker) don't support it.
7 years ago
David Wilson 88c198ea05 issue #141: copy Ansible's connect_timeout for sudo too. 7 years ago
David Wilson 63c3fc623c docs: note the semantic difference in Mitogen vs. Ansible timeouts
Related to issue #141.
7 years ago
David Wilson 8f85943083 issue #139: mention relating buffering issue 7 years ago
David Wilson 1f1d691a28 docs: update to match @moreati's code golf birdies :) 7 years ago
Alex Willmer 9aa83ef77f docs: First round of Pickle-likes survey 7 years ago
Alex Willmer a1fc21bb06 docs: Maximum size of pencode values 7 years ago
Alex Willmer e24db89f3a docs: Disco comparison 7 years ago
Alex Willmer 04f4851138 docs: multiprocessing comparison
Not strictly a rival, but has enough commonalities to be worth noting
7 years ago
Alex Willmer 8c227b2bdd docs: More detail about Baker 7 years ago
Alex Willmer e06e438228 docs: More detail about execnet 7 years ago
Alex Willmer da58f8595d docs: More detail about chopsticks 7 years ago
Alex Willmer d7fbb9aef6 docs: Link compared projects to their website
All outgoing links checked with

```bash
cd docs
make linkcheck
```
7 years ago
Alex Willmer 4615ab1a8e docs: Enable sphinx-autobuild
```bash
cd docs
make
```

to run a webserver that automatically rerenders whenever the rST is
modified.
7 years ago
David Wilson f8c85b549f docs: typo 7 years ago
David Wilson dc9dda08e1 Add link to IRC; closes #116 7 years ago
David Wilson b2a433dcc4 ssh: CompressionLevel is a v1-only option.
It's ignored by newer SSHes, which only pay attention to Compression.
7 years ago
Alex Willmer 1c20c61605 docs: Convert all URLs that support https://
Excluded: graphml XML namespaces, links to e.g. Fabric homepage

Fixes #128
7 years ago
David Wilson 089f39716e docs: beginnings of testimonials section 7 years ago
David Wilson 500c4f419c docs: update limitations list for Ansible. 7 years ago
David Wilson 537a263261 docs: So many typos 7 years ago
David Wilson 6bd4c5ead5 docs: Ansible logging update (#111) 7 years ago
David Wilson 4f352d7d4b Pin Sphinx version 7 years ago
David Wilson 53e51c4af8 docs: update Python 3 support to match reality. 7 years ago
David Wilson ce9a1fef26 docs: fix float 7 years ago
David Wilson 04bb5881b6 ansible: doc updates 7 years ago
David Wilson 9515291024 docs: small fix 7 years ago
David Wilson 3ddbf1a503 ansible: basic support for ssh_args 7 years ago
David Wilson 1b28252ad0 docs: get rid of "medium risk" category 7 years ago
David Wilson da00437f1e ansible: Support ansible_ssh_private_key_file 7 years ago
David Wilson a87b665099 ansible: limited support for become_flags, more docs. 7 years ago
David Wilson 235e1df987 sudo: support parsing sudo flags back out into parameters 7 years ago
David Wilson cd53023768 docs: Fix install docs now Ansible package layout changed. 7 years ago
David Wilson 03e51fdaa7 docs: mitogen.core.Latch docs 7 years ago
David Wilson 712e5dfca1 docs: fix image labels 7 years ago
David Wilson c2793b7102 docs: import but don't link compared.rst 7 years ago
David Wilson 2ecc6f43a4 docs: Import, but do not yet link signals.rst 7 years ago
David Wilson 7cf2edc3a8 ansible: Support many more common playbook variables. 7 years ago
David Wilson ba644e184d docs: Split up limitations list, add warning 7 years ago
David Wilson 3183dd4147 ansible: initial support for async jobs
Running in a thread to begin with, but this must change.
7 years ago
David Wilson e913c11e50 docs: new Ansible limitation 7 years ago
David Wilson 860a635500 docs: remove delegate_to limitation 7 years ago
David Wilson dfdb7ab128 docs: more Ansible 7 years ago
David Wilson cf7f8ff490 docs: document one more Ansible limitation 7 years ago
David Wilson 7179ca163b docs: Replace shifty hacker dude with plebey skater dude
Farewell, shifty hacker dude! May your 0days be many and your
indictments be few
7 years ago
David Wilson 0c056ddbe2 docs: new Ansible limitation, add new heading
Some differences are eventually likely to become permanent, because the
existing behaviour is unforgiveable.
7 years ago
David Wilson 14251539cd docs: Docker typo. 7 years ago
David Wilson 5036a9448f Add Docker method. 7 years ago
David Wilson 77f6b31e5c docs: more images. 7 years ago
David Wilson 56b563161b docs: single-threaded connection is the most important limitation 7 years ago
David Wilson c1bea0dd49 docs: store blog images for fear of laptop crash
There is only so much trackpad-driven yEd work one man can handle
7 years ago
David Wilson dd9d3494e0 docs: small ansible.rst updates 7 years ago
David Wilson ce6648da51 docs: mention CPU usage reduction 7 years ago
David Wilson 270db025c0 docs: fix up Ansible install docs. 7 years ago
David Wilson e9060b3b7f docs: more modest and accurate numbers for Ansible 7 years ago
David Wilson b221eaaa10 ansible: log call timings 7 years ago
David Wilson a817f5e9e8 docs: more marketing, add lots of drama bold. 7 years ago
David Wilson 5933723d70 docs: a little more marketing 7 years ago
David Wilson de1b3602e5 docs: tidy up ansible.rst 7 years ago
David Wilson 52da0d882e ansible: add a new limitation 7 years ago
David Wilson 14b5c4d504 docs: remove fixed Ansible limitation
The solution was that Mitogen's loader should emulate the behaviour of
ansible.executor.module_common, which restricts dependency scanning to
the ansible.module_utils namespace.
7 years ago
David Wilson 91116810a1 ansible: delete utils.py and promote cast() to mitogen.utils 7 years ago
David Wilson 18eaf14dca ansible: migrate logging variables into utils. 7 years ago
David Wilson 10c2de2845 examples: rename playbooks for clarity. 7 years ago
David Wilson ea126bb7e8 docs: add stats tracking 7 years ago
David Wilson 875ee29d4f docs: initial Ansible extension docs. 7 years ago
David Wilson bdd8648bae ssh: enable compression by default
Using the same test as in 7af97c0365,
transmitted wire bytes drops from 135,531 to 133,071 (-1.81%), while
received drops from 21,073 to 14,775 (-30%).

Combined, both changes shave 13,914 bytes (-8.6%) off aggregate
bandwidth usage.

Make it configurable as compression hurts in some scenarios.
7 years ago
David Wilson 4d940f08ae importer: drop redundant prefix from pkg_present
For the 52 submodules of ansible.modules.system, this produced a 1602
byte pkg_present list. After stripping it becomes 406 bytes, and the
entire LOAD_MODULE size drops from 1988 bytes to 792 bytes (-60%).

For the 68 submodules of ansible.module_utils, 1902 bytes pkg_present
becomes 474 bytes (-75%), and LOAD_MODULE size drops from 2867 bytes to
1439 bytes (-49%).

In a simple test running Ansible's "setup" module followed by its "apt"
module, wire bytes sent drops from 140,357 to 135,531 (-3.4%).
7 years ago
David Wilson 71a6b9e32a master: Select.all() sugar 7 years ago
David Wilson ac4a42a43e docs: fix Select() example for new style API. 7 years ago
Alex Willmer a69d9650ad Fix indentation of Router.route anchor 7 years ago
Alex Willmer eb1ad66469 docs: Fix out of date get_data() snippets
As of adc8fe3aed Receiver objects do not
have a get_data() method and Receiver.get() does not unpickle the
message.
7 years ago
David Wilson 20afa5b90c Latch v2: combined queue + one self-pipe-per-thread
Turns out it is far too easy to burn through available file descriptors,
so try something else: self-pipes are per thread, and only temporarily
associated with a Lack that wishes to sleep.

Reduce pointless locking by giving Latch its own queue, and removing
Queue.Queue() use in some places.

Temporarily undo merging of of Waker and Latch, let's do this one step
at a time.
7 years ago
David Wilson e63e9d299e docs: add Message documentation 7 years ago
David Wilson dd088908df select: clean up API. 7 years ago
David Wilson a39cd44bf2 core: add auth_id field. 7 years ago
David Wilson 07d4d799f1 Add mitogen.main() decorator mainly for docs and demo use. 7 years ago
David Wilson 09eb3fd9ba utils: support log_to_file(usec=True) 7 years ago
David Wilson 592ebd59c2 docs: reorder sections 7 years ago
David Wilson dee856f6f4 docs: remove obsolete warning 7 years ago
David Wilson b941bce964 docs: sphinx-autobuild default target. 7 years ago
David Wilson 4d9d21c808 docs: fix typo 7 years ago
David Wilson 4a6d55ced6 docs: vastly simplify importer concurrency docs 7 years ago
David Wilson 984b39180e importer: Beginnings of howitworks section. 7 years ago
David Wilson b580ada89f Tidy up and correct the new example 7 years ago
David Wilson eb2aef866b Add shell replacement example. 7 years ago
David Wilson 641498e7f6 docs: Fix function name. 7 years ago
David Wilson f1f36cec35 Simplify the API, make Broker optional and auto-shutdown on main thread exit. 7 years ago
Alex Willmer 3831ac360f Replace all calls to file() with open()
Although these are synonyms in Python 2.x, when using MyPy to typecheck
code use of file() causes spurious errors.

This commit also serves as one small step to Python 3.x compatibility,
since 3.x removes the file() builtin.
7 years ago
David Wilson 038ab04908 docs: convert paragraph to footnote. 7 years ago
David Wilson e3d2c8d649 issue #49: update howitworks.rst for command line change 7 years ago
David Wilson 58102c8179 docs: waiting on multiple calls 7 years ago
David Wilson b1ad04330b docs: move Router.route() into Sphinx. 7 years ago
David Wilson cf0668b2b1 docs: Add warning to preloading section. 7 years ago
David Wilson fb759f7c16 docs: move Broker docstrings into Sphinx. 7 years ago
David Wilson ac7cada323 docs: more getting started guide 7 years ago
David Wilson ffa063cc01 docs: annother barriage of cross-reference fixes. 7 years ago
David Wilson 9372d2c3de docs: Fix up tons of references, document trust chain 7 years ago
David Wilson 051285437f importer: module preloading docs 7 years ago
David Wilson b3bd075d37 docs: getting started 7 years ago
David Wilson 792e789908 More. 7 years ago
David Wilson c15d227da4 docs: way more appropriate image 7 years ago
David Wilson b94e7b3678 docs: more getting started 7 years ago
David Wilson 709b38c36e docs: Getting started tidyups 7 years ago
David Wilson dcc45bc7de docs: Get started on Getting Started 7 years ago
David Wilson 0299f86bc9 docs: fix reference to with: 7 years ago
David Wilson c9daa2ff30 docs: move fakessh docs into Sphinx. 7 years ago
David Wilson 6a10ab605e docs: move mitogen.utils docs to Sphinx. 7 years ago
David Wilson 270b242ec5 docs: fix decorator types. 7 years ago
David Wilson ec66152e37 docs: better io_op doc, move Side docs to Sphinx. 7 years ago
David Wilson 0767abf26f docs: move BasicStream docs into Sphinx. 7 years ago
David Wilson b7a9aa46cf core: More robust shutdown
Now there is a separate SHUTDOWN message that relies only on being
received by the broker thread, the main thread can be hung horribly and
the process will still eventually receive a SIGTERM.
7 years ago
David Wilson 79dd00db5a master: hack to avoid executing __main__. 7 years ago
David Wilson 416ab90e02 docs: Fix index generation everywhere. 7 years ago
David Wilson 976c643f21 docs: remove note, unpickling can no longer trigger module loads 7 years ago
David Wilson decc9a900c docs: s/random/pseudorandom/ 7 years ago
David Wilson 83f8f1863e docs: fix pickler docs, begin relabelling master/slave->parent/child 7 years ago
David Wilson 4327baabfa docs: remove final references to call_with_deadline(). 7 years ago
David Wilson dd69b8feeb docs: with_context element is replaced by a decorator. 7 years ago
David Wilson 12fe884f1a Reference Select 7 years ago
David Wilson a81f804e92 Accidentally hecked in incomplete paragraph. 7 years ago
David Wilson 83e3e1b82a Typo. 7 years ago
David Wilson e9af841e60 docs: fix :return: syntax. 7 years ago
David Wilson f1d82c7284 More API documentation. 7 years ago
David Wilson 9d2a11e70f Fix function reference. 7 years ago
David Wilson 5772e06368 Typo. 7 years ago
David Wilson b7f95e558f Better document Router API and constructors. 7 years ago
David Wilson 3eb60d777a Add mitogen.parent_ids to docs. 7 years ago
David Wilson 0c573d0e47 Tidy up select examples even more. 7 years ago
David Wilson e624295d63 Document channel class 7 years ago
David Wilson c4d9f124c6 Document Sender and Receiver classes. 7 years ago
David Wilson 33643222de docs: Add spacing between list items in body text. 7 years ago
David Wilson 265d9f0293 Tidy up select docs. 7 years ago
David Wilson 112f76fe89 More concise text. 7 years ago
David Wilson 19869da087 issue #20: Return type fix. 7 years ago
David Wilson 0565784164 Package rename cleanup. 7 years ago
David Wilson f869e088f8 issue #20: tests and fixes for mitogen.master.Select(). 7 years ago
David Wilson e3d967ebeb issue #20: initial implementation of mitogen.master.Select(). 7 years ago
David Wilson abfa85d7da Remove set from permitted types. 7 years ago
David Wilson 9c4bf37cfc Remove final vestiges of context.key. 7 years ago
David Wilson 0ff65c9780 docs: Use sphinx-programoutput plugin for preamble_size.py 7 years ago
David Wilson 2af1830323 Update preamble_size output. 7 years ago
David Wilson 04ea5306e9 Document a bunch of mitogen.master and move more docstrings into Sphinx. 7 years ago
David Wilson 092bbef0b3 Reorder table of contents. 7 years ago
David Wilson caa1fde4fb Add serialization rules section. 7 years ago
David Wilson 4720eb1c55 core: add ALLOCATE_ID message for fakessh. 7 years ago
David Wilson 6f2d514813 Import unused images. 7 years ago
David Wilson b809d43865 Move more docstrings out of core.py. 7 years ago
David Wilson 4f50707b82 core: support takes_econtext and takes_router decorators. 7 years ago
David Wilson 025954af32 Typo. 7 years ago
David Wilson 6b9e46ecd6 Add subheading to describe each import hack. 7 years ago
David Wilson 36f0584087 Link to six bug. 7 years ago
David Wilson 6ff2a49f10 Pad out shame.rst. 7 years ago
David Wilson fa427d83c8 Tidy. 7 years ago
David Wilson 38a9482860 Add hacks to allow Mock to be imported. 7 years ago
David Wilson 1f99dcb435 fix unbelievably dumb variable shadowing 7 years ago
David Wilson 25e4724bac A few more mitogen.slave/master changes 7 years ago
David Wilson 708b742a9c Mention git. 7 years ago
David Wilson e7ff6259a3 Initial commit. 7 years ago