Commit Graph

3267 Commits (60fbea4b59cb558ffc3c8cfe9d1531367fc4be71)
 

Author SHA1 Message Date
David Wilson 07a8994ff5 issue #156: waking thread result dictionary with an integer. 7 years ago
David Wilson 001e0163fe issue #156: handle multiple _put() before wake of first sleeper
- If latch.get() is called and the queue is empty, a thread is put to
  sleep.

- If Latch.put() from another thread then appends an item to the queue and
  wakes the sleeping thread, and

- If a subsequent Latch.put() from the same or another thread manages to
  acquire `lock` before the sleeping thread is scheduled,

- The sleeping thread's wake socket would have multiple bytes written to
  it.

Therefore create a new _pending variable to track the only item assigned
to each thread (keyed by its write socket), and remove the socket from
`sleeping` from within put.
7 years ago
David Wilson 168a954d90 issue #156: prefix Latch private variables 7 years ago
David Wilson b5398bd17f issue #156: docs typo 7 years ago
David Wilson 512ff77a46 issue #156: prevent non-sleeping threads from starving sleeping threads.
See new docs
7 years ago
David Wilson 9e514240a1 issue #156: always enable microsecond logging 7 years ago
David Wilson c51eee3c7f issue #156: make Pool repr log thread too. 7 years ago
David Wilson c20c2587d9 issue #156: make Latch() repr match Pool() repr. 7 years ago
David Wilson 7f4b89b7bb issue #156: log worker thread crashes in mitogen.pool 7 years ago
David Wilson 6e368d37da issue #156: log queue size too 7 years ago
David Wilson 037b461c39 issue #156: yet more logging :( 7 years ago
David Wilson 653c73c8f0 issue #156: also log target of wakes 7 years ago
David Wilson 4d96d0c1af issue #156: fix duplicate -vvvv logging 7 years ago
David Wilson a5cc7cb43c issue #156: add extra debugging around Latch
Change from writing '\x00' to writing '\x7f', and verify that is the
byte that woke the sleeping thread. Add a bunch more IO logging.
7 years ago
David Wilson ac7a64dfa3 core: assign common expression to a variable. 7 years ago
David Wilson 148ce1d703 issue #155: increase context ID width to 32 bits
Needed to make large range allocations (1000 per ALLOCATE_ID roundtrip)
feasible.
7 years ago
David Wilson 3579b6806b issue #152: reproduction for second problem 7 years ago
David Wilson c183f06dfb issue #152: respect the Ansible-selected interpreter for local connections too. 7 years ago
David Wilson 89b0faae2f Workaround for global state in yum_repository module; closes #154. 7 years ago
David Wilson 305e024819 issue #154: import user's reproduction 7 years ago
David Wilson 071d9fbfb3 docs: tidy ansible docs. 7 years ago
David Wilson 4d8ccab2ca ansible: docstring fixes 7 years ago
David Wilson 2132c311b2 tests: mark some tests as skipped 7 years ago
David Wilson f241eac5ce parent: allow Python to determine its install prefix from argv[0]
Fixes support for virtualenv. Closes #152.
7 years ago
David Wilson 088fd76109 issue #152: import reproduction 7 years ago
David Wilson dec3af375a issue #144: ansible: increase default pool size to 16. 7 years ago
David Wilson 9cf889b846 issue #144: master: public/private Pool attributes. 7 years ago
David Wilson 19632473dc issue #144: ansible: use service.Pool with default size=1. 7 years ago
David Wilson fe900087a2 issue #144: service: working service.Pool object.
It knows how to dispatch messages from multiple receivers (associated
with multiple services) to multiple threads, where the service
implementation is invoked on the message.

It wakes a maximum of one thread per received message.

It knows how to shut down gracefully.

Implication: due to the latch use, there are 2 file descriptors burned
for every thread. We don't need interruptibility here, so in future, it
might be nice to allow swapping a diferent queueing primitive into
Select (maybe a subclass?) just for this case.
7 years ago
David Wilson 4f361be7e7 issue #144: teach Select() to close its latch
Causes all threads sleeping on the select to wake.
7 years ago
David Wilson 8aada2646c core: support throwing LatchError in every sleeping thread
This is to allow Select() to be used as a generic queueing primitive
that supports graceful shutdown.
7 years ago
David Wilson ebfe733914 core: tidy up Stream.on_receive() branches. 7 years ago
David Wilson 7a74bb0a39 docs: update ansible risks/differences. 7 years ago
David Wilson 4541bc76a0 Add Google Cloud client to dev requirements
Will be used more heavily for CI later, but it's already in use by
gcloud-ansible-playbook.py.
7 years ago
David Wilson bcc15987fc docs: extra ansible paragraph. 7 years ago
David Wilson 858b01e78b issue #150: add docstrings. 7 years ago
David Wilson 6940b23013 issue #150: ansible: mark worker/child sock as CLOEXEC. 7 years ago
David Wilson 7a394dc73e ansible: allow establishment of duplicate SSH connections 7 years ago
David Wilson 86ede62241 issue #150: introduce separate connection multiplexer process
This is a work in progress.
7 years ago
David Wilson eee5423dd9 issue #150: tidy up mitogen.debug output for use next time 7 years ago
David Wilson 9adadb5c3a issue #150: import stack.py hack as mitogen.debug
Usage:
  - insert a call to mitogen.debug() in the desired process
  - kill -USR2 that process
  - observe its controlling TTY produces thread stack dumps
7 years ago
David Wilson df488237d4 core: fix race in PidfulStreamHandler
Need to re-test with the lock held, else >1 threads can end up waiting
for lock then reopening the log repeatedly.
7 years ago
David Wilson a06c92d285 core: enable_debug_logging() should reopen file post-fork. 7 years ago
David Wilson 051fb85d2d issue #150: 100 target docker inventory 7 years ago
David Wilson 4691ce0b95 issue #150: ansible: add basic Docker support. 7 years ago
David Wilson b64e52b1fd issue #150: tweak script for running without external IPs 7 years ago
David Wilson 8607680730 issue #150: quick script to run ansible against gcloud instance group 7 years ago
David Wilson 67ff762ba5 issue #139: docs: remove note about bad buffering 7 years ago
David Wilson eba12e2ee2 issue #139: bump kernel socket buffer size to 128kb
This allows us to write 128kb at a time towards SSH, but it doesn't help
with sudo, where the ancient tty layer is always used.
7 years ago
David Wilson 728a0da8a4 issue #139: eliminate quadratic behaviour from transmit path
Implication: the entire message remains buffered until its last byte is
transmitted. Not wasting time on it, as there are pieces of work like
issue #6 that might invalidate these problems on the transmit path
entirely.
7 years ago