Commit Graph

576 Commits (e3209d1de07bac6ad126b2de180c96be313abeb9)
 

Author SHA1 Message Date
David Wilson e3209d1de0 core: log Broker's id in repr. 7 years ago
David Wilson f4ba66e3ee issue #155: allocate child IDs in batches of 1000.
Avoids a roundtrip for every fork.
7 years ago
David Wilson 0c77107041 issue #96: fail test.sh if any test fails 7 years ago
David Wilson 1ed86774b5 issue #156: document select exception 7 years ago
David Wilson 7ec02f9bb0 issue #156: ensure Latch state is cleaned up if select throws. 7 years ago
David Wilson 20f5d89dfa issue #156: fix several more races
* Don't need to sleep if queue>sleepers, can just pop the right queue
  element and return it.

* If queue>sleeping and waking==sleeping, no mechanism existed to ensure
  a thread newly added to sleeping would ever be woken. Above change
  fixes that.

* Cannot trust select() return value, scheduler might sleep us
  indefinitely while put() writes a byte.

* Sleeping threads didn't pop FIFO, they popped in whatever order
  scheduler woke them up. Must recover index and use it to pick the pop
  index.
7 years ago
David Wilson 526b0a514b issue #156: prevent Latch.close() triggering spurious wakeups 7 years ago
David Wilson 18e2977baf docs: annoying phrasing 7 years ago
David Wilson 2c22c41819 issue #156: don't decrement `waking` if we timed out rather than being woken. 7 years ago
David Wilson 07a8994ff5 issue #156: waking thread result dictionary with an integer. 7 years ago
David Wilson 001e0163fe issue #156: handle multiple _put() before wake of first sleeper
- If latch.get() is called and the queue is empty, a thread is put to
  sleep.

- If Latch.put() from another thread then appends an item to the queue and
  wakes the sleeping thread, and

- If a subsequent Latch.put() from the same or another thread manages to
  acquire `lock` before the sleeping thread is scheduled,

- The sleeping thread's wake socket would have multiple bytes written to
  it.

Therefore create a new _pending variable to track the only item assigned
to each thread (keyed by its write socket), and remove the socket from
`sleeping` from within put.
7 years ago
David Wilson 168a954d90 issue #156: prefix Latch private variables 7 years ago
David Wilson b5398bd17f issue #156: docs typo 7 years ago
David Wilson 512ff77a46 issue #156: prevent non-sleeping threads from starving sleeping threads.
See new docs
7 years ago
David Wilson 9e514240a1 issue #156: always enable microsecond logging 7 years ago
David Wilson c51eee3c7f issue #156: make Pool repr log thread too. 7 years ago
David Wilson c20c2587d9 issue #156: make Latch() repr match Pool() repr. 7 years ago
David Wilson 7f4b89b7bb issue #156: log worker thread crashes in mitogen.pool 7 years ago
David Wilson 6e368d37da issue #156: log queue size too 7 years ago
David Wilson 037b461c39 issue #156: yet more logging :( 7 years ago
David Wilson 653c73c8f0 issue #156: also log target of wakes 7 years ago
David Wilson 4d96d0c1af issue #156: fix duplicate -vvvv logging 7 years ago
David Wilson a5cc7cb43c issue #156: add extra debugging around Latch
Change from writing '\x00' to writing '\x7f', and verify that is the
byte that woke the sleeping thread. Add a bunch more IO logging.
7 years ago
David Wilson ac7a64dfa3 core: assign common expression to a variable. 7 years ago
David Wilson 148ce1d703 issue #155: increase context ID width to 32 bits
Needed to make large range allocations (1000 per ALLOCATE_ID roundtrip)
feasible.
7 years ago
David Wilson 3579b6806b issue #152: reproduction for second problem 7 years ago
David Wilson c183f06dfb issue #152: respect the Ansible-selected interpreter for local connections too. 7 years ago
David Wilson 89b0faae2f Workaround for global state in yum_repository module; closes #154. 7 years ago
David Wilson 305e024819 issue #154: import user's reproduction 7 years ago
David Wilson 071d9fbfb3 docs: tidy ansible docs. 7 years ago
David Wilson 4d8ccab2ca ansible: docstring fixes 7 years ago
David Wilson 2132c311b2 tests: mark some tests as skipped 7 years ago
David Wilson f241eac5ce parent: allow Python to determine its install prefix from argv[0]
Fixes support for virtualenv. Closes #152.
7 years ago
David Wilson 088fd76109 issue #152: import reproduction 7 years ago
David Wilson dec3af375a issue #144: ansible: increase default pool size to 16. 7 years ago
David Wilson 9cf889b846 issue #144: master: public/private Pool attributes. 7 years ago
David Wilson 19632473dc issue #144: ansible: use service.Pool with default size=1. 7 years ago
David Wilson fe900087a2 issue #144: service: working service.Pool object.
It knows how to dispatch messages from multiple receivers (associated
with multiple services) to multiple threads, where the service
implementation is invoked on the message.

It wakes a maximum of one thread per received message.

It knows how to shut down gracefully.

Implication: due to the latch use, there are 2 file descriptors burned
for every thread. We don't need interruptibility here, so in future, it
might be nice to allow swapping a diferent queueing primitive into
Select (maybe a subclass?) just for this case.
7 years ago
David Wilson 4f361be7e7 issue #144: teach Select() to close its latch
Causes all threads sleeping on the select to wake.
7 years ago
David Wilson 8aada2646c core: support throwing LatchError in every sleeping thread
This is to allow Select() to be used as a generic queueing primitive
that supports graceful shutdown.
7 years ago
David Wilson ebfe733914 core: tidy up Stream.on_receive() branches. 7 years ago
David Wilson 7a74bb0a39 docs: update ansible risks/differences. 7 years ago
David Wilson 4541bc76a0 Add Google Cloud client to dev requirements
Will be used more heavily for CI later, but it's already in use by
gcloud-ansible-playbook.py.
7 years ago
David Wilson bcc15987fc docs: extra ansible paragraph. 7 years ago
David Wilson 858b01e78b issue #150: add docstrings. 7 years ago
David Wilson 6940b23013 issue #150: ansible: mark worker/child sock as CLOEXEC. 7 years ago
David Wilson 7a394dc73e ansible: allow establishment of duplicate SSH connections 7 years ago
David Wilson 86ede62241 issue #150: introduce separate connection multiplexer process
This is a work in progress.
7 years ago
David Wilson eee5423dd9 issue #150: tidy up mitogen.debug output for use next time 7 years ago
David Wilson 9adadb5c3a issue #150: import stack.py hack as mitogen.debug
Usage:
  - insert a call to mitogen.debug() in the desired process
  - kill -USR2 that process
  - observe its controlling TTY produces thread stack dumps
7 years ago