Commit Graph

594 Commits (878e7a0902787a2f04bc6c3067177fb9d4e6108c)
 

Author SHA1 Message Date
David Wilson 878e7a0902 issue #155: pass reference to existing Router into Stream constructor
This is a hacky layering violation, but it seems the simplest approach
for now: fork needs access to Router, in order to recover the existing
Importer instance.
7 years ago
David Wilson 21ef540cd8 issue #155: parent: split create_socketpair() from create_child() 7 years ago
David Wilson c31a177ebe issue #155: parent: split get_main_kwargs() from get_preamble() 7 years ago
David Wilson b51365209e parent: needless duplicate ADD_ROUTE message
notice_stream() does that already.
7 years ago
David Wilson 27175e3126 docs: tidy up signals.rst a little. 7 years ago
David Wilson a956aa409e Remove duplicate set_cloexec calls everywhere
Now it's handled in Side() constructor, it can disappear elsewhere.
7 years ago
David Wilson 732a610246 docs: add beginnings of section on func decorators 7 years ago
David Wilson 2abe87472c issue #162: docs: begin documenting mitogen.service 7 years ago
David Wilson 1777b8f42e ansible: use DeduplicatingService for ContextService; closes #162. 7 years ago
David Wilson f6b5d9f2f6 issue #162: implement mitogen.service.DeduplicatingService
This abstracts the pattern found in parent.ModuleForwarder and to a
lesser degree master.ModuleResponser. We can probably use it in those
contexts later.
7 years ago
David Wilson 65fcef2374 core: mark every side O_CLOEXEC
Not sure why this wasn't done before, seems it should have always been
this way, and can't see any reason it wasn't. Without it, many fds are
leaked into at least .local() children. Closes #163.
7 years ago
David Wilson 2d7821b824 tests: test_stream_name: fix non-localhost Docker 7 years ago
David Wilson 08612d4ca2 issue #155: fix call_function_test regression
It's entirely unclear how test_aborted_on_local_context_disconnect ever
passed, but it was broken by the previous commit.
7 years ago
David Wilson 54ff1c90fa issue #155: add DEL_ROUTE, propagate ADD_ROUTE upwards
* IDs are allocated by the parent responsible for contructing a new
  child, using ALLOCATE_ID to the master as necessary to allocate new ID
  ranges.

* ADD_ROUTE is sent up the tree rather than down. This permits
  construction of the new context to complete concurrent to parent
  contexts learning about its existence. Since all streams are strictly
  ordered, it's not possible for any parent to observe messages from the
  new context prior to arrival of an ADD_ROUTE from the parent notifying
  of its existence.

  If the new context, for example, implements an Ansible async task, its
  parent can start executing that without waiting for any synchronous
  confirmation from any parent or the master.

* Since routes propagate up, it's no longer possible for a plain
  non-parent child to ever receive ADD_ROUTE, so that code can be moved
  out of core.py and into parent.py (-0.2kb compressed).

* Add a .routes attribute to parent.Stream, and respond to disconnection
  signal on the stream by propagating DEL_ROUTE for any ADD_ROUTE ever
  received from that stream.

* Centralize route management in a new parent.RouteMonitor class
7 years ago
David Wilson aeeeb45ccb docs: farewell, glorious iframe! 7 years ago
David Wilson adfd827531 test.sh: enhancements
* Explicitly name every test to run, I have lots of unchecked in stuff
* Allow SIGINT to stop the process
7 years ago
David Wilson 23e279b617 tests: get import_test limping back to health. 7 years ago
David Wilson 469279d9ca master: refactor ThreadWatcher
In order to support a .remove() method, to prevent a minor but annoying
(log visible) memory leak while running the tests.
7 years ago
David Wilson e3209d1de0 core: log Broker's id in repr. 7 years ago
David Wilson f4ba66e3ee issue #155: allocate child IDs in batches of 1000.
Avoids a roundtrip for every fork.
7 years ago
David Wilson 0c77107041 issue #96: fail test.sh if any test fails 7 years ago
David Wilson 1ed86774b5 issue #156: document select exception 7 years ago
David Wilson 7ec02f9bb0 issue #156: ensure Latch state is cleaned up if select throws. 7 years ago
David Wilson 20f5d89dfa issue #156: fix several more races
* Don't need to sleep if queue>sleepers, can just pop the right queue
  element and return it.

* If queue>sleeping and waking==sleeping, no mechanism existed to ensure
  a thread newly added to sleeping would ever be woken. Above change
  fixes that.

* Cannot trust select() return value, scheduler might sleep us
  indefinitely while put() writes a byte.

* Sleeping threads didn't pop FIFO, they popped in whatever order
  scheduler woke them up. Must recover index and use it to pick the pop
  index.
7 years ago
David Wilson 526b0a514b issue #156: prevent Latch.close() triggering spurious wakeups 7 years ago
David Wilson 18e2977baf docs: annoying phrasing 7 years ago
David Wilson 2c22c41819 issue #156: don't decrement `waking` if we timed out rather than being woken. 7 years ago
David Wilson 07a8994ff5 issue #156: waking thread result dictionary with an integer. 7 years ago
David Wilson 001e0163fe issue #156: handle multiple _put() before wake of first sleeper
- If latch.get() is called and the queue is empty, a thread is put to
  sleep.

- If Latch.put() from another thread then appends an item to the queue and
  wakes the sleeping thread, and

- If a subsequent Latch.put() from the same or another thread manages to
  acquire `lock` before the sleeping thread is scheduled,

- The sleeping thread's wake socket would have multiple bytes written to
  it.

Therefore create a new _pending variable to track the only item assigned
to each thread (keyed by its write socket), and remove the socket from
`sleeping` from within put.
7 years ago
David Wilson 168a954d90 issue #156: prefix Latch private variables 7 years ago
David Wilson b5398bd17f issue #156: docs typo 7 years ago
David Wilson 512ff77a46 issue #156: prevent non-sleeping threads from starving sleeping threads.
See new docs
7 years ago
David Wilson 9e514240a1 issue #156: always enable microsecond logging 7 years ago
David Wilson c51eee3c7f issue #156: make Pool repr log thread too. 7 years ago
David Wilson c20c2587d9 issue #156: make Latch() repr match Pool() repr. 7 years ago
David Wilson 7f4b89b7bb issue #156: log worker thread crashes in mitogen.pool 7 years ago
David Wilson 6e368d37da issue #156: log queue size too 7 years ago
David Wilson 037b461c39 issue #156: yet more logging :( 7 years ago
David Wilson 653c73c8f0 issue #156: also log target of wakes 7 years ago
David Wilson 4d96d0c1af issue #156: fix duplicate -vvvv logging 7 years ago
David Wilson a5cc7cb43c issue #156: add extra debugging around Latch
Change from writing '\x00' to writing '\x7f', and verify that is the
byte that woke the sleeping thread. Add a bunch more IO logging.
7 years ago
David Wilson ac7a64dfa3 core: assign common expression to a variable. 7 years ago
David Wilson 148ce1d703 issue #155: increase context ID width to 32 bits
Needed to make large range allocations (1000 per ALLOCATE_ID roundtrip)
feasible.
7 years ago
David Wilson 3579b6806b issue #152: reproduction for second problem 7 years ago
David Wilson c183f06dfb issue #152: respect the Ansible-selected interpreter for local connections too. 7 years ago
David Wilson 89b0faae2f Workaround for global state in yum_repository module; closes #154. 7 years ago
David Wilson 305e024819 issue #154: import user's reproduction 7 years ago
David Wilson 071d9fbfb3 docs: tidy ansible docs. 7 years ago
David Wilson 4d8ccab2ca ansible: docstring fixes 7 years ago
David Wilson 2132c311b2 tests: mark some tests as skipped 7 years ago