Commit Graph

3610 Commits (b91407a7792e1e827082cbfda0076e5ef45c99f9)
 

Author SHA1 Message Date
Alex Willmer 0d725f9cd8 Fix DeprecationWarning in mitogen.utils.run_with_router()
```
14:20:32.186270 W py.warnings:
/home/alex/src/mitogen/mitogen/utils.py:152: DeprecationWarning: invalid
escape sequence \*
  """
```
6 years ago
Alex Willmer e7d70b109b Generate coverage report even if some tests fail
`set -o errexit` was exiting run_tests prematurely if any test failed,
so the coverage report was not always generated.
6 years ago
David Wilson d763570ca2 Merge remote-tracking branch 'origin/dmw'
* origin/dmw:
  ci: fix incorrect partition/rpartition from 8a4caea84f
  issue #260: hide force-disconnect messages.
  issue #498: fix shutdown crash
  issue #260: avoid start_transmit()/on_transmit()/stop_transmit()
  core: ensure broker profiling output reaches disk
  master: keep is_stdlib_path() result as negative cache entry
  ci: Allow DISTROS="debian*32" variable, and KEEP=1
6 years ago
David Wilson 49a8745a45 ci: fix incorrect partition/rpartition from 8a4caea84f 6 years ago
David Wilson 696cee57dd issue #260: hide force-disconnect messages.
Parent is always force-disconnected, and now it shows up quite visibly
on every exit.
6 years ago
David Wilson 26c6c6d048 issue #498: fix shutdown crash
Traceback (most recent call last):
      File "<stdin>", line 2707, in _invoke
      File "<stdin>", line 2480, in _on_del_route
    NameError: global name 'target_id' is not defined
6 years ago
David Wilson a18a083c94 issue #260: avoid start_transmit()/on_transmit()/stop_transmit()
Previous transmit sequence was:
        Router._async_route -> Stream._send -> Broker._start_transmit ->
        Broker.loop -> Stream.on_transmit -> socket.write ->
        Broker.stop_transmit

New sequence, when socket buffer can hold message is:
        Router._async_route -> Stream._send -> socket.write

bench/roundtrip.py
        Before: 240 usec
        after: 178 usec

Stat before:
       5088.276050      task-clock (msec)         #    0.997 CPUs utilized
           185,568      context-switches          #    0.036 M/sec
                 0      cpu-migrations            #    0.000 K/sec
            18,923      page-faults               #    0.004 M/sec
    13,063,871,501      cycles                    #    2.567 GHz
    12,834,579,684      instructions              #    0.98  insn per cycle
     2,669,820,684      branches                  #  524.700 M/sec
       107,296,033      branch-misses             #    4.02% of all branches

       5.105018296 seconds time elapsed

       2.350970000 seconds user
       0.345497000 seconds sys

Stat after:
       4019.208047      task-clock (msec)         #    0.998 CPUs utilized
           249,471      context-switches          #    0.062 M/sec
                 0      cpu-migrations            #    0.000 K/sec
            20,990      page-faults               #    0.005 M/sec
    10,312,535,979      cycles                    #    2.566 GHz
    11,586,365,995      instructions              #    1.12  insn per cycle
     2,392,933,370      branches                  #  595.374 M/sec
        75,432,205      branch-misses             #    3.15% of all branches

       4.028763347 seconds time elapsed

       3.367051000 seconds user
       0.652962000 seconds sys
6 years ago
David Wilson b161408598 Merge commit 'fe74577'
* commit 'fe74577':
  Use develop mode in tox
  issue #429: fix sudo regression.
  misc: rename to scripts. tab completion!!
  core: Latch._wake improvements
  issue #498: prevent crash on double 'disconnect' signal.
  issue #413: don't double-propagate DEL_ROUTE to parent.
  issue #498: wrap Router dict mutations in a lock
  issue #429: enable en_US locale to unbreak debops test.
  issue #499: fix another mind-numbingly stupid vanilla inconsistency
  issue #497: do our best to cope with crap upstream code
  ssh: fix test to match updated log format.
  issue #429: update Changelog.
  issue #429: update Changelog.
  issue #429: teach sudo about every know i18n password string.
  issue #429: install i18n-related bits in test images.
  ssh: tidy up logs and stream names.
  tests: ensure file is closed in connection_test.
  gcloud: small updates
  tests: give ansible/gcloud/ its own requirements file.
6 years ago
David Wilson d4a0b70e15 core: ensure broker profiling output reaches disk
Profiler hasn't been used much since the hard exit was added.
6 years ago
David Wilson dc4b27c6bf master: keep is_stdlib_path() result as negative cache entry
On 32x Docker run of issue_140__thread_pileup.yml

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1218500    1.716    0.000    7.325    0.000 /home/dmw/src/mitogen/mitogen/master.py:118(is_stdlib_path)

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      166    0.000    0.000    0.001    0.000 /home/dmw/src/mitogen/mitogen/master.py:123(is_stdlib_path)
6 years ago
David Wilson 8a4caea84f ci: Allow DISTROS="debian*32" variable, and KEEP=1 6 years ago
David Wilson fe745779ba Use develop mode in tox 6 years ago
David Wilson fca5513610 issue #429: fix sudo regression. 6 years ago
David Wilson ae8173a29f misc: rename to scripts. tab completion!! 6 years ago
David Wilson 0c0d34241b core: Latch._wake improvements
os.write() can fail with EINTR due to signals, so wrap it in
io_op(). Closes #483.

Masking EBADF looks like it is/was almost certainly papering over a bug,
remove it and suffer the bug reports. Closes #495.
6 years ago
David Wilson 407294cd79 issue #498: prevent crash on double 'disconnect' signal.
Fixes:

ERROR! [pid 1096] 23:31:48.363215 E mitogen: _broker_main() crashed
Traceback (most recent call last):
  File "/home/dmw/src/mitogen/mitogen/core.py", line 2917, in _broker_main
    self._loop_once()
  File "/home/dmw/src/mitogen/mitogen/core.py", line 2875, in _loop_once
    self._call(side.stream, func)
  File "/home/dmw/src/mitogen/mitogen/core.py", line 2860, in _call
    stream.on_disconnect(self)
  File "/home/dmw/src/mitogen/mitogen/parent.py", line 1161, in on_disconnect
    super(Stream, self).on_disconnect(broker)
  File "/home/dmw/src/mitogen/mitogen/core.py", line 1534, in on_disconnect
    fire(self, 'disconnect')
  File "/home/dmw/src/mitogen/mitogen/core.py", line 390, in fire
    func(*args, **kwargs)
  File "/home/dmw/src/mitogen/mitogen/parent.py", line 1794, in <lambda>
    func=lambda: self._on_stream_disconnect(stream),
  File "/home/dmw/src/mitogen/mitogen/parent.py", line 1810, in _on_stream_disconnect
    routes = self._routes_by_stream.pop(stream)
KeyError: mitogen.ssh.Stream('ssh.localhost:2236')
6 years ago
David Wilson 9aa845669c issue #413: don't double-propagate DEL_ROUTE to parent.
propagate_up() sends ADD_ROUTE and DEL_ROUTE

propagate_down() sends only DEL_ROUTE, but didn't bother checking if
up() had sent it already.

Fixes:

     ERROR! [pid 41060] 17:55:30.739159 E mitogen.ctx.ssh.localhost:
     mitogen: RouteMonitor(): received DEL_ROUTE for 6081 from
     mitogen.fork.Stream(u'fork.41142'), expected
     mitogen.core.Stream('parent')
6 years ago
David Wilson e2478dcb9f issue #498: wrap Router dict mutations in a lock 6 years ago
David Wilson 60fe3fd6f5 issue #429: enable en_US locale to unbreak debops test. 6 years ago
David Wilson 7531af3ee0 issue #499: fix another mind-numbingly stupid vanilla inconsistency 6 years ago
David Wilson a40946297f issue #497: do our best to cope with crap upstream code 6 years ago
David Wilson 2fdbd0cfcd ssh: fix test to match updated log format. 6 years ago
David Wilson 77e7cadd22 issue #429: update Changelog. 6 years ago
David Wilson cd1e5e0138 issue #429: update Changelog. 6 years ago
David Wilson 6e9f8e829e issue #429: teach sudo about every know i18n password string. 6 years ago
David Wilson 960e505f07 issue #429: install i18n-related bits in test images. 6 years ago
David Wilson ec789513dc ssh: tidy up logs and stream names. 6 years ago
David Wilson eb93f82d05 tests: ensure file is closed in connection_test. 6 years ago
David Wilson 73979043ad gcloud: small updates 6 years ago
David Wilson 59068ca955 tests: give ansible/gcloud/ its own requirements file. 6 years ago
David Wilson fba06b0bde Merge remote-tracking branch 'origin/issue499'
* origin/issue499:
  issue #499: another totally moronic implementation difference
  issue #499: disable new test on vanilla.
  docs: update Changelog; closes #499.
  issue #499: respect C.BECOME_ALLOW_SAME_USER.
6 years ago
David Wilson 9df314f9c5 issue #499: another totally moronic implementation difference 6 years ago
David Wilson 53794469a0 issue #499: disable new test on vanilla. 6 years ago
David Wilson 60e9596e7d docs: update Changelog; closes #499. 6 years ago
David Wilson a1121c5a84 issue #499: respect C.BECOME_ALLOW_SAME_USER. 6 years ago
David Wilson 16a2de65c4 issue #493: another Py3.x fix. 6 years ago
David Wilson 2a70b3d5f4 issue #493: Py3.x fix. 6 years ago
David Wilson 9f7db66f67 tox: add py37 target. 6 years ago
David Wilson 3158c4ae9d docs: update Changelog; closes #493. 6 years ago
David Wilson bc84d1e950 issue #493: less CPU-intensive cookie format. 6 years ago
David Wilson 14d393765b issue #488: 2.4-compatible syntax in unix.py. 6 years ago
David Wilson b254051416 ansible: add test to ensure UNIX socket is cleaned on exit; closes #488. 6 years ago
David Wilson 628e8f4466 Bump PyYaML to a 3.7-compatible version. 6 years ago
David Wilson be6ab52fe1 issue #488: fix shutdown damage caused in 6ca2677de5
os._exit() subverted calm shutdown, meaning unix.Listener never had a
chance to cleanup its socket.

Move unix.Listener socket cleanup into its class so it is automatic
during shutdown, rather than cutpasted for each consumer.

Disable the watcher thread in the MuxProcess, it is useless.

Add .sock extension to /tmp/mitogen_unix_*, so we can write a test.
6 years ago
David Wilson 104fa463af Merge remote-tracking branch 'origin/issue479'
* origin/issue479:
  issue #490: log mitogen.unix server-side accept.
  issue #490: have Side._on_fork() empty _fork_refs
  issue #490: prevent double close() destroying unrelated Connection.
  docs: update Changelog; closes #479.
  issue #479: ModuleFinder special case for __main__ on Py3.x.
6 years ago
David Wilson b370076689 issue #490: log mitogen.unix server-side accept. 6 years ago
David Wilson 7dae88f0f5 issue #490: have Side._on_fork() empty _fork_refs
This is mostly to avoid ugly debugging that depends on the state of GC.
Discard sides from _fork_refs after they have been closed.
6 years ago
David Wilson 38a553d42d issue #490: prevent double close() destroying unrelated Connection. 6 years ago
David Wilson bf676aacfe docs: update Changelog; closes #479. 6 years ago
David Wilson 3435f24e8d issue #479: ModuleFinder special case for __main__ on Py3.x. 6 years ago