Commit Graph

2431 Commits (d51e70636d39f871d91829d10cfc6a7a1478de69)
 

Author SHA1 Message Date
David Wilson e77048ec2d utils: pad out reset_affinity() and integrate with detach_popen() 6 years ago
David Wilson b0e3807b16 utils: import reset_affinity() function. 6 years ago
David Wilson cafdfdb8e2 master: set Router.profiling if MITOGEN_PROFILING variable present. 6 years ago
David Wilson 568044438d parent: don't kill children when profiling is active. 6 years ago
David Wilson e587396e70 ansible: hook strategy and worker processes into profiler 6 years ago
David Wilson 5dc85c05f9 profiler: import from linear2 branch 6 years ago
David Wilson 960392cf50 core: tidy up existing profiling code and support MITOGEN_PROFILE_FMT 6 years ago
David Wilson 3606580f05 issue #260: redundant if statement. 6 years ago
David Wilson 84944a9a61 ansible: ensure MuxProcess MITOGEN_PROFILING results reach disk.
This has been broken for quite some time.
6 years ago
David Wilson 2209d880e4 ansible/bench: make end= configurable. 6 years ago
David Wilson b8a0e0f929 master: cache sent/forwarded module names
On 32x Docker run of issue_140__thread_pileup.yml

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   384807    3.595    0.000   12.207    0.000 /home/dmw/src/mitogen/mitogen/master.py:896(_forward_one_module)
  1218352    4.867    0.000    7.302    0.000 /home/dmw/src/mitogen/mitogen/master.py:853(_send_module_and_related)

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   384807    3.839    0.000    6.543    0.000 /home/dmw/src/mitogen/mitogen/master.py:902(_forward_one_module)
  1218352    0.723    0.000    0.898    0.000 /home/dmw/src/mitogen/mitogen/master.py:856(_send_module_and_related)
6 years ago
dw 9a41e17270
Merge pull request #530 from moreati/cover-us
Aggregate code coverage data across all tox runs
6 years ago
Alex Willmer 2b1035b500
Merge branch 'master' into cover-us 6 years ago
Alex Willmer 2fa84e096f Aggregate code coverage data across tox all runs
Fixes #529
6 years ago
David Wilson a6fd080b3a Merge remote-tracking branch 'origin/issue527'
* origin/issue527:
  docs: update Changelog; closes #527.
  issue #527: catch new-style module tracebacks like vanilla.
6 years ago
Alex Willmer e7d9ec86f9 Allow independant control of coverage erase and reporting 6 years ago
Alex Willmer 6f17ede28b Fix incorrect attempt to use coverage
Looks like this was a copy and paste mistake
6 years ago
David Wilson 8a2dcbf674 docs: update Changelog; closes #527. 6 years ago
David Wilson 954f874085 issue #527: catch new-style module tracebacks like vanilla. 6 years ago
dw e8e66d6fd0
Merge pull request #526 from moreati/py36-deprecationwarning
Py36 deprecationwarning
6 years ago
dw ea83a1b677
Merge pull request #525 from moreati/coverage_report_always
Generate coverage report even if some tests fail
6 years ago
Alex Willmer 0d725f9cd8 Fix DeprecationWarning in mitogen.utils.run_with_router()
```
14:20:32.186270 W py.warnings:
/home/alex/src/mitogen/mitogen/utils.py:152: DeprecationWarning: invalid
escape sequence \*
  """
```
6 years ago
Alex Willmer e7d70b109b Generate coverage report even if some tests fail
`set -o errexit` was exiting run_tests prematurely if any test failed,
so the coverage report was not always generated.
6 years ago
David Wilson d763570ca2 Merge remote-tracking branch 'origin/dmw'
* origin/dmw:
  ci: fix incorrect partition/rpartition from 8a4caea84f
  issue #260: hide force-disconnect messages.
  issue #498: fix shutdown crash
  issue #260: avoid start_transmit()/on_transmit()/stop_transmit()
  core: ensure broker profiling output reaches disk
  master: keep is_stdlib_path() result as negative cache entry
  ci: Allow DISTROS="debian*32" variable, and KEEP=1
6 years ago
David Wilson 49a8745a45 ci: fix incorrect partition/rpartition from 8a4caea84f 6 years ago
David Wilson 696cee57dd issue #260: hide force-disconnect messages.
Parent is always force-disconnected, and now it shows up quite visibly
on every exit.
6 years ago
David Wilson 26c6c6d048 issue #498: fix shutdown crash
Traceback (most recent call last):
      File "<stdin>", line 2707, in _invoke
      File "<stdin>", line 2480, in _on_del_route
    NameError: global name 'target_id' is not defined
6 years ago
David Wilson a18a083c94 issue #260: avoid start_transmit()/on_transmit()/stop_transmit()
Previous transmit sequence was:
        Router._async_route -> Stream._send -> Broker._start_transmit ->
        Broker.loop -> Stream.on_transmit -> socket.write ->
        Broker.stop_transmit

New sequence, when socket buffer can hold message is:
        Router._async_route -> Stream._send -> socket.write

bench/roundtrip.py
        Before: 240 usec
        after: 178 usec

Stat before:
       5088.276050      task-clock (msec)         #    0.997 CPUs utilized
           185,568      context-switches          #    0.036 M/sec
                 0      cpu-migrations            #    0.000 K/sec
            18,923      page-faults               #    0.004 M/sec
    13,063,871,501      cycles                    #    2.567 GHz
    12,834,579,684      instructions              #    0.98  insn per cycle
     2,669,820,684      branches                  #  524.700 M/sec
       107,296,033      branch-misses             #    4.02% of all branches

       5.105018296 seconds time elapsed

       2.350970000 seconds user
       0.345497000 seconds sys

Stat after:
       4019.208047      task-clock (msec)         #    0.998 CPUs utilized
           249,471      context-switches          #    0.062 M/sec
                 0      cpu-migrations            #    0.000 K/sec
            20,990      page-faults               #    0.005 M/sec
    10,312,535,979      cycles                    #    2.566 GHz
    11,586,365,995      instructions              #    1.12  insn per cycle
     2,392,933,370      branches                  #  595.374 M/sec
        75,432,205      branch-misses             #    3.15% of all branches

       4.028763347 seconds time elapsed

       3.367051000 seconds user
       0.652962000 seconds sys
6 years ago
David Wilson b161408598 Merge commit 'fe74577'
* commit 'fe74577':
  Use develop mode in tox
  issue #429: fix sudo regression.
  misc: rename to scripts. tab completion!!
  core: Latch._wake improvements
  issue #498: prevent crash on double 'disconnect' signal.
  issue #413: don't double-propagate DEL_ROUTE to parent.
  issue #498: wrap Router dict mutations in a lock
  issue #429: enable en_US locale to unbreak debops test.
  issue #499: fix another mind-numbingly stupid vanilla inconsistency
  issue #497: do our best to cope with crap upstream code
  ssh: fix test to match updated log format.
  issue #429: update Changelog.
  issue #429: update Changelog.
  issue #429: teach sudo about every know i18n password string.
  issue #429: install i18n-related bits in test images.
  ssh: tidy up logs and stream names.
  tests: ensure file is closed in connection_test.
  gcloud: small updates
  tests: give ansible/gcloud/ its own requirements file.
6 years ago
David Wilson d4a0b70e15 core: ensure broker profiling output reaches disk
Profiler hasn't been used much since the hard exit was added.
6 years ago
David Wilson dc4b27c6bf master: keep is_stdlib_path() result as negative cache entry
On 32x Docker run of issue_140__thread_pileup.yml

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1218500    1.716    0.000    7.325    0.000 /home/dmw/src/mitogen/mitogen/master.py:118(is_stdlib_path)

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      166    0.000    0.000    0.001    0.000 /home/dmw/src/mitogen/mitogen/master.py:123(is_stdlib_path)
6 years ago
David Wilson 8a4caea84f ci: Allow DISTROS="debian*32" variable, and KEEP=1 6 years ago
David Wilson fe745779ba Use develop mode in tox 6 years ago
David Wilson fca5513610 issue #429: fix sudo regression. 6 years ago
David Wilson ae8173a29f misc: rename to scripts. tab completion!! 6 years ago
David Wilson 0c0d34241b core: Latch._wake improvements
os.write() can fail with EINTR due to signals, so wrap it in
io_op(). Closes #483.

Masking EBADF looks like it is/was almost certainly papering over a bug,
remove it and suffer the bug reports. Closes #495.
6 years ago
David Wilson 407294cd79 issue #498: prevent crash on double 'disconnect' signal.
Fixes:

ERROR! [pid 1096] 23:31:48.363215 E mitogen: _broker_main() crashed
Traceback (most recent call last):
  File "/home/dmw/src/mitogen/mitogen/core.py", line 2917, in _broker_main
    self._loop_once()
  File "/home/dmw/src/mitogen/mitogen/core.py", line 2875, in _loop_once
    self._call(side.stream, func)
  File "/home/dmw/src/mitogen/mitogen/core.py", line 2860, in _call
    stream.on_disconnect(self)
  File "/home/dmw/src/mitogen/mitogen/parent.py", line 1161, in on_disconnect
    super(Stream, self).on_disconnect(broker)
  File "/home/dmw/src/mitogen/mitogen/core.py", line 1534, in on_disconnect
    fire(self, 'disconnect')
  File "/home/dmw/src/mitogen/mitogen/core.py", line 390, in fire
    func(*args, **kwargs)
  File "/home/dmw/src/mitogen/mitogen/parent.py", line 1794, in <lambda>
    func=lambda: self._on_stream_disconnect(stream),
  File "/home/dmw/src/mitogen/mitogen/parent.py", line 1810, in _on_stream_disconnect
    routes = self._routes_by_stream.pop(stream)
KeyError: mitogen.ssh.Stream('ssh.localhost:2236')
6 years ago
David Wilson 9aa845669c issue #413: don't double-propagate DEL_ROUTE to parent.
propagate_up() sends ADD_ROUTE and DEL_ROUTE

propagate_down() sends only DEL_ROUTE, but didn't bother checking if
up() had sent it already.

Fixes:

     ERROR! [pid 41060] 17:55:30.739159 E mitogen.ctx.ssh.localhost:
     mitogen: RouteMonitor(): received DEL_ROUTE for 6081 from
     mitogen.fork.Stream(u'fork.41142'), expected
     mitogen.core.Stream('parent')
6 years ago
David Wilson e2478dcb9f issue #498: wrap Router dict mutations in a lock 6 years ago
David Wilson 60fe3fd6f5 issue #429: enable en_US locale to unbreak debops test. 6 years ago
David Wilson 7531af3ee0 issue #499: fix another mind-numbingly stupid vanilla inconsistency 6 years ago
David Wilson a40946297f issue #497: do our best to cope with crap upstream code 6 years ago
David Wilson 2fdbd0cfcd ssh: fix test to match updated log format. 6 years ago
David Wilson 77e7cadd22 issue #429: update Changelog. 6 years ago
David Wilson cd1e5e0138 issue #429: update Changelog. 6 years ago
David Wilson 6e9f8e829e issue #429: teach sudo about every know i18n password string. 6 years ago
David Wilson 960e505f07 issue #429: install i18n-related bits in test images. 6 years ago
David Wilson ec789513dc ssh: tidy up logs and stream names. 6 years ago
David Wilson eb93f82d05 tests: ensure file is closed in connection_test. 6 years ago
David Wilson 73979043ad gcloud: small updates 6 years ago