os.write() can fail with EINTR due to signals, so wrap it in
io_op(). Closes#483.
Masking EBADF looks like it is/was almost certainly papering over a bug,
remove it and suffer the bug reports. Closes#495.
Fixes:
ERROR! [pid 1096] 23:31:48.363215 E mitogen: _broker_main() crashed
Traceback (most recent call last):
File "/home/dmw/src/mitogen/mitogen/core.py", line 2917, in _broker_main
self._loop_once()
File "/home/dmw/src/mitogen/mitogen/core.py", line 2875, in _loop_once
self._call(side.stream, func)
File "/home/dmw/src/mitogen/mitogen/core.py", line 2860, in _call
stream.on_disconnect(self)
File "/home/dmw/src/mitogen/mitogen/parent.py", line 1161, in on_disconnect
super(Stream, self).on_disconnect(broker)
File "/home/dmw/src/mitogen/mitogen/core.py", line 1534, in on_disconnect
fire(self, 'disconnect')
File "/home/dmw/src/mitogen/mitogen/core.py", line 390, in fire
func(*args, **kwargs)
File "/home/dmw/src/mitogen/mitogen/parent.py", line 1794, in <lambda>
func=lambda: self._on_stream_disconnect(stream),
File "/home/dmw/src/mitogen/mitogen/parent.py", line 1810, in _on_stream_disconnect
routes = self._routes_by_stream.pop(stream)
KeyError: mitogen.ssh.Stream('ssh.localhost:2236')
propagate_up() sends ADD_ROUTE and DEL_ROUTE
propagate_down() sends only DEL_ROUTE, but didn't bother checking if
up() had sent it already.
Fixes:
ERROR! [pid 41060] 17:55:30.739159 E mitogen.ctx.ssh.localhost:
mitogen: RouteMonitor(): received DEL_ROUTE for 6081 from
mitogen.fork.Stream(u'fork.41142'), expected
mitogen.core.Stream('parent')
os._exit() subverted calm shutdown, meaning unix.Listener never had a
chance to cleanup its socket.
Move unix.Listener socket cleanup into its class so it is automatic
during shutdown, rather than cutpasted for each consumer.
Disable the watcher thread in the MuxProcess, it is useless.
Add .sock extension to /tmp/mitogen_unix_*, so we can write a test.
* origin/dmw: (135 commits)
tests: just disable the test.
tests: hopefully fix this dumb test for the final time
docs: update Changelog; closes#477.
issue #477: use MITOGEN_INVENTORY_FILE everywhere.
issue #477: hacksmash weird 2.3 inventory_file var issue.
issue #477: travis.yml typo.
issue #477: fix sudo_args selection.
issue #477: one more conditional test.
issue #477: enable Ansible 2.3.3 CI.
issue #477: some more conditional tests.
docs: update Changelog.
issue #477 / ansible: avoid a race in async job startup.
issue #477: use assert_equal for nicer debug.
issue #477: fix source of become_flags on 2.3.
issue #477: add Connection.homedir test.
core: docstring tidyups.
core: ensure early debug messages are logged correctly.
core: log disconnection reason.
issue #477: target.file_exists() wrapper.
issue #477: introduce subprocess isolation.
ansible: docstring fixes.
issue #477: paper over Ansible 2.3 flag handling difference
issue #477: update forking_correct_parent for subprocess isolation
issue #477: shlex.split() in 2.4 required bytes input.
issue #477: get rid of perl JSON module requirement.
issue #477: Ansible 2.3 did not support gather_facts min subset.
issue #477: CentOS 5 image requires perl installed too.
issue #477: missing stub-su.py from 137f5fa6c5
issue #477: 2.4-compatible syntax.
issue #477: clearing glibc caches is not possible on Py2.4.
parent: --with-pydebug bootstrap could fail due to corrupted stream
issue #477: install simplejson for vanilla tests.
docs: update Changelog.
ansible: synchronize module needs '.docker_cmd' attr for Docker plugin.
issue #477: add basic su_test and Py2.4 polyfill.
issue #477: import updated Python build scripts
ci: don't use the TTY->pipe hack except on Travis where it's needed.
WIP first run of py24 CI
issue #477: initial Python 2.4.6 build for CI.
issue #477: enable git-lfs for tests/data/*.tar.bz2.
issue #477: import build script for Python 2.4.6.
issue #477: add mitogen_py24 CI test type.
issue #477: disable Django parts of module_finder_test on 2.4.
issue #477: clean up globals after unix_test.
issue #477: remove unused pytest bits from importer_test.
issue #477: remove fork use from unix_test.
parent: don't kill child when profiling=True
issue #485: import new throuhgput bench
issue #477: more fork removal
issue #477: Py2.4 startswith() did not support tuples.
issue #477: util/fakessh/two_three_compat fixes.
issue #477: call_function_test fixes for 2.4.
issue #477: promote setup_gil() to mitogen.utils
issue #477: fix lxc_test any polyfill import.
issue #477: stop using fork in responder_test.
issue #477: stop using fork in service_test.
issue #477: Python<2.5 ioctl() request parameter was signed.
issue #477: stop using fork() in parent_test, compatible enumerate().
issue #477: Popen.terminate() polyfill for Py2.4.
issue #477: stop using .fork() in router_test, one small 2.4 fix.
issue #477: document master.Router.max_message_size.
issue #477: old Py zlib did not include extended exception text.
issue #477: stop using router.fork() in receiver_test
issue #477: any() polyfill for lxc_test.
issue #477: replace type(e) -> __class__ for an exception
issue #477: old Mock does not throw side_effect exceptions from a list
issue #477: 2.4 stat() returned int timestamps not float.
issue #477: set().union(a, b, ..) unsupported on Py2.4.
issue #477: Logger.log(extra=) unsupported on Py2.4.
issue #477: fix another Threading.getName() call.
issue #477: %f date format requires Py2.6 or newer.
issue #477: make mitogen.fork unsupported on Py<2.6.
issue #477: Py2.4 dep scanner bytecode difference
Drop 'alpha' trove classifier
issue #477: fix another str/bytes mixup.
issue #477: blacklist 'thread' module to avoid roundtrip on 2.x->3.x
issue #477: fix 3.x failure in new target.set_file_mode() function.
issue #477: fix 3.x failure in new target.set_file_mode() function.
issue #477: fix 2 runner tests on Ansible 2.7.
issue #477: fix 3.x test regressions.
issue #477: fix new KwargsTest on Python 3.x.
issue #477: ModuleFinder now returns Unicode module names.
issue #477: Python3 does not have Pickler.dispatch.
issue #477: ModuleFinder test fixes.
issue #477: Ansible 2.3 compatible regression/all.yml.
issue #477: Ansible 2.3 requires placeholder module for assert_equals
issue #477: build a CentOS 5/Py2.4 container + playbook compat fixes.
issue #477: use PY24 constant rather than explicit test.
issue #477: backport mitogen.master to Python 2.4.
issue #477: parent: make iter_read() log disconnect reason.
issue #477: backport ansible_mitogen.runner to 2.4.
issue #477: backport various test modules to Python 2.4.
issue #477: backport ansible_mitogen/target.py to Python2.4
issue #477: add all() polyfill to custom_python_detect_environmnet
issue #477: polyfill partition() use in mitogen.parent.
issue #477: polyfill partition() use in mitogen.service.
issue #477: polyfill partition() use in mitogen.ssh.
issue #477: vendorize the last 2.4-compatible simplejson
issue #477: _update_linecache() must append newlines.
issue #415, #477: Poller must handle POLLHUP too.
...
Ansible 2.3/Python 2.4 work revealed there is no guarantee a slow target
will have written the initial job status file out before a fast
controller makes an initial check for it. Therefore, provide AsyncRunner
with a sender it should send a message to when the initial job file has
been written.
As a bonus, also catch and report exceptions happening early in
AsyncRunner, rather than leaving them to end up in -vvv output.
David Wilson