There has always been a race in PushFileService since given a parent
asked to forward modules to two children via some intermediary:
interm = router.local()
c1 = router.local(via=interm)
c2 = router.local(via=interm)
service.propagate_to(c1, 'foo/bar.py')
service.propagate_to(c2, 'foo/bar.py')
Two calls will be emitted to 'interm':
PushFileService.store_and_forward(c1, 'foo/bar.py', [blob])
PushFileService.store(c2, 'foo/bar.py')
Which will be processed in-order up to the point where service pool
threads in 'interm' are woken to process the message.
While it is guaranteed store_and_forward() will be processed first, no
guarantee existed that its assigned pool thread would wake and take
_lock first, thus it was possible for forward() to win the race, and for
a request to arrive to forward a file that had not been placed in local
cache yet.
Here we get rid of SerializedInvoker entirely, as it is partially to
blame for hiding the race: SerializedInvoker can only ensure no two
messages are processed simultaneously, it cannot ensure the messages are
processed in their intended order.
Instead, teach forward() that it may be called before
store_and_forward(), and if that is the case, to place the forward
request on to _waiters alongside any local threads blocked in get().
* origin/dmw:
docs: update Changelog.
tests/ansible: Spec.port() test & mitogen_via= fix.
Update copyright year everywhere.
tests/ansible: Spec.become_pass() test.
docs: remove top "Table of Contents" link
docs: remove a little more top margin wastage
tests/ansible: Spec.become_user() test.
docs: update Changelog; closes#539.
issue #539: disable logger propagation.
* origin/dmw:
docs: update Changelog.
issue #536: rework how 2.3-compatible simplejson is served
.github: add some more questions to issue template
docs: duplicate word
docs: update Changelog.
tests/ansible: Spec.become_method() test & mitogen_via= fix.
setup.py: include LICENSE; closes#538.
tests/ansible: Spec.become() test
tests/ansible: Spec.password() test, document interactive pw limitation.
tests/ansible: Spec.remote_user() test & mitogen_via= fix.
tests/ansible: Spec.remote_addr() test & mitogen_via= fix.
tests/ansible: Spec.transport() test.
docs: lighter pink
docs: add 'Fixes' heading
docs: more margin tweaks for changelog
docs: tighter <p> margins, even less shouting, red headings
docs: tidy up footer and GitHub link
docs: enable fixed_sidebar
docs: sans-serif fonts, reduce shouty headings
issue #536: add mitogen_via= tests too.
ansible: fix a crash on 2.3 when mitogen_via= host is missing.
tests: for 2.3 compatibility, disable gcloud.py for now
Regardless of the version of simplejson loaded in the master, load up
the ModuleResponder cache with our 2.4-compatible version.
To cope with simplejson being loaded due to modules like ec2_group that
try to import it before importing 'json', also update target.py to
remove it from the whitelist if a local 'json' module import succeeds.
This was needed at some point in the past, but the tests don't seem to
care about it any more. We'll fix any CI breakage by changing the tests,
since verifying implicit localhost behaviour is important.
David Wilson