strip_comments() currently ignores comments on lines 1 and 2, in order
to preserve lines such as
The comments test had normal comments on those lines, hence it was
failing.
This permits graceful shutdown of individual contexts, without tearing
down everything.
Update mitogen.parent.Stream to also wait for the child to exit, to
prevent the buildup of zombie processes. This introduces a blocking wait
for process exit on the Broker thread, let's see if we can get away with
it. Chances are reasonable that it'll cause needless hangs on heavily
loaded machines.
Full output of failed test
```
ERROR: test_okay (__main__.FakeSshTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/ssh_test.py", line 16, in test_okay
ssh_path=testlib.data_path('fakessh.py'),
File "/home/alex/src/mitogen/mitogen/master.py", line 650, in ssh
return self.connect('ssh', **kwargs)
File "/home/alex/src/mitogen/mitogen/parent.py", line 463, in connect
return self._connect(context_id, klass, name=name, **kwargs)
File "/home/alex/src/mitogen/mitogen/parent.py", line 449, in _connect
stream.connect()
File "/home/alex/src/mitogen/mitogen/ssh.py", line 104, in connect
super(Stream, self).connect()
File "/home/alex/src/mitogen/mitogen/parent.py", line 395, in connect
self._connect_bootstrap()
File "/home/alex/src/mitogen/mitogen/ssh.py", line 116, in
_connect_bootstrap
time.time() + 10.0):
File "/home/alex/src/mitogen/mitogen/parent.py", line 207, in
iter_read
(''.join(bits)[-300:],)
mitogen.core.StreamError: EOF on stream; last 300 bytes received:
'Usage: fakessh.py [options]\n\nfakessh.py: error: no such option: -o\n'
```
e.g. assert x == y -> self.assertEqual(x, y);
self.assertTrue(isinstance(x, y)) -> self.assertIsInstance(x, y)
These specific methods give more useful errors in the case of a test
failure.
Although these are synonyms in Python 2.x, when using MyPy to typecheck
code use of file() causes spurious errors.
This commit also serves as one small step to Python 3.x compatibility,
since 3.x removes the file() builtin.
On my laptop (Ubuntu 17.10, Python 2.7.14 in a virtualenv),
`test_regular_mod` fails with
```
AssertionError: "\nimport sys\n\n\ndef say_hi():\n print 'hi'\n" !=
'\x03\xf3\r\n\xbbW\xd5Yc\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00@\x00\x00\x00s\x19\x00\x00\x00d\x00\x00d\x01\x00l\x00\x00Z\x00\x00d\x02\x00\x84\x00\x00Z\x01\x00d\x01\x00S(\x03\x00\x00\x00i\xff\xff\xff\xffNc\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00C\x00\x00\x00s\t\x00\x00\x00d\x01\x00GHd\x00\x00S(\x02\x00\x00\x00Nt\x02\x00\x00\x00hi(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00sF\x00\x00\x00/home/alex/src/mitogen/tests/data/module_finder_testmod/regular_mod.pyt\x06\x00\x00\x00say_hi\x05\x00\x00\x00s\x02\x00\x00\x00\x00\x01(\x02\x00\x00\x00t\x03\x00\x00\x00sysR\x01\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00sF\x00\x00\x00/home/alex/src/mitogen/tests/data/module_finder_testmod/regular_mod.pyt\x08\x00\x00\x00<module>\x02\x00\x00\x00s\x02\x00\x00\x00\x0c\x03'
```
`__file__` contains the path of the compiled `.pyc`, not the `.py`
source file.
Ubuntu 17.04 provides Docker 1.12.6, which has API version 1.24.
`dev_requirements.txt` specifies the docker-py 2.5.1, which by default
requests API version 1.30.
Hence when the SSH unit tests try to run the container specified in
`DockerizedSshDaemon` an error occurs
```
APIError: 400 Client Error: Bad Request ("client is newer than server
(client API version: 1.30, server API version: 1.24)")
```
On Ubuntu 17.10 something (probably Docker) appears to be accepting
connections, before sshd is fully ready. This results in a race
condition, and hence connection errors for the first few tests (2-3 on
my laptop).
testlib.wait_for_port() checks not only that the port can be connected
to, but also something resembling the sshd banner is sent.
Fixes#51
Can't figure out what it's supposed to do any more, and can't find a
version of Ansible before August 2016 (when I wrote that code) that
seems to need it.
Add some more mitigations to avoid sending dylibs.
Now there is a separate SHUTDOWN message that relies only on being
received by the broker thread, the main thread can be hung horribly and
the process will still eventually receive a SIGTERM.
* Support passing Context() objects in function calls and return values.
Now the fakessh demo from the documentation index would work
correctly.
* Since slaves can communicate with each other now, they should also use
the same approach to unpickling as the master already used. Collapse
away all the unpickle extension crap and hard-wire just the 3 types
that support unpickling.
Alex Willmer