Merge remote-tracking branch 'origin/stream-refactor'
* origin/stream-refactor: [stream-refactor] Py3.x test fixes [stream-refactor] mark py24 as allow-fail [stream-refactor] Debian Docker container image initctl [stream-refactor] replace cutpaste with Stream.accept() in mitogen.unix [stream-refactor] fix flake8 errors [stream-refactor] fix testlib assertion format string [stream-refactor] make mitogen-fuse work on Linux [stream-refactor] repair preamble_size.py again [stream-refactor] don't abort Connection until all buffers are empty Normalize docstring formatting [stream-refactor] fix LogHandler.uncork() race [stream-refactor] BufferedWriter must disconenct Stream, not Protocol [stream-refactor] statically link doas binary using musl [stream-refactor] stop writing to /tmp/foo in fd_check.py. [stream-refactor] yet another 2.4 issue in create_child_test [stream-refactor] fix Py2.4 failure by implementing missing Timer method [stream-refactor] allow up to 30 seconds to connect in unix_test [stream-refactor] mark setns module as requiring Python >2.4 [stream-refactor] another 2.4 fix for create_child_test .travis.yml: Add reverse shell spawn for Travis too core: better Side attribute docstrings [stream-refactor] remove one more getuser() usage [stream-refactor] allow doas_test to succeed on CentOS Pin idna==2.7 when running on Python<2.7. [stream-refactor] Py2.4 compat fix for iter_split_test. [stream-refactor] add descriptive task names to _container_prep [stream-refactor] 3.x socket.send() requires bytes [stream-refactor] fix 2.4 syntax error. [stream-refactor] avoid os.wait3() for Py2.4. Allow specifying -vvv to debops_tests. [stream-refactor] send MITO002 earlier module_finder: pass raw file to compile() [stream-refactor] merge stdout+stderr when reporting EofError [stream-refactor] fix crash in detach() / during async/multiple_items_loop.yml [stream-refactor] fix crash in runner/forking_active.yml [stream-refactor] replace old detach_popen() reference ansible: fixturize creation of MuxProcess unix: ensure mitogen.context_id is reset when client disconnects [stream-refactor] make syntax 2.4 compatible [stream-refactor] make trusty our Travis dist. [stream-refactor] fix su_test failure (issue #363) [stream-refactor] more readable log string format [stream-refactor] dont doubly log last partial line [stream-refactor] import fd_check.py used by create_child_test [stream-refactor] port mitogen.buildah, added to master since work began [stream-refactor] fix unix.Listener construction [stream-refactor] fix crash when no stderr present. [stream-refactor] fix Process constructor invocation Add tests/ansible/.*.pid to gitignore (for ansible_mitogen/process.py) Add extra/ to gitignore import release-notes script. [stream-refactor] repaired rest of create_child_test. [stream-refactor] rename Process attrs, fix up more create_child_test [stream-refactor] import incomplete create_child_test issue #482: tests: check for zombie process after test. issue #363: add test. tests: clean up old-style SSH exception catch issue #271: add mitogen__permdenied user to Docker image. ssh: fix issue #271 regression due to refactor, add test. Refactor Stream, introduce quasi-asynchronous connect, much more core: teach iter_split() to break on callback returning False. issue #507: log fatal errors to syslog. testlib: have LogCapturer.raw() return unicode on 2.x. core/master: docstring, repr, and debug log message cleanups parent: remove unused Timer parameter. tests: jail_test fixes. parent: docstring improvements, cfmakeraw() regression. core: introduce Protocol, DelimitedProtocol and BufferedWriter. core: introduce mitogen.core.pipe() tests/bench: import ssh-roundtrip.py. tests: note location of related tests. tests: add real test for doas. tests: install OpenBSD doas port in Debian image. tests: add setns_test that works if password localhost sudo works. Import minimal jail_test. core: move message encoding to Message.pack(), add+refactor tests. master: expect forwarded logs to be in UTF-8. tests: add some UTF-8 to ssh_login_banner to encourage breakage. core: bootstrap FD management improvements core: pending timers should keep broker alive. core: more succinct iter_split(). core: replace UTF8_CODEC with encodings.utf_8.encode() function. docs: remove bytearray from supported types list. core: docstring style cleanups, dead code. testlib: disable lsof warnings due to Docker crap parent: discard cancelled events in TimerList.get_timeout(). core: split out iter_split() for use in parent.py. parent: various style cleanups, remove unused function. issue #170: add TimerList docstrings. core: eliminate some quadratric behaviour from IoLogger issue #170: update Changelog; closes #170. issue #170: add timers to internals.rst. issue #170: implement timers.pull/607/head
David Wilson
5 years ago
commit
a5619a62bf
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,35 @@
|
||||
"""
|
||||
Measure latency of SSH RPC.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import time
|
||||
|
||||
import mitogen
|
||||
import mitogen.utils
|
||||
import ansible_mitogen.affinity
|
||||
|
||||
mitogen.utils.setup_gil()
|
||||
ansible_mitogen.affinity.policy.assign_worker()
|
||||
|
||||
try:
|
||||
xrange
|
||||
except NameError:
|
||||
xrange = range
|
||||
|
||||
def do_nothing():
|
||||
pass
|
||||
|
||||
@mitogen.main()
|
||||
def main(router):
|
||||
f = router.ssh(hostname=sys.argv[1])
|
||||
f.call(do_nothing)
|
||||
t0 = time.time()
|
||||
end = time.time() + 5.0
|
||||
i = 0
|
||||
while time.time() < end:
|
||||
f.call(do_nothing)
|
||||
i += 1
|
||||
t1 = time.time()
|
||||
|
||||
print('++', float(1e3 * (t1 - t0) / (1.0+i)), 'ms')
|
||||
@ -0,0 +1,284 @@
|
||||
|
||||
import fcntl
|
||||
import os
|
||||
import stat
|
||||
import sys
|
||||
import time
|
||||
import tempfile
|
||||
|
||||
import mock
|
||||
import unittest2
|
||||
|
||||
import mitogen.parent
|
||||
from mitogen.core import b
|
||||
|
||||
import testlib
|
||||
|
||||
|
||||
def run_fd_check(func, fd, mode, on_start=None):
|
||||
tf = tempfile.NamedTemporaryFile()
|
||||
args = [
|
||||
sys.executable,
|
||||
testlib.data_path('fd_check.py'),
|
||||
tf.name,
|
||||
str(fd),
|
||||
mode,
|
||||
]
|
||||
|
||||
proc = func(args=args)
|
||||
os = None
|
||||
if on_start:
|
||||
os = on_start(proc)
|
||||
proc.proc.wait()
|
||||
try:
|
||||
return proc, eval(tf.read()), os
|
||||
finally:
|
||||
tf.close()
|
||||
|
||||
|
||||
def close_proc(proc):
|
||||
proc.stdin.close()
|
||||
proc.stdout.close()
|
||||
if proc.stderr:
|
||||
prco.stderr.close()
|
||||
|
||||
|
||||
def wait_read(fp, n):
|
||||
poller = mitogen.core.Poller()
|
||||
try:
|
||||
poller.start_receive(fp.fileno())
|
||||
for _ in poller.poll():
|
||||
return os.read(fp.fileno(), n)
|
||||
assert False
|
||||
finally:
|
||||
poller.close()
|
||||
|
||||
|
||||
class StdinSockMixin(object):
|
||||
def test_stdin(self):
|
||||
proc, info, _ = run_fd_check(self.func, 0, 'read',
|
||||
lambda proc: proc.stdin.send(b('TEST')))
|
||||
st = os.fstat(proc.stdin.fileno())
|
||||
self.assertTrue(stat.S_ISSOCK(st.st_mode))
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
flags = fcntl.fcntl(proc.stdin.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(info['buf'], 'TEST')
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
|
||||
class StdoutSockMixin(object):
|
||||
def test_stdout(self):
|
||||
proc, info, buf = run_fd_check(self.func, 1, 'write',
|
||||
lambda proc: wait_read(proc.stdout, 4))
|
||||
st = os.fstat(proc.stdout.fileno())
|
||||
self.assertTrue(stat.S_ISSOCK(st.st_mode))
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
flags = fcntl.fcntl(proc.stdout.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
|
||||
class CreateChildTest(StdinSockMixin, StdoutSockMixin, testlib.TestCase):
|
||||
func = staticmethod(mitogen.parent.create_child)
|
||||
|
||||
def test_stderr(self):
|
||||
proc, info, _ = run_fd_check(self.func, 2, 'write')
|
||||
st = os.fstat(sys.stderr.fileno())
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
self.assertEquals(st.st_ino, info['st_ino'])
|
||||
|
||||
|
||||
class CreateChildMergedTest(StdinSockMixin, StdoutSockMixin,
|
||||
testlib.TestCase):
|
||||
def func(self, *args, **kwargs):
|
||||
kwargs['merge_stdio'] = True
|
||||
return mitogen.parent.create_child(*args, **kwargs)
|
||||
|
||||
def test_stderr(self):
|
||||
proc, info, buf = run_fd_check(self.func, 2, 'write',
|
||||
lambda proc: wait_read(proc.stdout, 4))
|
||||
self.assertEquals(None, proc.stderr)
|
||||
st = os.fstat(proc.stdout.fileno())
|
||||
self.assertTrue(stat.S_ISSOCK(st.st_mode))
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
flags = fcntl.fcntl(proc.stdout.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
|
||||
class CreateChildStderrPipeTest(StdinSockMixin, StdoutSockMixin,
|
||||
testlib.TestCase):
|
||||
def func(self, *args, **kwargs):
|
||||
kwargs['stderr_pipe'] = True
|
||||
return mitogen.parent.create_child(*args, **kwargs)
|
||||
|
||||
def test_stderr(self):
|
||||
proc, info, buf = run_fd_check(self.func, 2, 'write',
|
||||
lambda proc: wait_read(proc.stderr, 4))
|
||||
st = os.fstat(proc.stderr.fileno())
|
||||
self.assertTrue(stat.S_ISFIFO(st.st_mode))
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
flags = fcntl.fcntl(proc.stderr.fileno(), fcntl.F_GETFL)
|
||||
self.assertFalse(flags & os.O_WRONLY)
|
||||
self.assertFalse(flags & os.O_RDWR)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
self.assertTrue(info['flags'] & os.O_WRONLY)
|
||||
|
||||
|
||||
class TtyCreateChildTest(testlib.TestCase):
|
||||
func = staticmethod(mitogen.parent.tty_create_child)
|
||||
|
||||
def test_stdin(self):
|
||||
proc, info, _ = run_fd_check(self.func, 0, 'read',
|
||||
lambda proc: proc.stdin.write(b('TEST')))
|
||||
st = os.fstat(proc.stdin.fileno())
|
||||
self.assertTrue(stat.S_ISCHR(st.st_mode))
|
||||
self.assertTrue(stat.S_ISCHR(info['st_mode']))
|
||||
|
||||
self.assertTrue(isinstance(info['ttyname'],
|
||||
mitogen.core.UnicodeType))
|
||||
os.ttyname(proc.stdin.fileno()) # crashes if not TTY
|
||||
|
||||
flags = fcntl.fcntl(proc.stdin.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
self.assertNotEquals(st.st_dev, info['st_dev'])
|
||||
self.assertTrue(info['buf'], 'TEST')
|
||||
|
||||
def test_stdout(self):
|
||||
proc, info, buf = run_fd_check(self.func, 1, 'write',
|
||||
lambda proc: wait_read(proc.stdout, 4))
|
||||
|
||||
st = os.fstat(proc.stdout.fileno())
|
||||
self.assertTrue(stat.S_ISCHR(st.st_mode))
|
||||
self.assertTrue(stat.S_ISCHR(info['st_mode']))
|
||||
|
||||
self.assertTrue(isinstance(info['ttyname'],
|
||||
mitogen.core.UnicodeType))
|
||||
os.ttyname(proc.stdout.fileno()) # crashes if wrong
|
||||
|
||||
flags = fcntl.fcntl(proc.stdout.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
self.assertNotEquals(st.st_dev, info['st_dev'])
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
|
||||
def test_stderr(self):
|
||||
proc, info, buf = run_fd_check(self.func, 2, 'write',
|
||||
lambda proc: wait_read(proc.stdout, 4))
|
||||
|
||||
st = os.fstat(proc.stdout.fileno())
|
||||
self.assertTrue(stat.S_ISCHR(st.st_mode))
|
||||
self.assertTrue(stat.S_ISCHR(info['st_mode']))
|
||||
|
||||
self.assertTrue(isinstance(info['ttyname'],
|
||||
mitogen.core.UnicodeType))
|
||||
os.ttyname(proc.stdin.fileno()) # crashes if not TTY
|
||||
|
||||
flags = fcntl.fcntl(proc.stdout.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
self.assertNotEquals(st.st_dev, info['st_dev'])
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
|
||||
def test_dev_tty_open_succeeds(self):
|
||||
# In the early days of UNIX, a process that lacked a controlling TTY
|
||||
# would acquire one simply by opening an existing TTY. Linux and OS X
|
||||
# continue to follow this behaviour, however at least FreeBSD moved to
|
||||
# requiring an explicit ioctl(). Linux supports it, but we don't yet
|
||||
# use it there and anyway the behaviour will never change, so no point
|
||||
# in fixing things that aren't broken. Below we test that
|
||||
# getpass-loving apps like sudo and ssh get our slave PTY when they
|
||||
# attempt to open /dev/tty, which is what they both do on attempting to
|
||||
# read a password.
|
||||
tf = tempfile.NamedTemporaryFile()
|
||||
try:
|
||||
proc = self.func([
|
||||
'bash', '-c', 'exec 2>%s; echo hi > /dev/tty' % (tf.name,)
|
||||
])
|
||||
deadline = time.time() + 5.0
|
||||
self.assertEquals(mitogen.core.b('hi\n'), wait_read(proc.stdout, 3))
|
||||
waited_pid, status = os.waitpid(proc.pid, 0)
|
||||
self.assertEquals(proc.pid, waited_pid)
|
||||
self.assertEquals(0, status)
|
||||
self.assertEquals(mitogen.core.b(''), tf.read())
|
||||
proc.stdout.close()
|
||||
finally:
|
||||
tf.close()
|
||||
|
||||
|
||||
class StderrDiagTtyMixin(object):
|
||||
def test_stderr(self):
|
||||
proc, info, buf = run_fd_check(self.func, 2, 'write',
|
||||
lambda proc: wait_read(proc.stderr, 4))
|
||||
|
||||
st = os.fstat(proc.stderr.fileno())
|
||||
self.assertTrue(stat.S_ISCHR(st.st_mode))
|
||||
self.assertTrue(stat.S_ISCHR(info['st_mode']))
|
||||
|
||||
self.assertTrue(isinstance(info['ttyname'],
|
||||
mitogen.core.UnicodeType))
|
||||
os.ttyname(proc.stderr.fileno()) # crashes if wrong
|
||||
|
||||
flags = fcntl.fcntl(proc.stderr.fileno(), fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(info['flags'] & os.O_RDWR)
|
||||
|
||||
self.assertNotEquals(st.st_dev, info['st_dev'])
|
||||
self.assertTrue(flags & os.O_RDWR)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
|
||||
|
||||
class HybridTtyCreateChildTest(StdinSockMixin, StdoutSockMixin,
|
||||
StderrDiagTtyMixin, testlib.TestCase):
|
||||
func = staticmethod(mitogen.parent.hybrid_tty_create_child)
|
||||
|
||||
|
||||
|
||||
if 0:
|
||||
# issue #410
|
||||
class SelinuxHybridTtyCreateChildTest(StderrDiagTtyMixin, testlib.TestCase):
|
||||
func = staticmethod(mitogen.parent.selinux_hybrid_tty_create_child)
|
||||
|
||||
def test_stdin(self):
|
||||
proc, info, buf = run_fd_check(self.func, 0, 'read',
|
||||
lambda proc: proc.transmit_side.write('TEST'))
|
||||
st = os.fstat(proc.transmit_side.fd)
|
||||
self.assertTrue(stat.S_ISFIFO(st.st_mode))
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
flags = fcntl.fcntl(proc.transmit_side.fd, fcntl.F_GETFL)
|
||||
self.assertTrue(flags & os.O_WRONLY)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
self.assertFalse(info['flags'] & os.O_WRONLY)
|
||||
self.assertFalse(info['flags'] & os.O_RDWR)
|
||||
|
||||
def test_stdout(self):
|
||||
proc, info, buf = run_fd_check(self.func, 1, 'write',
|
||||
lambda proc: wait_read(proc.receive_side, 4))
|
||||
st = os.fstat(proc.receive_side.fd)
|
||||
self.assertTrue(stat.S_ISFIFO(st.st_mode))
|
||||
self.assertEquals(st.st_dev, info['st_dev'])
|
||||
self.assertEquals(st.st_mode, info['st_mode'])
|
||||
flags = fcntl.fcntl(proc.receive_side.fd, fcntl.F_GETFL)
|
||||
self.assertFalse(flags & os.O_WRONLY)
|
||||
self.assertFalse(flags & os.O_RDWR)
|
||||
self.assertTrue(info['flags'] & os.O_WRONLY)
|
||||
self.assertTrue(buf, 'TEST')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -0,0 +1,7 @@
|
||||
|
||||
|
||||
# doas-debian.tar.gz
|
||||
|
||||
A dynamically linked copy of the OpenBSD ``doas`` tool for Debian, port is from
|
||||
https://github.com/multiplexd/doas (the slicer69 port is broken, it reads the
|
||||
password from stdin).
|
||||
Binary file not shown.
@ -0,0 +1,4 @@
|
||||
|
||||
mkdir -p bad
|
||||
chmod 0 bad
|
||||
cd bad
|
||||
@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import fcntl
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def ttyname(fd):
|
||||
try:
|
||||
t = os.ttyname(fd)
|
||||
if hasattr(t, 'decode'):
|
||||
t = t.decode()
|
||||
return t
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
|
||||
def controlling_tty():
|
||||
try:
|
||||
fp = open('/dev/tty')
|
||||
try:
|
||||
return ttyname(fp.fileno())
|
||||
finally:
|
||||
fp.close()
|
||||
except (IOError, OSError):
|
||||
return None
|
||||
|
||||
|
||||
fd = int(sys.argv[2])
|
||||
st = os.fstat(fd)
|
||||
|
||||
if sys.argv[3] == 'write':
|
||||
os.write(fd, u'TEST'.encode())
|
||||
buf = u''
|
||||
else:
|
||||
buf = os.read(fd, 4).decode()
|
||||
|
||||
open(sys.argv[1], 'w').write(repr({
|
||||
'buf': buf,
|
||||
'flags': fcntl.fcntl(fd, fcntl.F_GETFL),
|
||||
'st_mode': st.st_mode,
|
||||
'st_dev': st.st_dev,
|
||||
'st_ino': st.st_ino,
|
||||
'ttyname': ttyname(fd),
|
||||
'controlling_tty': controlling_tty(),
|
||||
}))
|
||||
@ -1,13 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# I produce text every 100ms, for testing mitogen.core.iter_read()
|
||||
|
||||
import sys
|
||||
import time
|
||||
|
||||
|
||||
i = 0
|
||||
while True:
|
||||
i += 1
|
||||
sys.stdout.write(str(i))
|
||||
sys.stdout.flush()
|
||||
time.sleep(0.1)
|
||||
@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
os.environ['ORIGINAL_ARGV'] = json.dumps(sys.argv)
|
||||
os.environ['THIS_IS_STUB_JEXEC'] = '1'
|
||||
|
||||
# This must be a child process and not exec() since Mitogen replaces its stderr
|
||||
# descriptor, causing the last user of the slave PTY to close it, resulting in
|
||||
# the master side indicating EIO.
|
||||
subprocess.call(sys.argv[sys.argv.index('somejail') + 1:])
|
||||
os._exit(0)
|
||||
@ -1,9 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# I consume 65535 bytes every 10ms, for testing mitogen.core.write_all()
|
||||
|
||||
import os
|
||||
import time
|
||||
|
||||
while True:
|
||||
os.read(0, 65535)
|
||||
time.sleep(0.01)
|
||||
@ -0,0 +1,66 @@
|
||||
|
||||
import mock
|
||||
import unittest2
|
||||
|
||||
import mitogen.core
|
||||
|
||||
import testlib
|
||||
|
||||
try:
|
||||
next
|
||||
except NameError:
|
||||
def next(it):
|
||||
return it.next()
|
||||
|
||||
|
||||
class IterSplitTest(unittest2.TestCase):
|
||||
func = staticmethod(mitogen.core.iter_split)
|
||||
|
||||
def test_empty_buffer(self):
|
||||
lst = []
|
||||
trailer, cont = self.func(buf='', delim='\n', func=lst.append)
|
||||
self.assertTrue(cont)
|
||||
self.assertEquals('', trailer)
|
||||
self.assertEquals([], lst)
|
||||
|
||||
def test_empty_line(self):
|
||||
lst = []
|
||||
trailer, cont = self.func(buf='\n', delim='\n', func=lst.append)
|
||||
self.assertTrue(cont)
|
||||
self.assertEquals('', trailer)
|
||||
self.assertEquals([''], lst)
|
||||
|
||||
def test_one_line(self):
|
||||
buf = 'xxxx\n'
|
||||
lst = []
|
||||
trailer, cont = self.func(buf=buf, delim='\n', func=lst.append)
|
||||
self.assertTrue(cont)
|
||||
self.assertEquals('', trailer)
|
||||
self.assertEquals(lst, ['xxxx'])
|
||||
|
||||
def test_one_incomplete(self):
|
||||
buf = 'xxxx\nyy'
|
||||
lst = []
|
||||
trailer, cont = self.func(buf=buf, delim='\n', func=lst.append)
|
||||
self.assertTrue(cont)
|
||||
self.assertEquals('yy', trailer)
|
||||
self.assertEquals(lst, ['xxxx'])
|
||||
|
||||
def test_returns_false_immediately(self):
|
||||
buf = 'xxxx\nyy'
|
||||
func = lambda buf: False
|
||||
trailer, cont = self.func(buf=buf, delim='\n', func=func)
|
||||
self.assertFalse(cont)
|
||||
self.assertEquals('yy', trailer)
|
||||
|
||||
def test_returns_false_second_call(self):
|
||||
buf = 'xxxx\nyy\nzz'
|
||||
it = iter([True, False])
|
||||
func = lambda buf: next(it)
|
||||
trailer, cont = self.func(buf=buf, delim='\n', func=func)
|
||||
self.assertFalse(cont)
|
||||
self.assertEquals('zz', trailer)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -0,0 +1,33 @@
|
||||
|
||||
import os
|
||||
|
||||
import mitogen
|
||||
import mitogen.parent
|
||||
|
||||
import unittest2
|
||||
|
||||
import testlib
|
||||
|
||||
|
||||
class ConstructorTest(testlib.RouterMixin, testlib.TestCase):
|
||||
jexec_path = testlib.data_path('stubs/stub-jexec.py')
|
||||
|
||||
def test_okay(self):
|
||||
context = self.router.jail(
|
||||
jexec_path=self.jexec_path,
|
||||
container='somejail',
|
||||
)
|
||||
stream = self.router.stream_by_id(context.context_id)
|
||||
|
||||
argv = eval(context.call(os.getenv, 'ORIGINAL_ARGV'))
|
||||
self.assertEquals(argv[:4], [
|
||||
self.jexec_path,
|
||||
'somejail',
|
||||
stream.conn.options.python_path,
|
||||
'-c',
|
||||
])
|
||||
self.assertEquals('1', context.call(os.getenv, 'THIS_IS_STUB_JEXEC'))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -0,0 +1,545 @@
|
||||
|
||||
import sys
|
||||
import struct
|
||||
|
||||
import mock
|
||||
import unittest2
|
||||
|
||||
import mitogen.core
|
||||
import mitogen.master
|
||||
import testlib
|
||||
|
||||
from mitogen.core import b
|
||||
|
||||
|
||||
class ConstructorTest(testlib.TestCase):
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_dst_id_default(self):
|
||||
self.assertEquals(self.klass().dst_id, None)
|
||||
|
||||
def test_dst_id_explicit(self):
|
||||
self.assertEquals(self.klass(dst_id=1111).dst_id, 1111)
|
||||
|
||||
@mock.patch('mitogen.context_id', 1234)
|
||||
def test_src_id_default(self):
|
||||
self.assertEquals(self.klass().src_id, 1234)
|
||||
|
||||
def test_src_id_explicit(self):
|
||||
self.assertEquals(self.klass(src_id=4321).src_id, 4321)
|
||||
|
||||
@mock.patch('mitogen.context_id', 5555)
|
||||
def test_auth_id_default(self):
|
||||
self.assertEquals(self.klass().auth_id, 5555)
|
||||
|
||||
def test_auth_id_explicit(self):
|
||||
self.assertEquals(self.klass(auth_id=2222).auth_id, 2222)
|
||||
|
||||
def test_handle_default(self):
|
||||
self.assertEquals(self.klass().handle, None)
|
||||
|
||||
def test_handle_explicit(self):
|
||||
self.assertEquals(self.klass(handle=1234).handle, 1234)
|
||||
|
||||
def test_reply_to_default(self):
|
||||
self.assertEquals(self.klass().reply_to, None)
|
||||
|
||||
def test_reply_to_explicit(self):
|
||||
self.assertEquals(self.klass(reply_to=8888).reply_to, 8888)
|
||||
|
||||
def test_data_default(self):
|
||||
m = self.klass()
|
||||
self.assertEquals(m.data, b(''))
|
||||
self.assertTrue(isinstance(m.data, mitogen.core.BytesType))
|
||||
|
||||
def test_data_explicit(self):
|
||||
m = self.klass(data=b('asdf'))
|
||||
self.assertEquals(m.data, b('asdf'))
|
||||
self.assertTrue(isinstance(m.data, mitogen.core.BytesType))
|
||||
|
||||
def test_data_hates_unicode(self):
|
||||
self.assertRaises(Exception,
|
||||
lambda: self.klass(data=u'asdf'))
|
||||
|
||||
|
||||
class PackTest(testlib.TestCase):
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_header_format_sanity(self):
|
||||
self.assertEquals(self.klass.HEADER_LEN,
|
||||
struct.calcsize(self.klass.HEADER_FMT))
|
||||
|
||||
def test_header_length_correct(self):
|
||||
s = self.klass(dst_id=123, handle=123).pack()
|
||||
self.assertEquals(len(s), self.klass.HEADER_LEN)
|
||||
|
||||
def test_magic(self):
|
||||
s = self.klass(dst_id=123, handle=123).pack()
|
||||
magic, = struct.unpack('>h', s[:2])
|
||||
self.assertEquals(self.klass.HEADER_MAGIC, magic)
|
||||
|
||||
def test_dst_id(self):
|
||||
s = self.klass(dst_id=123, handle=123).pack()
|
||||
dst_id, = struct.unpack('>L', s[2:6])
|
||||
self.assertEquals(123, dst_id)
|
||||
|
||||
def test_src_id(self):
|
||||
s = self.klass(src_id=5432, dst_id=123, handle=123).pack()
|
||||
src_id, = struct.unpack('>L', s[6:10])
|
||||
self.assertEquals(5432, src_id)
|
||||
|
||||
def test_auth_id(self):
|
||||
s = self.klass(auth_id=1919, src_id=5432, dst_id=123, handle=123).pack()
|
||||
auth_id, = struct.unpack('>L', s[10:14])
|
||||
self.assertEquals(1919, auth_id)
|
||||
|
||||
def test_handle(self):
|
||||
s = self.klass(dst_id=123, handle=9999).pack()
|
||||
handle, = struct.unpack('>L', s[14:18])
|
||||
self.assertEquals(9999, handle)
|
||||
|
||||
def test_reply_to(self):
|
||||
s = self.klass(dst_id=1231, handle=7777, reply_to=9132).pack()
|
||||
reply_to, = struct.unpack('>L', s[18:22])
|
||||
self.assertEquals(9132, reply_to)
|
||||
|
||||
def test_data_length_empty(self):
|
||||
s = self.klass(dst_id=1231, handle=7777).pack()
|
||||
data_length, = struct.unpack('>L', s[22:26])
|
||||
self.assertEquals(0, data_length)
|
||||
|
||||
def test_data_length_present(self):
|
||||
s = self.klass(dst_id=1231, handle=7777, data=b('hello')).pack()
|
||||
data_length, = struct.unpack('>L', s[22:26])
|
||||
self.assertEquals(5, data_length)
|
||||
|
||||
def test_data_empty(self):
|
||||
s = self.klass(dst_id=1231, handle=7777).pack()
|
||||
data = s[26:]
|
||||
self.assertEquals(b(''), data)
|
||||
|
||||
def test_data_present(self):
|
||||
s = self.klass(dst_id=11, handle=77, data=b('hello')).pack()
|
||||
data = s[26:]
|
||||
self.assertEquals(b('hello'), data)
|
||||
|
||||
|
||||
class IsDeadTest(testlib.TestCase):
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_is_dead(self):
|
||||
msg = self.klass(reply_to=mitogen.core.IS_DEAD)
|
||||
self.assertTrue(msg.is_dead)
|
||||
|
||||
def test_is_not_dead(self):
|
||||
msg = self.klass(reply_to=5555)
|
||||
self.assertFalse(msg.is_dead)
|
||||
|
||||
|
||||
class DeadTest(testlib.TestCase):
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_no_reason(self):
|
||||
msg = self.klass.dead()
|
||||
self.assertEquals(msg.reply_to, mitogen.core.IS_DEAD)
|
||||
self.assertTrue(msg.is_dead)
|
||||
self.assertEquals(msg.data, b(''))
|
||||
|
||||
def test_with_reason(self):
|
||||
msg = self.klass.dead(reason=u'oh no')
|
||||
self.assertEquals(msg.reply_to, mitogen.core.IS_DEAD)
|
||||
self.assertTrue(msg.is_dead)
|
||||
self.assertEquals(msg.data, b('oh no'))
|
||||
|
||||
|
||||
class EvilObject(object):
|
||||
pass
|
||||
|
||||
|
||||
class PickledTest(testlib.TestCase):
|
||||
# getting_started.html#rpc-serialization-rules
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def roundtrip(self, v, router=None):
|
||||
msg = self.klass.pickled(v)
|
||||
msg2 = self.klass(data=msg.data)
|
||||
msg2.router = router
|
||||
return msg2.unpickle()
|
||||
|
||||
def test_bool(self):
|
||||
for b in True, False:
|
||||
self.assertEquals(b, self.roundtrip(b))
|
||||
|
||||
@unittest2.skipIf(condition=sys.version_info < (2, 6),
|
||||
reason='bytearray missing on <2.6')
|
||||
def test_bytearray(self):
|
||||
ba = bytearray(b('123'))
|
||||
self.assertRaises(mitogen.core.StreamError,
|
||||
lambda: self.roundtrip(ba)
|
||||
)
|
||||
|
||||
def test_bytes(self):
|
||||
by = b('123')
|
||||
self.assertEquals(by, self.roundtrip(by))
|
||||
|
||||
def test_dict(self):
|
||||
d = {1: 2, u'a': 3, b('b'): 4, 'c': {}}
|
||||
roundtrip = self.roundtrip(d)
|
||||
self.assertEquals(d, roundtrip)
|
||||
self.assertTrue(isinstance(roundtrip, dict))
|
||||
for k in d:
|
||||
self.assertTrue(isinstance(roundtrip[k], type(d[k])))
|
||||
|
||||
def test_int(self):
|
||||
self.assertEquals(123, self.klass.pickled(123).unpickle())
|
||||
|
||||
def test_list(self):
|
||||
l = [1, u'b', b('c')]
|
||||
roundtrip = self.roundtrip(l)
|
||||
self.assertTrue(isinstance(roundtrip, list))
|
||||
self.assertEquals(l, roundtrip)
|
||||
for k in range(len(l)):
|
||||
self.assertTrue(isinstance(roundtrip[k], type(l[k])))
|
||||
|
||||
@unittest2.skipIf(condition=sys.version_info > (3, 0),
|
||||
reason='long missing in >3.x')
|
||||
def test_long(self):
|
||||
l = long(0xffffffffffff)
|
||||
roundtrip = self.roundtrip(l)
|
||||
self.assertEquals(l, roundtrip)
|
||||
self.assertTrue(isinstance(roundtrip, long))
|
||||
|
||||
def test_tuple(self):
|
||||
l = (1, u'b', b('c'))
|
||||
roundtrip = self.roundtrip(l)
|
||||
self.assertEquals(l, roundtrip)
|
||||
self.assertTrue(isinstance(roundtrip, tuple))
|
||||
for k in range(len(l)):
|
||||
self.assertTrue(isinstance(roundtrip[k], type(l[k])))
|
||||
|
||||
def test_unicode(self):
|
||||
u = u'abcd'
|
||||
roundtrip = self.roundtrip(u)
|
||||
self.assertEquals(u, roundtrip)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.UnicodeType))
|
||||
|
||||
#### custom types. see also: types_test.py, call_error_test.py
|
||||
|
||||
# Python 3 pickle protocol 2 does weird stuff depending on whether an empty
|
||||
# or nonempty bytes is being serialized. For non-empty, it yields a
|
||||
# _codecs.encode() call. For empty, it yields a bytes() call.
|
||||
|
||||
def test_blob_nonempty(self):
|
||||
v = mitogen.core.Blob(b('dave'))
|
||||
roundtrip = self.roundtrip(v)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.Blob))
|
||||
self.assertEquals(b('dave'), roundtrip)
|
||||
|
||||
def test_blob_empty(self):
|
||||
v = mitogen.core.Blob(b(''))
|
||||
roundtrip = self.roundtrip(v)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.Blob))
|
||||
self.assertEquals(b(''), v)
|
||||
|
||||
def test_secret_nonempty(self):
|
||||
s = mitogen.core.Secret(u'dave')
|
||||
roundtrip = self.roundtrip(s)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.Secret))
|
||||
self.assertEquals(u'dave', roundtrip)
|
||||
|
||||
def test_secret_empty(self):
|
||||
s = mitogen.core.Secret(u'')
|
||||
roundtrip = self.roundtrip(s)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.Secret))
|
||||
self.assertEquals(u'', roundtrip)
|
||||
|
||||
def test_call_error(self):
|
||||
ce = mitogen.core.CallError('nope')
|
||||
ce2 = self.assertRaises(mitogen.core.CallError,
|
||||
lambda: self.roundtrip(ce))
|
||||
self.assertEquals(ce.args[0], ce2.args[0])
|
||||
|
||||
def test_context(self):
|
||||
router = mitogen.master.Router()
|
||||
try:
|
||||
c = router.context_by_id(1234)
|
||||
roundtrip = self.roundtrip(c)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.Context))
|
||||
self.assertEquals(c.context_id, 1234)
|
||||
finally:
|
||||
router.broker.shutdown()
|
||||
router.broker.join()
|
||||
|
||||
def test_sender(self):
|
||||
router = mitogen.master.Router()
|
||||
try:
|
||||
recv = mitogen.core.Receiver(router)
|
||||
sender = recv.to_sender()
|
||||
roundtrip = self.roundtrip(sender, router=router)
|
||||
self.assertTrue(isinstance(roundtrip, mitogen.core.Sender))
|
||||
self.assertEquals(roundtrip.context.context_id, mitogen.context_id)
|
||||
self.assertEquals(roundtrip.dst_handle, sender.dst_handle)
|
||||
finally:
|
||||
router.broker.shutdown()
|
||||
router.broker.join()
|
||||
|
||||
####
|
||||
|
||||
def test_custom_object_deserialization_fails(self):
|
||||
self.assertRaises(mitogen.core.StreamError,
|
||||
lambda: self.roundtrip(EvilObject())
|
||||
)
|
||||
|
||||
|
||||
class ReplyTest(testlib.TestCase):
|
||||
# getting_started.html#rpc-serialization-rules
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_reply_calls_router_route(self):
|
||||
msg = self.klass(src_id=1234, reply_to=9191)
|
||||
router = mock.Mock()
|
||||
msg.reply(123, router=router)
|
||||
self.assertEquals(1, router.route.call_count)
|
||||
|
||||
def test_reply_pickles_object(self):
|
||||
msg = self.klass(src_id=1234, reply_to=9191)
|
||||
router = mock.Mock()
|
||||
msg.reply(123, router=router)
|
||||
_, (reply,), _ = router.route.mock_calls[0]
|
||||
self.assertEquals(reply.dst_id, 1234)
|
||||
self.assertEquals(reply.unpickle(), 123)
|
||||
|
||||
def test_reply_uses_preformatted_message(self):
|
||||
msg = self.klass(src_id=1234, reply_to=9191)
|
||||
router = mock.Mock()
|
||||
my_reply = mitogen.core.Message.pickled(4444)
|
||||
msg.reply(my_reply, router=router)
|
||||
_, (reply,), _ = router.route.mock_calls[0]
|
||||
self.assertTrue(my_reply is reply)
|
||||
self.assertEquals(reply.dst_id, 1234)
|
||||
self.assertEquals(reply.unpickle(), 4444)
|
||||
|
||||
def test_reply_sets_dst_id(self):
|
||||
msg = self.klass(src_id=1234, reply_to=9191)
|
||||
router = mock.Mock()
|
||||
msg.reply(123, router=router)
|
||||
_, (reply,), _ = router.route.mock_calls[0]
|
||||
self.assertEquals(reply.dst_id, 1234)
|
||||
|
||||
def test_reply_sets_handle(self):
|
||||
msg = self.klass(src_id=1234, reply_to=9191)
|
||||
router = mock.Mock()
|
||||
msg.reply(123, router=router)
|
||||
_, (reply,), _ = router.route.mock_calls[0]
|
||||
self.assertEquals(reply.handle, 9191)
|
||||
|
||||
|
||||
class UnpickleTest(testlib.TestCase):
|
||||
# mostly done by PickleTest, just check behaviour of parameters
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_throw(self):
|
||||
ce = mitogen.core.CallError('nope')
|
||||
m = self.klass.pickled(ce)
|
||||
ce2 = self.assertRaises(mitogen.core.CallError,
|
||||
lambda: m.unpickle())
|
||||
self.assertEquals(ce.args[0], ce2.args[0])
|
||||
|
||||
def test_no_throw(self):
|
||||
ce = mitogen.core.CallError('nope')
|
||||
m = self.klass.pickled(ce)
|
||||
ce2 = m.unpickle(throw=False)
|
||||
self.assertEquals(ce.args[0], ce2.args[0])
|
||||
|
||||
def test_throw_dead(self):
|
||||
m = self.klass.pickled('derp', reply_to=mitogen.core.IS_DEAD)
|
||||
self.assertRaises(mitogen.core.ChannelError,
|
||||
lambda: m.unpickle())
|
||||
|
||||
def test_no_throw_dead(self):
|
||||
m = self.klass.pickled('derp', reply_to=mitogen.core.IS_DEAD)
|
||||
self.assertEquals('derp', m.unpickle(throw_dead=False))
|
||||
|
||||
|
||||
class UnpickleCompatTest(testlib.TestCase):
|
||||
# try weird variations of pickles from different Python versions.
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def check(self, value, encoded, **kwargs):
|
||||
if isinstance(encoded, mitogen.core.UnicodeType):
|
||||
encoded = encoded.encode('latin1')
|
||||
m = self.klass(data=encoded)
|
||||
m.router = mitogen.master.Router()
|
||||
try:
|
||||
return m.unpickle(**kwargs)
|
||||
finally:
|
||||
m.router.broker.shutdown()
|
||||
m.router.broker.join()
|
||||
|
||||
def test_py24_bytes(self):
|
||||
self.check('test',
|
||||
('\x80\x02U\x04testq\x00.'))
|
||||
|
||||
def test_py24_unicode(self):
|
||||
self.check(u'test',
|
||||
('\x80\x02X\x04\x00\x00\x00testq\x00.'))
|
||||
|
||||
def test_py24_int(self):
|
||||
self.check(123,
|
||||
('\x80\x02K{.'))
|
||||
|
||||
def test_py24_long(self):
|
||||
self.check(17592186044415,
|
||||
('\x80\x02\x8a\x06\xff\xff\xff\xff\xff\x0f.'))
|
||||
|
||||
def test_py24_dict(self):
|
||||
self.check({},
|
||||
('\x80\x02}q\x00.'))
|
||||
|
||||
def test_py24_tuple(self):
|
||||
self.check((1, 2, u'b'),
|
||||
('\x80\x02K\x01K\x02X\x01\x00\x00\x00bq\x00\x87q\x01.'))
|
||||
|
||||
def test_py24_bool(self):
|
||||
self.check(True,
|
||||
('\x80\x02\x88.'))
|
||||
|
||||
def test_py24_list(self):
|
||||
self.check([1, 2, u'b'],
|
||||
('\x80\x02]q\x00(K\x01K\x02X\x01\x00\x00\x00bq\x01e.'))
|
||||
|
||||
def test_py24_blob(self):
|
||||
self.check(mitogen.core.mitogen.core.Blob(b('bigblob')),
|
||||
('\x80\x02cmitogen.core\nBlob\nq\x00U\x07bigblobq\x01\x85q\x02Rq\x03.'))
|
||||
|
||||
def test_py24_secret(self):
|
||||
self.check(mitogen.core.Secret(u'mypassword'),
|
||||
('\x80\x02cmitogen.core\nSecret\nq\x00X\n\x00\x00\x00mypasswordq\x01\x85q\x02Rq\x03.'))
|
||||
|
||||
def test_py24_call_error(self):
|
||||
self.check(mitogen.core.CallError('big error'),
|
||||
('\x80\x02cmitogen.core\n_unpickle_call_error\nq\x00X\t\x00\x00\x00big errorq\x01\x85q\x02R.'), throw=False)
|
||||
|
||||
def test_py24_context(self):
|
||||
self.check(mitogen.core.Context(1234, None),
|
||||
('\x80\x02cmitogen.core\n_unpickle_context\nq\x00M\xd2\x04N\x86q\x01Rq\x02.'))
|
||||
|
||||
def test_py24_sender(self):
|
||||
self.check(mitogen.core.Sender(mitogen.core.Context(55555, None), 4444),
|
||||
('\x80\x02cmitogen.core\n_unpickle_sender\nq\x00M\x03\xd9M\\\x11\x86q\x01Rq\x02.'))
|
||||
|
||||
def test_py27_bytes(self):
|
||||
self.check(b('test'),
|
||||
('\x80\x02U\x04testq\x01.'))
|
||||
|
||||
def test_py27_unicode(self):
|
||||
self.check(u'test',
|
||||
('\x80\x02X\x04\x00\x00\x00testq\x01.'))
|
||||
|
||||
def test_py27_int(self):
|
||||
self.check(123,
|
||||
('\x80\x02K{.'))
|
||||
|
||||
def test_py27_long(self):
|
||||
self.check(17592186044415,
|
||||
('\x80\x02\x8a\x06\xff\xff\xff\xff\xff\x0f.'))
|
||||
|
||||
def test_py27_dict(self):
|
||||
self.check({},
|
||||
('\x80\x02}q\x01.'))
|
||||
|
||||
def test_py27_tuple(self):
|
||||
self.check((1, 2, u'b'),
|
||||
('\x80\x02K\x01K\x02X\x01\x00\x00\x00b\x87q\x01.'))
|
||||
|
||||
def test_py27_bool(self):
|
||||
self.check(True,
|
||||
('\x80\x02\x88.'))
|
||||
|
||||
def test_py27_list(self):
|
||||
self.check([1, 2, u'b'],
|
||||
('\x80\x02]q\x01(K\x01K\x02X\x01\x00\x00\x00be.'))
|
||||
|
||||
def test_py27_blob(self):
|
||||
self.check(mitogen.core.mitogen.core.Blob(b('bigblob')),
|
||||
('\x80\x02cmitogen.core\nBlob\nq\x01U\x07bigblob\x85Rq\x02.'))
|
||||
|
||||
def test_py27_secret(self):
|
||||
self.check(mitogen.core.Secret(u'mypassword'),
|
||||
('\x80\x02cmitogen.core\nSecret\nq\x01X\n\x00\x00\x00mypassword\x85Rq\x02.'))
|
||||
|
||||
def test_py27_call_error(self):
|
||||
self.check(mitogen.core.CallError(u'big error',),
|
||||
('\x80\x02cmitogen.core\n_unpickle_call_error\nq\x01X\t\x00\x00\x00big errorq\x02\x85Rq\x03.'), throw=False)
|
||||
|
||||
def test_py27_context(self):
|
||||
self.check(mitogen.core.Context(1234, None),
|
||||
('\x80\x02cmitogen.core\n_unpickle_context\nq\x01M\xd2\x04N\x86Rq\x02.'))
|
||||
|
||||
def test_py27_sender(self):
|
||||
self.check(mitogen.core.Sender(mitogen.core.Context(55555, None), 4444),
|
||||
('\x80\x02cmitogen.core\n_unpickle_sender\nq\x01M\x03\xd9M\\\x11\x86Rq\x02.'))
|
||||
|
||||
def test_py36_bytes(self):
|
||||
self.check(b('test'),
|
||||
('\x80\x02c_codecs\nencode\nq\x00X\x04\x00\x00\x00testq\x01X\x06\x00\x00\x00latin1q\x02\x86q\x03Rq\x04.'))
|
||||
|
||||
def test_py36_unicode(self):
|
||||
self.check('test',
|
||||
('\x80\x02X\x04\x00\x00\x00testq\x00.'))
|
||||
|
||||
def test_py36_int(self):
|
||||
self.check(123,
|
||||
('\x80\x02K{.'))
|
||||
|
||||
def test_py36_long(self):
|
||||
self.check(17592186044415,
|
||||
('\x80\x02\x8a\x06\xff\xff\xff\xff\xff\x0f.'))
|
||||
|
||||
def test_py36_dict(self):
|
||||
self.check({},
|
||||
('\x80\x02}q\x00.'))
|
||||
|
||||
def test_py36_tuple(self):
|
||||
self.check((1, 2, u'b'),
|
||||
('\x80\x02K\x01K\x02X\x01\x00\x00\x00bq\x00\x87q\x01.'))
|
||||
|
||||
def test_py36_bool(self):
|
||||
self.check(True,
|
||||
('\x80\x02\x88.'))
|
||||
|
||||
def test_py36_list(self):
|
||||
self.check([1, 2, u'b'],
|
||||
('\x80\x02]q\x00(K\x01K\x02X\x01\x00\x00\x00bq\x01e.'))
|
||||
|
||||
def test_py36_blob(self):
|
||||
self.check(mitogen.core.mitogen.core.Blob(b('bigblob')),
|
||||
('\x80\x02cmitogen.core\nBlob\nq\x00c_codecs\nencode\nq\x01X\x07\x00\x00\x00bigblobq\x02X\x06\x00\x00\x00latin1q\x03\x86q\x04Rq\x05\x85q\x06Rq\x07.'))
|
||||
|
||||
def test_py36_secret(self):
|
||||
self.check(mitogen.core.Secret('mypassword'),
|
||||
('\x80\x02cmitogen.core\nSecret\nq\x00X\n\x00\x00\x00mypasswordq\x01\x85q\x02Rq\x03.'))
|
||||
|
||||
def test_py36_call_error(self):
|
||||
self.check(mitogen.core.CallError('big error'),
|
||||
('\x80\x02cmitogen.core\n_unpickle_call_error\nq\x00X\t\x00\x00\x00big errorq\x01\x85q\x02Rq\x03.'), throw=False)
|
||||
|
||||
def test_py36_context(self):
|
||||
self.check(mitogen.core.Context(1234, None),
|
||||
('\x80\x02cmitogen.core\n_unpickle_context\nq\x00M\xd2\x04N\x86q\x01Rq\x02.'))
|
||||
|
||||
def test_py36_sender(self):
|
||||
self.check(mitogen.core.Sender(mitogen.core.Context(55555, None), 4444),
|
||||
('\x80\x02cmitogen.core\n_unpickle_sender\nq\x00M\x03\xd9M\\\x11\x86q\x01Rq\x02.'))
|
||||
|
||||
|
||||
class ReprTest(testlib.TestCase):
|
||||
klass = mitogen.core.Message
|
||||
|
||||
def test_repr(self):
|
||||
# doesn't crash
|
||||
repr(self.klass.pickled('test'))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -0,0 +1,34 @@
|
||||
|
||||
import unittest2
|
||||
import mock
|
||||
|
||||
import mitogen.core
|
||||
|
||||
import testlib
|
||||
|
||||
|
||||
class ReceiveOneTest(testlib.TestCase):
|
||||
klass = mitogen.core.MitogenProtocol
|
||||
|
||||
def test_corruption(self):
|
||||
broker = mock.Mock()
|
||||
router = mock.Mock()
|
||||
stream = mock.Mock()
|
||||
|
||||
protocol = self.klass(router, 1)
|
||||
protocol.stream = stream
|
||||
|
||||
junk = mitogen.core.b('x') * mitogen.core.Message.HEADER_LEN
|
||||
|
||||
capture = testlib.LogCapturer()
|
||||
capture.start()
|
||||
protocol.on_receive(broker, junk)
|
||||
capture.stop()
|
||||
|
||||
self.assertEquals(1, stream.on_disconnect.call_count)
|
||||
expect = self.klass.corrupt_msg % (stream.name, junk)
|
||||
self.assertTrue(expect in capture.raw())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -0,0 +1,46 @@
|
||||
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
|
||||
import mitogen
|
||||
import mitogen.parent
|
||||
|
||||
import unittest2
|
||||
|
||||
import testlib
|
||||
|
||||
|
||||
class DockerTest(testlib.DockerMixin, testlib.TestCase):
|
||||
def test_okay(self):
|
||||
# Magic calls must happen as root.
|
||||
try:
|
||||
root = self.router.sudo()
|
||||
except mitogen.core.StreamError:
|
||||
raise unittest2.SkipTest("requires sudo to localhost root")
|
||||
|
||||
via_ssh = self.docker_ssh(
|
||||
username='mitogen__has_sudo',
|
||||
password='has_sudo_password',
|
||||
)
|
||||
|
||||
via_setns = self.router.setns(
|
||||
kind='docker',
|
||||
container=self.dockerized_ssh.container_name,
|
||||
via=root,
|
||||
)
|
||||
|
||||
self.assertEquals(
|
||||
via_ssh.call(socket.gethostname),
|
||||
via_setns.call(socket.gethostname),
|
||||
)
|
||||
|
||||
|
||||
DockerTest = unittest2.skipIf(
|
||||
condition=sys.version_info < (2, 5),
|
||||
reason="mitogen.setns unsupported on Python <2.4"
|
||||
)(DockerTest)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -1,33 +0,0 @@
|
||||
|
||||
import unittest2
|
||||
import mock
|
||||
|
||||
import mitogen.core
|
||||
|
||||
import testlib
|
||||
|
||||
|
||||
class ReceiveOneTest(testlib.TestCase):
|
||||
klass = mitogen.core.Stream
|
||||
|
||||
def test_corruption(self):
|
||||
broker = mock.Mock()
|
||||
router = mock.Mock()
|
||||
|
||||
stream = self.klass(router, 1)
|
||||
junk = mitogen.core.b('x') * stream.HEADER_LEN
|
||||
stream._input_buf = [junk]
|
||||
stream._input_buf_len = len(junk)
|
||||
|
||||
capture = testlib.LogCapturer()
|
||||
capture.start()
|
||||
ret = stream._receive_one(broker)
|
||||
#self.assertEquals(1, broker.stop_receive.mock_calls)
|
||||
capture.stop()
|
||||
|
||||
self.assertFalse(ret)
|
||||
self.assertTrue((self.klass.corrupt_msg % (junk,)) in capture.raw())
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
@ -0,0 +1,189 @@
|
||||
|
||||
import time
|
||||
|
||||
import mock
|
||||
import unittest2
|
||||
|
||||
import mitogen.core
|
||||
import mitogen.parent
|
||||
|
||||
import testlib
|
||||
|
||||
|
||||
class TimerListMixin(object):
|
||||
klass = mitogen.parent.TimerList
|
||||
|
||||
def setUp(self):
|
||||
self.list = self.klass()
|
||||
|
||||
|
||||
class GetTimeoutTest(TimerListMixin, testlib.TestCase):
|
||||
def test_empty(self):
|
||||
self.assertEquals(None, self.list.get_timeout())
|
||||
|
||||
def test_one_event(self):
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list._now = lambda: 1
|
||||
self.assertEquals(1, self.list.get_timeout())
|
||||
|
||||
def test_two_events_same_moment(self):
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list._now = lambda: 1
|
||||
self.assertEquals(1, self.list.get_timeout())
|
||||
|
||||
def test_two_events(self):
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list.schedule(3, lambda: None)
|
||||
self.list._now = lambda: 1
|
||||
self.assertEquals(1, self.list.get_timeout())
|
||||
|
||||
def test_two_events_expired(self):
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list.schedule(3, lambda: None)
|
||||
self.list._now = lambda: 3
|
||||
self.assertEquals(0, self.list.get_timeout())
|
||||
|
||||
def test_two_events_in_past(self):
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list.schedule(3, lambda: None)
|
||||
self.list._now = lambda: 30
|
||||
self.assertEquals(0, self.list.get_timeout())
|
||||
|
||||
def test_two_events_in_past(self):
|
||||
self.list.schedule(2, lambda: None)
|
||||
self.list.schedule(3, lambda: None)
|
||||
self.list._now = lambda: 30
|
||||
self.assertEquals(0, self.list.get_timeout())
|
||||
|
||||
def test_one_cancelled(self):
|
||||
t1 = self.list.schedule(2, lambda: None)
|
||||
t2 = self.list.schedule(3, lambda: None)
|
||||
self.list._now = lambda: 0
|
||||
t1.cancel()
|
||||
self.assertEquals(3, self.list.get_timeout())
|
||||
|
||||
def test_two_cancelled(self):
|
||||
t1 = self.list.schedule(2, lambda: None)
|
||||
t2 = self.list.schedule(3, lambda: None)
|
||||
self.list._now = lambda: 0
|
||||
t1.cancel()
|
||||
t2.cancel()
|
||||
self.assertEquals(None, self.list.get_timeout())
|
||||
|
||||
|
||||
class ScheduleTest(TimerListMixin, testlib.TestCase):
|
||||
def test_in_past(self):
|
||||
self.list._now = lambda: 30
|
||||
timer = self.list.schedule(29, lambda: None)
|
||||
self.assertEquals(29, timer.when)
|
||||
self.assertEquals(0, self.list.get_timeout())
|
||||
|
||||
def test_in_future(self):
|
||||
self.list._now = lambda: 30
|
||||
timer = self.list.schedule(31, lambda: None)
|
||||
self.assertEquals(31, timer.when)
|
||||
self.assertEquals(1, self.list.get_timeout())
|
||||
|
||||
def test_same_moment(self):
|
||||
self.list._now = lambda: 30
|
||||
timer = self.list.schedule(31, lambda: None)
|
||||
timer2 = self.list.schedule(31, lambda: None)
|
||||
self.assertEquals(31, timer.when)
|
||||
self.assertEquals(31, timer2.when)
|
||||
self.assertTrue(timer is not timer2)
|
||||
self.assertEquals(1, self.list.get_timeout())
|
||||
|
||||
|
||||
class ExpireTest(TimerListMixin, testlib.TestCase):
|
||||
def test_in_past(self):
|
||||
timer = self.list.schedule(29, mock.Mock())
|
||||
self.list._now = lambda: 30
|
||||
self.list.expire()
|
||||
self.assertEquals(1, len(timer.func.mock_calls))
|
||||
|
||||
def test_in_future(self):
|
||||
timer = self.list.schedule(29, mock.Mock())
|
||||
self.list._now = lambda: 28
|
||||
self.list.expire()
|
||||
self.assertEquals(0, len(timer.func.mock_calls))
|
||||
|
||||
def test_same_moment(self):
|
||||
timer = self.list.schedule(29, mock.Mock())
|
||||
timer2 = self.list.schedule(29, mock.Mock())
|
||||
self.list._now = lambda: 29
|
||||
self.list.expire()
|
||||
self.assertEquals(1, len(timer.func.mock_calls))
|
||||
self.assertEquals(1, len(timer2.func.mock_calls))
|
||||
|
||||
def test_cancelled(self):
|
||||
self.list._now = lambda: 29
|
||||
timer = self.list.schedule(29, mock.Mock())
|
||||
timer.cancel()
|
||||
self.assertEquals(None, self.list.get_timeout())
|
||||
self.list._now = lambda: 29
|
||||
self.list.expire()
|
||||
self.assertEquals(0, len(timer.func.mock_calls))
|
||||
self.assertEquals(None, self.list.get_timeout())
|
||||
|
||||
|
||||
class CancelTest(TimerListMixin, testlib.TestCase):
|
||||
def test_single_cancel(self):
|
||||
self.list._now = lambda: 29
|
||||
timer = self.list.schedule(29, mock.Mock())
|
||||
timer.cancel()
|
||||
self.list.expire()
|
||||
self.assertEquals(0, len(timer.func.mock_calls))
|
||||
|
||||
def test_double_cancel(self):
|
||||
self.list._now = lambda: 29
|
||||
timer = self.list.schedule(29, mock.Mock())
|
||||
timer.cancel()
|
||||
timer.cancel()
|
||||
self.list.expire()
|
||||
self.assertEquals(0, len(timer.func.mock_calls))
|
||||
|
||||
|
||||
@mitogen.core.takes_econtext
|
||||
def do_timer_test_econtext(econtext):
|
||||
do_timer_test(econtext.broker)
|
||||
|
||||
|
||||
def do_timer_test(broker):
|
||||
now = time.time()
|
||||
latch = mitogen.core.Latch()
|
||||
broker.defer(lambda:
|
||||
broker.timers.schedule(
|
||||
now + 0.250,
|
||||
lambda: latch.put('hi'),
|
||||
)
|
||||
)
|
||||
|
||||
assert 'hi' == latch.get()
|
||||
assert time.time() > (now + 0.250)
|
||||
|
||||
|
||||
class BrokerTimerTest(testlib.TestCase):
|
||||
klass = mitogen.master.Broker
|
||||
|
||||
def test_call_later(self):
|
||||
broker = self.klass()
|
||||
try:
|
||||
do_timer_test(broker)
|
||||
finally:
|
||||
broker.shutdown()
|
||||
broker.join()
|
||||
|
||||
def test_child_upgrade(self):
|
||||
router = mitogen.master.Router()
|
||||
try:
|
||||
c = router.local()
|
||||
c.call(mitogen.parent.upgrade_router)
|
||||
c.call(do_timer_test_econtext)
|
||||
finally:
|
||||
router.broker.shutdown()
|
||||
router.broker.join()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest2.main()
|
||||
Loading…
Reference in New Issue