Commit Graph

2646 Commits (54987100b2e74ff836c94df98e1f57dc43005172)
 

Author SHA1 Message Date
David Wilson 54987100b2 [stream-refactor] allow up to 30 seconds to connect in unix_test
It reliably fails when running on a (intentionally) heavily loaded
machine
5 years ago
David Wilson db9066fbfb [stream-refactor] mark setns module as requiring Python >2.4 5 years ago
David Wilson 856dfcebcd [stream-refactor] another 2.4 fix for create_child_test 5 years ago
David Wilson 054643783c .travis.yml: Add reverse shell spawn for Travis too 5 years ago
David Wilson 11ae6f3873 core: better Side attribute docstrings 5 years ago
David Wilson bed5931194 [stream-refactor] remove one more getuser() usage 5 years ago
David Wilson 2f950b3bda [stream-refactor] allow doas_test to succeed on CentOS
Unlike on Debian, some environment variables that tickle
getpass.getuser() are being inherited. So use getuid() instead.

Also install the doas binary on CentOS. CI was changed (I believe) to
shrink the configuration matrix, and now these tests run on CentOS too.
5 years ago
David Wilson 2ba3973bc5 Pin idna==2.7 when running on Python<2.7. 5 years ago
David Wilson 07f3179e58 [stream-refactor] Py2.4 compat fix for iter_split_test. 5 years ago
David Wilson f0065d76d8 [stream-refactor] add descriptive task names to _container_prep 5 years ago
David Wilson fdf3484a2a [stream-refactor] 3.x socket.send() requires bytes 5 years ago
David Wilson c09bbdc2f9 [stream-refactor] fix 2.4 syntax error. 5 years ago
David Wilson bd80d4b0af [stream-refactor] avoid os.wait3() for Py2.4. 5 years ago
David Wilson 321dac3046 Allow specifying -vvv to debops_tests. 5 years ago
David Wilson b1379e6f45 [stream-refactor] send MITO002 earlier
Prevents 2.4 bootstrap from attempting to fetch os_fork too early.

Connection(None).connect(): pid:25098 stdin:81 stdout:81 stderr:79
ssh.localhost:2201: (partial): mitogen__has_sudo_nopw@localhost's password:
ssh.localhost:2201: (password prompt): mitogen__has_sudo_nopw@localhost's password:
ssh.localhost:2201: (unrecognized): mitogen__has_sudo_nopw@localhost's password:
BootstrapProtocol(ssh.localhost:2201): first stage started succcessfully
BootstrapProtocol(ssh.localhost:2201): first stage received bootstrap
ssh.localhost:2201: (partial): MIdmitogen.os_fork
ssh.localhost:2201: (unrecognized partial): MIdmitogen.os_fork
ssh.localhost:2201: failing connection due to TimeoutError(u'Failed to setup connection after 10.00 seconds',)
5 years ago
David Wilson 402dba4197 module_finder: pass raw file to compile()
Newer Ansibles have e.g. UTF-8 present in apt.py.
5 years ago
David Wilson 4eecc08047 [stream-refactor] merge stdout+stderr when reporting EofError
Fixes sudo regression
5 years ago
David Wilson 1d2bfc28da [stream-refactor] fix crash in detach() / during async/multiple_items_loop.yml 5 years ago
David Wilson 93abbcaf7a [stream-refactor] fix crash in runner/forking_active.yml 5 years ago
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 5 years ago
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
5 years ago
David Wilson 6e33de7cd2 unix: ensure mitogen.context_id is reset when client disconnects
To ensure a test process can successfully recreate an Ansible
MuxProcess, reset fork-inherited globals during disconnection.

There is basically no good place for this. Per the comments on #91, it
would be far better if the context's identity was tied to its router,
rather than some global variable.
5 years ago
David Wilson 7c4621a010 [stream-refactor] make syntax 2.4 compatible 5 years ago
David Wilson 2ce3383a01 [stream-refactor] make trusty our Travis dist.
They updated to xenial recently, and it no longer supports Py2.6.
5 years ago
David Wilson 0ff5fb8fc4 [stream-refactor] fix su_test failure (issue #363) 5 years ago
David Wilson 8769c3ce24 [stream-refactor] more readable log string format 5 years ago
David Wilson d411003b64 [stream-refactor] dont doubly log last partial line 5 years ago
David Wilson 869e04af10 [stream-refactor] import fd_check.py used by create_child_test 5 years ago
David Wilson 1069ca43d6 [stream-refactor] port mitogen.buildah, added to master since work began 5 years ago
David Wilson 26b6333787 [stream-refactor] fix unix.Listener construction 5 years ago
David Wilson 1fb3852fa6 [stream-refactor] fix crash when no stderr present. 5 years ago
David Wilson 4b0870aa6e [stream-refactor] fix Process constructor invocation 5 years ago
David Wilson 8e9f47a2e9 Add tests/ansible/.*.pid to gitignore (for ansible_mitogen/process.py) 5 years ago
David Wilson 2e371c96b4 Add extra/ to gitignore 5 years ago
David Wilson 29c63f56ae import release-notes script. 5 years ago
David Wilson f43e24e970 [stream-refactor] repaired rest of create_child_test. 5 years ago
David Wilson f039c81bb0 [stream-refactor] rename Process attrs, fix up more create_child_test 5 years ago
David Wilson cfe337b3c0 [stream-refactor] import incomplete create_child_test 5 years ago
David Wilson 9c38093aa7 issue #482: tests: check for zombie process after test. 5 years ago
David Wilson 3a1125a7bd issue #363: add test. 5 years ago
David Wilson baafc746fe tests: clean up old-style SSH exception catch 5 years ago
David Wilson 4524f03a48 issue #271: add mitogen__permdenied user to Docker image. 5 years ago
David Wilson acade4ce88 ssh: fix issue #271 regression due to refactor, add test. 5 years ago
David Wilson 8d1b01d8ef Refactor Stream, introduce quasi-asynchronous connect, much more
Split Stream into many, many classes

  * mitogen.parent.Connection: Handles connection setup logic only.
    * Maintain references to stdout and stderr streams.
    * Manages TimerList timer to cancel connection attempt after
      deadline
    * Blocking setup code replaced by async equivalents running on the
      broker

  * mitogen.parent.Options: Tracks connection-specific options. This
    keeps the connection class small, but more importantly, it is
    generic to the future desire to build and execute command lines
    without starting a full connection.

  * mitogen.core.Protocol: Handles program behaviour relating to events
    on a stream. Protocol performs no IO of its own, instead deferring
    it to Stream and Side. This makes testing much easier, and means
    libssh can reimplement Stream and Side to reuse MitogenProtocol

  * mitogen.core.MitogenProtocol: Guts of the old Mitogen stream
    implementtion

  * mitogen.core.BufferedWriter: Guts of the old Mitogen buffered
    transmit implementation, made generic

  * mitogen.core.DelineatedProtocol: Guts of the old IoLogger, knows how
    to split up input and pass it on to a
    on_line_received()/on_partial_line_received() callback.

  * mitogen.parent.BootstrapProtocol: Asynchronous equivalent of the old
    blocking connect code. Waits for various prompts (MITO001 etc) and
    writes the bootstrap using a BufferedWriter. On success, switches
    the stream to MitogenProtocol.

  * mitogen.core.Message: move encoding parts of MitogenProtocol out to
    Message (where it belongs) and write a bunch of new tests for
    pickling.

  * The bizarre Stream.construct() is gone now, Option.__init__ is its
    own constructor. Should fix many LGTM errors.

* Update all connection methods:  Every connection method is updated to
  use async logic, defining protocols as required to handle interactive
  prompts like in SSH or su. Add new real integration tests for at least
  doas and su.

* Eliminate manual fd management: File descriptors are trapped in file
  objects at their point of origin, and Side is updated to use file
  objects rather than raw descriptors. This eliminates a whole class of
  bugs where unrelated FDs could be closed by the wrong component. Now
  an FD's open/closed status is fused to it everywhere in the library.

* Halve file descriptor usage: now FD open/close state is tracked by
  its file object, we don't need to duplicate FDs everywhere so that
  receive/transmit side can be closed independently. Instead both sides
  back on to the same file object. Closes #26, Closes #470.

* Remove most uses of dup/dup2: Closes #256. File descriptors are
  trapped in a common file object and shared among classes. The
  remaining few uses for dup/dup2 are as close to minimal as possible.

* Introduce mitogen.parent.Process: uniform interface for subprocesses
  created either via mitogen.fork or the subprocess module. Remove all
  the crap where we steal a pid from subprocess guts. Now we use
  subprocess to manage its processes as it should be. Closes #169 by
  using the new Timers facility to poll for a slow-to-exit subprocess.

* Fix su password race: Closes #363. DelineatedProtocol naturally
  retries partially received lines, preventing the cause of the original
  race.

* Delete old blocking IO utility functions
  iter_read()/write_all()/discard_until().

Closes #26
Closes #147
Closes #169
Closes #256
Closes #363
Closes #419
Closes #470
5 years ago
David Wilson 37beb3a5c5 core: teach iter_split() to break on callback returning False. 5 years ago
David Wilson 33ecc8a5d2 issue #507: log fatal errors to syslog.
Next round should log entire exception text, but this is useful enough
already.
5 years ago
David Wilson bf77d4ab1d testlib: have LogCapturer.raw() return unicode on 2.x. 5 years ago
David Wilson 46ebd56c7a core/master: docstring, repr, and debug log message cleanups
Debug output is vastly more readable now.
5 years ago
David Wilson 0f7bbcece9 parent: remove unused Timer parameter. 5 years ago
David Wilson fb23ecae01 tests: jail_test fixes. 5 years ago