Commit Graph

757 Commits (462c4ff59fa0b4fbeafe288ed276f21e2b404a7e)

Author SHA1 Message Date
David Wilson 462c4ff59f [stream-refactor] stop writing to /tmp/foo in fd_check.py. 5 years ago
David Wilson cd0a557602 [stream-refactor] yet another 2.4 issue in create_child_test 5 years ago
David Wilson 54987100b2 [stream-refactor] allow up to 30 seconds to connect in unix_test
It reliably fails when running on a (intentionally) heavily loaded
machine
5 years ago
David Wilson db9066fbfb [stream-refactor] mark setns module as requiring Python >2.4 5 years ago
David Wilson 856dfcebcd [stream-refactor] another 2.4 fix for create_child_test 5 years ago
David Wilson bed5931194 [stream-refactor] remove one more getuser() usage 5 years ago
David Wilson 2f950b3bda [stream-refactor] allow doas_test to succeed on CentOS
Unlike on Debian, some environment variables that tickle
getpass.getuser() are being inherited. So use getuid() instead.

Also install the doas binary on CentOS. CI was changed (I believe) to
shrink the configuration matrix, and now these tests run on CentOS too.
5 years ago
David Wilson 2ba3973bc5 Pin idna==2.7 when running on Python<2.7. 5 years ago
David Wilson 07f3179e58 [stream-refactor] Py2.4 compat fix for iter_split_test. 5 years ago
David Wilson f0065d76d8 [stream-refactor] add descriptive task names to _container_prep 5 years ago
David Wilson bd80d4b0af [stream-refactor] avoid os.wait3() for Py2.4. 5 years ago
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 5 years ago
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
5 years ago
David Wilson 869e04af10 [stream-refactor] import fd_check.py used by create_child_test 5 years ago
David Wilson 1069ca43d6 [stream-refactor] port mitogen.buildah, added to master since work began 5 years ago
David Wilson f43e24e970 [stream-refactor] repaired rest of create_child_test. 5 years ago
David Wilson f039c81bb0 [stream-refactor] rename Process attrs, fix up more create_child_test 5 years ago
David Wilson cfe337b3c0 [stream-refactor] import incomplete create_child_test 5 years ago
David Wilson 9c38093aa7 issue #482: tests: check for zombie process after test. 5 years ago
David Wilson 3a1125a7bd issue #363: add test. 5 years ago
David Wilson baafc746fe tests: clean up old-style SSH exception catch 5 years ago
David Wilson 4524f03a48 issue #271: add mitogen__permdenied user to Docker image. 5 years ago
David Wilson acade4ce88 ssh: fix issue #271 regression due to refactor, add test. 5 years ago
David Wilson 8d1b01d8ef Refactor Stream, introduce quasi-asynchronous connect, much more
Split Stream into many, many classes

  * mitogen.parent.Connection: Handles connection setup logic only.
    * Maintain references to stdout and stderr streams.
    * Manages TimerList timer to cancel connection attempt after
      deadline
    * Blocking setup code replaced by async equivalents running on the
      broker

  * mitogen.parent.Options: Tracks connection-specific options. This
    keeps the connection class small, but more importantly, it is
    generic to the future desire to build and execute command lines
    without starting a full connection.

  * mitogen.core.Protocol: Handles program behaviour relating to events
    on a stream. Protocol performs no IO of its own, instead deferring
    it to Stream and Side. This makes testing much easier, and means
    libssh can reimplement Stream and Side to reuse MitogenProtocol

  * mitogen.core.MitogenProtocol: Guts of the old Mitogen stream
    implementtion

  * mitogen.core.BufferedWriter: Guts of the old Mitogen buffered
    transmit implementation, made generic

  * mitogen.core.DelineatedProtocol: Guts of the old IoLogger, knows how
    to split up input and pass it on to a
    on_line_received()/on_partial_line_received() callback.

  * mitogen.parent.BootstrapProtocol: Asynchronous equivalent of the old
    blocking connect code. Waits for various prompts (MITO001 etc) and
    writes the bootstrap using a BufferedWriter. On success, switches
    the stream to MitogenProtocol.

  * mitogen.core.Message: move encoding parts of MitogenProtocol out to
    Message (where it belongs) and write a bunch of new tests for
    pickling.

  * The bizarre Stream.construct() is gone now, Option.__init__ is its
    own constructor. Should fix many LGTM errors.

* Update all connection methods:  Every connection method is updated to
  use async logic, defining protocols as required to handle interactive
  prompts like in SSH or su. Add new real integration tests for at least
  doas and su.

* Eliminate manual fd management: File descriptors are trapped in file
  objects at their point of origin, and Side is updated to use file
  objects rather than raw descriptors. This eliminates a whole class of
  bugs where unrelated FDs could be closed by the wrong component. Now
  an FD's open/closed status is fused to it everywhere in the library.

* Halve file descriptor usage: now FD open/close state is tracked by
  its file object, we don't need to duplicate FDs everywhere so that
  receive/transmit side can be closed independently. Instead both sides
  back on to the same file object. Closes #26, Closes #470.

* Remove most uses of dup/dup2: Closes #256. File descriptors are
  trapped in a common file object and shared among classes. The
  remaining few uses for dup/dup2 are as close to minimal as possible.

* Introduce mitogen.parent.Process: uniform interface for subprocesses
  created either via mitogen.fork or the subprocess module. Remove all
  the crap where we steal a pid from subprocess guts. Now we use
  subprocess to manage its processes as it should be. Closes #169 by
  using the new Timers facility to poll for a slow-to-exit subprocess.

* Fix su password race: Closes #363. DelineatedProtocol naturally
  retries partially received lines, preventing the cause of the original
  race.

* Delete old blocking IO utility functions
  iter_read()/write_all()/discard_until().

Closes #26
Closes #147
Closes #169
Closes #256
Closes #363
Closes #419
Closes #470
5 years ago
David Wilson 37beb3a5c5 core: teach iter_split() to break on callback returning False. 5 years ago
David Wilson bf77d4ab1d testlib: have LogCapturer.raw() return unicode on 2.x. 5 years ago
David Wilson fb23ecae01 tests: jail_test fixes. 5 years ago
David Wilson 12569ad70a tests/bench: import ssh-roundtrip.py. 5 years ago
David Wilson bb0c98f451 tests: note location of related tests. 5 years ago
David Wilson eff8179591 tests: add real test for doas. 5 years ago
David Wilson d8dc5420ce tests: install OpenBSD doas port in Debian image.
To allow fancy new improved doas_test.
5 years ago
David Wilson de9a8b2a80 tests: add setns_test that works if password localhost sudo works. 5 years ago
David Wilson bbf0b22493 Import minimal jail_test. 5 years ago
David Wilson d1f5e0663d core: move message encoding to Message.pack(), add+refactor tests.
The old inline pack is still present in the old location but will be
removed in a followup commit.
5 years ago
David Wilson 3ab7998861 tests: add some UTF-8 to ssh_login_banner to encourage breakage. 5 years ago
David Wilson f6d26c5acb testlib: disable lsof warnings due to Docker crap 5 years ago
David Wilson 70ff4b674c parent: discard cancelled events in TimerList.get_timeout().
Otherwise get_timeout() keeps broker alive via keep_alive() for a
cancelled timer during shutdown.
5 years ago
David Wilson 5aca9d6c3f core: split out iter_split() for use in parent.py. 5 years ago
David Wilson 2fbc77a155 issue #170: implement timers. 5 years ago
Jordan Webb d71bdd1694
Add buildah test and stub 6 years ago
David Wilson ee7dae7514 ci: Another round of fixes for random Ansible UI breakage in 2.7/2.8 6 years ago
David Wilson ad5a80f200 Use virtualenv Python for stub connections to workaround problem
../data/stubs/stub-kubectl.py exec -it localhost -- /usr/bin/python -c "...":

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    LookupError: unknown encoding: base64

It's not clear why this is happening. "stub-kubectl.py" is executed with
the 2.7 virtualenv, while the exec() that happens inside stub-kubectl
was for "/usr/bin/python".

That second Python can't find chunks of its stdlib:

    stat("/usr/lib/python2.7/encodings/base64", 0x7ffde8744c60) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.so", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64module.so", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.py", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.pyc", O_RDONLY) = -1 ENOENT (No such file or directory)
    write(2, "Traceback (most recent call last):\n", 35) = 35
    write(2, "  File \"<string>\", line 1, in <module>\n", 39) = 39
6 years ago
David Wilson 08e7fe4f80 tests: add 2.8 format async error timeout message 6 years ago
David Wilson 418fc15e80 tests: allow running without hdrhistograms library. 6 years ago
David Wilson 3620fce071 issue #593: expose configurables for SSH keepalive and increase the default 6 years ago
David Wilson e90c05dc9d issue #591: fix test for Ansible 2.3. 6 years ago
David Wilson 0b7fd3f290 issue #591: ansible: restore CWD prior to AnsibleModule initialization. 6 years ago
David Wilson fe7c3610ff issue #590: disable distro test on vanilla 6 years ago
David Wilson f1287e6e9a tests: Py3.x fix. 6 years ago
David Wilson a1f2ec222d issue #590: fix test for <2.8 Ansibles. 6 years ago