Commit Graph

557 Commits (4388e794ce9fe69c3243d28e923d5c23b381136e)

Author SHA1 Message Date
David Wilson 802de6a8d5 issue #406: clean up DiagLogStream handling and connect() failure.
When Stream.connect() fails, have it just use on_disconnect(). Now there
is a single disconnect cleanup path.

Remove cutpasted DiagLogStream setup/destruction, and move it into the
base class (temporarily), and only manage the lifetime of its underlying
FD via Side.close().  This cures another EBADF failure.
6 years ago
David Wilson e01c8f2891 issue #406: 3.x syntax fixes. 6 years ago
David Wilson 003526ef7b issue #406: fix thread leaks in unix_test too. 6 years ago
David Wilson eae1bdba4e tests: make minify_test print something useful on failure 6 years ago
David Wilson 802efa6ea6 issue #406: ensure broker_test waits for broker exit. 6 years ago
David Wilson 175fc377d2 tests: remove hard-wired SSL paths from fork_test. 6 years ago
David Wilson 3da4b1a420 tests: verify only main/watcher threads exist at teardown 6 years ago
David Wilson b0dd628f07 issue #406: parent_test fixes, NameError in log_fd_calls(). 6 years ago
David Wilson 6ff1e001da issue #406: log socketpair calls too. 6 years ago
David Wilson 70c550f50c issue #406: close stdout pipes in parent_test 6 years ago
David Wilson 9b3cb55a8b issue #4096: import log_fd_calls() helper. 6 years ago
David Wilson 8a0b343760 issue #406: test for FD leak after every TestCase 6 years ago
David Wilson e9a6e4c3d2 issue #406: add test. 6 years ago
David Wilson 5eff8ea4fb tests: make result_shell_echo_hi compare less of the timedelta; closes #361
Assuming less than one second is too much to ask from Travis.
6 years ago
David Wilson 5be9a55bf4 core: allow Context to be pickled by non-Mitogen pickler. 6 years ago
David Wilson c148c869e6 issue #76, #370: add disconnect cleanup test 6 years ago
David Wilson aeec2b5054 tests: pad out minify_test to verify all internal modules 6 years ago
David Wilson f2d288bb1e tests: ensure minify() result can be compiled for all of core. 6 years ago
David Wilson fd326f5ad7 tests: stub tests for doas/mitogen_doas 6 years ago
David Wilson cbd4129cb9 tests: fix paramiko_unblemished.yml 6 years ago
David Wilson e8fc9e490f tests: update osa_delegate_to_self to match connection parameters 6 years ago
David Wilson 6c71c5bfef issue #369: disable reset_connection on Ansible<2.5.6
https://github.com/ansible/ansible/issues/27520
6 years ago
David Wilson 8ed72e7e7b issue #369: avoid Ansible 2.5 bug (cond_reset_warn missing method) 6 years ago
David Wilson ab4ccc6b92 issue #369: don't mass-kill all SSH clients in reconnection.yml
It breaks my new development environment :)
6 years ago
David Wilson fcdf4a0f35 Import missing stub-sudo.py. 6 years ago
David Wilson a6dd8bb2d0 issue #409: stub test for mitogen_sudo method. 6 years ago
David Wilson fadb9181bc issue #410: support sudo --user and SELinux options, add stub test. 6 years ago
David Wilson 144685a327 issue #409: more lxc/lxd stub tests, let tests run on vanilla. 6 years ago
David Wilson 0e8f451190 issue #409: add stub LXC test, refactor playbooks. 6 years ago
David Wilson e832ddec13 issue #409: mark sudo test noninteractive 6 years ago
David Wilson a77f07659e issue #409: make setns test to work anywhere sudo works. 6 years ago
David Wilson f2294c1678 issue #409: add new stub_connections/ test type. 6 years ago
David Wilson 429832b8f7 issue #409: add kubectl stub and constructor test. 6 years ago
David Wilson c51b67b863 issue #409: add test stubs to the PATH in run_ansible_playbook.py 6 years ago
David Wilson 51658fdd66 issue #409: name-prefix stubs so they can be added to PATH
Allows us to reference them from playbooks easily.
6 years ago
David Wilson 5b916fc556 issue #409: Pythonize run_ansible_playbook.sh
And add git_basedir extra variable.
6 years ago
David Wilson 48e8f1f7aa tests: Pythonize write_all_consumer.py 6 years ago
David Wilson f53e7dd637 tests: Pythonize another shell script. 6 years ago
David Wilson 332d128651 tests: get rid of some more shell 6 years ago
David Wilson 71f9e84ab3 Add EOF error hints for LXC/LXD; closes #373. 6 years ago
David Wilson d5a8293c91 issue #333: closure & data distinctness tests. 6 years ago
David Wilson 73cda2994f issue #333: add versioning, initial batch of poller tests
Now poller is start enough to know a start_receive() during an iteration
does not cause events yielded by that iteration to associate with the
wrong descriptor.

These changes are tangentially related to the associated ticket, but
event versioning is still the underlying issue.
6 years ago
David Wilson 96f000c5ea ansible: tilde-expand SSH key before passing to SSH; closes #334. 6 years ago
David Wilson 9aa76cf9ce tests: better Docker test key comment. 6 years ago
David Wilson 1cbff1011e core: send dead message if max message size exceeded; closes #405 6 years ago
David Wilson 1eae594e32 ssh: fix check_host_keys="accept" and test; closes #411
Add real accept/enforce tests.
6 years ago
David Wilson 16ca111ebd ssh: better OpenSSH 7.5+ permission denied handling
The user@host prefix in new-style OpenSSH messages unfortunately takes
the host part from ~/.ssh/config and friends. There is no way to know
which hostname will appear in this string without parsing the OpenSSH
config, nor which username will appear.

Instead just regex it.

Add SSH stub modes to print the new/old errors and add some simple
tests.

This extends the work done in b9112a9cbb
6 years ago
David Wilson 519faa3b3b issue #369: add Connection.reset() test. 6 years ago
David Wilson 89852db163 issue #370: add 'disconnect resets connection' test 6 years ago
David Wilson c510e58f9b issue #352: add test for disconnect message. 6 years ago
David Wilson 53d882dcbd tests: activate faulthandler if available 6 years ago
David Wilson 9ec360c26d core: split out & extend Broker.sync_call() 6 years ago
David Wilson 592d6fc8d3 tests: fix CaptureStreamHandler on 2.6. 6 years ago
David Wilson 3429e57825 tests: fix target_test 3.x compat. 6 years ago
David Wilson 3585ee74f7 tests: split out ansible_tests requirements
Also remove hard-coded Ansible version, the tests don't need it, nor
does local testing most of the time
6 years ago
David Wilson b70c57a2cb tests: fix wstatus_to_str() test on 3.x
Now they use enums.
6 years ago
David Wilson c1c7e5171d tests: fix fork FD sharing in unix_test. 6 years ago
David Wilson 0cf6019bac tests: rename one more stubs/ssh.py reference. 6 years ago
David Wilson 778892eaaa issue #76: call_function_test fix. 6 years ago
David Wilson a7d635dff8 tests: import ara_env helper script. 6 years ago
David Wilson 4e3830d75e tests: add basic unix_test.py. 6 years ago
David Wilson 918f709420 tests: import a bunch more random unchecked in pieces. 6 years ago
David Wilson 0d70fc1324 tests: import z hostfile 6 years ago
David Wilson 36e5ca4115 tests: import missing main_with_no_exec_guard.py. 6 years ago
David Wilson 3943634fa6 tests: import bench/large_messages.py. 6 years ago
David Wilson 73055150f3 tests: move stub tools, into subdir, import docker_test. 6 years ago
David Wilson 7647c95f34 issue #76: add one more test for indirect siblings 6 years ago
David Wilson 431051f69b issue #76: parent: broadcast DEL_ROUTE to interested parties
Now rather than simply propagate DEL_ROUTE upwards towards the parent,
we broadcast it downward to any stream that ever sent a message toward
any of the routes that have just become disconnected.
6 years ago
David Wilson babe3eec31 issue #76: record egress context IDs
Used in a subsequent change to broadcast DEL_ROUTE to potentially
interested children.
6 years ago
David Wilson fd5066d671 tests: teach various tests to cleanup /tmp when they're done. 6 years ago
David Wilson 7fd9fb0014 issue #397: fix another case where stray tmpdirs can be left behind.
Newer Ansibles use atexit.register() to invoke cleanup, so we need to
run those registrations after each run.
6 years ago
David Wilson 1b17aa1d1a ansible: fix temp cleanup regression and add test; closes #397. 6 years ago
David Wilson e45e5d3e06 tests: Document Python versions in build_docker_images.py 6 years ago
David Wilson 3aa5c4c53d issue #373: parse the child process wait status
Don't log the raw waitpid() result, convert it to a useful string first.
6 years ago
David Wilson 9d070541d9 ansible: try to create tempdir if missing.
Closes #358.
6 years ago
David Wilson 6dd1001d7a tests: move kubectl into new subdir
Fixes tab completion with tests/ dir :)

CC @yannig
6 years ago
David Wilson 130e42a932 tests: prevent compare_output_test running on import. 6 years ago
David Wilson 4c81eba599 Merge commit 'refs/pull/377/head' of github.com:dw/mitogen into dmw
(Pull #377)

Changes:
- additional_parameters -> extra_args
- Merge with kubectl changes from dmw branch
- Update docs
- Remove unused username class member
- Avoid mutable kubectl_args class member
- Use six.iteritems
6 years ago
David Wilson 4146648759 master: log error an refuse __main__ import if no guard detected.
Closes #366.
6 years ago
David Wilson 6dddef0c45 Make image_prep work on Ubuntu. 6 years ago
David Wilson 2eb3ea78d6 tests: remove a bunch of stray debug 6 years ago
David Wilson 21a7aac220 tests: import custom binaries for tests
Same for async tests.
6 years ago
David Wilson dfb4930fce tests: import custom binaries for tests
It means Linux<->OS X runs work fine without manual hackery.
6 years ago
David Wilson 564113874b tests: explicitly define localhost in common-hosts 6 years ago
David Wilson 5521945bd2 ansible: temporary files take 5. 6 years ago
David Wilson f6b74992e1 tests: fix apparently erroneous localhost delegation.
The stack delegates to localhost, which has ansible_python_interpreter
set.
6 years ago
David Wilson e85760477b tests: fix connection/_put_file.yml
Was statting wrong destination path, and comparing floats that don't
roundtrip serialization reliably.
6 years ago
David Wilson 638e473ff1 tests: hacksmash synchronize test to work
Avoid password typing idiocy.
6 years ago
Yannig Perré 17548d1e49 [Enhancement] handle kubectl vars from Ansible connector.
This change allows the kubectl connector to support the same options as
Ansible's original connector.

The playbook sample comes with an example of a pod containing two containers
and checking that moving from one container to another, the version of Python
changes as expected.
6 years ago
dw 1a5dadf732
Merge pull request #383 from moreati/pytz-2018.05
Update pytz to 2018.05 (needed by babel 2.6.0)
6 years ago
Alex Willmer 03be0afeeb tests: Add tests of mitogen.utils.cast() 6 years ago
Alex Willmer 9fbcb67665 Update pytz to 2018.05 (needed by babel 2.6.0)
On Ubuntu 18.04 (others not tested) installing the dev requirements
fails with the following error

babel 2.6.0 has requirement pytz>=0a, but you'll have pytz 2012d which
is incompatible.

Despite the comment in dev_requirements.txt pytz-2012d is not the most
recent version to support Python 2.6. In fact the latest release of pytz
supports Python 2.6.
6 years ago
dw 4356fdf027
Merge pull request #376 from Yannig/kubectl
Kubernetes connection support for mitogen.
6 years ago
Yannig Perré 6828926a36 Kubernetes connection support for mitogen. 6 years ago
Brian Candler d1c84552ec Use `lxc exec --mode=noninteractive` which is more widely compatible
Closes #371
6 years ago
David Wilson 638b196a45 ansible: fix put_file() for large temporary files.
Reverts 49736b3a, large file copies can't avoid the RTT.

The parent stack must be blocked while FileService progresses, as unlike
the small file path, it does not make a snapshot of the (possibly
temporary) file passed by the action plug-in. So we need to keep that
file alive while the service runs.

Add a new integration test and a new soak test to cover both.
6 years ago
David Wilson 001b63074c tests: fix another typo. 6 years ago
David Wilson d5524178bf tests: fix bonehead syntax error. 6 years ago
David Wilson 65f03e03f5 tests: remote_tmp test fixes. 6 years ago
David Wilson 5eb41751f5 tests: import missing connection_passthrough 6 years ago
David Wilson 90f89f95fb ansible: fix exec_command() regression. 6 years ago
David Wilson 7cd4d0828d tests: data/fakessh.py 3.x fixes. 6 years ago
David Wilson 24a44499ca tests: verify Connection.put_file() for small/large files. 6 years ago
David Wilson 4d3873c784 core: call chains v3: abstract it into a new CallChain class. 6 years ago
David Wilson 705d77a9be ansible: remove a bunch more aliasing from connection.py. 6 years ago
David Wilson 66142e7d75 ansible: fork isolated tasks from correct parent.
Closes #355.
6 years ago
David Wilson a3957d6aaf parent: add Context.forget_chain(). 6 years ago
David Wilson 42b1b3d286 core: support mitogen_chain dispatcher option. 6 years ago
David Wilson e5d421e5f4 Update k3 inventory. 6 years ago
David Wilson 426cffd9f5 tests: set no_target_syslog 6 years ago
David Wilson 7d62a53264 issue #337: ssh: disabling PTYs round 2: make it automatic. 6 years ago
David Wilson 5004207705 issue #337: ssh: support disabling PTY allocation
`.ssh(batch_mode=True)`
6 years ago
David Wilson c4c6ae88a4 parent: raise a descriptive error when openpty fails. 6 years ago
David Wilson acf7fe56ee tests: a few more inventory helpers. 6 years ago
David Wilson b0ffc4e209 Copy random test setup changes out of linear2 branch. 6 years ago
David Wilson e52684c186 tests: enable display_args_to_stdout 6 years ago
David Wilson 0a2ae4d597 tests: tidy up issue_140.yml 6 years ago
David Wilson 72fa129f8a tests: fix clash when localhost is test-target 6 years ago
David Wilson 5bac246676 tests: make nice_stdout print failing task line number 6 years ago
David Wilson 9792b8b54f ansible: use template-expanded delegate_to hostname in one more location. 6 years ago
David Wilson 3b012e5bce tests: allow plugging in pprint/pprintpp via env. 6 years ago
Jesse London 3453d4d7d0 Python 3 support for classmethod call targets
There were two problems with detection and handling of class methods as call targets in Python 3:

* Methods no longer define `im_self` -- this is now only `__self__`
* The `types` module no longer defines a `ClassType`

The universally-compatible (v2.6+) solution was to switch to using the `inspect` module -- whose interface has been stable -- and to checking the method attribute `__self__`.

(It doesn't hurt that `inspect` checks are more brief and we now no longer need the `types` module here.)
6 years ago
David Wilson 90c2ed03d0 ansible: fix synchronize module
Broken by recent connection delegation fixes.
6 years ago
David Wilson 8e9b5ad576 tests: import template benchmark script. 6 years ago
David Wilson 084c0ac065 ansible: avoid roundtrip in copy action due to fixup_perms2().
On top of existing temporary files work, this reduces the number of
roundtrips required for "copy" and "template" actions from 6 to 3.
6 years ago
David Wilson 4098d45dac tests: disable delegation tests on vanilla. 6 years ago
David Wilson 50e285f7ba tests: update for identities_only change. 6 years ago
David Wilson a4ed27fa63 tests: add playbook step to ensure key file perms. 6 years ago
David Wilson ac9b84d237 issue #321: 2.4+ compatibility fixes, disable test on Vanilla. 6 years ago
David Wilson f24f02ba06 issue #321: take remote_tmp and system_tmpdirs into account.
Can't simply ignore these settings as some users may have weird noexec
filesystems.
6 years ago
David Wilson a2686b1a2c issue #321: simplify temp directory handling. 6 years ago
David Wilson 06e2e846c5 parent: don't generate illegal default remote names.
getpass.getuser() output may contain slashes, which must be avoided as
they break virtualenv when present in argv[0].

Closes #344.
6 years ago
David Wilson 3d588323ff issue #340: use expanded delegate_to hostname, not template.
PlayContext.delegate_to is the unexpanded template, Ansible doesn't keep
a copy of it around anywhere convenient. We either need to re-expand it
or take the expanded version that was stored on the Task, which is what
is done here.
6 years ago
David Wilson 154dc2e119 tests: fix integration/runner/missing_module.yml on Travis. 6 years ago
David Wilson 3113bf6228 tests: fix debops tests (py-apt broken if /var/lbi/apt missing) 6 years ago
David Wilson a6995a5288 issue #338: refactor env handling into class and fix tests. 6 years ago
David Wilson 06ae59702c tests: rationalize matrix and rewrite ansible_tests
Now all distros run in parallel.
6 years ago
David Wilson da391f0542 tests: fix host limit. 6 years ago
David Wilson ce058eb8bd Add 'clean' target to makefile. 6 years ago
David Wilson 8eb288856c issue #338: run /etc/environment test with become:true. 6 years ago
David Wilson ad365dad56 issue #340: one more test, update Changelog. 6 years ago
David Wilson 916e46621b issue #340: add connection delegation tests. 6 years ago
David Wilson aed8fb531b tests: unused imports 6 years ago
David Wilson d39efd9f54 tests: add new users for conndel tests. 6 years ago
David Wilson e48e32cd0c tests: image_prep fixes. 6 years ago
David Wilson e1306bb03d tests: build Docker images in parallel 6 years ago
David Wilson df112be704 tests: teach controller.yml to configure git too 6 years ago
David Wilson 49c804937d tests: import 2 more simple benchmarks. 6 years ago
David Wilson b521f215fd ansible: handle >2.6 magic exceptions + sys.excepthook damage
Closes #332.
6 years ago
David Wilson a192935daf tests: merge build_docker_images.py with osx_setup.yml
Hooray!
6 years ago