Commit Graph

360 Commits (26a9fed396a981082e68288f019e82a88e3497f5)

Author SHA1 Message Date
David Wilson 599da0689a issue #477 / ansible: avoid a race in async job startup.
Ansible 2.3/Python 2.4 work revealed there is no guarantee a slow target
will have written the initial job status file out before a fast
controller makes an initial check for it. Therefore, provide AsyncRunner
with a sender it should send a message to when the initial job file has
been written.

As a bonus, also catch and report exceptions happening early in
AsyncRunner, rather than leaving them to end up in -vvv output.
6 years ago
David Wilson c61498202e issue #477: use assert_equal for nicer debug. 6 years ago
David Wilson 5e4066bb64 issue #477: add Connection.homedir test. 6 years ago
David Wilson 8385c38cff issue #477: paper over Ansible 2.3 flag handling difference 6 years ago
David Wilson b67e4e118e issue #477: update forking_correct_parent for subprocess isolation 6 years ago
David Wilson abb0319bbb issue #477: get rid of perl JSON module requirement. 6 years ago
David Wilson 7d1b0e0807 issue #477: Ansible 2.3 did not support gather_facts min subset. 6 years ago
David Wilson 2f087859bc issue #477: 2.4-compatible syntax. 6 years ago
David Wilson 7abe5ff564 issue #477: clearing glibc caches is not possible on Py2.4. 6 years ago
David Wilson 51294db52d issue #477: fix 2 runner tests on Ansible 2.7. 6 years ago
David Wilson c443244986 issue #477: Ansible 2.3 compatible regression/all.yml. 6 years ago
David Wilson 532ae92029 issue #477: Ansible 2.3 requires placeholder module for assert_equals 6 years ago
David Wilson 0b0ae5c971 issue #477: backport various test modules to Python 2.4. 6 years ago
David Wilson 462a8567e5 issue #477: add all() polyfill to custom_python_detect_environmnet 6 years ago
David Wilson dc01f9e47e issue #477: Ansible 2.3 module output format difference.
Don't test for keys it doesn't set.
6 years ago
David Wilson b6062afe24 issue #477: Ansible 2.3 cannot use when: on an include.
So just use our magic action module to reset the connection instead.
6 years ago
David Wilson c1a4597e32 issue #477: tests: use Ansible 2.3-compatible include rather than import
This will break with Ansible 2.8. Probably going to end up with a regex
hack for CI.
6 years ago
David Wilson be8562ce09 issue #477: Ansible 2.3 had stricter arg spec format. 6 years ago
David Wilson 66b64cae2c issue #477: older Ansibles had no vars plugin base class. 6 years ago
David Wilson d6945443b7 tests: add exact test for issue 251; closes #251. 6 years ago
David Wilson 23866084d7 issue #412: promote "mitogen_get_stack" to the main extension.
This is to make it easier for users to diagnose their own problems.
6 years ago
David Wilson 2ad05f1238 issue #251, #412, #434: fix connection configuration brainwrong
This refactors connection.py to pull the two huge dict-building
functions out into new transport_transport_config.PlayContextSpec and
MitogenViaSpec classes, leaving a lot more room to breath in both files
to figure out exactly how connection configuration should work.

The changes made in 1f21a30 / 3d58832 are updated or completely removed,
the original change was misguided, in a bid to fix connection delegation
taking variables from the wrong place when delegate_to was active.

The Python path no longer defaults to '/usr/bin/python', this does not
appear to be Ansible's normal behaviour. This has changed several times,
so it may have to change again, and it may cause breakage after release.

Connection delegation respects the c.DEFAULT_REMOTE_USER whereas the
previous version simply tried to fetch whatever was in the
'ansible_user' hostvar. Many more connection delegation variables closer
match vanilla's handling, but this still requires more work. Some of the
variables need access to the command line, and upstream are in the
process of changing all that stuff around.
6 years ago
David Wilson 8414ff21ca issue #434: tests: set a default remote_user in ansible.cfg. 6 years ago
David Wilson 1b8748a8d9 tests: use assert_equal in more places. 6 years ago
David Wilson d72567b15b tests: make assert_equal work on newer Ansibles. 6 years ago
David Wilson 8891b48080 tests: convert stack_construction.yml to assert_equal. 6 years ago
David Wilson 4256d2aa4b tests: make fork_histogram optional 6 years ago
David Wilson 17eff064b0 tests: use assert_equal in delegate_to_template.yml. 6 years ago
David Wilson 91c9aff9ff tests: import assert_equal action. 6 years ago
David Wilson 630c058a89 tests: rename 'delegation/' to 'connection_delegation/' 6 years ago
David Wilson 0ba8cc7b61 tests: clean up / deduplicate Ansible inventory. 6 years ago
David Wilson 9d9add3339 issue #414: reenable test. 6 years ago
David Wilson 4bdf60326c issue #424: ansible: make put_file() raise AnsibleFileNotFound 6 years ago
David Wilson 8380baf41b issue #461: Ansible 2.3 placeholder modules for action plug-ins. 6 years ago
David Wilson b0e7c1a315 tests: fix /etc/environment test on vanilla
Use "meta: reset_connection" to ensure /etc/environment is reloaded.
Looks like this entire feature can be ripped out!
6 years ago
David Wilson a8921bb6ff tests: fix scaling in fork_histogram 6 years ago
David Wilson 6936b93b43 tests: import fork_histogram.py. 6 years ago
David Wilson 5a96d13fd8 issue #426: fix all.yml sorting, one more delegate_to 6 years ago
David Wilson bd82fa1986 issue #426: fix low_level_execute_command.yml breakage. 6 years ago
David Wilson a6e6bc4c71 issue #426: to_text filter. 6 years ago
David Wilson 9d87f03ab4 issue #426: disable Ansible smart transport. 6 years ago
David Wilson 49d37bfa2d issue #426: remove LANG and LC_ALL during tests. 6 years ago
David Wilson a6e6fd14be issue #426: more 2->3 test fixes. 6 years ago
David Wilson 67f710fd9b issue #426: use delegate_to in fixup_perms2 and copy.yml 6 years ago
David Wilson 3179951f5c issue #454: fix AttributeError and atexit.yml test. 6 years ago
David Wilson 22de7f0e72 issue #436: fix string parsing of mitogen_ssh_debug_level
It can be a string when specified on the command line.
6 years ago
David Wilson 44d6ca771a issue #426: fix local/delegate_to issue 6 years ago
David Wilson bef4b0c962 tests: fix copy.yml title 6 years ago
David Wilson b60a6d0f3b Split dev_requirements.txt up according to test mode. 6 years ago
David Wilson fccf424140 tests: Pythonize print_env.sh. 6 years ago
David Wilson a717c5406c tests: split etc_environment test in two
Turns out nobody supports ~/.pam_environment any more. Keep the
behaviour around for the time being.
6 years ago
David Wilson 374fd72dbb tests: disable mtime test on vanilla 6 years ago
David Wilson 79ca67aadd tests: disable connection tests for non-Mitogen 6 years ago
David Wilson 8972dbb7b9 tests: more Ansible fixes. 6 years ago
David Wilson acf0b04876 tests: run some playbooks against only one target. 6 years ago
David Wilson ee2d10375d tests: don't run reset_connection tests on <2.5.6. 6 years ago
David Wilson fcdfd5f107 tests: fix disconnect_cleanup.yml target count assumption 6 years ago
David Wilson 1bb239189b tests: another attempt at working paths. 6 years ago
David Wilson 5f815ec6c4 tests: try to fix PATH problem on Travis. 6 years ago
David Wilson f5f72b958f tests: avoid -u command line parameter conflict 6 years ago
David Wilson 0c3e48468b tests: run disconnect_during_module.yml in subprocess
Avoid entire run failing with unreachable
6 years ago
David Wilson 35092c5d35 tests: Unicode/bytes fixes for integration/connection/exec_command.yml 6 years ago
David Wilson 4d443e654b tests: replace another shell script. 6 years ago
David Wilson 9ad022107e issue #414: disable test until rest of CI is healthy 6 years ago
David Wilson f87553b165 tests: must set ansible_become_pass in synchronize.yml. 6 years ago
David Wilson 574fc27a9c issue #414: import test / reproduction. 6 years ago
David Wilson c286f4f107 Add tests/ansible/requirements.txt to Tox. 6 years ago
David Wilson 5eff8ea4fb tests: make result_shell_echo_hi compare less of the timedelta; closes #361
Assuming less than one second is too much to ask from Travis.
6 years ago
David Wilson c148c869e6 issue #76, #370: add disconnect cleanup test 6 years ago
David Wilson fd326f5ad7 tests: stub tests for doas/mitogen_doas 6 years ago
David Wilson cbd4129cb9 tests: fix paramiko_unblemished.yml 6 years ago
David Wilson e8fc9e490f tests: update osa_delegate_to_self to match connection parameters 6 years ago
David Wilson 6c71c5bfef issue #369: disable reset_connection on Ansible<2.5.6
https://github.com/ansible/ansible/issues/27520
6 years ago
David Wilson 8ed72e7e7b issue #369: avoid Ansible 2.5 bug (cond_reset_warn missing method) 6 years ago
David Wilson ab4ccc6b92 issue #369: don't mass-kill all SSH clients in reconnection.yml
It breaks my new development environment :)
6 years ago
David Wilson a6dd8bb2d0 issue #409: stub test for mitogen_sudo method. 6 years ago
David Wilson 144685a327 issue #409: more lxc/lxd stub tests, let tests run on vanilla. 6 years ago
David Wilson 0e8f451190 issue #409: add stub LXC test, refactor playbooks. 6 years ago
David Wilson e832ddec13 issue #409: mark sudo test noninteractive 6 years ago
David Wilson a77f07659e issue #409: make setns test to work anywhere sudo works. 6 years ago
David Wilson f2294c1678 issue #409: add new stub_connections/ test type. 6 years ago
David Wilson c51b67b863 issue #409: add test stubs to the PATH in run_ansible_playbook.py 6 years ago
David Wilson 5b916fc556 issue #409: Pythonize run_ansible_playbook.sh
And add git_basedir extra variable.
6 years ago
David Wilson 96f000c5ea ansible: tilde-expand SSH key before passing to SSH; closes #334. 6 years ago
David Wilson 519faa3b3b issue #369: add Connection.reset() test. 6 years ago
David Wilson 89852db163 issue #370: add 'disconnect resets connection' test 6 years ago
David Wilson c510e58f9b issue #352: add test for disconnect message. 6 years ago
David Wilson 3429e57825 tests: fix target_test 3.x compat. 6 years ago
David Wilson 3585ee74f7 tests: split out ansible_tests requirements
Also remove hard-coded Ansible version, the tests don't need it, nor
does local testing most of the time
6 years ago
David Wilson a7d635dff8 tests: import ara_env helper script. 6 years ago
David Wilson 918f709420 tests: import a bunch more random unchecked in pieces. 6 years ago
David Wilson 0d70fc1324 tests: import z hostfile 6 years ago
David Wilson fd5066d671 tests: teach various tests to cleanup /tmp when they're done. 6 years ago
David Wilson 7fd9fb0014 issue #397: fix another case where stray tmpdirs can be left behind.
Newer Ansibles use atexit.register() to invoke cleanup, so we need to
run those registrations after each run.
6 years ago
David Wilson 1b17aa1d1a ansible: fix temp cleanup regression and add test; closes #397. 6 years ago
David Wilson 9d070541d9 ansible: try to create tempdir if missing.
Closes #358.
6 years ago
David Wilson 6dd1001d7a tests: move kubectl into new subdir
Fixes tab completion with tests/ dir :)

CC @yannig
6 years ago
David Wilson 130e42a932 tests: prevent compare_output_test running on import. 6 years ago
David Wilson 4c81eba599 Merge commit 'refs/pull/377/head' of github.com:dw/mitogen into dmw
(Pull #377)

Changes:
- additional_parameters -> extra_args
- Merge with kubectl changes from dmw branch
- Update docs
- Remove unused username class member
- Avoid mutable kubectl_args class member
- Use six.iteritems
6 years ago
David Wilson 2eb3ea78d6 tests: remove a bunch of stray debug 6 years ago
David Wilson 21a7aac220 tests: import custom binaries for tests
Same for async tests.
6 years ago
David Wilson dfb4930fce tests: import custom binaries for tests
It means Linux<->OS X runs work fine without manual hackery.
6 years ago
David Wilson 564113874b tests: explicitly define localhost in common-hosts 6 years ago
David Wilson 5521945bd2 ansible: temporary files take 5. 6 years ago
David Wilson f6b74992e1 tests: fix apparently erroneous localhost delegation.
The stack delegates to localhost, which has ansible_python_interpreter
set.
6 years ago
David Wilson e85760477b tests: fix connection/_put_file.yml
Was statting wrong destination path, and comparing floats that don't
roundtrip serialization reliably.
6 years ago
David Wilson 638e473ff1 tests: hacksmash synchronize test to work
Avoid password typing idiocy.
6 years ago
Yannig Perré 17548d1e49 [Enhancement] handle kubectl vars from Ansible connector.
This change allows the kubectl connector to support the same options as
Ansible's original connector.

The playbook sample comes with an example of a pod containing two containers
and checking that moving from one container to another, the version of Python
changes as expected.
6 years ago
Yannig Perré 6828926a36 Kubernetes connection support for mitogen. 6 years ago
David Wilson 638b196a45 ansible: fix put_file() for large temporary files.
Reverts 49736b3a, large file copies can't avoid the RTT.

The parent stack must be blocked while FileService progresses, as unlike
the small file path, it does not make a snapshot of the (possibly
temporary) file passed by the action plug-in. So we need to keep that
file alive while the service runs.

Add a new integration test and a new soak test to cover both.
6 years ago
David Wilson 001b63074c tests: fix another typo. 6 years ago
David Wilson d5524178bf tests: fix bonehead syntax error. 6 years ago
David Wilson 65f03e03f5 tests: remote_tmp test fixes. 6 years ago
David Wilson 5eb41751f5 tests: import missing connection_passthrough 6 years ago
David Wilson 90f89f95fb ansible: fix exec_command() regression. 6 years ago
David Wilson 24a44499ca tests: verify Connection.put_file() for small/large files. 6 years ago
David Wilson 705d77a9be ansible: remove a bunch more aliasing from connection.py. 6 years ago
David Wilson 66142e7d75 ansible: fork isolated tasks from correct parent.
Closes #355.
6 years ago
David Wilson e5d421e5f4 Update k3 inventory. 6 years ago
David Wilson 426cffd9f5 tests: set no_target_syslog 6 years ago
David Wilson acf7fe56ee tests: a few more inventory helpers. 6 years ago
David Wilson b0ffc4e209 Copy random test setup changes out of linear2 branch. 6 years ago
David Wilson e52684c186 tests: enable display_args_to_stdout 6 years ago
David Wilson 0a2ae4d597 tests: tidy up issue_140.yml 6 years ago
David Wilson 72fa129f8a tests: fix clash when localhost is test-target 6 years ago
David Wilson 5bac246676 tests: make nice_stdout print failing task line number 6 years ago
David Wilson 9792b8b54f ansible: use template-expanded delegate_to hostname in one more location. 6 years ago
David Wilson 3b012e5bce tests: allow plugging in pprint/pprintpp via env. 6 years ago
David Wilson 90c2ed03d0 ansible: fix synchronize module
Broken by recent connection delegation fixes.
6 years ago
David Wilson 8e9b5ad576 tests: import template benchmark script. 6 years ago
David Wilson 084c0ac065 ansible: avoid roundtrip in copy action due to fixup_perms2().
On top of existing temporary files work, this reduces the number of
roundtrips required for "copy" and "template" actions from 6 to 3.
6 years ago
David Wilson 4098d45dac tests: disable delegation tests on vanilla. 6 years ago
David Wilson 50e285f7ba tests: update for identities_only change. 6 years ago
David Wilson a4ed27fa63 tests: add playbook step to ensure key file perms. 6 years ago
David Wilson ac9b84d237 issue #321: 2.4+ compatibility fixes, disable test on Vanilla. 6 years ago
David Wilson f24f02ba06 issue #321: take remote_tmp and system_tmpdirs into account.
Can't simply ignore these settings as some users may have weird noexec
filesystems.
6 years ago
David Wilson a2686b1a2c issue #321: simplify temp directory handling. 6 years ago
David Wilson 3d588323ff issue #340: use expanded delegate_to hostname, not template.
PlayContext.delegate_to is the unexpanded template, Ansible doesn't keep
a copy of it around anywhere convenient. We either need to re-expand it
or take the expanded version that was stored on the Task, which is what
is done here.
6 years ago
David Wilson 154dc2e119 tests: fix integration/runner/missing_module.yml on Travis. 6 years ago
David Wilson a6995a5288 issue #338: refactor env handling into class and fix tests. 6 years ago
David Wilson 06ae59702c tests: rationalize matrix and rewrite ansible_tests
Now all distros run in parallel.
6 years ago
David Wilson da391f0542 tests: fix host limit. 6 years ago
David Wilson ce058eb8bd Add 'clean' target to makefile. 6 years ago
David Wilson 8eb288856c issue #338: run /etc/environment test with become:true. 6 years ago
David Wilson ad365dad56 issue #340: one more test, update Changelog. 6 years ago
David Wilson 916e46621b issue #340: add connection delegation tests. 6 years ago
David Wilson aed8fb531b tests: unused imports 6 years ago
David Wilson df112be704 tests: teach controller.yml to configure git too 6 years ago
David Wilson 49c804937d tests: import 2 more simple benchmarks. 6 years ago
David Wilson b521f215fd ansible: handle >2.6 magic exceptions + sys.excepthook damage
Closes #332.
6 years ago