Commit Graph

526 Commits (master)

Author SHA1 Message Date
Steven Robertson de114be821 try and bump ansible to the latest 2.8 version to try and fix tests 4 years ago
Steven Robertson 552ae9c6d3 need become: yes for package install 4 years ago
Steven Robertson c705645ade need to ensure we can install rh-python for testing 4 years ago
Steven Robertson 51b43104e0 adding interpreter_discovery tests to ci 4 years ago
Steven Robertson 371ef07683 with statement doesn't exist in early python 4 years ago
Steven Robertson 9eedff97a6 closed fp after writing to temp files 4 years ago
Steven Robertson c9dd36dbc4 pegging back to ans 2.8 4 years ago
Steven Robertson 90d51fcbbd 'fix' test for virtualenv on mac 4 years ago
Steven Robertson 488ab3553e upgrade ansible back to 2.9.4, and fix issue with virtualenv on Mac 4 years ago
Steven Robertson e85720f4d5 peg to a previously-passing ansible version 4 years ago
Steven Robertson 6d179da480 ported over tests from ansible validating python interpreter discovery 4 years ago
Steven Robertson 868de4e5ba added ability to repro current virtualenv bug behind a proxy, and adding verbosity to see why azure is unhappy 4 years ago
David Wilson d6329f3446 Merge devel/290 @ 79b979ec8544ef5d8620c64068d4a42fabf50415 5 years ago
David Wilson 3023ab3b7b issue #633: skip test on older Ansibles. 5 years ago
David Wilson efd82dd35a issue #633: various task_vars fixes
- take host_vars from task_vars too
- make missing task_vars a hard error
- update tests to provide stub task_vars
5 years ago
David Wilson fc09b81949 issue #633: handle meta: reset_connection when become is active
- don't create a new connection during reset if no existing connection
  exists
- strip off last hop in connection stack if PlayContext.become is True.
- log a debug message if reset cannot find an existing connection
5 years ago
David Wilson 8a870f1402 issue #625: use exec() instead of subprocess in mitogen_ansible_playbook
This is just to make CTRL+C handling less confusing. Alternate would be
ignoring SIGINT, but this is simpler.
5 years ago
David Wilson 0e489625ed issue #615: regression test 5 years ago
David Wilson bc275b2526 tests: another random string changed in 2.8.3 5 years ago
David Wilson 7e0c2fd1af tests: fix sudo_flags_failure for Ansible 2.8.3 5 years ago
David Wilson 206a8d4aeb issue #558: disable test on OSX to cope with boundless mediocrity 5 years ago
David Wilson 8dfb3966df issue #558, #582: preserve remote tmpdir if caller did not supply one
The undocumented 'tmp' parameter controls whether _execute_module()
would delete anything on 2.3, so mimic that. This means
_execute_remove_stat() calls will not blow away the temp directory,
which broke the unarchive plugin.
5 years ago
David Wilson 4b9b1ca24d tests: work around AnsibleModule.run_command() race.
See https://github.com/ansible/ansible/issues/51393
5 years ago
David Wilson 9b45872246 issue #598: allow disabling preempt in terraform 5 years ago
David Wilson 588859423a issue #615: another Py3x fix. 5 years ago
David Wilson 9e1e1ba015 issue #615: Py3x fix. 5 years ago
David Wilson 5af6c9b26f issue #615: use FileService for target->controll file transfers 5 years ago
David Wilson 22a07a043b tests: fix error in affinity_test 5 years ago
David Wilson e4321f81a0 issue #600: /etc/environment may be non-ASCII in an unknown encoding 5 years ago
David Wilson 28b4d63e49 tests: terraform tweaks 5 years ago
David Wilson 5ed0b93669 tests: update gcloud.py to match terraform config 5 years ago
David Wilson de2e1ec184 tests/ansible/gcloud: terraform conf for load testing 5 years ago
David Wilson 108015aa22 ansible: gracefully handle failure to connect to MuxProcess
It's possible to hit an ugly exception during early CTRL+C
5 years ago
David Wilson a9d3fdf6b7 ansible: fix affinity tests for 5ae45f6612390bbc888b65964fb5c218feed1679 5 years ago
David Wilson 17d0e1b315 issue #543: skip test that's hard to do on Mac 5 years ago
David Wilson 501cfca82b issue #543: make localhost_ansible_tests run locally 5 years ago
David Wilson 115145555e [linear2] fix another test relying on Connection.parent 5 years ago
David Wilson a39169ff16 [linear2] fix mitogen_shutdown_all service context access 5 years ago
David Wilson 1fca0b7a94 [linear2] fix MuxProcess test fixture and some merge fallout 5 years ago
David Wilson 9035884c77 ansible: abstract worker process model.
Move all details of broker/router setup out of connection.py, instead
deferring it to a WorkerModel class exported by process.py via
get_worker_model(). The running strategy can override the configured
worker model via _get_worker_model().

ClassicWorkerModel is installed by default, which implements the
extension's existing process model.

Add optional support for the third party setproctitle module, so
children have pretty names in ps output.

Add optional support for per-CPU multiplexers to classic runs.
5 years ago
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 5 years ago
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
5 years ago
David Wilson ee7dae7514 ci: Another round of fixes for random Ansible UI breakage in 2.7/2.8 5 years ago
David Wilson ad5a80f200 Use virtualenv Python for stub connections to workaround problem
../data/stubs/stub-kubectl.py exec -it localhost -- /usr/bin/python -c "...":

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    LookupError: unknown encoding: base64

It's not clear why this is happening. "stub-kubectl.py" is executed with
the 2.7 virtualenv, while the exec() that happens inside stub-kubectl
was for "/usr/bin/python".

That second Python can't find chunks of its stdlib:

    stat("/usr/lib/python2.7/encodings/base64", 0x7ffde8744c60) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.so", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64module.so", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.py", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.pyc", O_RDONLY) = -1 ENOENT (No such file or directory)
    write(2, "Traceback (most recent call last):\n", 35) = 35
    write(2, "  File \"<string>\", line 1, in <module>\n", 39) = 39
5 years ago
David Wilson 08e7fe4f80 tests: add 2.8 format async error timeout message 5 years ago
David Wilson 418fc15e80 tests: allow running without hdrhistograms library. 5 years ago
David Wilson 3620fce071 issue #593: expose configurables for SSH keepalive and increase the default 5 years ago
David Wilson e90c05dc9d issue #591: fix test for Ansible 2.3. 5 years ago
David Wilson 0b7fd3f290 issue #591: ansible: restore CWD prior to AnsibleModule initialization. 5 years ago
David Wilson fe7c3610ff issue #590: disable distro test on vanilla 5 years ago
David Wilson a1f2ec222d issue #590: fix test for <2.8 Ansibles. 5 years ago
David Wilson e94200aeb9 issue #590: actually run Ansible test. 5 years ago
David Wilson 7a5c436a39 issue #590: Ansible test for module_utils.distro use. 5 years ago
David Wilson 61b651bdd2 issue #587: workaround for ansible/ansible#56629 5 years ago
David Wilson 8a18d1862e issue #587: "state: absent" was removed in 2.8.0
ansible/cc9c72d6f845710b24e952670b534a57f6948513
5 years ago
David Wilson 92b4724010 issue #587: consistent become_exe() behaviour for older Ansibles. 5 years ago
David Wilson a25fa566a1 issue #587: update stub_connections/ test to use new doas var 5 years ago
David Wilson a3be746865 issue #587: update MODULE FAILURE message format for post >2.7 5 years ago
David Wilson 54b5fdf761 issue #587: fix syntax error due to presence of comment 5 years ago
David Wilson ce80e326fe issue #587: disable deprecation_warnings for CI.
Our use case justifies it
5 years ago
David Wilson b8e1b4df51 issue #587: 2.8 whitespace handling was improved.
b72e989e18
5 years ago
David Wilson c616359a17 Import include processing bench 5 years ago
David Wilson f30a4c05c8 issue #581: expose mitogen_mask_remote_name variable. 5 years ago
David Wilson 65deb3feac issue #575: fix exception text rendering 5 years ago
David Wilson 3ff6123483 issue #557: support correct cpu_set_t size 5 years ago
David Wilson 2bd0bbd4df issue #555: ansible: workaround ancient reload(sys) hack.
This is the most minimal change for what might be relatively minimal
edge case. Alternative is replacing reload(), but let's not do that yet.

Closes #555
5 years ago
David Wilson 30b8172573 issue #554: mitogen_action_script fix 5 years ago
David Wilson 6309774be2 issue #554: fix Ansible 2.4 compatibility 5 years ago
David Wilson f36b4b47bf issue #554: don't rely on tmp_path autoremoval in test.
Ansible doesn't do this, so we shouldn't either.
5 years ago
David Wilson 7743e57ff3 issue #554: track and remove multiple make_tmp_path() calls. 5 years ago
David Wilson 26e6194d0a issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
The idea behind transport=smart is to select between paramiko and
OpenSSH given the availability of connection multiplexing and/or OSX
kernel bugs. We need to make no such choice.
5 years ago
David Wilson 0aa4c9d8fc issue #542: .ci: move some tests to Azure and enable Mac job. 5 years ago
David Wilson 5ed445c4aa issue #537: disable just the trivial LinuxPolicyTest on Travis. 5 years ago
David Wilson 8f9c67daf1 ansible: refactor affinity class and add abstract tests. 5 years ago
David Wilson 7fd0d34910 tests/ansible: Spec.port() test & mitogen_via= fix.
ansible_ssh_port was not respected.
5 years ago
David Wilson b5b23e8f3d tests/ansible: Spec.become_pass() test. 5 years ago
David Wilson 7b3ed52fd8 tests/ansible: Spec.become_user() test. 5 years ago
David Wilson 1c955a9876 ansible: capture stderr stream of async tasks. Closes #540. 5 years ago
David Wilson 8ae6ca1d5b tests/ansible: Spec.become_method() test & mitogen_via= fix.
ansible_become_method hostvar was not taken into account.
5 years ago
David Wilson 8ba75d82ec tests/ansible: Spec.become() test 5 years ago
David Wilson d1cadf8ac8 tests/ansible: Spec.password() test, document interactive pw limitation. 5 years ago
David Wilson 21ad299d7b tests/ansible: Spec.remote_user() test & mitogen_via= fix.
ansible_ssh_user precedence was incorrect.
5 years ago
David Wilson 748f5f675d tests/ansible: Spec.remote_addr() test & mitogen_via= fix.
ansible_ssh_host was not respected.
5 years ago
David Wilson 8fd641c442 tests/ansible: Spec.transport() test. 5 years ago
David Wilson e1df98168c issue #536: add mitogen_via= tests too. 5 years ago
David Wilson 2f29c76eec tests: for 2.3 compatibility, disable gcloud.py for now 5 years ago
David Wilson b3f20f54e6 issue #536: disable transport_config tests on vanilla 5 years ago
David Wilson 1d43e187e8 issue #536: connection_delegation/ tests were erroneously broken
While fixing delegate_to, this un-hardwiring of /usr/bin/python
happened. It was always incorrect.
5 years ago
David Wilson 0dfcf5560b tests: define MITOGEN_INVENTORY_FILE even if -i unspecified.
To fix running tests locally.
5 years ago
David Wilson 01faed708d issue #536: add tests for each ansible_python_interpreter case. 5 years ago
David Wilson 86f9572ef7 issue #536: stop defining explicit localhost in inventory.
This was needed at some point in the past, but the tests don't seem to
care about it any more. We'll fix any CI breakage by changing the tests,
since verifying implicit localhost behaviour is important.
5 years ago
David Wilson 81e386586f tests: allow running Ansible tests locally without -udmw again. 5 years ago
David Wilson 2aed0a0c6c tests: add new compression parameter to mitogen_get_stack results 5 years ago
David Wilson 6859186069 tests: disable affinity_test on Travis :/ 5 years ago
David Wilson 8026e54b9b tests: move affinity_test to Ansible tests. 5 years ago
David Wilson b2bae9deed tests: pad out localhost-* 5 years ago
David Wilson 29adfbea59 tests: add localhost-x100 5 years ago
David Wilson b72d139bd2 tests: make soak testing work reliably on vanilla.
It claims to disable host key checking, but it doesn't.
5 years ago
David Wilson b59a47aecb tests: make mitogen_shutdown_all be run_once by default. 5 years ago
David Wilson 2209d880e4 ansible/bench: make end= configurable. 5 years ago
David Wilson 954f874085 issue #527: catch new-style module tracebacks like vanilla. 5 years ago
David Wilson a40946297f issue #497: do our best to cope with crap upstream code 5 years ago
David Wilson eb93f82d05 tests: ensure file is closed in connection_test. 5 years ago
David Wilson 73979043ad gcloud: small updates 5 years ago
David Wilson 59068ca955 tests: give ansible/gcloud/ its own requirements file. 5 years ago
David Wilson 9df314f9c5 issue #499: another totally moronic implementation difference 5 years ago
David Wilson 53794469a0 issue #499: disable new test on vanilla. 5 years ago
David Wilson a1121c5a84 issue #499: respect C.BECOME_ALLOW_SAME_USER. 5 years ago
David Wilson b254051416 ansible: add test to ensure UNIX socket is cleaned on exit; closes #488. 5 years ago
David Wilson 628e8f4466 Bump PyYaML to a 3.7-compatible version. 5 years ago
David Wilson a7da1b048b tests: just disable the test. 5 years ago
David Wilson cdb1434809 tests: hopefully fix this dumb test for the final time 5 years ago
David Wilson a67cc85bdb issue #477: use MITOGEN_INVENTORY_FILE everywhere. 5 years ago
David Wilson ea23d7fd54 issue #477: hacksmash weird 2.3 inventory_file var issue. 5 years ago
David Wilson 9aff8edf50 issue #477: one more conditional test. 5 years ago
David Wilson e133b8dbb1 issue #477: some more conditional tests. 5 years ago
David Wilson 599da0689a issue #477 / ansible: avoid a race in async job startup.
Ansible 2.3/Python 2.4 work revealed there is no guarantee a slow target
will have written the initial job status file out before a fast
controller makes an initial check for it. Therefore, provide AsyncRunner
with a sender it should send a message to when the initial job file has
been written.

As a bonus, also catch and report exceptions happening early in
AsyncRunner, rather than leaving them to end up in -vvv output.
5 years ago
David Wilson c61498202e issue #477: use assert_equal for nicer debug. 5 years ago
David Wilson 5e4066bb64 issue #477: add Connection.homedir test. 5 years ago
David Wilson 8385c38cff issue #477: paper over Ansible 2.3 flag handling difference 5 years ago
David Wilson b67e4e118e issue #477: update forking_correct_parent for subprocess isolation 5 years ago
David Wilson abb0319bbb issue #477: get rid of perl JSON module requirement. 5 years ago
David Wilson 7d1b0e0807 issue #477: Ansible 2.3 did not support gather_facts min subset. 5 years ago
David Wilson 2f087859bc issue #477: 2.4-compatible syntax. 5 years ago
David Wilson 7abe5ff564 issue #477: clearing glibc caches is not possible on Py2.4. 5 years ago
David Wilson 51294db52d issue #477: fix 2 runner tests on Ansible 2.7. 5 years ago
David Wilson c443244986 issue #477: Ansible 2.3 compatible regression/all.yml. 5 years ago
David Wilson 532ae92029 issue #477: Ansible 2.3 requires placeholder module for assert_equals 5 years ago
David Wilson 0b0ae5c971 issue #477: backport various test modules to Python 2.4. 5 years ago
David Wilson 462a8567e5 issue #477: add all() polyfill to custom_python_detect_environmnet 5 years ago
David Wilson dc01f9e47e issue #477: Ansible 2.3 module output format difference.
Don't test for keys it doesn't set.
5 years ago
David Wilson b6062afe24 issue #477: Ansible 2.3 cannot use when: on an include.
So just use our magic action module to reset the connection instead.
5 years ago
David Wilson c1a4597e32 issue #477: tests: use Ansible 2.3-compatible include rather than import
This will break with Ansible 2.8. Probably going to end up with a regex
hack for CI.
5 years ago
David Wilson be8562ce09 issue #477: Ansible 2.3 had stricter arg spec format. 5 years ago
David Wilson 66b64cae2c issue #477: older Ansibles had no vars plugin base class. 5 years ago
David Wilson d6945443b7 tests: add exact test for issue 251; closes #251. 5 years ago
David Wilson 23866084d7 issue #412: promote "mitogen_get_stack" to the main extension.
This is to make it easier for users to diagnose their own problems.
5 years ago
David Wilson 2ad05f1238 issue #251, #412, #434: fix connection configuration brainwrong
This refactors connection.py to pull the two huge dict-building
functions out into new transport_transport_config.PlayContextSpec and
MitogenViaSpec classes, leaving a lot more room to breath in both files
to figure out exactly how connection configuration should work.

The changes made in 1f21a30 / 3d58832 are updated or completely removed,
the original change was misguided, in a bid to fix connection delegation
taking variables from the wrong place when delegate_to was active.

The Python path no longer defaults to '/usr/bin/python', this does not
appear to be Ansible's normal behaviour. This has changed several times,
so it may have to change again, and it may cause breakage after release.

Connection delegation respects the c.DEFAULT_REMOTE_USER whereas the
previous version simply tried to fetch whatever was in the
'ansible_user' hostvar. Many more connection delegation variables closer
match vanilla's handling, but this still requires more work. Some of the
variables need access to the command line, and upstream are in the
process of changing all that stuff around.
5 years ago
David Wilson 8414ff21ca issue #434: tests: set a default remote_user in ansible.cfg. 5 years ago
David Wilson 1b8748a8d9 tests: use assert_equal in more places. 5 years ago
David Wilson d72567b15b tests: make assert_equal work on newer Ansibles. 5 years ago
David Wilson 8891b48080 tests: convert stack_construction.yml to assert_equal. 5 years ago
David Wilson 4256d2aa4b tests: make fork_histogram optional 5 years ago
David Wilson 17eff064b0 tests: use assert_equal in delegate_to_template.yml. 5 years ago
David Wilson 91c9aff9ff tests: import assert_equal action. 5 years ago
David Wilson 630c058a89 tests: rename 'delegation/' to 'connection_delegation/' 5 years ago
David Wilson 0ba8cc7b61 tests: clean up / deduplicate Ansible inventory. 5 years ago
David Wilson 9d9add3339 issue #414: reenable test. 5 years ago
David Wilson 4bdf60326c issue #424: ansible: make put_file() raise AnsibleFileNotFound 5 years ago
David Wilson 8380baf41b issue #461: Ansible 2.3 placeholder modules for action plug-ins. 5 years ago
David Wilson b0e7c1a315 tests: fix /etc/environment test on vanilla
Use "meta: reset_connection" to ensure /etc/environment is reloaded.
Looks like this entire feature can be ripped out!
5 years ago
David Wilson a8921bb6ff tests: fix scaling in fork_histogram 5 years ago
David Wilson 6936b93b43 tests: import fork_histogram.py. 5 years ago
David Wilson 5a96d13fd8 issue #426: fix all.yml sorting, one more delegate_to 5 years ago
David Wilson bd82fa1986 issue #426: fix low_level_execute_command.yml breakage. 5 years ago
David Wilson a6e6bc4c71 issue #426: to_text filter. 5 years ago
David Wilson 9d87f03ab4 issue #426: disable Ansible smart transport. 5 years ago
David Wilson 49d37bfa2d issue #426: remove LANG and LC_ALL during tests. 5 years ago
David Wilson a6e6fd14be issue #426: more 2->3 test fixes. 5 years ago
David Wilson 67f710fd9b issue #426: use delegate_to in fixup_perms2 and copy.yml 5 years ago
David Wilson 3179951f5c issue #454: fix AttributeError and atexit.yml test. 6 years ago
David Wilson 22de7f0e72 issue #436: fix string parsing of mitogen_ssh_debug_level
It can be a string when specified on the command line.
6 years ago
David Wilson 44d6ca771a issue #426: fix local/delegate_to issue 6 years ago
David Wilson bef4b0c962 tests: fix copy.yml title 6 years ago
David Wilson b60a6d0f3b Split dev_requirements.txt up according to test mode. 6 years ago
David Wilson fccf424140 tests: Pythonize print_env.sh. 6 years ago
David Wilson a717c5406c tests: split etc_environment test in two
Turns out nobody supports ~/.pam_environment any more. Keep the
behaviour around for the time being.
6 years ago
David Wilson 374fd72dbb tests: disable mtime test on vanilla 6 years ago
David Wilson 79ca67aadd tests: disable connection tests for non-Mitogen 6 years ago
David Wilson 8972dbb7b9 tests: more Ansible fixes. 6 years ago
David Wilson acf0b04876 tests: run some playbooks against only one target. 6 years ago
David Wilson ee2d10375d tests: don't run reset_connection tests on <2.5.6. 6 years ago
David Wilson fcdfd5f107 tests: fix disconnect_cleanup.yml target count assumption 6 years ago
David Wilson 1bb239189b tests: another attempt at working paths. 6 years ago
David Wilson 5f815ec6c4 tests: try to fix PATH problem on Travis. 6 years ago
David Wilson f5f72b958f tests: avoid -u command line parameter conflict 6 years ago
David Wilson 0c3e48468b tests: run disconnect_during_module.yml in subprocess
Avoid entire run failing with unreachable
6 years ago
David Wilson 35092c5d35 tests: Unicode/bytes fixes for integration/connection/exec_command.yml 6 years ago
David Wilson 4d443e654b tests: replace another shell script. 6 years ago
David Wilson 9ad022107e issue #414: disable test until rest of CI is healthy 6 years ago
David Wilson f87553b165 tests: must set ansible_become_pass in synchronize.yml. 6 years ago
David Wilson 574fc27a9c issue #414: import test / reproduction. 6 years ago
David Wilson c286f4f107 Add tests/ansible/requirements.txt to Tox. 6 years ago
David Wilson 5eff8ea4fb tests: make result_shell_echo_hi compare less of the timedelta; closes #361
Assuming less than one second is too much to ask from Travis.
6 years ago
David Wilson c148c869e6 issue #76, #370: add disconnect cleanup test 6 years ago
David Wilson fd326f5ad7 tests: stub tests for doas/mitogen_doas 6 years ago
David Wilson cbd4129cb9 tests: fix paramiko_unblemished.yml 6 years ago
David Wilson e8fc9e490f tests: update osa_delegate_to_self to match connection parameters 6 years ago
David Wilson 6c71c5bfef issue #369: disable reset_connection on Ansible<2.5.6
https://github.com/ansible/ansible/issues/27520
6 years ago
David Wilson 8ed72e7e7b issue #369: avoid Ansible 2.5 bug (cond_reset_warn missing method) 6 years ago
David Wilson ab4ccc6b92 issue #369: don't mass-kill all SSH clients in reconnection.yml
It breaks my new development environment :)
6 years ago
David Wilson a6dd8bb2d0 issue #409: stub test for mitogen_sudo method. 6 years ago
David Wilson 144685a327 issue #409: more lxc/lxd stub tests, let tests run on vanilla. 6 years ago
David Wilson 0e8f451190 issue #409: add stub LXC test, refactor playbooks. 6 years ago
David Wilson e832ddec13 issue #409: mark sudo test noninteractive 6 years ago
David Wilson a77f07659e issue #409: make setns test to work anywhere sudo works. 6 years ago
David Wilson f2294c1678 issue #409: add new stub_connections/ test type. 6 years ago
David Wilson c51b67b863 issue #409: add test stubs to the PATH in run_ansible_playbook.py 6 years ago
David Wilson 5b916fc556 issue #409: Pythonize run_ansible_playbook.sh
And add git_basedir extra variable.
6 years ago
David Wilson 96f000c5ea ansible: tilde-expand SSH key before passing to SSH; closes #334. 6 years ago