Commit Graph

460 Commits (b5353aa6e0f653e0a96b776577357c92aa29661b)

Author SHA1 Message Date
David Wilson 8a870f1402 issue #625: use exec() instead of subprocess in mitogen_ansible_playbook
This is just to make CTRL+C handling less confusing. Alternate would be
ignoring SIGINT, but this is simpler.
5 years ago
David Wilson 0e489625ed issue #615: regression test 5 years ago
David Wilson bc275b2526 tests: another random string changed in 2.8.3 5 years ago
David Wilson 7e0c2fd1af tests: fix sudo_flags_failure for Ansible 2.8.3 5 years ago
David Wilson 206a8d4aeb issue #558: disable test on OSX to cope with boundless mediocrity 5 years ago
David Wilson 8dfb3966df issue #558, #582: preserve remote tmpdir if caller did not supply one
The undocumented 'tmp' parameter controls whether _execute_module()
would delete anything on 2.3, so mimic that. This means
_execute_remove_stat() calls will not blow away the temp directory,
which broke the unarchive plugin.
5 years ago
David Wilson 4b9b1ca24d tests: work around AnsibleModule.run_command() race.
See https://github.com/ansible/ansible/issues/51393
5 years ago
David Wilson 9b45872246 issue #598: allow disabling preempt in terraform 5 years ago
David Wilson 588859423a issue #615: another Py3x fix. 5 years ago
David Wilson 9e1e1ba015 issue #615: Py3x fix. 5 years ago
David Wilson 5af6c9b26f issue #615: use FileService for target->controll file transfers 5 years ago
David Wilson 22a07a043b tests: fix error in affinity_test 5 years ago
David Wilson e4321f81a0 issue #600: /etc/environment may be non-ASCII in an unknown encoding 5 years ago
David Wilson 28b4d63e49 tests: terraform tweaks 5 years ago
David Wilson 5ed0b93669 tests: update gcloud.py to match terraform config 5 years ago
David Wilson de2e1ec184 tests/ansible/gcloud: terraform conf for load testing 5 years ago
David Wilson 108015aa22 ansible: gracefully handle failure to connect to MuxProcess
It's possible to hit an ugly exception during early CTRL+C
5 years ago
David Wilson a9d3fdf6b7 ansible: fix affinity tests for 5ae45f6612390bbc888b65964fb5c218feed1679 5 years ago
David Wilson 17d0e1b315 issue #543: skip test that's hard to do on Mac 5 years ago
David Wilson 501cfca82b issue #543: make localhost_ansible_tests run locally 5 years ago
David Wilson 115145555e [linear2] fix another test relying on Connection.parent 5 years ago
David Wilson a39169ff16 [linear2] fix mitogen_shutdown_all service context access 5 years ago
David Wilson 1fca0b7a94 [linear2] fix MuxProcess test fixture and some merge fallout 5 years ago
David Wilson 9035884c77 ansible: abstract worker process model.
Move all details of broker/router setup out of connection.py, instead
deferring it to a WorkerModel class exported by process.py via
get_worker_model(). The running strategy can override the configured
worker model via _get_worker_model().

ClassicWorkerModel is installed by default, which implements the
extension's existing process model.

Add optional support for the third party setproctitle module, so
children have pretty names in ps output.

Add optional support for per-CPU multiplexers to classic runs.
5 years ago
David Wilson 1aceacf89e [stream-refactor] replace old detach_popen() reference 5 years ago
David Wilson 300f8b2ff9 ansible: fixturize creation of MuxProcess
This relies on the previous commit resetting global variables.

Update clean_shutdown() to handle duplicate calls, due to tests
repeatedly installing it.
5 years ago
David Wilson ee7dae7514 ci: Another round of fixes for random Ansible UI breakage in 2.7/2.8 5 years ago
David Wilson ad5a80f200 Use virtualenv Python for stub connections to workaround problem
../data/stubs/stub-kubectl.py exec -it localhost -- /usr/bin/python -c "...":

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    LookupError: unknown encoding: base64

It's not clear why this is happening. "stub-kubectl.py" is executed with
the 2.7 virtualenv, while the exec() that happens inside stub-kubectl
was for "/usr/bin/python".

That second Python can't find chunks of its stdlib:

    stat("/usr/lib/python2.7/encodings/base64", 0x7ffde8744c60) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.so", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64module.so", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.py", O_RDONLY) = -1 ENOENT (No such file or directory)
    open("/usr/lib/python2.7/encodings/base64.pyc", O_RDONLY) = -1 ENOENT (No such file or directory)
    write(2, "Traceback (most recent call last):\n", 35) = 35
    write(2, "  File \"<string>\", line 1, in <module>\n", 39) = 39
5 years ago
David Wilson 08e7fe4f80 tests: add 2.8 format async error timeout message 5 years ago
David Wilson 418fc15e80 tests: allow running without hdrhistograms library. 5 years ago
David Wilson 3620fce071 issue #593: expose configurables for SSH keepalive and increase the default 5 years ago
David Wilson e90c05dc9d issue #591: fix test for Ansible 2.3. 5 years ago
David Wilson 0b7fd3f290 issue #591: ansible: restore CWD prior to AnsibleModule initialization. 5 years ago
David Wilson fe7c3610ff issue #590: disable distro test on vanilla 5 years ago
David Wilson a1f2ec222d issue #590: fix test for <2.8 Ansibles. 5 years ago
David Wilson e94200aeb9 issue #590: actually run Ansible test. 5 years ago
David Wilson 7a5c436a39 issue #590: Ansible test for module_utils.distro use. 5 years ago
David Wilson 61b651bdd2 issue #587: workaround for ansible/ansible#56629 5 years ago
David Wilson 8a18d1862e issue #587: "state: absent" was removed in 2.8.0
ansible/cc9c72d6f845710b24e952670b534a57f6948513
5 years ago
David Wilson 92b4724010 issue #587: consistent become_exe() behaviour for older Ansibles. 5 years ago
David Wilson a25fa566a1 issue #587: update stub_connections/ test to use new doas var 5 years ago
David Wilson a3be746865 issue #587: update MODULE FAILURE message format for post >2.7 5 years ago
David Wilson 54b5fdf761 issue #587: fix syntax error due to presence of comment 5 years ago
David Wilson ce80e326fe issue #587: disable deprecation_warnings for CI.
Our use case justifies it
5 years ago
David Wilson b8e1b4df51 issue #587: 2.8 whitespace handling was improved.
b72e989e18
5 years ago
David Wilson c616359a17 Import include processing bench 5 years ago
David Wilson f30a4c05c8 issue #581: expose mitogen_mask_remote_name variable. 5 years ago
David Wilson 65deb3feac issue #575: fix exception text rendering 5 years ago
David Wilson 3ff6123483 issue #557: support correct cpu_set_t size 5 years ago
David Wilson 2bd0bbd4df issue #555: ansible: workaround ancient reload(sys) hack.
This is the most minimal change for what might be relatively minimal
edge case. Alternative is replacing reload(), but let's not do that yet.

Closes #555
5 years ago
David Wilson 30b8172573 issue #554: mitogen_action_script fix 5 years ago
David Wilson 6309774be2 issue #554: fix Ansible 2.4 compatibility 5 years ago
David Wilson f36b4b47bf issue #554: don't rely on tmp_path autoremoval in test.
Ansible doesn't do this, so we shouldn't either.
5 years ago
David Wilson 7743e57ff3 issue #554: track and remove multiple make_tmp_path() calls. 5 years ago
David Wilson 26e6194d0a issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
The idea behind transport=smart is to select between paramiko and
OpenSSH given the availability of connection multiplexing and/or OSX
kernel bugs. We need to make no such choice.
5 years ago
David Wilson 0aa4c9d8fc issue #542: .ci: move some tests to Azure and enable Mac job. 5 years ago
David Wilson 5ed445c4aa issue #537: disable just the trivial LinuxPolicyTest on Travis. 5 years ago
David Wilson 8f9c67daf1 ansible: refactor affinity class and add abstract tests. 5 years ago
David Wilson 7fd0d34910 tests/ansible: Spec.port() test & mitogen_via= fix.
ansible_ssh_port was not respected.
5 years ago
David Wilson b5b23e8f3d tests/ansible: Spec.become_pass() test. 5 years ago
David Wilson 7b3ed52fd8 tests/ansible: Spec.become_user() test. 5 years ago
David Wilson 1c955a9876 ansible: capture stderr stream of async tasks. Closes #540. 5 years ago
David Wilson 8ae6ca1d5b tests/ansible: Spec.become_method() test & mitogen_via= fix.
ansible_become_method hostvar was not taken into account.
5 years ago
David Wilson 8ba75d82ec tests/ansible: Spec.become() test 5 years ago
David Wilson d1cadf8ac8 tests/ansible: Spec.password() test, document interactive pw limitation. 5 years ago
David Wilson 21ad299d7b tests/ansible: Spec.remote_user() test & mitogen_via= fix.
ansible_ssh_user precedence was incorrect.
5 years ago
David Wilson 748f5f675d tests/ansible: Spec.remote_addr() test & mitogen_via= fix.
ansible_ssh_host was not respected.
5 years ago
David Wilson 8fd641c442 tests/ansible: Spec.transport() test. 5 years ago
David Wilson e1df98168c issue #536: add mitogen_via= tests too. 5 years ago
David Wilson 2f29c76eec tests: for 2.3 compatibility, disable gcloud.py for now 5 years ago
David Wilson b3f20f54e6 issue #536: disable transport_config tests on vanilla 5 years ago
David Wilson 1d43e187e8 issue #536: connection_delegation/ tests were erroneously broken
While fixing delegate_to, this un-hardwiring of /usr/bin/python
happened. It was always incorrect.
5 years ago
David Wilson 0dfcf5560b tests: define MITOGEN_INVENTORY_FILE even if -i unspecified.
To fix running tests locally.
5 years ago
David Wilson 01faed708d issue #536: add tests for each ansible_python_interpreter case. 5 years ago
David Wilson 86f9572ef7 issue #536: stop defining explicit localhost in inventory.
This was needed at some point in the past, but the tests don't seem to
care about it any more. We'll fix any CI breakage by changing the tests,
since verifying implicit localhost behaviour is important.
5 years ago
David Wilson 81e386586f tests: allow running Ansible tests locally without -udmw again. 5 years ago
David Wilson 2aed0a0c6c tests: add new compression parameter to mitogen_get_stack results 5 years ago
David Wilson 6859186069 tests: disable affinity_test on Travis :/ 5 years ago
David Wilson 8026e54b9b tests: move affinity_test to Ansible tests. 5 years ago
David Wilson b2bae9deed tests: pad out localhost-* 5 years ago
David Wilson 29adfbea59 tests: add localhost-x100 5 years ago
David Wilson b72d139bd2 tests: make soak testing work reliably on vanilla.
It claims to disable host key checking, but it doesn't.
5 years ago
David Wilson b59a47aecb tests: make mitogen_shutdown_all be run_once by default. 5 years ago
David Wilson 2209d880e4 ansible/bench: make end= configurable. 5 years ago
David Wilson 954f874085 issue #527: catch new-style module tracebacks like vanilla. 5 years ago
David Wilson a40946297f issue #497: do our best to cope with crap upstream code 6 years ago
David Wilson eb93f82d05 tests: ensure file is closed in connection_test. 6 years ago
David Wilson 73979043ad gcloud: small updates 6 years ago
David Wilson 59068ca955 tests: give ansible/gcloud/ its own requirements file. 6 years ago
David Wilson 9df314f9c5 issue #499: another totally moronic implementation difference 6 years ago
David Wilson 53794469a0 issue #499: disable new test on vanilla. 6 years ago
David Wilson a1121c5a84 issue #499: respect C.BECOME_ALLOW_SAME_USER. 6 years ago
David Wilson b254051416 ansible: add test to ensure UNIX socket is cleaned on exit; closes #488. 6 years ago
David Wilson 628e8f4466 Bump PyYaML to a 3.7-compatible version. 6 years ago
David Wilson a7da1b048b tests: just disable the test. 6 years ago
David Wilson cdb1434809 tests: hopefully fix this dumb test for the final time 6 years ago
David Wilson a67cc85bdb issue #477: use MITOGEN_INVENTORY_FILE everywhere. 6 years ago
David Wilson ea23d7fd54 issue #477: hacksmash weird 2.3 inventory_file var issue. 6 years ago
David Wilson 9aff8edf50 issue #477: one more conditional test. 6 years ago
David Wilson e133b8dbb1 issue #477: some more conditional tests. 6 years ago
David Wilson 599da0689a issue #477 / ansible: avoid a race in async job startup.
Ansible 2.3/Python 2.4 work revealed there is no guarantee a slow target
will have written the initial job status file out before a fast
controller makes an initial check for it. Therefore, provide AsyncRunner
with a sender it should send a message to when the initial job file has
been written.

As a bonus, also catch and report exceptions happening early in
AsyncRunner, rather than leaving them to end up in -vvv output.
6 years ago
David Wilson c61498202e issue #477: use assert_equal for nicer debug. 6 years ago
David Wilson 5e4066bb64 issue #477: add Connection.homedir test. 6 years ago
David Wilson 8385c38cff issue #477: paper over Ansible 2.3 flag handling difference 6 years ago
David Wilson b67e4e118e issue #477: update forking_correct_parent for subprocess isolation 6 years ago
David Wilson abb0319bbb issue #477: get rid of perl JSON module requirement. 6 years ago
David Wilson 7d1b0e0807 issue #477: Ansible 2.3 did not support gather_facts min subset. 6 years ago
David Wilson 2f087859bc issue #477: 2.4-compatible syntax. 6 years ago
David Wilson 7abe5ff564 issue #477: clearing glibc caches is not possible on Py2.4. 6 years ago
David Wilson 51294db52d issue #477: fix 2 runner tests on Ansible 2.7. 6 years ago
David Wilson c443244986 issue #477: Ansible 2.3 compatible regression/all.yml. 6 years ago
David Wilson 532ae92029 issue #477: Ansible 2.3 requires placeholder module for assert_equals 6 years ago
David Wilson 0b0ae5c971 issue #477: backport various test modules to Python 2.4. 6 years ago
David Wilson 462a8567e5 issue #477: add all() polyfill to custom_python_detect_environmnet 6 years ago
David Wilson dc01f9e47e issue #477: Ansible 2.3 module output format difference.
Don't test for keys it doesn't set.
6 years ago
David Wilson b6062afe24 issue #477: Ansible 2.3 cannot use when: on an include.
So just use our magic action module to reset the connection instead.
6 years ago
David Wilson c1a4597e32 issue #477: tests: use Ansible 2.3-compatible include rather than import
This will break with Ansible 2.8. Probably going to end up with a regex
hack for CI.
6 years ago
David Wilson be8562ce09 issue #477: Ansible 2.3 had stricter arg spec format. 6 years ago
David Wilson 66b64cae2c issue #477: older Ansibles had no vars plugin base class. 6 years ago
David Wilson d6945443b7 tests: add exact test for issue 251; closes #251. 6 years ago
David Wilson 23866084d7 issue #412: promote "mitogen_get_stack" to the main extension.
This is to make it easier for users to diagnose their own problems.
6 years ago
David Wilson 2ad05f1238 issue #251, #412, #434: fix connection configuration brainwrong
This refactors connection.py to pull the two huge dict-building
functions out into new transport_transport_config.PlayContextSpec and
MitogenViaSpec classes, leaving a lot more room to breath in both files
to figure out exactly how connection configuration should work.

The changes made in 1f21a30 / 3d58832 are updated or completely removed,
the original change was misguided, in a bid to fix connection delegation
taking variables from the wrong place when delegate_to was active.

The Python path no longer defaults to '/usr/bin/python', this does not
appear to be Ansible's normal behaviour. This has changed several times,
so it may have to change again, and it may cause breakage after release.

Connection delegation respects the c.DEFAULT_REMOTE_USER whereas the
previous version simply tried to fetch whatever was in the
'ansible_user' hostvar. Many more connection delegation variables closer
match vanilla's handling, but this still requires more work. Some of the
variables need access to the command line, and upstream are in the
process of changing all that stuff around.
6 years ago
David Wilson 8414ff21ca issue #434: tests: set a default remote_user in ansible.cfg. 6 years ago
David Wilson 1b8748a8d9 tests: use assert_equal in more places. 6 years ago
David Wilson d72567b15b tests: make assert_equal work on newer Ansibles. 6 years ago
David Wilson 8891b48080 tests: convert stack_construction.yml to assert_equal. 6 years ago
David Wilson 4256d2aa4b tests: make fork_histogram optional 6 years ago
David Wilson 17eff064b0 tests: use assert_equal in delegate_to_template.yml. 6 years ago
David Wilson 91c9aff9ff tests: import assert_equal action. 6 years ago
David Wilson 630c058a89 tests: rename 'delegation/' to 'connection_delegation/' 6 years ago
David Wilson 0ba8cc7b61 tests: clean up / deduplicate Ansible inventory. 6 years ago
David Wilson 9d9add3339 issue #414: reenable test. 6 years ago
David Wilson 4bdf60326c issue #424: ansible: make put_file() raise AnsibleFileNotFound 6 years ago
David Wilson 8380baf41b issue #461: Ansible 2.3 placeholder modules for action plug-ins. 6 years ago
David Wilson b0e7c1a315 tests: fix /etc/environment test on vanilla
Use "meta: reset_connection" to ensure /etc/environment is reloaded.
Looks like this entire feature can be ripped out!
6 years ago
David Wilson a8921bb6ff tests: fix scaling in fork_histogram 6 years ago
David Wilson 6936b93b43 tests: import fork_histogram.py. 6 years ago
David Wilson 5a96d13fd8 issue #426: fix all.yml sorting, one more delegate_to 6 years ago
David Wilson bd82fa1986 issue #426: fix low_level_execute_command.yml breakage. 6 years ago
David Wilson a6e6bc4c71 issue #426: to_text filter. 6 years ago
David Wilson 9d87f03ab4 issue #426: disable Ansible smart transport. 6 years ago
David Wilson 49d37bfa2d issue #426: remove LANG and LC_ALL during tests. 6 years ago
David Wilson a6e6fd14be issue #426: more 2->3 test fixes. 6 years ago
David Wilson 67f710fd9b issue #426: use delegate_to in fixup_perms2 and copy.yml 6 years ago
David Wilson 3179951f5c issue #454: fix AttributeError and atexit.yml test. 6 years ago
David Wilson 22de7f0e72 issue #436: fix string parsing of mitogen_ssh_debug_level
It can be a string when specified on the command line.
6 years ago
David Wilson 44d6ca771a issue #426: fix local/delegate_to issue 6 years ago
David Wilson bef4b0c962 tests: fix copy.yml title 6 years ago
David Wilson b60a6d0f3b Split dev_requirements.txt up according to test mode. 6 years ago
David Wilson fccf424140 tests: Pythonize print_env.sh. 6 years ago