Commit Graph

919 Commits (60fbea4b59cb558ffc3c8cfe9d1531367fc4be71)

Author SHA1 Message Date
David Wilson 572db1a385 issue #477: build a CentOS 5/Py2.4 container + playbook compat fixes. 6 years ago
David Wilson 0b0ae5c971 issue #477: backport various test modules to Python 2.4. 6 years ago
David Wilson 462a8567e5 issue #477: add all() polyfill to custom_python_detect_environmnet 6 years ago
David Wilson e9706a4a09 issue #477: _update_linecache() must append newlines. 6 years ago
David Wilson 19b708e141 issue #415, #477: Poller must handle POLLHUP too.
Linux will fire poll() with simply the POLLHUP bit set even though it
was not requested, resulting in an infinite loop.
6 years ago
David Wilson 97a96f5dd8 issue #477: rename and add tests for polyfill functions. 6 years ago
David Wilson dc01f9e47e issue #477: Ansible 2.3 module output format difference.
Don't test for keys it doesn't set.
6 years ago
David Wilson b6062afe24 issue #477: Ansible 2.3 cannot use when: on an include.
So just use our magic action module to reset the connection instead.
6 years ago
David Wilson c1a4597e32 issue #477: tests: use Ansible 2.3-compatible include rather than import
This will break with Ansible 2.8. Probably going to end up with a regex
hack for CI.
6 years ago
David Wilson 83aff538b3 issue #477: testlib: Py2.4 did not have BaseException. 6 years ago
David Wilson be8562ce09 issue #477: Ansible 2.3 had stricter arg spec format. 6 years ago
David Wilson 0ee8ee78b8 issue #477: Py2.4 cannot tolerate unicode kwargs. 6 years ago
David Wilson 4388e794ce issue #477: Py2.4: enumerate() may return stopped threads. 6 years ago
David Wilson 07401d767a issue #477: Python 2.4 type(exc) returns old-style instance. 6 years ago
David Wilson 2ead451277 issue #477: Python 2.4 lacked str.partition. 6 years ago
David Wilson 2ec19f2374 issue #477: Python 2.4 lacked Thread.name. 6 years ago
David Wilson 2f016b5c7b issue #477: Python 2.4 lacked context managers. 6 years ago
David Wilson 66b64cae2c issue #477: older Ansibles had no vars plugin base class. 6 years ago
David Wilson d6945443b7 tests: add exact test for issue 251; closes #251. 6 years ago
David Wilson 23866084d7 issue #412: promote "mitogen_get_stack" to the main extension.
This is to make it easier for users to diagnose their own problems.
6 years ago
David Wilson 2ad05f1238 issue #251, #412, #434: fix connection configuration brainwrong
This refactors connection.py to pull the two huge dict-building
functions out into new transport_transport_config.PlayContextSpec and
MitogenViaSpec classes, leaving a lot more room to breath in both files
to figure out exactly how connection configuration should work.

The changes made in 1f21a30 / 3d58832 are updated or completely removed,
the original change was misguided, in a bid to fix connection delegation
taking variables from the wrong place when delegate_to was active.

The Python path no longer defaults to '/usr/bin/python', this does not
appear to be Ansible's normal behaviour. This has changed several times,
so it may have to change again, and it may cause breakage after release.

Connection delegation respects the c.DEFAULT_REMOTE_USER whereas the
previous version simply tried to fetch whatever was in the
'ansible_user' hostvar. Many more connection delegation variables closer
match vanilla's handling, but this still requires more work. Some of the
variables need access to the command line, and upstream are in the
process of changing all that stuff around.
6 years ago
David Wilson 8414ff21ca issue #434: tests: set a default remote_user in ansible.cfg. 6 years ago
David Wilson 1b8748a8d9 tests: use assert_equal in more places. 6 years ago
David Wilson d72567b15b tests: make assert_equal work on newer Ansibles. 6 years ago
David Wilson 8891b48080 tests: convert stack_construction.yml to assert_equal. 6 years ago
David Wilson 4256d2aa4b tests: make fork_histogram optional 6 years ago
David Wilson 17eff064b0 tests: use assert_equal in delegate_to_template.yml. 6 years ago
David Wilson 91c9aff9ff tests: import assert_equal action. 6 years ago
David Wilson 630c058a89 tests: rename 'delegation/' to 'connection_delegation/' 6 years ago
David Wilson f5d9af80ef tests: some more utility function tests + flake8. 6 years ago
David Wilson 0ba8cc7b61 tests: clean up / deduplicate Ansible inventory. 6 years ago
David Wilson 1555f23215 tests: add some more helper function tests. 6 years ago
David Wilson 9d9add3339 issue #414: reenable test. 6 years ago
David Wilson fcc403cc2f core: make Receiver.to_sender() use Router.myself(). 6 years ago
David Wilson 4bdf60326c issue #424: ansible: make put_file() raise AnsibleFileNotFound 6 years ago
David Wilson 06415bb720 issue #310: fix test failures, teach old import method new tricks
- don't try anything unless something really lives in sys.modules by
  that name
- non-ASCII files are possible
- the unimportable thing might be an extension module, we don't want
  that
6 years ago
David Wilson 6af1a64cce master: handle crazy non-modules in sys.modules again; closes #310. 6 years ago
David Wilson 7c33979e25 issue #415: add IPC latency bench. 6 years ago
David Wilson a8c056c27b issue #436: decode debug log lines on Python 3. 6 years ago
David Wilson 8380baf41b issue #461: Ansible 2.3 placeholder modules for action plug-ins. 6 years ago
David Wilson 822978520f issue #446: update Receiver.__iter__ to match
iter() previously relied on the fake dead message being enqueued.
6 years ago
David Wilson 386d869d5f tests: fix responder_test after removing unused imports 6 years ago
David Wilson 6592598999 tests: stray socket file left during unix_test. 6 years ago
David Wilson cce1dbf3b1 tests: quieten a bunch of spam printed during run 6 years ago
David Wilson b0e7c1a315 tests: fix /etc/environment test on vanilla
Use "meta: reset_connection" to ensure /etc/environment is reloaded.
Looks like this entire feature can be ripped out!
6 years ago
David Wilson 26fdbf70e0 tests: good_load_module_size check can't be exact
Since message contains pathnames that vary across machines
6 years ago
David Wilson ae03d5f4f1 issue #446: basic tests for Receiver._on_receive(). 6 years ago
David Wilson bcd9827c3b core: Latch.empty() improvements
- throw LatchError if the latch is closed.
- wrap with the lock to avoid unexpected weirdness.
6 years ago
David Wilson 388649df97 core: Receiver.close() now wakes all threads; closes #446. 6 years ago
David Wilson f2f41809ae issue #459: initial get_stats() implementation 6 years ago
David Wilson 1d97493fcd tests: fallout from #447. 6 years ago
David Wilson de719fa249 core: throw error on duplicate add_handler(); closes #447. 6 years ago
David Wilson dc92e529bc service: unregister receiver at shutdown; closes #445. 6 years ago
David Wilson 85e965118d Merge remote-tracking branch 'origin/issue462'
* origin/issue462:
  issue #462: docs: update Changelog.
  parent: cope with broken /dev/pts on Linux; closes #462.
6 years ago
David Wilson 9da6e6a9a8 tests: don't call Router.shutdown() twice.
Due to 497234e7 / issue #456.
6 years ago
David Wilson b2b7e7b06b tests: file_service_test fixes
Due to 6713b90acc
6 years ago
David Wilson a8921bb6ff tests: fix scaling in fork_histogram 6 years ago
David Wilson 6936b93b43 tests: import fork_histogram.py. 6 years ago
David Wilson 5a96d13fd8 issue #426: fix all.yml sorting, one more delegate_to 6 years ago
David Wilson bd82fa1986 issue #426: fix low_level_execute_command.yml breakage. 6 years ago
David Wilson a6e6bc4c71 issue #426: to_text filter. 6 years ago
David Wilson 9d87f03ab4 issue #426: disable Ansible smart transport. 6 years ago
David Wilson 49d37bfa2d issue #426: remove LANG and LC_ALL during tests. 6 years ago
David Wilson a6e6fd14be issue #426: more 2->3 test fixes. 6 years ago
David Wilson 67f710fd9b issue #426: use delegate_to in fixup_perms2 and copy.yml 6 years ago
David Wilson a4c7a98dd9 parent: cope with broken /dev/pts on Linux; closes #462. 6 years ago
David Wilson 3179951f5c issue #454: fix AttributeError and atexit.yml test. 6 years ago
David Wilson 1eb08fb5c5 core: docstring tidyups 6 years ago
David Wilson 497234e782 issue #456: core: raise error during defer() if Broker shutdown 6 years ago
David Wilson f20e0bbac1 service: Allow registering path prefixes with FileService.
e.g. service.register_prefix('/') disables all security checks.
6 years ago
David Wilson 6fafc0a631 select: Python 2.x required __nonzero__ for bool(Select) 6 years ago
David Wilson 300cb41e2e core: detect stream corruption. Closes #438. 6 years ago
David Wilson 3876590aa1 parent: add descriptive errors for unsupported call() types.
Closes #439.
6 years ago
David Wilson 22de7f0e72 issue #436: fix string parsing of mitogen_ssh_debug_level
It can be a string when specified on the command line.
6 years ago
David Wilson 44d6ca771a issue #426: fix local/delegate_to issue 6 years ago
David Wilson 934d8ac139 issue #426: fix 2->3 issue in plain_old_module. 6 years ago
David Wilson 3206d59c87 issue #426: teach DockerMixin to allow selecting interpreter 6 years ago
David Wilson f1661abe4e tests: make IterReadTest a little more robust 6 years ago
David Wilson bef4b0c962 tests: fix copy.yml title 6 years ago
David Wilson 2a6dbb038f Pythonize, split out and parallelize all jobs. 6 years ago
David Wilson b60a6d0f3b Split dev_requirements.txt up according to test mode. 6 years ago
David Wilson fccf424140 tests: Pythonize print_env.sh. 6 years ago
David Wilson 8f1f3de123 tests: Pythonize env_wrapper.sh, clean up local_test 6 years ago
David Wilson a717c5406c tests: split etc_environment test in two
Turns out nobody supports ~/.pam_environment any more. Keep the
behaviour around for the time being.
6 years ago
David Wilson 4b61e5af02 tests: run FD and thread checks on every test case.
Trying to hunt down weirdness on Azure.
6 years ago
David Wilson 374fd72dbb tests: disable mtime test on vanilla 6 years ago
David Wilson 79ca67aadd tests: disable connection tests for non-Mitogen 6 years ago
David Wilson 8972dbb7b9 tests: more Ansible fixes. 6 years ago
David Wilson acf0b04876 tests: run some playbooks against only one target. 6 years ago
David Wilson ee2d10375d tests: don't run reset_connection tests on <2.5.6. 6 years ago
David Wilson fcdfd5f107 tests: fix disconnect_cleanup.yml target count assumption 6 years ago
David Wilson 1bb239189b tests: another attempt at working paths. 6 years ago
David Wilson 5f815ec6c4 tests: try to fix PATH problem on Travis. 6 years ago
David Wilson f5f72b958f tests: avoid -u command line parameter conflict 6 years ago
David Wilson 0c3e48468b tests: run disconnect_during_module.yml in subprocess
Avoid entire run failing with unreachable
6 years ago
David Wilson 35092c5d35 tests: Unicode/bytes fixes for integration/connection/exec_command.yml 6 years ago
David Wilson 4d443e654b tests: replace another shell script. 6 years ago
David Wilson 9ad022107e issue #414: disable test until rest of CI is healthy 6 years ago
David Wilson b29c8eaf2a tests: allow passing -vvv to build_docker_images. 6 years ago
David Wilson f87553b165 tests: must set ansible_become_pass in synchronize.yml. 6 years ago
David Wilson 816da64df5 tests: show task args in image_prep 6 years ago
David Wilson 174b685d16 tests: CentOS 6 lacks %wheel in sudo by default. 6 years ago
David Wilson 574fc27a9c issue #414: import test / reproduction. 6 years ago
David Wilson c286f4f107 Add tests/ansible/requirements.txt to Tox. 6 years ago
David Wilson cd6486b0e9 tests: fix more DisconnectTest raciness. 6 years ago
David Wilson e180d310b5 tests: fix fork_test compat on 3.x. 6 years ago
David Wilson 3f46c9569c tests: 3.x syntax compat for tests/data/stubs/ 6 years ago
David Wilson 27a4001f4f tests: handle NameError when faulthandler is not installed. 6 years ago
David Wilson 5482b4d528 tests: poller_test 3.x fix. 6 years ago
David Wilson 3836c6a220 tests/bench: run roundtrip.py a ton more to reduce variance 6 years ago
David Wilson d1b7c232bf tests: image_prep needs sudo 6 years ago
David Wilson cf97932fad core: dead messages have optional body, use it everywhere; closes #387. 6 years ago
David Wilson 802de6a8d5 issue #406: clean up DiagLogStream handling and connect() failure.
When Stream.connect() fails, have it just use on_disconnect(). Now there
is a single disconnect cleanup path.

Remove cutpasted DiagLogStream setup/destruction, and move it into the
base class (temporarily), and only manage the lifetime of its underlying
FD via Side.close().  This cures another EBADF failure.
6 years ago
David Wilson e01c8f2891 issue #406: 3.x syntax fixes. 6 years ago
David Wilson 003526ef7b issue #406: fix thread leaks in unix_test too. 6 years ago
David Wilson eae1bdba4e tests: make minify_test print something useful on failure 6 years ago
David Wilson 802efa6ea6 issue #406: ensure broker_test waits for broker exit. 6 years ago
David Wilson 175fc377d2 tests: remove hard-wired SSL paths from fork_test. 6 years ago
David Wilson 3da4b1a420 tests: verify only main/watcher threads exist at teardown 6 years ago
David Wilson b0dd628f07 issue #406: parent_test fixes, NameError in log_fd_calls(). 6 years ago
David Wilson 6ff1e001da issue #406: log socketpair calls too. 6 years ago
David Wilson 70c550f50c issue #406: close stdout pipes in parent_test 6 years ago
David Wilson 9b3cb55a8b issue #4096: import log_fd_calls() helper. 6 years ago
David Wilson 8a0b343760 issue #406: test for FD leak after every TestCase 6 years ago
David Wilson e9a6e4c3d2 issue #406: add test. 6 years ago
David Wilson 5eff8ea4fb tests: make result_shell_echo_hi compare less of the timedelta; closes #361
Assuming less than one second is too much to ask from Travis.
6 years ago
David Wilson 5be9a55bf4 core: allow Context to be pickled by non-Mitogen pickler. 6 years ago
David Wilson c148c869e6 issue #76, #370: add disconnect cleanup test 6 years ago
David Wilson aeec2b5054 tests: pad out minify_test to verify all internal modules 6 years ago
David Wilson f2d288bb1e tests: ensure minify() result can be compiled for all of core. 6 years ago
David Wilson fd326f5ad7 tests: stub tests for doas/mitogen_doas 6 years ago
David Wilson cbd4129cb9 tests: fix paramiko_unblemished.yml 6 years ago
David Wilson e8fc9e490f tests: update osa_delegate_to_self to match connection parameters 6 years ago
David Wilson 6c71c5bfef issue #369: disable reset_connection on Ansible<2.5.6
https://github.com/ansible/ansible/issues/27520
6 years ago
David Wilson 8ed72e7e7b issue #369: avoid Ansible 2.5 bug (cond_reset_warn missing method) 6 years ago
David Wilson ab4ccc6b92 issue #369: don't mass-kill all SSH clients in reconnection.yml
It breaks my new development environment :)
6 years ago
David Wilson fcdf4a0f35 Import missing stub-sudo.py. 6 years ago
David Wilson a6dd8bb2d0 issue #409: stub test for mitogen_sudo method. 6 years ago
David Wilson fadb9181bc issue #410: support sudo --user and SELinux options, add stub test. 6 years ago
David Wilson 144685a327 issue #409: more lxc/lxd stub tests, let tests run on vanilla. 6 years ago
David Wilson 0e8f451190 issue #409: add stub LXC test, refactor playbooks. 6 years ago
David Wilson e832ddec13 issue #409: mark sudo test noninteractive 6 years ago
David Wilson a77f07659e issue #409: make setns test to work anywhere sudo works. 6 years ago
David Wilson f2294c1678 issue #409: add new stub_connections/ test type. 6 years ago
David Wilson 429832b8f7 issue #409: add kubectl stub and constructor test. 6 years ago
David Wilson c51b67b863 issue #409: add test stubs to the PATH in run_ansible_playbook.py 6 years ago
David Wilson 51658fdd66 issue #409: name-prefix stubs so they can be added to PATH
Allows us to reference them from playbooks easily.
6 years ago
David Wilson 5b916fc556 issue #409: Pythonize run_ansible_playbook.sh
And add git_basedir extra variable.
6 years ago
David Wilson 48e8f1f7aa tests: Pythonize write_all_consumer.py 6 years ago
David Wilson f53e7dd637 tests: Pythonize another shell script. 6 years ago
David Wilson 332d128651 tests: get rid of some more shell 6 years ago
David Wilson 71f9e84ab3 Add EOF error hints for LXC/LXD; closes #373. 6 years ago
David Wilson d5a8293c91 issue #333: closure & data distinctness tests. 6 years ago
David Wilson 73cda2994f issue #333: add versioning, initial batch of poller tests
Now poller is start enough to know a start_receive() during an iteration
does not cause events yielded by that iteration to associate with the
wrong descriptor.

These changes are tangentially related to the associated ticket, but
event versioning is still the underlying issue.
6 years ago
David Wilson 96f000c5ea ansible: tilde-expand SSH key before passing to SSH; closes #334. 6 years ago
David Wilson 9aa76cf9ce tests: better Docker test key comment. 6 years ago
David Wilson 1cbff1011e core: send dead message if max message size exceeded; closes #405 6 years ago
David Wilson 1eae594e32 ssh: fix check_host_keys="accept" and test; closes #411
Add real accept/enforce tests.
6 years ago
David Wilson 16ca111ebd ssh: better OpenSSH 7.5+ permission denied handling
The user@host prefix in new-style OpenSSH messages unfortunately takes
the host part from ~/.ssh/config and friends. There is no way to know
which hostname will appear in this string without parsing the OpenSSH
config, nor which username will appear.

Instead just regex it.

Add SSH stub modes to print the new/old errors and add some simple
tests.

This extends the work done in b9112a9cbb
6 years ago
David Wilson 519faa3b3b issue #369: add Connection.reset() test. 6 years ago
David Wilson 89852db163 issue #370: add 'disconnect resets connection' test 6 years ago
David Wilson c510e58f9b issue #352: add test for disconnect message. 6 years ago
David Wilson 53d882dcbd tests: activate faulthandler if available 6 years ago
David Wilson 9ec360c26d core: split out & extend Broker.sync_call() 6 years ago
David Wilson 592d6fc8d3 tests: fix CaptureStreamHandler on 2.6. 6 years ago
David Wilson 3429e57825 tests: fix target_test 3.x compat. 6 years ago
David Wilson 3585ee74f7 tests: split out ansible_tests requirements
Also remove hard-coded Ansible version, the tests don't need it, nor
does local testing most of the time
6 years ago
David Wilson b70c57a2cb tests: fix wstatus_to_str() test on 3.x
Now they use enums.
6 years ago
David Wilson c1c7e5171d tests: fix fork FD sharing in unix_test. 6 years ago
David Wilson 0cf6019bac tests: rename one more stubs/ssh.py reference. 6 years ago
David Wilson 778892eaaa issue #76: call_function_test fix. 6 years ago
David Wilson a7d635dff8 tests: import ara_env helper script. 6 years ago
David Wilson 4e3830d75e tests: add basic unix_test.py. 6 years ago
David Wilson 918f709420 tests: import a bunch more random unchecked in pieces. 6 years ago
David Wilson 0d70fc1324 tests: import z hostfile 6 years ago
David Wilson 36e5ca4115 tests: import missing main_with_no_exec_guard.py. 6 years ago
David Wilson 3943634fa6 tests: import bench/large_messages.py. 6 years ago
David Wilson 73055150f3 tests: move stub tools, into subdir, import docker_test. 6 years ago
David Wilson 7647c95f34 issue #76: add one more test for indirect siblings 6 years ago
David Wilson 431051f69b issue #76: parent: broadcast DEL_ROUTE to interested parties
Now rather than simply propagate DEL_ROUTE upwards towards the parent,
we broadcast it downward to any stream that ever sent a message toward
any of the routes that have just become disconnected.
6 years ago
David Wilson babe3eec31 issue #76: record egress context IDs
Used in a subsequent change to broadcast DEL_ROUTE to potentially
interested children.
6 years ago
David Wilson fd5066d671 tests: teach various tests to cleanup /tmp when they're done. 6 years ago
David Wilson 7fd9fb0014 issue #397: fix another case where stray tmpdirs can be left behind.
Newer Ansibles use atexit.register() to invoke cleanup, so we need to
run those registrations after each run.
6 years ago
David Wilson 1b17aa1d1a ansible: fix temp cleanup regression and add test; closes #397. 6 years ago
David Wilson e45e5d3e06 tests: Document Python versions in build_docker_images.py 6 years ago
David Wilson 3aa5c4c53d issue #373: parse the child process wait status
Don't log the raw waitpid() result, convert it to a useful string first.
6 years ago
David Wilson 9d070541d9 ansible: try to create tempdir if missing.
Closes #358.
6 years ago
David Wilson 6dd1001d7a tests: move kubectl into new subdir
Fixes tab completion with tests/ dir :)

CC @yannig
6 years ago
David Wilson 130e42a932 tests: prevent compare_output_test running on import. 6 years ago
David Wilson 4c81eba599 Merge commit 'refs/pull/377/head' of github.com:dw/mitogen into dmw
(Pull #377)

Changes:
- additional_parameters -> extra_args
- Merge with kubectl changes from dmw branch
- Update docs
- Remove unused username class member
- Avoid mutable kubectl_args class member
- Use six.iteritems
6 years ago
David Wilson 4146648759 master: log error an refuse __main__ import if no guard detected.
Closes #366.
6 years ago
David Wilson 6dddef0c45 Make image_prep work on Ubuntu. 6 years ago
David Wilson 2eb3ea78d6 tests: remove a bunch of stray debug 6 years ago
David Wilson 21a7aac220 tests: import custom binaries for tests
Same for async tests.
6 years ago
David Wilson dfb4930fce tests: import custom binaries for tests
It means Linux<->OS X runs work fine without manual hackery.
6 years ago
David Wilson 564113874b tests: explicitly define localhost in common-hosts 6 years ago
David Wilson 5521945bd2 ansible: temporary files take 5. 6 years ago
David Wilson f6b74992e1 tests: fix apparently erroneous localhost delegation.
The stack delegates to localhost, which has ansible_python_interpreter
set.
6 years ago
David Wilson e85760477b tests: fix connection/_put_file.yml
Was statting wrong destination path, and comparing floats that don't
roundtrip serialization reliably.
6 years ago
David Wilson 638e473ff1 tests: hacksmash synchronize test to work
Avoid password typing idiocy.
6 years ago
Yannig Perré 17548d1e49 [Enhancement] handle kubectl vars from Ansible connector.
This change allows the kubectl connector to support the same options as
Ansible's original connector.

The playbook sample comes with an example of a pod containing two containers
and checking that moving from one container to another, the version of Python
changes as expected.
6 years ago
dw 1a5dadf732
Merge pull request #383 from moreati/pytz-2018.05
Update pytz to 2018.05 (needed by babel 2.6.0)
6 years ago
Alex Willmer 03be0afeeb tests: Add tests of mitogen.utils.cast() 6 years ago
Alex Willmer 9fbcb67665 Update pytz to 2018.05 (needed by babel 2.6.0)
On Ubuntu 18.04 (others not tested) installing the dev requirements
fails with the following error

babel 2.6.0 has requirement pytz>=0a, but you'll have pytz 2012d which
is incompatible.

Despite the comment in dev_requirements.txt pytz-2012d is not the most
recent version to support Python 2.6. In fact the latest release of pytz
supports Python 2.6.
6 years ago
dw 4356fdf027
Merge pull request #376 from Yannig/kubectl
Kubernetes connection support for mitogen.
6 years ago
Yannig Perré 6828926a36 Kubernetes connection support for mitogen. 6 years ago
Brian Candler d1c84552ec Use `lxc exec --mode=noninteractive` which is more widely compatible
Closes #371
6 years ago
David Wilson 638b196a45 ansible: fix put_file() for large temporary files.
Reverts 49736b3a, large file copies can't avoid the RTT.

The parent stack must be blocked while FileService progresses, as unlike
the small file path, it does not make a snapshot of the (possibly
temporary) file passed by the action plug-in. So we need to keep that
file alive while the service runs.

Add a new integration test and a new soak test to cover both.
6 years ago
David Wilson 001b63074c tests: fix another typo. 6 years ago
David Wilson d5524178bf tests: fix bonehead syntax error. 6 years ago
David Wilson 65f03e03f5 tests: remote_tmp test fixes. 6 years ago
David Wilson 5eb41751f5 tests: import missing connection_passthrough 6 years ago
David Wilson 90f89f95fb ansible: fix exec_command() regression. 6 years ago
David Wilson 7cd4d0828d tests: data/fakessh.py 3.x fixes. 6 years ago
David Wilson 24a44499ca tests: verify Connection.put_file() for small/large files. 6 years ago
David Wilson 4d3873c784 core: call chains v3: abstract it into a new CallChain class. 6 years ago
David Wilson 705d77a9be ansible: remove a bunch more aliasing from connection.py. 6 years ago
David Wilson 66142e7d75 ansible: fork isolated tasks from correct parent.
Closes #355.
6 years ago
David Wilson a3957d6aaf parent: add Context.forget_chain(). 6 years ago
David Wilson 42b1b3d286 core: support mitogen_chain dispatcher option. 6 years ago
David Wilson e5d421e5f4 Update k3 inventory. 6 years ago
David Wilson 426cffd9f5 tests: set no_target_syslog 6 years ago
David Wilson 7d62a53264 issue #337: ssh: disabling PTYs round 2: make it automatic. 6 years ago
David Wilson 5004207705 issue #337: ssh: support disabling PTY allocation
`.ssh(batch_mode=True)`
6 years ago
David Wilson c4c6ae88a4 parent: raise a descriptive error when openpty fails. 6 years ago
David Wilson acf7fe56ee tests: a few more inventory helpers. 6 years ago
David Wilson b0ffc4e209 Copy random test setup changes out of linear2 branch. 6 years ago
David Wilson e52684c186 tests: enable display_args_to_stdout 6 years ago
David Wilson 0a2ae4d597 tests: tidy up issue_140.yml 6 years ago
David Wilson 72fa129f8a tests: fix clash when localhost is test-target 6 years ago
David Wilson 5bac246676 tests: make nice_stdout print failing task line number 6 years ago
David Wilson 9792b8b54f ansible: use template-expanded delegate_to hostname in one more location. 6 years ago
David Wilson 3b012e5bce tests: allow plugging in pprint/pprintpp via env. 6 years ago
Jesse London 3453d4d7d0 Python 3 support for classmethod call targets
There were two problems with detection and handling of class methods as call targets in Python 3:

* Methods no longer define `im_self` -- this is now only `__self__`
* The `types` module no longer defines a `ClassType`

The universally-compatible (v2.6+) solution was to switch to using the `inspect` module -- whose interface has been stable -- and to checking the method attribute `__self__`.

(It doesn't hurt that `inspect` checks are more brief and we now no longer need the `types` module here.)
6 years ago
David Wilson 90c2ed03d0 ansible: fix synchronize module
Broken by recent connection delegation fixes.
6 years ago
David Wilson 8e9b5ad576 tests: import template benchmark script. 6 years ago
David Wilson 084c0ac065 ansible: avoid roundtrip in copy action due to fixup_perms2().
On top of existing temporary files work, this reduces the number of
roundtrips required for "copy" and "template" actions from 6 to 3.
6 years ago
David Wilson 4098d45dac tests: disable delegation tests on vanilla. 6 years ago
David Wilson 50e285f7ba tests: update for identities_only change. 6 years ago
David Wilson a4ed27fa63 tests: add playbook step to ensure key file perms. 6 years ago
David Wilson ac9b84d237 issue #321: 2.4+ compatibility fixes, disable test on Vanilla. 6 years ago
David Wilson f24f02ba06 issue #321: take remote_tmp and system_tmpdirs into account.
Can't simply ignore these settings as some users may have weird noexec
filesystems.
6 years ago
David Wilson a2686b1a2c issue #321: simplify temp directory handling. 6 years ago
David Wilson 06e2e846c5 parent: don't generate illegal default remote names.
getpass.getuser() output may contain slashes, which must be avoided as
they break virtualenv when present in argv[0].

Closes #344.
6 years ago
David Wilson 3d588323ff issue #340: use expanded delegate_to hostname, not template.
PlayContext.delegate_to is the unexpanded template, Ansible doesn't keep
a copy of it around anywhere convenient. We either need to re-expand it
or take the expanded version that was stored on the Task, which is what
is done here.
6 years ago
David Wilson 154dc2e119 tests: fix integration/runner/missing_module.yml on Travis. 6 years ago
David Wilson 3113bf6228 tests: fix debops tests (py-apt broken if /var/lbi/apt missing) 6 years ago
David Wilson a6995a5288 issue #338: refactor env handling into class and fix tests. 6 years ago
David Wilson 06ae59702c tests: rationalize matrix and rewrite ansible_tests
Now all distros run in parallel.
6 years ago
David Wilson da391f0542 tests: fix host limit. 6 years ago
David Wilson ce058eb8bd Add 'clean' target to makefile. 6 years ago
David Wilson 8eb288856c issue #338: run /etc/environment test with become:true. 6 years ago
David Wilson ad365dad56 issue #340: one more test, update Changelog. 6 years ago
David Wilson 916e46621b issue #340: add connection delegation tests. 6 years ago
David Wilson aed8fb531b tests: unused imports 6 years ago
David Wilson d39efd9f54 tests: add new users for conndel tests. 6 years ago
David Wilson e48e32cd0c tests: image_prep fixes. 6 years ago
David Wilson e1306bb03d tests: build Docker images in parallel 6 years ago
David Wilson df112be704 tests: teach controller.yml to configure git too 6 years ago
David Wilson 49c804937d tests: import 2 more simple benchmarks. 6 years ago
David Wilson b521f215fd ansible: handle >2.6 magic exceptions + sys.excepthook damage
Closes #332.
6 years ago
David Wilson a192935daf tests: merge build_docker_images.py with osx_setup.yml
Hooray!
6 years ago
David Wilson 6f524d3ff8 issue #339: minimal tests for lxc/lxd modules. 6 years ago
David Wilson 2c2fc73b0a tests: whups, s/ansible_hostname/ansible_host/ 6 years ago
David Wilson 6c4b01642c ansible: don't crash when adhoc tries to run a missing module.
ansible-playbook prints a separate error during parsing stage, adhoc
performs no such check.
6 years ago
David Wilson b964e647d2 tests: split inventory up slightly.
This makes it easier to run connection delegation tests against either
the local machine or a container.
6 years ago
David Wilson 1473f49505 ansible: emulate /etc/environment reloading behaviour of vanilla.
This change is relatively incomplete -- ideally we could snapshot
os.environ and /etc/environment at startup and respect key deletions
too, but that's a lot more work. Wait for a bug report instead.

Closes #338.
6 years ago
David Wilson a05835c46e tests: more stable roundtrip.py. 6 years ago
David Wilson 3e0de9790c issue #324: fix Python 3 fallout for custom module_utils.
Also enable at last one of its tests.
6 years ago
David Wilson 1008cda93b tests: add missing debops installs tep 6 years ago
David Wilson b44b823c4a ansible: make _remote_expand_user() pay attention to sudoable=.. 6 years ago
David Wilson a8e4dcc98d issue #301: correct remote_tmp evaluation context.
Vanilla Ansible expands remote_tmp variables in the context of the login
account, not any become_user account.
6 years ago
David Wilson 9410903f20 issue #301: add related test. 6 years ago
David Wilson b6d6468c92 issue #301: support expandvars() for remote_tmp only.
Vanilla Ansible support expandvars-like expansions widely in a variety
of places. Prefer to whitelist those we need, rather than sprinkling
hellish semantics everywhere.
6 years ago
David Wilson 1a74938ee0 issue #291: missing env_wrapper.sh test script. 6 years ago
David Wilson e39c602fd3 issue #291: support UNIX hashbang syntax for ansible_*_interpreter.
Closes #291.
6 years ago
David Wilson f977be2868 issue #291: permit supplying a full Python argv. 6 years ago
David Wilson 15d68b3c32 issue #309: fix environment cleanup regression.
Closes #309.
6 years ago
David Wilson 830a133ad6 issue #307: require partial line when matching interactive prompt.
This is a best-effort attempt to avoid SSHd banner spam from breaking
our password entry loop.

Closes #307.
6 years ago
David Wilson 8ce51ec96c issue #307: add SSH login banner to Docker containers 6 years ago
David Wilson 0b1f92b59f issue #297: test fixes to match fixed CWD. 6 years ago
David Wilson d8e0c9e12c issue #297: local commands must execute with WorkerProcess environment. 6 years ago
David Wilson 012745efea issue #297: local actions must execute with fixed directory.
Local actions must execute in the the parent directory of the playbook
that defines the action.
6 years ago
David Wilson 26ba3e4d83 issue #294: stronger integration test
Run the tests in a brand new ansible-playbook invocation, and try both
linear->mitogen->linear and mitogen->linear->mitogen.
6 years ago
David Wilson 5b03e06457 issue #294: ansible: fix mixed vanilla/Mitogen runs.
Don't bother trying to understand what damage PluginLoader has done to
ansible.plugins.* namespace, just ask it for the base class instead.
6 years ago
David Wilson 5668267f47 tests: import debug_ansible_playbook.sh. 6 years ago
David Wilson e1402b2214 tests: import various bench/ scripts. 6 years ago
David Wilson 8a2e8964a9 tests: import gce.yml 6 years ago
David Wilson 079dc95aef tests: use nice_stdout in gcloud/ too 6 years ago
David Wilson b21d51be47 ansible: fix test for 3.x. 6 years ago
David Wilson 04e138e060 core: fix serialization of empty bytes() on 3.x. 6 years ago
David Wilson b4f4b66679 tests: make cpu_load.py more random. 6 years ago
David Wilson 46ec0ffcaf tests: log original traceback in assertRaises(). 6 years ago
David Wilson 3e48f95439 tests: import cpu_load.py. 6 years ago
David Wilson 7ff9d573b6 tests: remove stray enable_debug() 6 years ago
David Wilson 4ff47d6a93 parent: more 2/3x format fixes 6 years ago
David Wilson 90f2f823db tests: remove stray hard-coded interpreter. 6 years ago
David Wilson 6b4e047017 tests: 3.x parent_test fixes. 6 years ago
David Wilson 6cbf34d5fe tests: another smattering of 3.x type/API fixes. 6 years ago
David Wilson 7697861011 tests: skip one test that triggers bug in ancient vendered six.py 6 years ago
David Wilson c141dd10ec master: fix resolve_relpath()
looks like this was just as broken on 2.x, and suddenly we're
finding a bunch more legit Django deps. It seems anywhere
absolute_import appeared in 2.x, we skipped some imports.
6 years ago
David Wilson b0404bef40 tests: fix get_module_via_* encoding issues 6 years ago
David Wilson 9903692811 master: update scan_code_imports to cope with wordcode
Constant-sized opcodes were introduced as an optimization in Python 3.6.
See https://bugs.python.org/issue26647
6 years ago
David Wilson 826c477061 tests: built-in exceptions live in a different module in 3.x 6 years ago
David Wilson cbcd8267d1 tests: must decode() zlib result for 3.x. 6 years ago
David Wilson ec176bf1f8 tests: unittest2 import behaviour appears to differ between 2/3.
In 2 the module is loaded as __main__
In 3 it has its actual name.
6 years ago
David Wilson 410016ff47 Initial Python 3.x port work.
* ansible: use unicode_literals everywhere since it only needs to be
  compatible back to 2.6.
* compat/collections.py: delete this entirely and rip out the parts of
  functools that require it.
* Introduce serializable Kwargs dict subclass that translates keys to
  Unicode on instantiation.
* enable_debug_logging() must set _v/_vv globals.
* cStringIO does not exist in 3.x.
* Treat IOLogger and LogForwarder input as latin-1.
* Avoid ResourceWarnings in first stage by explicitly closing fps.
* Fix preamble_size.py syntax errors.
6 years ago
David Wilson 4f57c59b7e issue #275: Don't run virtualnv test on 2.6. 6 years ago
David Wilson cec564654e issue #275: tests: fix module_finder_test for 2.6. 6 years ago
David Wilson 4be8afa3d3 issue #275: tests: fix test_simple for 2.6. 6 years ago
David Wilson 7b84a2c2e4 issue #275: tests: use same EC0_MARKER as parent.py 6 years ago
David Wilson 6e0883f369 issue #275: tests: fix bug in 2.6 compat check_output(), ignore it for >2.6. 6 years ago
David Wilson 3b1cc3676c issue #275: ssh_debug_level=3 for tests 6 years ago
David Wilson 6d618593f3 issue #275: Python 2.6 reports linux as 'linux3'. 6 years ago
David Wilson 1d04a99adb issue #275: missing check_output() call 6 years ago
David Wilson 38d69a6ecd issue #275: tests: drop docker client dep, doesn't run on 2.6. 6 years ago
David Wilson b38318dfec issue #275: build for centos 6 too (python2.6) 6 years ago
dw 27ab051289
Merge pull request #282 from dw/issue278
Issue278
6 years ago
David Wilson 6d14652077 issue #278: tests: fix fakessh.
See source comment. This behaviour always existed, but it now seems to
be triggered since we started draining the master side input buffer,
which someone was prolonging the life of the PTY.
6 years ago
Alex Willmer 21199f290e Fix bash loop when add users to docker images 7 years ago
David Wilson 9617f4d7bf Revert "try to catch EINTR on travis"
This reverts commit 42797d5cff.
7 years ago
David Wilson fbb67e837e tests: import nice_stdout plugin 7 years ago
David Wilson 42797d5cff try to catch EINTR on travis 7 years ago
David Wilson ffc7306cf8 tests: better runner_two_simultaneous_jobs.yml. 7 years ago
David Wilson 1d96d80e8d tests: osx_setup.yml missing line 7 years ago
David Wilson 3994f1b30a ansible: implment async job time limit. 7 years ago
David Wilson df8fe59eda tests: replace hard-coded sleep with a polling loop 7 years ago
David Wilson 64b60be50c tests: split runner_new_process out of runner_one_job 7 years ago
David Wilson f7d2eace08 tests: importer fixes 7 years ago
David Wilson cee76ee7b9 tests: gcloud setup fixes. 7 years ago
David Wilson 3b0addcfb0 service: v2. Closes #213 7 years ago
David Wilson 633585524f tests: don't need separate module for id_allocation_test any more
This used to be because everything would explode while importing
__main__ under py.test, but that was fixed months ago.
7 years ago
David Wilson ddf28987a0 master: split Select() into new module to reduce wire size.
service.py currently imports master.py(+parent.py) just to get Select().
7 years ago
David Wilson b0ce6eecd7 fork: support on_start= argument. 7 years ago
David Wilson 00edf0d66d core: have ExternalContext accept a config dict rather than kwargs.
The parameter lists had gotten out of control.
7 years ago
David Wilson eac4cc7afe tests: speed things up 7 years ago
David Wilson b0aa413173 tests: import benchmark reproduction setup playbook
Incomplete, also needs to mess around with Gcloud routing and handle box
setup/teardown, because that's another "well engineered" waste of time.
7 years ago
David Wilson a99c7a12f9 tests: output split groups in gcloud.py 7 years ago
David Wilson dbcee4041a tests: refactor gcloud.py to be dynamic inventory. 7 years ago
David Wilson a3995f8e5f ansible: remove hard-coded dw username. 7 years ago
David Wilson 2c58591129 Bump Ansible version in dev_requirements.txt 7 years ago
David Wilson 78d375e0c5 core: CallError should handle any exception type.
Previously SystemExit would be pickled.
7 years ago
David Wilson faaac43a78 Disable SSH variables test on vanilla Ansible. 7 years ago
David Wilson 68d6e6bf34 ansible: tests for all SSH variables.
Closes #184.
7 years ago
David Wilson 356647bef4 issue #132: initial unidirectional routing mode. 7 years ago
David Wilson 2a4157723a tests: fix breakage from moving is_stdlib_name() 7 years ago
David Wilson 7d9b7eec0c tests: must wrap log_to_file() in is_master
Without it, end up with a billion loggers attached in some tests
7 years ago
David Wilson 2ad0d0521d ansible: reload /etc/resolv.conf for each task.
The OpenShift installer modifies /etc/resolv.conf then tests the new
resolver configuration, however, there was no mechanism to reload
resolv.conf in our reuseable interpreter.

https://github.com/openshift/openshift-ansible/blob/release-3.9/roles/openshift_web_console/tasks/install.yml#L137

This inserts an explicit call to res_init() for every new style
invocation, with an approximate cost of ~1usec on Linux since glibc
verifies resolv.conf has changed before reloading it.

There is little to be done for users of the thread-safe resolver APIs,
their state is hidden from us. If bugs like that manifest, whack-a-mole
style 'del sys.modules[thatmod]' patches may suffice.
7 years ago
David Wilson 60781ecab6 tests: fix test for vanilla Ansible. 7 years ago
David Wilson eb56a8f98d tests: add debug 7 years ago
David Wilson 267f787d20 issue #217: tests: import (unused) module_utils tests. 7 years ago
David Wilson ee741da2b3 tests: reenable remaining Ansible tests. 7 years ago
David Wilson cecef992b0 issue #218: core: add Secret and Blob types. 7 years ago
David Wilson be5c03c152 tests: many test fixes. Travis broken for >1 week. 7 years ago
David Wilson 69f58875c8 tests: more ansible fixes from recent brakage. 7 years ago
David Wilson b20174729d issue #199: fix readonly_homedir test. 7 years ago
David Wilson f98279fc95 tests: fix LRU test after splitting up Connection class.
The module the connection class is now loaded as is
"ansible.plugins.connection.mitogen_ssh", etc., which breaks the test.
Instead, check if the connection is an instance of the base Connection
class.
7 years ago
David Wilson f9e1905ec6 issue #199: ansible: stop writing temp files for new style modules
While adding support for non-new style module types, NewStyleRunner
began writing modules to a temporary file, and sys.argv was patched to
actually include the script filename. The argv change was never required
to fix any particular bug, and a search of the standard modules reveals
no argv users. Update argv[0] to be '', like an interactive interpreter
would have.

While fixing #210, new style runner began setting __file__ to the
temporary file path in order to allow apt.py to discover the Ansiballz
temporary directory. 5 out of 1,516 standard modules follow this
pattern, but in each case, none actually attempt to access __file__,
they just call dirname on it. Therefore do not write the contents of
file, simply set it to the path as it would exist, within a real
temporary directory.

Finally move temporary directory creation out of runner and into target.
Now a single directory exists for the duration of a run, and is emptied
by runner.py as necessary after each task invocation.

This could be further extended to stop rewriting non-new-style modules
in a with_items loop, but that's another step.

Finally the last bullet point in the documentation almost isn't a lie
again.
7 years ago
David Wilson 6109de51a0 tests: Ansible SSH timeout test
Also change Docker image to new 'mitogen' organization.
7 years ago
David Wilson d4d4f46006 tests: update check_host_keys parameter style in testlib. 7 years ago
David Wilson b7fe0e09f4 tests: su integration test. 7 years ago
David Wilson 3203846708 issue #239: ansible: ignore remote_tmp in new style runner. 7 years ago
David Wilson baa4e75526 tests: tidy up thread_pileup for use as a toy benchmark. 7 years ago
David Wilson 7c5bbc5168 setns: support changing user.
To match existing third party plugin.
7 years ago
David Wilson 1be03eb458 parent: change create_child interface.
To allow for additional arguments.
7 years ago
David Wilson fa4746f656 ansible: add connection delegation test targets.
For command line use only, no integration tests yet.
7 years ago
David Wilson 962ba862e9 tests: use test-targets group, not all group 7 years ago
David Wilson 86c9978e09 Ensure mod.__file__ is set; closes #210. 7 years ago
David Wilson e63ae4768e core: support Receiver.get(thread_dead=False)
For tests.
7 years ago
David Wilson 6541779dd6 tests: import Ansible file transfer benchmark 7 years ago
David Wilson 7c88e4d013 Move _DEAD into header, autogenerate dead messages
This change blocks off 2 common scenarios where a race condition is
upgraded to a hang, when the library could internally do better.

* Since we don't know whether the receiver of a `reply_to` is expecting
  a raw or pickled message, and since in the case of a raw reply, there
  is no way to signal "dead" to the receiver, override the reply_to
  field to explicitly mark a message as dead using a special handle.

  This replaces the serialized _DEAD sentinel value with a slightly
  neater interface, in the form of the reserved IS_DEAD handle, and
  enables an important subsequent change: when a context cannot route a
  message, it can send a generic 'dead' reply back towards the message
  source, ensuring any sleeping thread is woken with ChannelError.

  The use of this field could potentially be extended later on if
  additional flags are needed, but for now this seems to suffice.

* Teach Router._invoke() to reply with a dead message when it receives a
  message for an invalid local handle.

* Teach Router._async_route() to reply with a dead message when it
  receives an unroutable message.
7 years ago
David Wilson 8c3b1fcf15 ansible: disable script interpreter processing for new-style
Re: #199
7 years ago
David Wilson 2f02b5c0b1 ansible: prevent Unicode strings leaking into sys.argv
The module name comes from YAML via Jinja2.. it's always Unicode. Mixing
it into a temporary directory name produces a Unicode tempdir name,
which ends up in sys.argv via TemporaryArgv.
7 years ago
David Wilson e56608ab91 parent: don't wait for SIGTERM to complete. 7 years ago
David Wilson dc4433ace6 issue #202: ansible: forget all dependent contexts on Stream disconnect
This is a partial fix, there are still at least 2 cases needing covered:

- In-progress connections must have CallError or similar sent to any
  waiters
- Once connection delegation exists, it is possible for other worker
  processes to be active (and in any step in the process), trying to
  communicate with a context that we know can no longer be communicated
  with. The solution to that isn't clear yet.

Additionally ensure root has /bin/bash shell in both Docker images.
7 years ago
David Wilson 9fe14e841c parent: reap the child process if connection fails
For example if no response is received in :attr:`connect_timeout`
seconds, the child would be left running.
7 years ago
David Wilson e7831a801f issue #195: handle non-ASCII scripts in runner.py. 7 years ago
David Wilson 85e1f5f515 ansible: remove JobResultService, more compatible async jobs; closes #191.
And by "compatible" I mean "terrible". This does not implement async job
timeouts, but I'm not going to bother, upstream async implementation is
so buggy and inconsistent it resists even having its behaviour captured
in tests.
7 years ago
David Wilson 6ad18b6719 issue #191: move async tests to their own category 7 years ago
David Wilson e43c6c531b Mostly implement hybrid TTY/socket mode for sudo and SSH.
Presently there is still no mechanism to add :attr:`tty_stream` to the
multiplexer after connection is successful, but for now it's not
expected that anything will be logged to it anyway.

Closes #148.
7 years ago
David Wilson fca22efe90 tests: enable profile_tasks callback plugin 7 years ago
David Wilson c6284e00e9 Use subprocess to start child processes; closes #185. 7 years ago
David Wilson 1e882a72c0 tests: install virtualenv in the Docker images. 7 years ago
David Wilson b91785c9a1 tests: workaround for issue_109 7 years ago
David Wilson ce260933d9 tests: consistent play naming everywhere 7 years ago
David Wilson d827bdb848 tests: fixes to get regression/ running under vanilla Ansible 7 years ago
David Wilson b882b78d35 tests: clean up remaining reg tests
Delete some that are impossible to detect, fix up the remainder to
detect the condition they describe.
7 years ago
David Wilson aeeba54d96 tests: better regression test for issue #109 7 years ago
David Wilson f4d791662e ansible: better planner logging, try again 7 years ago
David Wilson b4bb1c5a2a tests: fix suspected readdir() ordering issue (!) 7 years ago
David Wilson 0046a3de0b tests: insert debug print for travis :( 7 years ago
David Wilson ef47347cdf tests: raft of fixes for vanilla Ansible 7 years ago
David Wilson 89e11ed25c tests: make osx_setup a little more generic again 7 years ago
David Wilson 2c3d2db4ec tests: fix output checks for Linux 7 years ago
David Wilson 7e49e73683 tests: add perl-JSON to the test images too.
Needed for the runner tests.
7 years ago
David Wilson ac80bf12f0 tests: start making osx_setup.yml more generic 7 years ago
David Wilson 16b5aa2d10 tests: disable host key checking for Paramiko. 7 years ago
David Wilson 41d2661ecb tests: fix sudo rule order. 7 years ago
David Wilson ed915b6e63 tests: magic mitogen_shutdown_all action
LRU tests break when run as part of the whole suite rather than
individually, because LRU stuff is already happening for earlier tests.
7 years ago
David Wilson 6614d17021 tests: explicit sudo rules to require password from sudo_nopw
This account is used on Travis. It needs to be forced to type passwords
in some cases
7 years ago