Commit Graph

232 Commits (f2f1a530a6ab3b830d0ac838dcb030350db29a90)

Author SHA1 Message Date
David Wilson 65f03e03f5 tests: remote_tmp test fixes. 6 years ago
David Wilson 90f89f95fb ansible: fix exec_command() regression. 6 years ago
David Wilson 24a44499ca tests: verify Connection.put_file() for small/large files. 6 years ago
David Wilson 705d77a9be ansible: remove a bunch more aliasing from connection.py. 6 years ago
David Wilson 66142e7d75 ansible: fork isolated tasks from correct parent.
Closes #355.
6 years ago
David Wilson 72fa129f8a tests: fix clash when localhost is test-target 6 years ago
David Wilson 9792b8b54f ansible: use template-expanded delegate_to hostname in one more location. 6 years ago
David Wilson 90c2ed03d0 ansible: fix synchronize module
Broken by recent connection delegation fixes.
6 years ago
David Wilson 084c0ac065 ansible: avoid roundtrip in copy action due to fixup_perms2().
On top of existing temporary files work, this reduces the number of
roundtrips required for "copy" and "template" actions from 6 to 3.
6 years ago
David Wilson 4098d45dac tests: disable delegation tests on vanilla. 6 years ago
David Wilson 50e285f7ba tests: update for identities_only change. 6 years ago
David Wilson a4ed27fa63 tests: add playbook step to ensure key file perms. 6 years ago
David Wilson ac9b84d237 issue #321: 2.4+ compatibility fixes, disable test on Vanilla. 6 years ago
David Wilson a2686b1a2c issue #321: simplify temp directory handling. 6 years ago
David Wilson 3d588323ff issue #340: use expanded delegate_to hostname, not template.
PlayContext.delegate_to is the unexpanded template, Ansible doesn't keep
a copy of it around anywhere convenient. We either need to re-expand it
or take the expanded version that was stored on the Task, which is what
is done here.
6 years ago
David Wilson 154dc2e119 tests: fix integration/runner/missing_module.yml on Travis. 6 years ago
David Wilson a6995a5288 issue #338: refactor env handling into class and fix tests. 6 years ago
David Wilson 06ae59702c tests: rationalize matrix and rewrite ansible_tests
Now all distros run in parallel.
6 years ago
David Wilson 8eb288856c issue #338: run /etc/environment test with become:true. 6 years ago
David Wilson ad365dad56 issue #340: one more test, update Changelog. 6 years ago
David Wilson 916e46621b issue #340: add connection delegation tests. 6 years ago
David Wilson 6c4b01642c ansible: don't crash when adhoc tries to run a missing module.
ansible-playbook prints a separate error during parsing stage, adhoc
performs no such check.
6 years ago
David Wilson 1473f49505 ansible: emulate /etc/environment reloading behaviour of vanilla.
This change is relatively incomplete -- ideally we could snapshot
os.environ and /etc/environment at startup and respect key deletions
too, but that's a lot more work. Wait for a bug report instead.

Closes #338.
6 years ago
David Wilson 3e0de9790c issue #324: fix Python 3 fallout for custom module_utils.
Also enable at last one of its tests.
6 years ago
David Wilson b44b823c4a ansible: make _remote_expand_user() pay attention to sudoable=.. 6 years ago
David Wilson a8e4dcc98d issue #301: correct remote_tmp evaluation context.
Vanilla Ansible expands remote_tmp variables in the context of the login
account, not any become_user account.
6 years ago
David Wilson 9410903f20 issue #301: add related test. 6 years ago
David Wilson e39c602fd3 issue #291: support UNIX hashbang syntax for ansible_*_interpreter.
Closes #291.
6 years ago
David Wilson 15d68b3c32 issue #309: fix environment cleanup regression.
Closes #309.
6 years ago
David Wilson 0b1f92b59f issue #297: test fixes to match fixed CWD. 6 years ago
David Wilson d8e0c9e12c issue #297: local commands must execute with WorkerProcess environment. 6 years ago
David Wilson 012745efea issue #297: local actions must execute with fixed directory.
Local actions must execute in the the parent directory of the playbook
that defines the action.
6 years ago
David Wilson 26ba3e4d83 issue #294: stronger integration test
Run the tests in a brand new ansible-playbook invocation, and try both
linear->mitogen->linear and mitogen->linear->mitogen.
6 years ago
David Wilson 5b03e06457 issue #294: ansible: fix mixed vanilla/Mitogen runs.
Don't bother trying to understand what damage PluginLoader has done to
ansible.plugins.* namespace, just ask it for the base class instead.
6 years ago
David Wilson 410016ff47 Initial Python 3.x port work.
* ansible: use unicode_literals everywhere since it only needs to be
  compatible back to 2.6.
* compat/collections.py: delete this entirely and rip out the parts of
  functools that require it.
* Introduce serializable Kwargs dict subclass that translates keys to
  Unicode on instantiation.
* enable_debug_logging() must set _v/_vv globals.
* cStringIO does not exist in 3.x.
* Treat IOLogger and LogForwarder input as latin-1.
* Avoid ResourceWarnings in first stage by explicitly closing fps.
* Fix preamble_size.py syntax errors.
6 years ago
David Wilson ffc7306cf8 tests: better runner_two_simultaneous_jobs.yml. 7 years ago
David Wilson 3994f1b30a ansible: implment async job time limit. 7 years ago
David Wilson df8fe59eda tests: replace hard-coded sleep with a polling loop 7 years ago
David Wilson 64b60be50c tests: split runner_new_process out of runner_one_job 7 years ago
David Wilson faaac43a78 Disable SSH variables test on vanilla Ansible. 7 years ago
David Wilson 68d6e6bf34 ansible: tests for all SSH variables.
Closes #184.
7 years ago
David Wilson 2ad0d0521d ansible: reload /etc/resolv.conf for each task.
The OpenShift installer modifies /etc/resolv.conf then tests the new
resolver configuration, however, there was no mechanism to reload
resolv.conf in our reuseable interpreter.

https://github.com/openshift/openshift-ansible/blob/release-3.9/roles/openshift_web_console/tasks/install.yml#L137

This inserts an explicit call to res_init() for every new style
invocation, with an approximate cost of ~1usec on Linux since glibc
verifies resolv.conf has changed before reloading it.

There is little to be done for users of the thread-safe resolver APIs,
their state is hidden from us. If bugs like that manifest, whack-a-mole
style 'del sys.modules[thatmod]' patches may suffice.
7 years ago
David Wilson 60781ecab6 tests: fix test for vanilla Ansible. 7 years ago
David Wilson eb56a8f98d tests: add debug 7 years ago
David Wilson 267f787d20 issue #217: tests: import (unused) module_utils tests. 7 years ago
David Wilson ee741da2b3 tests: reenable remaining Ansible tests. 7 years ago
David Wilson be5c03c152 tests: many test fixes. Travis broken for >1 week. 7 years ago
David Wilson b20174729d issue #199: fix readonly_homedir test. 7 years ago
David Wilson 6109de51a0 tests: Ansible SSH timeout test
Also change Docker image to new 'mitogen' organization.
7 years ago
David Wilson b7fe0e09f4 tests: su integration test. 7 years ago
David Wilson 3203846708 issue #239: ansible: ignore remote_tmp in new style runner. 7 years ago
David Wilson 962ba862e9 tests: use test-targets group, not all group 7 years ago
David Wilson 8c3b1fcf15 ansible: disable script interpreter processing for new-style
Re: #199
7 years ago
David Wilson 2f02b5c0b1 ansible: prevent Unicode strings leaking into sys.argv
The module name comes from YAML via Jinja2.. it's always Unicode. Mixing
it into a temporary directory name produces a Unicode tempdir name,
which ends up in sys.argv via TemporaryArgv.
7 years ago
David Wilson dc4433ace6 issue #202: ansible: forget all dependent contexts on Stream disconnect
This is a partial fix, there are still at least 2 cases needing covered:

- In-progress connections must have CallError or similar sent to any
  waiters
- Once connection delegation exists, it is possible for other worker
  processes to be active (and in any step in the process), trying to
  communicate with a context that we know can no longer be communicated
  with. The solution to that isn't clear yet.

Additionally ensure root has /bin/bash shell in both Docker images.
7 years ago
David Wilson e7831a801f issue #195: handle non-ASCII scripts in runner.py. 7 years ago
David Wilson 85e1f5f515 ansible: remove JobResultService, more compatible async jobs; closes #191.
And by "compatible" I mean "terrible". This does not implement async job
timeouts, but I'm not going to bother, upstream async implementation is
so buggy and inconsistent it resists even having its behaviour captured
in tests.
7 years ago
David Wilson 6ad18b6719 issue #191: move async tests to their own category 7 years ago
David Wilson ce260933d9 tests: consistent play naming everywhere 7 years ago
David Wilson 0046a3de0b tests: insert debug print for travis :( 7 years ago
David Wilson ef47347cdf tests: raft of fixes for vanilla Ansible 7 years ago
David Wilson 2c3d2db4ec tests: fix output checks for Linux 7 years ago
David Wilson ed915b6e63 tests: magic mitogen_shutdown_all action
LRU tests break when run as part of the whole suite rather than
individually, because LRU stuff is already happening for earlier tests.
7 years ago
David Wilson ba8022424b tests: more work to standardize user accounts. 7 years ago
David Wilson 63e2ec6406 tests: expanduser lookup runs locally 7 years ago
David Wilson 472610805f tests: fix remote_tmp test on 2.5.x. 7 years ago
David Wilson 9f94fb78c8 issue #159: make LRU size configurable. 7 years ago
David Wilson 6394226722 issue #159: verify no more than 20 tasks exist on the remote 7 years ago
David Wilson 633cd888a0 issue #182: test for previous commit. 7 years ago
David Wilson 0c803141a7 tests: fix filename. 7 years ago
David Wilson 98ee3e177a ansible: tests for sudo behaviour; closes #143. 7 years ago
David Wilson 3613162bc0 ansible: enable forking when requested and for async jobs.
Closes #105.
References #155.

mitogen/service.py:
    Refactor services to support individually exposed methods with
    different security policies for each method.

    - @mitogen.service.expose() to expose a method and set its policy
    - @mitogen.service.arg_spec() to validate input.
    - Require basic service message format to be a tuple of
      `(method, kwargs)`, where kwargs is always a dict.
    - Update DeduplicatingService to match the new scheme.

ansible_mitogen/connection.py:
    - Rename 'method' to 'method_name' to disambiguate it from the
      service.call()'s method= argument.

ansible_mitogen/planner.py:
    - Generate an ID for every job, sync or not, and fetch job results
      from JobResultService rather than via the initiating function
      call's return value.
    - Planner subclasses now get to select whether their Runner should
      run in a forked process. The base implementation requests this if
      the 'mitogen_isolation_mode=fork' task variable is present.

ansible_mitogen/runner.py:
    Teach runners to deliver their result via JobResultService executing
    in their indirect parent mux process.

ansible_mitogen/plugins/actions/mitogen_async_status.py:
    Split the implementation up into methods, and more compatibly
    emulate Ansible's existing output.

ansible_mitogen/process.py:
    Mux processes now host JobResultService.

ansible_mitogen/services.py:
    Update existing services to the new mitogen.service scheme, and
    implement JobResultService:

    * listen() method for synchronous jobs. planner.invoke() registers a
      Sender with the service prior to invoking the job, then sleeps
      waiting for the service to write the job result to the
      corresponding Receiver.

    * Non-blocking get() method for implementing mitogen_async_status
      action.

    * Child-accessible push() method for delivering task results.

ansible_mitogen/target.py:
    New helpers for spawning a virginal subprocess on startup, from
    which asynchronous and mitogen_task_isolation=fork jobs are forked.
    Necessary to avoid a task inheriting potentially
    polluted/monkey-patched parent environment, since remaining jobs
    continue to run in the original child process.

docs/ansible.rst:
    Add/merge/remove some behaviours/risks.

tests/ansible/integration:
    New tests for forking/async.
7 years ago
David Wilson b9d4ec57b3 issue #164: some more ActionMixin tests. 7 years ago
David Wilson db894478f8 issue #164: make become_flags work without FOO=2 env var. 7 years ago
David Wilson d068a36c1e issue #164: more dir layout contortions.
all.yml slurps in tests from each file/subdir in the CWD.
7 years ago
David Wilson 680dc1bf68 issue #164: basic connection loader tests. 7 years ago
David Wilson 6aeb4e9f05 issue #164: precisely emulate Ansible's stdio behaviour.
* Use identical logic to select when stdout/stderr are merged, so
  'stdout', 'stdout_lines', 'stderr', 'stderr_lines' contain the same
  output before/after the extension.

* When stdout/stderr are merged, synthesize carriage returns just like
  the TTY layer.

* Mimic the SSH connection multiplexing message on stderr. Not really
  for user code, but so compare_output_test.sh needs fewer fixups.
7 years ago
David Wilson c5ca2e87ea issue #164: stop tests on first failure 7 years ago
David Wilson 26cc0f2724 issue #164: fix remote_tmp handling on <2.5 7 years ago
David Wilson 3ebe600389 issue #164: convert "examples" into actual tests
- Add new Travis mode, "ansible_tests.sh" that runs
  integrations/all.yml. Slowly build this up over time to cover more of
  the existing junk.

- Add basic assertions on the output of the existing runner__* files.

- Wire up 2.4.3/2.5.0 jobs in Travis.
7 years ago
David Wilson ae75a0ca8c issue #164: rearrange playbooks a little more 7 years ago
David Wilson 49aa8834b0 issue #164: split "examples" out into regression/integration tests. 7 years ago