Commit Graph

81 Commits (e572922d9dfbf6eb51a32fd5dca33c32d05e656d)

Author SHA1 Message Date
Alex Willmer 090952a987 Python 3.14 support 3 months ago
Alex Willmer 3cba11a126 CI: Fix ansible_version comparison with ansible-core 2.19.0rc1
Note that tests/ansible/integration/ssh/templated_by_play_taskvar.yml was
previously erroniously being skipped with ansible-core 2.19.0a<N> and
2.19.0b<N>.

fixes #1293
refs #1175
5 months ago
Alex Willmer 0187418697 ansible_mitogen: alpha datatag handling & CI for Ansible 12 (ansible-core 2.19)
refs #1258
6 months ago
Alex Willmer a00263d963 CI: Fix cosmetic typos in get_with_context() regression test 6 months ago
Alex Willmer 4c41bf02f1 CI: Specify ANSIBLE_STRATEGY in tasks that run ansible or ansible-playbook
This makes the behaviour more consistent across jobs that run with
`mitogen_linear` or plain `linear`.
6 months ago
Alex Willmer 27b4b77bba CI: Upgrade Github jobs from Ubuntu 20.04 to 22.04 & 24.04
Python 2.7 (distro package) and 3.6 (pyenv managed) jobs run on Ubuntu 22.04.
More recent Pythons (distro or Github provided) run on 24.04.

fixes #1256

Ansible tasks that run locally (e.g. `connection: local`, `delegate_to:
localhost`) must now specify their `ansible_python_interpreter`, typically as
`{{ ansible_playbook_python }}`; otherwise the system Python on the controller
(e.g. `/usr/bin/python`) is likely to be used and this is often outside the
version range supported by the Ansible verison under test. If this occurs then
the symptom is often a failure to import a builtin from
`ansible.module_utils.six.moves`, e.g.

```
fatal: [target-centos6-1]: FAILED! => changed=true
  cmd:
  - ansible
  - -m
  - shell
  - -c
  - local
  - -a
  - whoami
  - -i
  - /tmp/mitogen_ci_ansibled3llejls/hosts
  - test-targets
  delta: '0:00:02.076385'
  end: '2025-04-17 17:27:02.561500'
  msg: non-zero return code
  rc: 8
  start: '2025-04-17 17:27:00.485115'
  stderr: |-
  stderr_lines: <omitted>
  stdout: |-
    An exception occurred during task execution. To see the full traceback,
    use -vvv. The error was:     from ansible.module_utils.six.moves import
    map, reduce, shlex_quote
```
7 months ago
Alex Willmer fdbd1a8c9b ci: Configure package managers using a role
This allows code sharing between integration tests and test image prep.
9 months ago
Jarl Gullberg 211079f130
Add regression tests for the new connection logic.
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
Co-authored-by: Mark Farrell <mark.a.farrell@team.telstra.com>
11 months ago
Alex Willmer 5e6d7bf4fb ansible_mitogen: Templated connection timeout
Ansible >= 4 (ansible-core >= 2.11) the SSH plugin has a `timeout` option and
with variable `ansible_ssh_timeout`, but not a `ansible_timeout` variable.
The local plugin has no such option or variable(s). However `ansible_timeout`
is backfilled for all conection plugins, by legacy mechanisms that populate
the play context attribute:
- `ansible.constants.COMMON_CONNECTION_VARS`
- `ansible.constants.MAGIC_VARIABLE_MAPPING`

The `timeout` keyword is for task completion timeout, not connection timeout.
12 months ago
Alex Willmer 6900e88dfd ansible_mitogen: Fix templated python interpreter with `meta: reset_connection`
refs #1079
1 year ago
Alex Willmer 53b4881628 ansible_mitogen: Fix wait_for_connection + templated ansible_python_interpreter
This tightens up our monkey patching `Connection._action` so it's only applied
during `meta: reset_connection` & promptly removed. This fixes "'int' object
has no attribute 'template'" when `ansible.plugins.action.wait_for_connection`
or other code calls `ansible.plugins.connection.ConnectionBase.reset()`.

This could also have switched to `templar=templar` on the temporary action,
rather than `templar=0`, but it's not strictly necessary to fix this bug. I
anticipate other changes doing so soon, to improve interpreter discovery &
templated python interpreter path support.
1 year ago
Alex Willmer 06df62c8b8 CI: Migrated macOS 12 runners to macOS 13, due to EOL.
macOS Python 2.7 jobs have been removed because the macOS 13 image doesn't
include CPython 2.7.
1 year ago
Alex Willmer 6accc87da1 tests: Improve Ansible fail_msg formatting
By switching to block style (`|`) with clip (no `-` or `+`) the failure
messages don't require quoting and gain a single trailing newline. This causes
Ansible to print them as block style, when using the yaml stdout callback
plugin. As a result the values have one less layer of quoting and quote
escaping, making them much easier to read.
1 year ago
Alex Willmer 5ab872f289 ansible_mitogen: Add regression test for ActionModuleMixin._remote_chmod()
Adapted from Jonathon's reproducer in #1087.
1 year ago
Alex Willmer 357fe38766 Ansible 10 (ansible-core 2.17) support
Notably
- Python 2.7 and 3.6 are no longer supported by Ansible on targets
- The yum module has been removed, and redirected to dnf
- _INTERPRETER_PYTHON_DISTRO_MAP has been neutered. Interpreter discovery
  always favours specific `python3.<x>` interpreters in decending version
  order, then generic `python3` or `python`.
- Add the ability for an action plugin to call self._execute_module(*,
  ignore_unknown_opts=True) to execute a module with options that may not be
  supported for the version being called.

https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_10.html
https://github.com/ansible-community/ansible-build-data/blob/main/10/CHANGELOG-v10.md
https://github.com/ansible/ansible/blob/stable-2.17/changelogs/CHANGELOG-v2.17.rst

fixes #1074, refs #1082

Co-authored-by: Claude Becker <becker@phys.ethz.ch>
1 year ago
Alex Willmer 85b1b4070a tests: Respect configured or detected Python more often
Relying on the virtualenv default or hardcoding "python" results in a Python
2.x virtualenv on some targets (e.g. debian10-test). This caused a failure
when testing with Ansible >= 10 (ansible-core >= 2.17), which have dropped
Python 2.x support.

refs #1074
1 year ago
Alex Willmer 924dbd6f0c CI: Migrate macOS integration tests to macOS 12, drop Python 2.7 jobs
macOS 11 is not longer an available runner on Azure Devops. The minimum is now
macOS 12. This runner does not have Python 2.7 installed, so running them
would require a custom install - which I'm declaring too much effort for too
little gain.

refs #1090
1 year ago
Alex Willmer f5a8840668 CI: Use archived RPMs on CentOS 8
CentOS 8 has reached EOL. Packages are no longer mirrored or maintained. A
historic snapshot of the packages is kept on vault.centos.org.

refs #1088, #1090
1 year ago
Alex Willmer 7079a07a15 tests: Fix duplicate local task executions
integration/ssh/timeouts.yml is noteworthy. It was an accidental N**2 in time
-  executing num hosts * num hosts tasks.
2 years ago
Alex Willmer 65c8a42c13 tests: Use same verbosity when re-executing Ansible inside a playbook 2 years ago
Alex Willmer 60f868290d
tests: Remove --limit when running Ansible localhost CI
Some tests were being incorrectly excluded. Including those that use
`add_host`.
refs #1066, #1069
2 years ago
Alex Willmer d2eefc06aa
tests: Add regression for add_host with host_key_checking
refs #1066
2 years ago
Alex Willmer d5e9186289 ansible_mitogen: Fix --ask-become-pass, add test coverage
Previously f1503874de fixed the priority of
ansible_become_pass over ansible_become_password, but broke --ask-become-pass.
Fixes #952.
2 years ago
Alex Willmer 45c42d386a tests: Replace uses of ``include:``, unify skipping of mitogen only tests
The tag mitogen_only is only informational for now. It may be possible to use
it with ANSIBLE_SKIP_TAGS in the future.
2 years ago
Alex Willmer 2839954559 tests: Account for /tmp symlink in virtualenv test on macOS 2 years ago
Alex Willmer 19b79f7ab5 CI: Fix tests on Linux, Ansible tests targetting Debian 9 & 11
Without Ubuntu 20.04 virtualenv package being installed pip was installing a
version of virtualenv that couldn't create the Tox environment for Python 2.7.

> Successfully installed distlib-0.3.6 filelock-3.12.2 platformdirs-3.8.0
> pluggy-1.2.0 py-1.11.0 tomli-2.0.1 tox-3.28.0 virtualenv-20.23.1
> Finishing: Install tooling
> ...
> py27-mode_mitogen-distro_centos6 create: /home/vsts/work/1/s/.tox/py27-
> mode_mitogen-distro_centos6
> ERROR: invocation failed (exit code 1), logfile: /home/vsts/work/1/s/.tox/
> py27-mode_mitogen-distro_centos6/log/py27-mode_mitogen-distro_centos6-0.log
> ================================== log start
> ===================================
> RuntimeError: failed to query /usr/bin/python2.7 with code 1 err:
> '  File "/home/vsts/.local/lib/python3.8/site-packages/virtualenv/discovery/
> py_info.py", line 24\n    return list(OrderedDict.fromkeys(["",
> *os.environ.get("PATHEXT", "").lower().split(os.pathsep)]))\n
> ^\nSyntaxError: invalid syntax\n'
2 years ago
Alex Willmer edd2868ef6 tests: Don't rely on facts when setting become
They won't be available if the play is first, and hence no facts have been
gather in previous play(s), e.g. due to --start-at-task
3 years ago
Alex Willmer 526422b74b tests: Name tasks
For easier grep, and easier identification in task_profiler summaries.
3 years ago
Alex Willmer 64819ecb5f tests: Regression test for #776 (package/yum/dnf module called twice) 3 years ago
Alex Willmer 24c845379a tests: Remove redundant regression tags
The tag is applied by the playbook that imports this one.
3 years ago
Alex Willmer db0ffae352 tests: Enable stricter error handling, fix resulting failures 3 years ago
Alex Willmer 216e7c9150 tests: Correct Ansible targets 3 years ago
Alex Willmer 8e79488768 tests: Mark or avoid sudo tasks on localhost 3 years ago
Alex Willmer f070767dad tests: Use meaningful play names 3 years ago
Alex Willmer 25ea6dde02 ansible_mitogen: Allow mitogen_fetch to bypass slurp module
This reapplies an earlier change, when this plugin was first introduced to
Mitogen. The plugin was updated to fix

[DEPRECATION WARNING]: The '_remote_checksum()' method is deprecated.

I've elected to short-circuit the if statemtn logic, rather than
deleting/unindenting, to make the code delta much smaller. This should make it
easier to maintain/update.

Fixes #915
4 years ago
Alex Willmer db9e52ce8e tests: Run containers on macOS with podman, instead of Docker 4 years ago
Klaus Zerwes b8dd348b1e Revert "Revert "fixed tests for ansible 2.12""
This reverts commit f82bd3d714.
4 years ago
Klaus Zerwes f82bd3d714 Revert "fixed tests for ansible 2.12"
include is deprecated and will be removed in 2.16, so there is some time to fix this left
This reverts commit 7bc2102d2b.
4 years ago
Klaus Zerwes 7bc2102d2b fixed tests for ansible 2.12 4 years ago
Alex Willmer e194a6367f ci: Fix version comparisons involving double digits
See https://gist.github.com/moreati/e7507c5b606b12ec0ddafcb7c8debbf1
4 years ago
Alex Willmer 3c58215a91 tests: Tag Ansible tests
This makes it easier to run subsets using ANSIBLE_RUN_TAGS=...
4 years ago
Alex Willmer cfa5888547 tests: Print variables on failure of assert tasks
Fixes #852
4 years ago
Alex Willmer 1a0951c664
Merge branch 'master' into tests-resource_intensive-tag 5 years ago
Alex Willmer bb271d8d00 tests: Tag resource intensive tasks, to skip when strategy=linear
The ansible_mitogen test suite takes over an hour when Ansible is not
accelerated by Mitogen. This change aims to reduce that by skipping
tests with a large number of iterations when the linear strategy is
chosen.

The tagged tests are intended to uncover Mitogen resource leaks. Since
Mitogen is not invoked when strategy=linear, the slight reduction in
test coverage is an acceptable trade off.
5 years ago
Alex Willmer a64e966ab9 tests: Add unofficial Tox environments for Ansible tests
These are not part of the official testing regime (tests run for pull
requests). I find them convenient for local development.

Limitations
- Python 2.7+ only. No Python 2.4, 2.5, or 2.6.
- Requires Pythons pre-installed (e.g. DeadSnakes, pyenv)
- No coverage of alternate controller OS (e.g. MacOS)

The environments tested by default are

py27-mode_ansible-ansible2.10
py36-mode_ansible-ansible2.10
py39-mode_ansible-ansible2.10
py27-mode_mitogen
py36-mode_mitogen
py39-mode_mitogen
py27-mode_mitogen-distro_centos7
py36-mode_mitogen-distro_centos7
py39-mode_mitogen-distro_centos7
5 years ago
Steven Robertson 1091cd7ca1 try and suppress mode warning clogging up logs 5 years ago
Steven Robertson acde13f9d6 handles a 'wait_for_connection' call right after a task caused a shutdown 6 years ago
Steven Robertson bb048cc03d accidentally deleted quotes 6 years ago
Steven Robertson faf7798c16 revert python3 setup + add TODO; python3 tests aren't working now but python2 is EOL so they should work soon 6 years ago
Steven Robertson 1157ff6f43 try fixing mac python symlink weirdness 6 years ago