Commit Graph

569 Commits (4b0954a441abdd8c383941c5578017f69a635e23)

Author SHA1 Message Date
Alex Willmer dd41ddf89b ansible_mitogen: Templated become flag
The code change to support this was already made in transport_config.py, as
part of templated become_user support (commit bf6607e27e, PR #1148). This
commit adds tests to confirm the functionality.
2 weeks ago
Alex Willmer e120cd2cae ansible_mitogen: Templated become method 2 weeks ago
Alex Willmer 905b87b71a tests: Test templated ansible_host_key_checking provided by task vars
missed by #1184
2 weeks ago
Alex Willmer f50a61f981 ansible_mitogen: Templated host option (e.g. ansible_host, ansible_ssh_host)
A twist - for the connection option "host" the corresponding legacy
PlayContext attribute is PlayContext.remote_addr. This may be the only case
where a connection option name and the PlayContext attribute name differ.
2 weeks ago
Alex Willmer 6d9f2e12d9 tests: Switch remaining tt_targets_inventory group vars to host vars
This is ground work for adding/testing templated hostnames and python
interpreters. The extreme wideness will hopefully be temporary, e.g. by
switching to YAML inventories. The INI inventory plugin doesn't support
multiline host entries.

> 640 K(olumns) should be enough for anyone
> -- Apocryphal, not Bill Gates
2 weeks ago
Alex Willmer 9e0dad2a1a ansible_mitogen: Templated SSH host key checking
refs #1083
2 weeks ago
Alex Willmer c7df5c97c1 ansible_mitogen: Templated SSH private key file 2 weeks ago
Alex Willmer 43cc937bc6 CI: Fix incorrect u=r,g=r,o=rw file permissions on mitogen__has_sudo_pubkey.key
The wrong base was used when calculating the mode. So the file became world
readable and writable on a CI runner, until
ansible/integration/ssh/variables.yml happened to correct it near the end of
the integration tests.

I believe this was the only instance.

```console
mitogen git:(issue1182) ✗ ag --python 'int\(.+7\)' . .ci | wc -l
       0
```

fixes #1182
2 weeks ago
Alex Willmer 06df62c8b8 CI: Migrated macOS 12 runners to macOS 13, due to EOL.
macOS Python 2.7 jobs have been removed because the macOS 13 image doesn't
include CPython 2.7.
3 weeks ago
Alex Willmer 833e2845e9 ansible_mitogen: Templated ssh executable, templated reset_connection fix
Adding a the tt-ssh-executable test target uncovered an Ansible bug during
`meta: reset_connection` tasks. So this commit includes a workaround for
affected versions of Ansible.
3 weeks ago
Alex Willmer 66ea10d577 ansible_mitogen: Template become command arguments (become_flags)
Uses the same fallback for (mitogen_sudo et al) as become_exe (see #1173).

The new `Spec.become_flags()` is not yet explicitly tested. Note that it
returns a string (matching the Ansible option of the same name), whereas
`Spec.sudo_args()` returns a list.

refs #1083
4 weeks ago
Alex Willmer ec9b3e5c5d ansible_mitogen: Support templated become_exe option
Some ansible_mitogen connection plugins look more like become plugins (e.g.
mitogen_sudo) & use become plugin options. For now there's special handling in
PlayContextSpec._become_option(). Further design/discussion can go in #1173.

Refs #1087.
4 weeks ago
Alex Willmer 7e5b064139 ansible_mitogen: Support templated become passwords 4 weeks ago
Alex Willmer 8a34b925a4 tests: Re-enable become/sudo tests, fix them on macOS runners
The tasks in tests/imageprep/_user_accounts.yml that create users did not
specify a primary group for those users - this left the decision to Ansible's
user module, and/or the underlying OS. In Ansible 9+ (ansible-core 2.16+ the
user module defaults to primary group "staff." Earlier don't supply a default,
which releases probably results in a primary group nameed "None" (due to
stringifying the Python singleton of the same name), or whatever the macOS
Directory Services has for no data/NULL.

The invalid GID 4294967295 (MAX_UINT32 == 2**32-1) in the sudo error probably
enters the mix via something similar to sudo CVE-2019-14287.

Fixes #692

See
- https://github.com/ansible/ansible/pull/79999
- https://github.com/ansible/ansible/commit/c69c83c962f987c78af98da0746527df
- https://www.sudo.ws/security/advisories/minus_1_uid/

> Bruce Wayne : [confused]  Am I meant to understand any of that?
> Lucius Fox : Not at all, I just wanted you to know how hard it was.
> -- Batman Begins
4 weeks ago
Alex Willmer cdfaf31ebc ansible_mitogen: Template ssh_*_args connection options
This expands support to setting them in Play scoped variables. Task scoped
variables are also very likely to work, but untested for now.

refs #905
4 weeks ago
Alex Willmer bf6607e27e ansible_mitogen: Support templated become_user
This reads the become username from the `become_user` attribute of the play
context, to the `"become_user"` option of the loaded become plugin. This has
been supported by vanilla Ansible since Ansible 2.10 (ansible-base 2.10).

To support this I've also switched from using the `play_context.become` (a
bool), to `connection.become` (an instance of the appropriate) become plugin.

New tests have been added, modelled on those for templated connection
parameters (see #1147, #1153, #1159).

See
- 480b106d65

refs #1083

Co-authored-by: mordek <m.pirog@bonasoft.pl>
1 month ago
Alex Willmer e9bddf0c03 CI: Use templated ansible_user for localhost Ansible tests
refs #1022, #1116
1 month ago
Alex Willmer 5e816be12c tests: Templated connection keywords with delegated_to 1 month ago
Alex Willmer 5d6a185242 tests: Templated "remote_user" provided as Ansible playbook keyword
The password is provided as a variable because there is no corresponding
keyword. I get the impression that keywords are considered a legacy mechanism,
so most (new) options are only overridable by variables.

The port is proved as a variable for now, to test remote_name in isolation.
1 month ago
Alex Willmer 77a01ff8d6 ansible_mitogen: Support templated SSH port
fixes #978
2 months ago
Alex Willmer 14cb8be7e5 ansible_mitogen: Test templated connection user (e.g. ansible_user) 2 months ago
Alex Willmer 0e9c890637 tests: Remove unused physical_hosts variable 2 months ago
Alex Willmer 1773c9aba6 trivia: Fix trailing whitespace 2 months ago
Alex Willmer 3504bea3bb tests: Ignore inventory files of inactive tests & benchmarks
These targets are not used by any active tests, and the large numbers of hosts
multiply the size of the taskvars disctionary in memory to many (10s) MiB.

refs #1058
2 months ago
Alex Willmer 6accc87da1 tests: Improve Ansible fail_msg formatting
By switching to block style (`|`) with clip (no `-` or `+`) the failure
messages don't require quoting and gain a single trailing newline. This causes
Ansible to print them as block style, when using the yaml stdout callback
plugin. As a result the values have one less layer of quoting and quote
escaping, making them much easier to read.
2 months ago
Alex Willmer 551690ee1d ansible_mitogen: Handle templated connection passwords and ansible_ssh_password
This switches `ansible_mitogen.transport_config.PlayContextSpec.password()` to
Ansible's plugin option framework. As a result
- The relatively recent `ansible_ssh_password` variable is now respected.
- The SSH connection password can be templated and specified as a play
  variable. Task variables will probably also work, but testing was blocked
  by #1132.

There is a chance this change will cause a regression in another connection
plugin (e.g. mitogen_docker), but nothing turned up in the test suite.
I intend ot migrate other connection configuration to
`ansible_mitogen.transport_config.PlayContextSpec._connect_option()`, the next
candidate is the remote port.

fixes #1106
2 months ago
Alex Willmer 3bdd3e237a tests: Coverage of support for ansible_ssh_password variable 2 months ago
Alex Willmer 27214517a7 tests: Use a subprocess to check discovered python == running
This replaces the use of `os.path.realpath()` which gave incorrect results on
macOS - depending on the exact Python build, Python version, macOS version,
installation method, and phase of the moon.

realpath information kept around to aid debugging.
2 months ago
Alex Willmer c6c8bfb690 tests: Skip vanilla Ansible on Linux unpriviliged -> unprivileged become
CI containers lack the necessary `setfacl` command. This has not previously
been noticed because no vanilla Ansible jobs were being run on Linux, only on
macOS.

refs #1118
2 months ago
Alex Willmer b1fd6038bf ansible_mitogen: Remove Python 2.4 and 2.5 backward compatibility fallbacks
Because ansible_mitogen >= 0.3 supports Ansible >= 2.10 and Ansible 2.10
requires supports Python >= 2.7 on controllers and Python >= 2.6 on targets
these are dead weight.

See
- https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#ansible-core-support-matrix
- tox.ini
2 months ago
Alex Willmer 79ed797bad tests: Test templating of ansible_ssh_common_args et al
refs #905
3 months ago
Alex Willmer f3097b5743 ci: Template Ansible test-targets inventory with Jinja2 3 months ago
Alex Willmer 46c9f772d8 tests: Simplify Ansible ssh password tests, test priority
This
- Removes the indirection of calling ansible in a sub-shell
- Includes vanilla Ansible, which was previously skipped
- Tests whether ansible_ssh_pass overrides ansible_password, as it should

As a one off I've the new tests against vanilla Ansible 2.10 through Ansible
10, to confirm the baseline priorities have remained unchanged all releases
currently supported by Mitogen 0.3.x.
3 months ago
Alex Willmer 5af6534a70 tests: Test AnsibleUnsafeText when copying files larger SMALL_FILE_LIMIT
The bug was fixed in a previous commit by Jonathan Rosser. This adds testing.
The bug is only triggered when the copy module is used inside a `with_items:`
loop and the destination filename has an extension. A `loop:` loop is not
sufficient.

refs #1110
3 months ago
Alex Willmer ce1accedbc tests: Refactor Ansible copy integration tests to be loop driven
This is in anticipation of #1110, which only exhibits inside a with_items:
loop. For this refactor `loop:` is used, to confirm the refactored tests are
still correct. A subsequent commit will change them to with_items.

The content of the files and their SHA1 checksums are unchanged.
3 months ago
Alex Willmer 5ab872f289 ansible_mitogen: Add regression test for ActionModuleMixin._remote_chmod()
Adapted from Jonathon's reproducer in #1087.
3 months ago
Alex Willmer 357fe38766 Ansible 10 (ansible-core 2.17) support
Notably
- Python 2.7 and 3.6 are no longer supported by Ansible on targets
- The yum module has been removed, and redirected to dnf
- _INTERPRETER_PYTHON_DISTRO_MAP has been neutered. Interpreter discovery
  always favours specific `python3.<x>` interpreters in decending version
  order, then generic `python3` or `python`.
- Add the ability for an action plugin to call self._execute_module(*,
  ignore_unknown_opts=True) to execute a module with options that may not be
  supported for the version being called.

https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_10.html
https://github.com/ansible-community/ansible-build-data/blob/main/10/CHANGELOG-v10.md
https://github.com/ansible/ansible/blob/stable-2.17/changelogs/CHANGELOG-v2.17.rst

fixes #1074, refs #1082

Co-authored-by: Claude Becker <becker@phys.ethz.ch>
3 months ago
Alex Willmer 85b1b4070a tests: Respect configured or detected Python more often
Relying on the virtualenv default or hardcoding "python" results in a Python
2.x virtualenv on some targets (e.g. debian10-test). This caused a failure
when testing with Ansible >= 10 (ansible-core >= 2.17), which have dropped
Python 2.x support.

refs #1074
3 months ago
Alex Willmer 863f923f14 tests: Bypass interpreter discovery on non-existant connection delegation targets
By setting ansible_python_interpreter for these fictious hosts we avoid
Ansible trying and failing to connect to them in a attempt to populate
ansible_facts.discovered_interpreter_python. This speeds up these tests by
avoiding a timeout.

It is also a necessary pre-requisite for Ansible 10 (ansible-core 2.17). In
that release no hardcoded fallback is used, failure to determine a valid
Python interpreter is a fatal error.

refs #1074
3 months ago
Alex Willmer 40695f413b ansible_mitogen: Respect ansible_facts.discovered_interpreter_python more
fixes #1097
3 months ago
Alex Willmer fe435bb7d0 CI: Workaround "No module named 'setuptools.command.test'"
Pip 72 was released yesterday (2024-07-28), dropping `setup.py test` support.
hdrhistogram 0.6.1 requires it to install.

For now constrain Pip to earlier releases, so our tests can be run.

refs #1090
4 months ago
Alex Willmer 924dbd6f0c CI: Migrate macOS integration tests to macOS 12, drop Python 2.7 jobs
macOS 11 is not longer an available runner on Azure Devops. The minimum is now
macOS 12. This runner does not have Python 2.7 installed, so running them
would require a custom install - which I'm declaring too much effort for too
little gain.

refs #1090
4 months ago
Alex Willmer f5a8840668 CI: Use archived RPMs on CentOS 8
CentOS 8 has reached EOL. Packages are no longer mirrored or maintained. A
historic snapshot of the packages is kept on vault.centos.org.

refs #1088, #1090
4 months ago
Alex Willmer 7079a07a15 tests: Fix duplicate local task executions
integration/ssh/timeouts.yml is noteworthy. It was an accidental N**2 in time
-  executing num hosts * num hosts tasks.
6 months ago
Alex Willmer 65c8a42c13 tests: Use same verbosity when re-executing Ansible inside a playbook 6 months ago
Alex Willmer 05d98e5b49 tests: Speed up ssh timeout tests 6 months ago
Alex Willmer 60f868290d
tests: Remove --limit when running Ansible localhost CI
Some tests were being incorrectly excluded. Including those that use
`add_host`.
refs #1066, #1069
7 months ago
Alex Willmer d2eefc06aa
tests: Add regression for add_host with host_key_checking
refs #1066
7 months ago
Alex Willmer 8c93973f98 tests: Use Android portal to check get_url
Should have higher uptime, and make us less of a burden. Refs #1058
7 months ago
Alex Willmer d5e9186289 ansible_mitogen: Fix --ask-become-pass, add test coverage
Previously f1503874de fixed the priority of
ansible_become_pass over ansible_become_password, but broke --ask-become-pass.
Fixes #952.
7 months ago