Some ansible_mitogen connection plugins look more like become plugins (e.g.
mitogen_sudo) & use become plugin options. For now there's special handling in
PlayContextSpec._become_option(). Further design/discussion can go in #1173.
Refs #1087.
The tasks in tests/imageprep/_user_accounts.yml that create users did not
specify a primary group for those users - this left the decision to Ansible's
user module, and/or the underlying OS. In Ansible 9+ (ansible-core 2.16+ the
user module defaults to primary group "staff." Earlier don't supply a default,
which releases probably results in a primary group nameed "None" (due to
stringifying the Python singleton of the same name), or whatever the macOS
Directory Services has for no data/NULL.
The invalid GID 4294967295 (MAX_UINT32 == 2**32-1) in the sudo error probably
enters the mix via something similar to sudo CVE-2019-14287.
Fixes#692
See
- https://github.com/ansible/ansible/pull/79999
- https://github.com/ansible/ansible/commit/c69c83c962f987c78af98da0746527df
- https://www.sudo.ws/security/advisories/minus_1_uid/
> Bruce Wayne : [confused] Am I meant to understand any of that?
> Lucius Fox : Not at all, I just wanted you to know how hard it was.
> -- Batman Begins
This reads the become username from the `become_user` attribute of the play
context, to the `"become_user"` option of the loaded become plugin. This has
been supported by vanilla Ansible since Ansible 2.10 (ansible-base 2.10).
To support this I've also switched from using the `play_context.become` (a
bool), to `connection.become` (an instance of the appropriate) become plugin.
New tests have been added, modelled on those for templated connection
parameters (see #1147, #1153, #1159).
See
- 480b106d65
refs #1083
Co-authored-by: mordek <m.pirog@bonasoft.pl>
This reduces the number of jobs from 48 to 24. The Mitogen part of the test
suite has been parameterized on the Linux container targets to be run against.
Both the Ansible tests & Mitogen tests now use the same source of truth to
control which targets to use: environment variable MITOGEN_TEST_DISTRO_SPECS.
This replaces the two mutually exclusive env vars DISTRO and DISTROS. I've
also removed vestgial traces of an unused env var MITOGEN_TEST_DISTRO.
Parameterization adapted from
https://eli.thegreenplace.net/2014/04/02/dynamically-generating-python-test-cases
refs #1058, #1059
The password is provided as a variable because there is no corresponding
keyword. I get the impression that keywords are considered a legacy mechanism,
so most (new) options are only overridable by variables.
The port is proved as a variable for now, to test remote_name in isolation.
These targets are not used by any active tests, and the large numbers of hosts
multiply the size of the taskvars disctionary in memory to many (10s) MiB.
refs #1058
By switching to block style (`|`) with clip (no `-` or `+`) the failure
messages don't require quoting and gain a single trailing newline. This causes
Ansible to print them as block style, when using the yaml stdout callback
plugin. As a result the values have one less layer of quoting and quote
escaping, making them much easier to read.
This switches `ansible_mitogen.transport_config.PlayContextSpec.password()` to
Ansible's plugin option framework. As a result
- The relatively recent `ansible_ssh_password` variable is now respected.
- The SSH connection password can be templated and specified as a play
variable. Task variables will probably also work, but testing was blocked
by #1132.
There is a chance this change will cause a regression in another connection
plugin (e.g. mitogen_docker), but nothing turned up in the test suite.
I intend ot migrate other connection configuration to
`ansible_mitogen.transport_config.PlayContextSpec._connect_option()`, the next
candidate is the remote port.
fixes#1106
This replaces the use of `os.path.realpath()` which gave incorrect results on
macOS - depending on the exact Python build, Python version, macOS version,
installation method, and phase of the moon.
realpath information kept around to aid debugging.
CI containers lack the necessary `setfacl` command. This has not previously
been noticed because no vanilla Ansible jobs were being run on Linux, only on
macOS.
refs #1118
Preperation for migrating from Azure DevOps with Amazon Elastic Container
Registry (AWS ECR), to GitHub Actions with GitHub Container Registry (GHCR).
DebOps tests are not currently being run, the updates to .ci/debops*.py are
best effort only.
Python 3.x emits `ResourceWarning`s if certains resources aren't correctly
closed. Due to the way Mitogen has been terminating child processes this has
been occurring.
```
test_dev_tty_open_succeeds
(create_child_test.TtyCreateChildTest.test_dev_tty_open_succeeds) ...
/opt/hostedtoolcache/Python/3.12.5/x64/lib/python3.12/subprocess.py:1127:
ResourceWarning: subprocess 3313 is still running
_warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok
```
During garbage collection subprocess.Popen() objects emit
ResourceWarning("subprocess 123 is still running")
if proc.returncode hasn't been set. Typically calling proc.wait() does so,
once the sub-process has exited. Calling os.waitpid(proc.pid, 0) also waits
for the sub-process to exit, but it doesn't update proc.returncode, so the
ResourceWarning is still emitted.
This change exposes `subprocess.Popen` methods on
`mitogen.parent.PopenProcess`, so that the returncode can be set.
See https://gist.github.com/moreati/b8d157ff82cb15234bece4033accc5e5
This
- Removes the indirection of calling ansible in a sub-shell
- Includes vanilla Ansible, which was previously skipped
- Tests whether ansible_ssh_pass overrides ansible_password, as it should
As a one off I've the new tests against vanilla Ansible 2.10 through Ansible
10, to confirm the baseline priorities have remained unchanged all releases
currently supported by Mitogen 0.3.x.
The bug was fixed in a previous commit by Jonathan Rosser. This adds testing.
The bug is only triggered when the copy module is used inside a `with_items:`
loop and the destination filename has an extension. A `loop:` loop is not
sufficient.
refs #1110
This is in anticipation of #1110, which only exhibits inside a with_items:
loop. For this refactor `loop:` is used, to confirm the refactored tests are
still correct. A subsequent commit will change them to with_items.
The content of the files and their SHA1 checksums are unchanged.
Relying on the virtualenv default or hardcoding "python" results in a Python
2.x virtualenv on some targets (e.g. debian10-test). This caused a failure
when testing with Ansible >= 10 (ansible-core >= 2.17), which have dropped
Python 2.x support.
refs #1074
By setting ansible_python_interpreter for these fictious hosts we avoid
Ansible trying and failing to connect to them in a attempt to populate
ansible_facts.discovered_interpreter_python. This speeds up these tests by
avoiding a timeout.
It is also a necessary pre-requisite for Ansible 10 (ansible-core 2.17). In
that release no hardcoded fallback is used, failure to determine a valid
Python interpreter is a fatal error.
refs #1074
Pip 72 was released yesterday (2024-07-28), dropping `setup.py test` support.
hdrhistogram 0.6.1 requires it to install.
For now constrain Pip to earlier releases, so our tests can be run.
refs #1090
macOS 11 is not longer an available runner on Azure Devops. The minimum is now
macOS 12. This runner does not have Python 2.7 installed, so running them
would require a custom install - which I'm declaring too much effort for too
little gain.
refs #1090
CentOS 8 has reached EOL. Packages are no longer mirrored or maintained. A
historic snapshot of the packages is kept on vault.centos.org.
refs #1088, #1090
Alex Willmer