It was necessary to split setup.yml because there is no common subset of
supported include/import keywords across Ansible 2.3 - 2.11. The yaml stdout
callback is unavailabe in Ansible 2.3.
This eliminates use of third-party *-vault images and performs repository
config during image prep.
The Apache httpd proxy is necessary because https://vault.centos.org now only
accepts TLS 1.x connections, and CentOS 5 can only do upto SSL 3.0. It is
developed to run on Debian 11.
The test currently fails with the following error:
$ PYTHONPATH=$(pwd)/tests:$PYTHONPATH python3 -m unittest -v log_handler_test
...
test_logrecordfactory (log_handler_test.LogRecordFactoryTest.test_logrecordfactory) ... --- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.12/logging/__init__.py", line 464, in format
return self._format(record)
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 460, in _format
return self._fmt % values
~~~~~~~~~~^~~~~~~~
KeyError: 'custom_attribute'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.12/logging/__init__.py", line 1160, in emit
msg = self.format(record)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 999, in format
return fmt.format(record)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 999, in format
return fmt.format(record)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 706, in format
s = self.formatMessage(record)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 675, in formatMessage
return self._style.format(record)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/logging/__init__.py", line 466, in format
raise ValueError('Formatting field not found in record: %s' % e)
ValueError: Formatting field not found in record: 'custom_attribute'
Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com>
This replaces `mitogen.master.scan_code_imports()` with
`mitogen.imports.codeobj_imports()`. The Python 3.x implementation now uses
`str.find()`, relying on Python >= 3.6 "widecode" format. Behaviour and
semantics should be unchanged. Now implementations are approx
- 1.5 x faster on Python 2.x
- 2 - 3 x faster on Python 3.x
Before
```console
$ ./tests/bench/scan_code
scan_code_imports python2.7 100 loops, best of 3: 3.19 msec per loop
scan_code_imports python3.9 500 loops, best of 5: 685 usec per loop
scan_code_imports python3.10 500 loops, best of 5: 727 usec per loop
scan_code_imports python3.11 500 loops, best of 5: 601 usec per loop
scan_code_imports python3.12 500 loops, best of 5: 609 usec per loop
scan_code_imports python3.13 500 loops, best of 5: 586 usec per loop
```
After
```console
codeobj_imports python2.7 1000 loops, best of 3: 1.98 msec per loop
codeobj_imports python3.9 1000 loops, best of 5: 302 usec per loop
codeobj_imports python3.10 1000 loops, best of 5: 297 usec per loop
codeobj_imports python3.11 1000 loops, best of 5: 243 usec per loop
codeobj_imports python3.12 1000 loops, best of 5: 278 usec per loop
codeobj_imports python3.13 1000 loops, best of 5: 259 usec per loop
```
```console
$ uname -a
Darwin kintha 24.6.0 Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:29 PDT
2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6000 arm64
```
```console
$ ./tests/bench/scan_code
scan_code_imports python2.7 100 loops, best of 3: 3.19 msec per loop
scan_code_imports python3.9 500 loops, best of 5: 685 usec per loop
scan_code_imports python3.10 500 loops, best of 5: 727 usec per loop
scan_code_imports python3.11 500 loops, best of 5: 601 usec per loop
scan_code_imports python3.12 500 loops, best of 5: 609 usec per loop
scan_code_imports python3.13 500 loops, best of 5: 586 usec per loop
```
This covers existing behaviours of `mitogen.master.scan_code_imports()` some
of which are relied on, some not, but regardless weren't tested. Notably
- Explicit relative imports return level > 0
- Imports inside `class` and `def` are excluded
- Imports inside other blocks are included
- Python 3.x prunes impossible if/else branches (previously unknown)
It also
- Decouples the test results from the implementation details of the unit test.
- Fixes a missing import
- Fixes at least one Python 2.4 incompatibility (use of with block)
Mitogen was leaving the stdout and stderr of subprocesses in non-blocking
mode. When Python code ran in the remote process created by Mitogen calls such
as `print(long_string)` or `os.stout.write(bigger_than_the_buffer)` sometimes
raised `BlockingIOError`, or similar.
This change
- Removes code in `mitogen.core.Side` that set blocking/non-blocking mode
- Adds blocking/non-blocking control to `os.mitogen.pipe()` and a new
function `mitogen.core.socketpair()`
- Replaces `mitogen.core.set_block` and `mitogen.core.set_nonblock`
with `mitogen.core.set_blocking`, mirroring `os.set_blocking`
- Updates call sites as appropriate
- Adds tests for new functions and arguments
- Adds a regression test for subprocess stdio blocking/non-blocking
fixes#712
In vanilla Ansible >= 12 (ansible-core 2.19)
- ssh connection plugin `verbosity` controls `ssh [-v[v[v]]]`
- config option `DEFAULT_VERBOSITY` controls whether that output is displayed
In vanilla Ansible <= 11 (ansible-core <= 2.18)
- `DEFAULT_VERBOSITY` controls both `ssh` verbosity & display verbositty
As of this change
- Mitogen + Ansible >= 12 behaviour matches vanilla Ansible >= 12.
- Mitogen + Ansible <= 11 behaviour remains unchanged
- `DEFAULT_VERBOSITY` only controls display verbosity.
- Mitogen + Ansible respect the Ansible variable `mitogen_ssh_debug_level`
I've chosen not to retroactively replicate the old vanilla Ansible behaviour
in Mitogen + Ansible <= 11 cases. I'm pretty sure it was an oversight,
rather than a design choice, but Ansible+Mitogen with `ANSIBLE_VERBOSITY=3`
is already very verbose.
fixes#1282
See
- https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-verbosity
- https://docs.ansible.com/ansible/devel/collections/ansible/builtin/ssh_connection.html#parameter-verbosity
Ansible 12 (ansible-core 2.19) has gained support for specifying an SSH
password, without requiring `sshpass`. It specifies the environment variable
`SSH_ASKPASS` such that `ansible` itself is called.
Mitogen is already able to support this. This change provides test coverage of
the new feature by not installing `sshpass` on macOS runners. when Ansible 12
is under test. Ubuntu runners come with `sshpass` pre-installed.
Required Ansible is also bumped to the latest pre-releases, for relevant
fixes.
Note that tests/ansible/integration/ssh/templated_by_play_taskvar.yml was
previously erroniously being skipped with ansible-core 2.19.0a<N> and
2.19.0b<N>.
fixes#1293
refs #1175
Ansible >= 12 (ansible-core >= 2.19) deprecates `stdout_callback=yaml`,
superceded by `callback_result_format=yaml`. There is a change in behaviour:
`callback_result_format` applies to output of both `ansible-playbook` _and_
`ansible`.
Tests that run `ansible` in a subprocess are now explicitly configured to use
json (even if they don't inspect that output yet) for more assert-able output
across all versions of Ansible.
The Van_* GitHub Actions jobs (corresponding to Tox factor strategy_linear,
environment variable ANSIBLE_STRATEGY=linear) were failing inside Mitogen
modules, which they should not touch. The jobs are intended as a cross
validation of the test suite, they should only fail if Ansible itself has a
problem.
Python 2.7 (distro package) and 3.6 (pyenv managed) jobs run on Ubuntu 22.04.
More recent Pythons (distro or Github provided) run on 24.04.
fixes#1256
Ansible tasks that run locally (e.g. `connection: local`, `delegate_to:
localhost`) must now specify their `ansible_python_interpreter`, typically as
`{{ ansible_playbook_python }}`; otherwise the system Python on the controller
(e.g. `/usr/bin/python`) is likely to be used and this is often outside the
version range supported by the Ansible verison under test. If this occurs then
the symptom is often a failure to import a builtin from
`ansible.module_utils.six.moves`, e.g.
```
fatal: [target-centos6-1]: FAILED! => changed=true
cmd:
- ansible
- -m
- shell
- -c
- local
- -a
- whoami
- -i
- /tmp/mitogen_ci_ansibled3llejls/hosts
- test-targets
delta: '0:00:02.076385'
end: '2025-04-17 17:27:02.561500'
msg: non-zero return code
rc: 8
start: '2025-04-17 17:27:00.485115'
stderr: |-
stderr_lines: <omitted>
stdout: |-
An exception occurred during task execution. To see the full traceback,
use -vvv. The error was: from ansible.module_utils.six.moves import
map, reduce, shlex_quote
```
Alex Willmer