A few changes are bundled in this
- Ansible 2.10.x and Mitogen 0.3.x are used to build nearly all images
(Ansile 2.3.x is retained for CentOS 5, because it uses Python 2.4).
- Tox is used to install/run Ansible, replacing build_docker_images.py
- A static inventory, identifying containers by name rather than ID.
- debian-test image is renamed to debian9-test
- debian9-test image is now based on debian:9
- centos6-test image is now based on moreati/centos6-vault
following the same scheme as centos5-test.
- Images are now uploaded to Amazon Elastic Container Registry (ECR).
See #809.
- Debian recommended packages aren't installed (e.g. build-essential)
- Python 2.x and Python 3.x are installed wherever available.
- Python Virtualenv is installed wherever available.
The Travis Linux distribution must be upgraded because the Trusty
(Ubuntu 14.04) image does not have Python 3.9. Xenial (Ubuntu 16.04) is
the earliest version that offers Python 3.9.
I have not chosen a later release, in order to aid restoration of Python
2.4 - 2.6 tests.
The ansible_mitogen test suite takes over an hour when Ansible is not
accelerated by Mitogen. This change aims to reduce that by skipping
tests with a large number of iterations when the linear strategy is
chosen.
The tagged tests are intended to uncover Mitogen resource leaks. Since
Mitogen is not invoked when strategy=linear, the slight reduction in
test coverage is an acceptable trade off.
These are not part of the official testing regime (tests run for pull
requests). I find them convenient for local development.
Limitations
- Python 2.7+ only. No Python 2.4, 2.5, or 2.6.
- Requires Pythons pre-installed (e.g. DeadSnakes, pyenv)
- No coverage of alternate controller OS (e.g. MacOS)
The environments tested by default are
py27-mode_ansible-ansible2.10
py36-mode_ansible-ansible2.10
py39-mode_ansible-ansible2.10
py27-mode_mitogen
py36-mode_mitogen
py39-mode_mitogen
py27-mode_mitogen-distro_centos7
py36-mode_mitogen-distro_centos7
py39-mode_mitogen-distro_centos7
The environment variable DISTRO is used to set the target Docker image
used when running the Mitogen unit tests. DISTROS is used for the
Ansible integration tests. VER sets the version of Ansible that is
installed on the controller.
When MODE=Mitogen
- there is no need to set VER, because nothing that installs Ansible
- it does not make sense to set DISTROS.
CALLBACK_VERSION et al are documented as required in
https://docs.ansible.com/ansible/2.10/dev_guide/developing_plugins.html#callback-plugins.
The need for document_fragment is noted in
cfa8075537/lib/ansible/plugins/callback/default.py (L28-L32)Fixes#758
This addresses the following error, seen while running
`ansible_tests.py`.
```
TASK [Gathering Facts gather_timeout=10, gather_subset=['all']]
****************
task path:
/home/alex/src/mitogen/tests/ansible/regression/issue_109__target_has_old_ansible_installed.yml:4
[WARNING]: Failure using method (v2_runner_on_start) in callback plugin
(<ansible.plugins.callback.nice_stdout.CallbackModule object at
0x7f76b3dad090>): 'show_per_host_start'
Callback Exception:
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py",
line 372, in send_callback
method(*new_args, **kwargs)
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/plugins/callback/default.py",
line 240, in v2_runner_on_start
if self.get_option('show_per_host_start'):
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/plugins/callback/__init__.py",
line 91, in get_option
return self._plugin_options[k]
Callback Exception:
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py",
line 372, in send_callback
method(*new_args, **kwargs)
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/plugins/callback/default.py",
line 240, in v2_runner_on_start
if self.get_option('show_per_host_start'):
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/plugins/callback/__init__.py",
line 91, in get_option
return self._plugin_options[k]
[task 339882] 00:00:08.172036 D ansible_mitogen.affinity: CPU mask for
WorkerProcess: 0x000004
Callback Exception:
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py",
line 372, in send_callback
method(*new_args, **kwargs)
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/plugins/callback/default.py",
line 240, in v2_runner_on_start
if self.get_option('show_per_host_start'):
File
"/home/alex/src/mitogen/.tox/py27-ansible2.10/lib/python2.7/site-packages/ansible/plugins/callback/__init__.py",
line 91, in get_option
return self._plugin_options[k]
```
- don't create a new connection during reset if no existing connection
exists
- strip off last hop in connection stack if PlayContext.become is True.
- log a debug message if reset cannot find an existing connection
Logs were broken because ADD_ROUTE was being sent *after* messages
started flowing from the new child. That's an explicit non-goal of the
design, so fix it.
The undocumented 'tmp' parameter controls whether _execute_module()
would delete anything on 2.3, so mimic that. This means
_execute_remove_stat() calls will not blow away the temp directory,
which broke the unarchive plugin.
When creating a context using Router.method(via=somechild),
unidirectional mode was set on the new child correctly, however if the
child were to call Router.method(), due to a typing mistake the new
child would start without it.
This doesn't impact the Ansible extension, as only forked tasks are
started directly by children, and they are not responsible for routing
messages.
Add test so it can't happen again.
Previously given something like:
l = mitogen.core.Latch()
l.put(1)
l.put(2)
s = mitogen.select.Select([l], oneshot=False)
assert 1 == s.get(block=False)
assert 2 == s.get(block=False)
The second call would throw TimeoutError, because Select.add() only
queued the receiver/latch once if it was non-empty, rather than once for
each item as should happen.
Alex Willmer