Follwing fixes in Ansible 7-9 for CVE-2023-5764 cating `AnsibleUnsafeBytes` &
`AnsibleUnsafeText` to `bytes()` or `str()` requires special handling. The
handling is Ansible specific, so it shouldn't go in the mitogen package but
rather the ansible_mitogen package.
`ansible_mitogen.utils.unsafe.cast()` is most like `mitogen.utils.cast()`.
During development it began as `ansible_mitogen.utils.unsafe.unwrap_var()`,
closer to an inverse of `ansible.utils.unsafe_procy.wrap_var()`. Future
enhancements may move in this direction.
refs #977, refs #1046
See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
- https://github.com/mitogen-hq/mitogen/wiki/AnsibleUnsafe-notes
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4
under PEP 451. They replace the find_module() API of PEP 302, which was
deprecated from Python 3.4. They were removed in Python 3.12 along with the
imp module.
This change adds support for the PEP 451 APIs. Mitogen should no longer import
imp on Python versions that support ModuleSpec. Tests have been added to cover
the new APIs.
CI jobs have been added to cover Python 3.x on macOS.
Refs #1033
Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
Using https:// requires certificate store management and additional parameter
passing that changed across Ansible and Python versions. Using http:// allows
the same tests to be used across wider spans of Python version on the
controller, and Python verison on the targets.
Python 3.12 on a target + get_uri needs Ansible >= 8 (ansible-core >= 2.15).
Python 3.12 removed deprecated httplib.HTTPSConnection() arguments.
https://github.com/ansible/ansible/pull/80751
To do so the test suite allows a weak cryptographic alogorithm (SHA1) to be
used, principally for CentOS 6 targets. This can be removed if/when support
for older (legacy) targets is dropped.
Only the test suite enables this known weak alogorithm. Mitogen as-shipped
doesn't enable or disable algorithms.
Until Ansible 2.9 it looks like ansible_become_password had higher priority.
From Ansible 2.10 ansible_become_pass has higher priority [1]. Mitogen was not
respecting this.
I may need to rework this further, instatiating the become plugin may have
slowed down execution.
[1] Based on testing with
```
[ubuntus]
become-pass-pass ansible_become_pass=1234
become-pass-password ansible_become_password=1234
become-pass-both ansible_become_password=wrong ansible_become_pass=1234
[ubuntus:vars]
ansible_host=ubuntu2004.local
ansible_user=ubuntu
```
```
- hosts: ubuntus
gather_facts: false
become: true
tasks:
- ping:
```
A few changes are bundled in this
- Ansible 2.10.x and Mitogen 0.3.x are used to build nearly all images
(Ansile 2.3.x is retained for CentOS 5, because it uses Python 2.4).
- Tox is used to install/run Ansible, replacing build_docker_images.py
- A static inventory, identifying containers by name rather than ID.
- debian-test image is renamed to debian9-test
- debian9-test image is now based on debian:9
- centos6-test image is now based on moreati/centos6-vault
following the same scheme as centos5-test.
- Images are now uploaded to Amazon Elastic Container Registry (ECR).
See #809.
- Debian recommended packages aren't installed (e.g. build-essential)
- Python 2.x and Python 3.x are installed wherever available.
- Python Virtualenv is installed wherever available.
These are not part of the official testing regime (tests run for pull
requests). I find them convenient for local development.
Limitations
- Python 2.7+ only. No Python 2.4, 2.5, or 2.6.
- Requires Pythons pre-installed (e.g. DeadSnakes, pyenv)
- No coverage of alternate controller OS (e.g. MacOS)
The environments tested by default are
py27-mode_ansible-ansible2.10
py36-mode_ansible-ansible2.10
py39-mode_ansible-ansible2.10
py27-mode_mitogen
py36-mode_mitogen
py39-mode_mitogen
py27-mode_mitogen-distro_centos7
py36-mode_mitogen-distro_centos7
py39-mode_mitogen-distro_centos7
Alex Willmer