Commit Graph

3696 Commits (v0.3.15)
 

Author SHA1 Message Date
Alex Willmer 8c93973f98 tests: Use Android portal to check get_url
Should have higher uptime, and make us less of a burden. Refs #1058
8 months ago
Alex Willmer d5e9186289 ansible_mitogen: Fix --ask-become-pass, add test coverage
Previously f1503874de fixed the priority of
ansible_become_pass over ansible_become_password, but broke --ask-become-pass.
Fixes #952.
8 months ago
Alex Willmer 3f7a0f74a5
Merge pull request #1057 from moreati/release-v0.3.7
Release v0.3.7
8 months ago
Alex Willmer f6ed546c38 Merge commit 'a3644963c40f117b2bef819cc8617bbd395c7f95' into release-v0.3.7 8 months ago
Alex Willmer c4cf0d5ba2 ci: Use profile_tasks callback as rough benchmark of Ansible tests 8 months ago
Alex Willmer a3644963c4 Prepare v0.3.7 8 months ago
Alex Willmer e1b2f38c8e tox: Add python2 & python3 to adhoc install hint 8 months ago
Alex Willmer 37ebce7e6e Begin 0.3.8dev 8 months ago
Alex Willmer cca651da1f ansible_mitogen: Ansible 9 (ansible-core 2.16) support 8 months ago
Alex Willmer 45c42d386a tests: Replace uses of ``include:``, unify skipping of mitogen only tests
The tag mitogen_only is only informational for now. It may be possible to use
it with ANSIBLE_SKIP_TAGS in the future.
8 months ago
Alex Willmer fa1d21747f ansible_mitogen: Declare Ansible 8 (ansible-core 2.15) support
refs #1021
8 months ago
Alex Willmer 2333b9aced ci: Exclude docs-master branch 8 months ago
Alexandre Detiste fe54b0ac3f prefer newer unittest.mock from the standad library 8 months ago
Alexandre Detiste 58235e3675 add Python3 compatibility 8 months ago
Alex Willmer b488baed25 Prepare 0.3.6 8 months ago
Alex Willmer d70ec4e540 ansible_mitogen: Handle AnsibleUnsafeText et al in Ansible >= 7
Follwing fixes in Ansible 7-9 for CVE-2023-5764 cating `AnsibleUnsafeBytes` &
`AnsibleUnsafeText` to `bytes()` or `str()` requires special handling. The
handling is Ansible specific, so it shouldn't go in the mitogen package but
rather the ansible_mitogen package.

`ansible_mitogen.utils.unsafe.cast()` is most like `mitogen.utils.cast()`.
During development it began as `ansible_mitogen.utils.unsafe.unwrap_var()`,
closer to an inverse of `ansible.utils.unsafe_procy.wrap_var()`. Future
enhancements may move in this direction.

refs #977, refs #1046

See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
- https://github.com/mitogen-hq/mitogen/wiki/AnsibleUnsafe-notes
8 months ago
Alex Willmer 8059be7160 ansible_mitogen: Make ansible_mitogens.utils a package
Prep work for ansible_mitogen.utils.unsafe
8 months ago
Alex Willmer 1c0253f201 mitogen: Raise TypeError on `mitogen.utils.cast(custom_str)` failures
If casting a string fails then raise a TypeError. This is potentially an API
breaking change; chosen as the lesser evil vs. allowing silent errors.

`cast()` relies on `bytes(obj)` & `str(obj)` returning the respective
supertype. That's no longer the case for `AnsibleUnsafeBytes` &
`AnsibleUnsafeText`; since fixes/mitigations for  CVE-2023-5764.

fixes #1046, refs #977

See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
8 months ago
Orion Poplawski 11e5529e44 ansible_mitogen: Add Ansible 7 support
Co-authored-by: Orion Poplawski <orion@nwra.com>
8 months ago
Alex Willmer 1fbf93203c Start v0.3.6 development 8 months ago
Alex Willmer 933477fcbe Begin 0.3.7dev 8 months ago
Alex Willmer 5d789faee5 Prepare 0.3.6 8 months ago
Alex Willmer b822f20007 ansible_mitogen: Handle AnsibleUnsafeText et al in Ansible >= 7
Follwing fixes in Ansible 7-9 for CVE-2023-5764 cating `AnsibleUnsafeBytes` &
`AnsibleUnsafeText` to `bytes()` or `str()` requires special handling. The
handling is Ansible specific, so it shouldn't go in the mitogen package but
rather the ansible_mitogen package.

`ansible_mitogen.utils.unsafe.cast()` is most like `mitogen.utils.cast()`.
During development it began as `ansible_mitogen.utils.unsafe.unwrap_var()`,
closer to an inverse of `ansible.utils.unsafe_procy.wrap_var()`. Future
enhancements may move in this direction.

refs #977, refs #1046

See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
- https://github.com/mitogen-hq/mitogen/wiki/AnsibleUnsafe-notes
8 months ago
Alex Willmer 813f253d6b ansible_mitogen: Make ansible_mitogens.utils a package
Prep work for ansible_mitogen.utils.unsafe
8 months ago
Alex Willmer d7979c3597 mitogen: Raise TypeError on `mitogen.utils.cast(custom_str)` failures
If casting a string fails then raise a TypeError. This is potentially an API
breaking change; chosen as the lesser evil vs. allowing silent errors.

`cast()` relies on `bytes(obj)` & `str(obj)` returning the respective
supertype. That's no longer the case for `AnsibleUnsafeBytes` &
`AnsibleUnsafeText`; since fixes/mitigations for  CVE-2023-5764.

fixes #1046, refs #977

See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
8 months ago
Orion Poplawski dfc3c7d516 ansible_mitogen: Add Ansible 7 support
Co-authored-by: Orion Poplawski <orion@nwra.com>
8 months ago
Alex Willmer 541f7c3247
Merge pull request #1044 from moreati/stable-0.3.5
Release v0.3.5
8 months ago
Alex Willmer 1572da1563 docs: Correct PEP 451 hyperlink
(cherry picked from commit 50efa53f8f)
8 months ago
Alex Willmer 21e874e60c
Merge pull request #1047 from mitogen-hq/changlog-pep451
docs: Correct PEP 451 hyperlink
8 months ago
Alex Willmer 50efa53f8f
docs: Correct PEP 451 hyperlink 8 months ago
Alex Willmer d83cfd6bed Merge v0.3.5 preperation 8 months ago
Alex Willmer a9d32a7708
Merge pull request #1043 from moreati/rel-0.3.5
Prepare 0.3.5 release, start 0.3.6 development
8 months ago
Alex Willmer fc24b3f25e Start v0.3.6 development 8 months ago
Alex Willmer e97ab2f597 Prepare v0.3.5 8 months ago
Alex Willmer a210c37f70
Merge pull request #1032 from moreati/docs-download-url
Python 3.12 support
8 months ago
Alex Willmer 123efa7510 mitogen: Support Python 3.12
Most of the necessary changes were made in recent PEP 451 commits. This bumps
the CI jobs, and declares the support. Test dependendancies are bumped to
latest supportted/available versions.

refs #1033
8 months ago
Alex Willmer fe8a3a71fc ansible_mitogen: Remove use of distutils, which was removed in Python 3.12 8 months ago
Alex Willmer 92c00d913e tests: Skip "discovered python matches invoked" on macOS 11/Python 2.7/Vanilla 8 months ago
Alex Willmer 5ad3d14ceb mitogen: Support PEP 451 ModuleSpec API, required for Python 3.12
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4
under PEP 451. They replace the find_module() API of PEP 302, which was
deprecated from Python 3.4. They were removed in Python 3.12 along with the
imp module.

This change adds support for the PEP 451 APIs. Mitogen should no longer import
imp on Python versions that support ModuleSpec. Tests have been added to cover
the new APIs.

CI jobs have been added to cover Python 3.x on macOS.

Refs #1033
Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
8 months ago
Alex Willmer 3a31a7d886 mitogen: Workaround CPython importlib PermissionError when cwd is unreadable
On macOS when using a become plugin as an unprivileged user, to another
unprivileged user it is likely that the current working directory can't be
read. In this case os.cwd() raises PermissionError.

On versions of Python currently in the wild (March 2024, CPython <= 3.13) if
any non-builtin or non-frozen module (e.g. zlib, base64) is imported then
`importlib._bootstrap_external.PathFinder._path_importer_cache()` attempts to
call os.cwd() without catching PermissionError.

The previous comment about needing an extra .encode() appears to be wrong,
atleast for Python 3.x >= 3.6.

Command size increased by 54 bytes, bootstrap by 804 bytes. Changed from
codecs module to binascii & zlib because they're extensions, and importing
them triggers fewer supporting imports (e.g. encodings module).

Before

```
✗ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17078 (16.68KiB)

                              Original          Minimized           Compressed
mitogen.parent            97884 95.6KiB  50515 49.3KiB 51.6%  12727 12.4KiB
13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB
19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB
19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB
18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB
7.8%
mitogen.service           41699 40.7KiB  22477 22.0KiB 53.9%   5885  5.7KiB
14.1%
mitogen.fakessh           15577 15.2KiB   7989  7.8KiB 51.3%   2623  2.6KiB
16.8%
mitogen.master            51398 50.2KiB  25715 25.1KiB 50.0%   6886  6.7KiB
13.4%
```

After

```
✗ ./preamble_size.py
SSH command size: 759
Bootstrap (mitogen.core) size: 17882 (17.46KiB)

                              Original          Minimized           Compressed
mitogen.parent            98173 95.9KiB  50571 49.4KiB 51.5%  12747 12.4KiB
13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB
19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB
19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB
18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB
7.8%
mitogen.service           41699 40.7KiB  22477 22.0KiB 53.9%   5885  5.7KiB
14.1%
mitogen.fakessh           15577 15.2KiB   7989  7.8KiB 51.3%   2623  2.6KiB
16.8%
mitogen.master            56116 54.8KiB  29427 28.7KiB 52.4%   7627  7.4KiB
13.6%
```

Fixes #885
Refs https://github.com/python/cpython/issues/115911
8 months ago
Alex Willmer 1031551dd9 tests: Clarify transport config tests optimisation & correct value
The ini inventory parser doesn't support comments after a value, so the value
parsed was "python3000  # Not expected to exist".
9 months ago
Alex Willmer 2973d90670 tests: Enable su tests under vanilla Ansible >= 2.11
cwd_show was useful when debugging these tests, worth keeping around.
9 months ago
Alex Willmer e2f4d9275c tests: Fix ansible_python_interpreter & discovered_interpreter_python tests on macOS
Should account for fiddling in mitogen.parent.Connection._first_stage() and
symlinks. I won't be surprised if it breaks again soon and often.
9 months ago
Alex Willmer c2ad52e54e tests: Fix tests using get_url across Python versions
Using https:// requires certificate store management and additional parameter
passing that changed across Ansible and Python versions. Using http:// allows
the same tests to be used across wider spans of Python version on the
controller, and Python verison on the targets.

Python 3.12 on a target + get_uri needs Ansible >= 8 (ansible-core >= 2.15).
Python 3.12 removed deprecated httplib.HTTPSConnection() arguments.
https://github.com/ansible/ansible/pull/80751
9 months ago
Alex Willmer a6a5c5bb97 tests: Clarify status/purpose of Python 2.x era Ansible Module workaround 9 months ago
Alex Willmer 2839954559 tests: Account for /tmp symlink in virtualenv test on macOS 9 months ago
Alex Willmer adfd4e17f3 tests: Declare inventory file types to Visual Studio Code and Vim
Works with the VS Code modeline extension. Enables syntax highlighting.
9 months ago
Alex Willmer 591152bef0 tests: Avoid intermittant 2 hour timeout in new style Ansible module tests
This has been lurking for years, raising it's head at unpredictable times.
This change doesn't fix it, but it should make it a lot less mysterious.
9 months ago
Alex Willmer a6c89751f9 tests: Cleanup ansible-lint errors & warnings in user creation playbook
Task " Install slow profile for one account" removed because it duplicates
earlier work.
9 months ago
Alex Willmer 8b574f234d tests: Report Ansible controller parameters before image prep & user creation 9 months ago