d70ec4e540
ansible_mitogen: Handle AnsibleUnsafeText et al in Ansible >= 7
...
Follwing fixes in Ansible 7-9 for CVE-2023-5764 cating `AnsibleUnsafeBytes` &
`AnsibleUnsafeText` to `bytes()` or `str()` requires special handling. The
handling is Ansible specific, so it shouldn't go in the mitogen package but
rather the ansible_mitogen package.
`ansible_mitogen.utils.unsafe.cast()` is most like `mitogen.utils.cast()`.
During development it began as `ansible_mitogen.utils.unsafe.unwrap_var()`,
closer to an inverse of `ansible.utils.unsafe_procy.wrap_var()`. Future
enhancements may move in this direction.
refs #977 , refs #1046
See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
- https://github.com/mitogen-hq/mitogen/wiki/AnsibleUnsafe-notes
8 months ago
1c0253f201
mitogen: Raise TypeError on `mitogen.utils.cast(custom_str)` failures
...
If casting a string fails then raise a TypeError. This is potentially an API
breaking change; chosen as the lesser evil vs. allowing silent errors.
`cast()` relies on `bytes(obj)` & `str(obj)` returning the respective
supertype. That's no longer the case for `AnsibleUnsafeBytes` &
`AnsibleUnsafeText`; since fixes/mitigations for CVE-2023-5764.
fixes #1046 , refs #977
See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
8 months ago
11e5529e44
ansible_mitogen: Add Ansible 7 support
...
Co-authored-by: Orion Poplawski <orion@nwra.com>
8 months ago
1fbf93203c
Start v0.3.6 development
8 months ago
1572da1563
docs: Correct PEP 451 hyperlink
...
(cherry picked from commit 50efa53f8f
8 months ago
d83cfd6bed
Merge v0.3.5 preperation
8 months ago
e97ab2f597
Prepare v0.3.5
8 months ago
123efa7510
mitogen: Support Python 3.12
...
Most of the necessary changes were made in recent PEP 451 commits. This bumps
the CI jobs, and declares the support. Test dependendancies are bumped to
latest supportted/available versions.
refs #1033
8 months ago
5ad3d14ceb
mitogen: Support PEP 451 ModuleSpec API, required for Python 3.12
...
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4
under PEP 451. They replace the find_module() API of PEP 302, which was
deprecated from Python 3.4. They were removed in Python 3.12 along with the
imp module.
This change adds support for the PEP 451 APIs. Mitogen should no longer import
imp on Python versions that support ModuleSpec. Tests have been added to cover
the new APIs.
CI jobs have been added to cover Python 3.x on macOS.
Refs #1033
Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
8 months ago
3a31a7d886
mitogen: Workaround CPython importlib PermissionError when cwd is unreadable
...
On macOS when using a become plugin as an unprivileged user, to another
unprivileged user it is likely that the current working directory can't be
read. In this case os.cwd() raises PermissionError.
On versions of Python currently in the wild (March 2024, CPython <= 3.13) if
any non-builtin or non-frozen module (e.g. zlib, base64) is imported then
`importlib._bootstrap_external.PathFinder._path_importer_cache()` attempts to
call os.cwd() without catching PermissionError.
The previous comment about needing an extra .encode() appears to be wrong,
atleast for Python 3.x >= 3.6.
Command size increased by 54 bytes, bootstrap by 804 bytes. Changed from
codecs module to binascii & zlib because they're extensions, and importing
them triggers fewer supporting imports (e.g. encodings module).
Before
```
✗ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17078 (16.68KiB)
Original Minimized Compressed
mitogen.parent 97884 95.6KiB 50515 49.3KiB 51.6% 12727 12.4KiB
13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB
19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB
19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB
18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB
7.8%
mitogen.service 41699 40.7KiB 22477 22.0KiB 53.9% 5885 5.7KiB
14.1%
mitogen.fakessh 15577 15.2KiB 7989 7.8KiB 51.3% 2623 2.6KiB
16.8%
mitogen.master 51398 50.2KiB 25715 25.1KiB 50.0% 6886 6.7KiB
13.4%
```
After
```
✗ ./preamble_size.py
SSH command size: 759
Bootstrap (mitogen.core) size: 17882 (17.46KiB)
Original Minimized Compressed
mitogen.parent 98173 95.9KiB 50571 49.4KiB 51.5% 12747 12.4KiB
13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB
19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB
19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB
18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB
7.8%
mitogen.service 41699 40.7KiB 22477 22.0KiB 53.9% 5885 5.7KiB
14.1%
mitogen.fakessh 15577 15.2KiB 7989 7.8KiB 51.3% 2623 2.6KiB
16.8%
mitogen.master 56116 54.8KiB 29427 28.7KiB 52.4% 7627 7.4KiB
13.6%
```
Fixes #885
Refs https://github.com/python/cpython/issues/115911
8 months ago
b7188c1cad
docs: Decouple website download version from package version
...
This prevents unreleased versions appearing on the website (e.g. 0.3.5.dev0),
but introduces the risk of forgetting to update the website after a release.
A better fix requires deeper design/workflow thought.
refs #1028
1 year ago
e580258071
docs: Bypass networkgenomics.com/try/ -> PyPI redirect
...
refs #1028
1 year ago
63457b4866
docs: Update external URLs (e.g. dw/mitogen -> mitogen-hq/mitogen)
...
Found with sphinx-build -b linkcheck. Not all flagged URLs have been changed,
e.g. Ansible plugins, deleted Github users.
1 year ago
6aa4fd3573
docs: Fix generation of static website
...
Bare minimum syntax errors and requirements constraints to work with Netlify
hosting.
1 year ago
4089e875a9
Add Python 3.11 support
...
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
1 year ago
b4d910ae1d
Merge commit 'f18f516' into release-0.3.4
1 year ago
455fd2bcdf
Bump version
1 year ago
f18f5165cd
Prep for v0.3.4
1 year ago
19b79f7ab5
CI: Fix tests on Linux, Ansible tests targetting Debian 9 & 11
...
Without Ubuntu 20.04 virtualenv package being installed pip was installing a
version of virtualenv that couldn't create the Tox environment for Python 2.7.
> Successfully installed distlib-0.3.6 filelock-3.12.2 platformdirs-3.8.0
> pluggy-1.2.0 py-1.11.0 tomli-2.0.1 tox-3.28.0 virtualenv-20.23.1
> Finishing: Install tooling
> ...
> py27-mode_mitogen-distro_centos6 create: /home/vsts/work/1/s/.tox/py27-
> mode_mitogen-distro_centos6
> ERROR: invocation failed (exit code 1), logfile: /home/vsts/work/1/s/.tox/
> py27-mode_mitogen-distro_centos6/log/py27-mode_mitogen-distro_centos6-0.log
> ================================== log start
> ===================================
> RuntimeError: failed to query /usr/bin/python2.7 with code 1 err:
> ' File "/home/vsts/.local/lib/python3.8/site-packages/virtualenv/discovery/
> py_info.py", line 24\n return list(OrderedDict.fromkeys(["",
> *os.environ.get("PATHEXT", "").lower().split(os.pathsep)]))\n
> ^\nSyntaxError: invalid syntax\n'
1 year ago
21cb4a3472
CI: Remove faulthandler fallback requirement
...
faulthandler is a stdlib module in Python 3.3+. For a long time a PyPI package
of the same name was available for earlier Python releases. That package has
since been removed from PyPI, and the source respoitory archived. So we should
not rely on it.
fixes #983 refs #970
2 years ago
1871f2a9b1
Remove vendored mitogen.compat.simplejson
...
Python 2.6 added json to the stdlib. We no longer support Python <= 2.7 in
Mitogen 0.3.x, so this fallback is unneeded complexity. Fixes #659
2 years ago
0af2ce8c30
Remove ansible_mitogen Connection.close() workaround
...
Refs #925 #969
I'm not 100% confident that merely removing this is the full fix,
without substituting something else. I am sure keeping it would be
the greater of two evils. __del__() should be avoided on general
principal, and it's associated with multiple intermittant CI
failures, plus multiple user reported issues.
2 years ago
e8ad12e881
Ansible 6 support
...
fixes #929
2 years ago
c1e72b8225
Fix typo changelog.rst
2 years ago
a30a743ce7
Add ansible.builtin.dnf to ALWAYS_FORK_MODULES
...
The new fully qualified name of the DNF module needs to also be added to the list.
Fixes #832
2 years ago
8ab84237af
Merge commit 'c0d3deeac571eb36dfccbe5ca2f1579f2aceca2a' into release-0.3.3
3 years ago
d71fb672e8
Begin v0.3.4.dev0
3 years ago
c0d3deeac5
Prepare v0.3.3
3 years ago
25ea6dde02
ansible_mitogen: Allow mitogen_fetch to bypass slurp module
...
This reapplies an earlier change, when this plugin was first introduced to
Mitogen. The plugin was updated to fix
[DEPRECATION WARNING]: The '_remote_checksum()' method is deprecated.
I've elected to short-circuit the if statemtn logic, rather than
deleting/unindenting, to make the code delta much smaller. This should make it
easier to maintain/update.
Fixes #915
3 years ago
e101cc4f44
mitogen.utils: Preserve docstring of functions decorated @with_router
...
Co-authored-by: Rezart Qelibari <gast-kontakt+mitogen@astzweig.de>
Replaces #837
Fixes #836
3 years ago
96e20a09d6
ansible_mitogen: Add podman connection plugin
3 years ago
0417d4d73a
Replace os.system() with subprocess.check_call()
...
Non-zero return codes should raise an exception, not pass silently.
3 years ago
2a95d039ab
Python 3.10 support
3 years ago
d2ca8a9423
master.ParentEnumerationMethod: Require matching pkg.__name__
...
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
When the requested module (e.g. ansible.module_utils.distro)
- is provided by another module *e.g. distro)
- that itself was a package (e.g. distro 1.7.0)
At runtime
- ansible/module_utils/distro/__init__.py executes
- if https://pypi.org/project/distro/ is present, it's loaded as
ansible.module_utils.distro
- otherwise ansible/module_utils/distro/_distro.py is loaded
ParentEnumerationMethod would wrongly use whatever was in
sys.modules['ansible.module_utils.distro]. Instead we should ascend to
the first parent that has fullname == sys.modules[fullname].__name__.
Then descend to the appropriate .py file on disk.
This bug didn't show up before because until distro 1.7.0 (Feb 2022) the
top-level distro module was a module (distro.py) not a package
(distro/__init__.py)
fixes #906
3 years ago
5b8f7dd1be
Start v0.3.3 development
3 years ago
e8c3fe7881
Fix Trove classifier, bump version
...
fixes #891
(cherry picked from commit 1a84184838
3 years ago
1a84184838
Fix Trove classifier, bump version
...
fixes #891
3 years ago
b19223c168
Release 0.3.1
3 years ago
61ccf055ad
Merge branch 'master' into prepare-0.3.1
3 years ago
54b9115f20
Update docs/ansible_detailed.rst
...
fixed supported ansible / python version listing
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
3 years ago
56c341790c
prepare support for ansible community 5 aka. core 2.12
3 years ago
552819e765
mitogen.parent: Detect and avoid Python2.7 wrapper on macOS 11 & 12
...
Without this errors such as the following occur
```
✗ MITOGEN_LOG_LEVEL=DEBUG python3 foo.py
Python: execv: (null): No such file or directory
Traceback (most recent call last):
File "foo.py", line 16, in <module>
target = router.local(python_path='/usr/bin/python2.7', debug=True)
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2486, in local
return self.connect(u'local', **kwargs)
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2446, in connect
return self._connect(klass, **mitogen.core.Kwargs(kwargs))
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2426, in _connect
conn.connect(context=context)
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 1708, in connect
raise self.exception
mitogen.parent.EofError: EOF on stream; last 100 lines received:
MITO000
MITO001
```
Before
```
$ ./preamble_size.py
SSH command size: 625
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97496 95.2KiB 50355 49.2KiB 51.6% 12663 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
After
```
$ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97885 95.6KiB 50516 49.3KiB 51.6% 12728 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48733 47.6KiB 24570 24.0KiB 50.4% 6771 6.6KiB 13.9%
```
3 years ago
465ac8abff
ansible: Fix AttributeError in kubectl connection
3 years ago
e194a6367f
ci: Fix version comparisons involving double digits
...
See https://gist.github.com/moreati/e7507c5b606b12ec0ddafcb7c8debbf1
3 years ago
e6bc53ca3a
mitogen.parent: Remove tabs from minimized first stage
...
Saves 4 bytes
Before
```
$ python preamble_size.py
SSH command size: 629
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97497 95.2KiB 50356 49.2KiB 51.6% 12665 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
After
```
$ python preamble_size.py
SSH command size: 625
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97496 95.2KiB 50355 49.2KiB 51.6% 12663 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
3 years ago
dd6d73db37
mitogen.parent: Eliminate use of platform module in first stage
...
This reduces the size of the initial SSH command by 204 bytes, & may fix errors
running Mitogen on macOS. AFAICT platform was used but not imported.
Before
```
$ python ./preamble_size.py
SSH command size: 833
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97565 95.3KiB 50427 49.2KiB 51.7% 12689 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
After
```
$ python preamble_size.py
SSH command size: 629
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97543 95.3KiB 50357 49.2KiB 51.6% 12665 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
3 years ago
c61c063b4f
Support for Ansible 3 & 4
...
fixes #834
Co-authored-by: Claude Becker (@upekkha)
Co-authored-by: Dolph Mathews (@dolph)
3 years ago
84c567e265
Add podman connection support
...
Shameless copy of buildah connection with modifications of invocation to
fit podman CLI.
Signed-off-by: Denis Zalevskiy <dez@aiven.io>
3 years ago
10caa4a104
Fix typo in buildah context creation docs
...
Signed-off-by: Denis Zalevskiy <dez@aiven.io>
3 years ago
c5c65ab0d2
ci: Switch Continuous Integration to Tox
...
Also
- Simplifies adding support for additional Ansible versions
- Unifies Python package versioning in CI and local test environments
- Matches Python versions tested, with those declared in setup.py
- Expands targets covered by automated Ansible tests to
- centos6, centos8
- debian9, debian11
- ubuntu1604, ubuntu2004
3 years ago
Alex Willmer