archive
/
mitogen
mirror of https://github.com/mitogen-hq/mitogen.git
1
0
Fork 0
Code Releases Activity
3,490 Commits
16 Branches
36 Tags
16 MiB
docs-master
stable
master
pr683
0.2-release
pull-request-checklist
dmw
circleci-project-setup
new-serialization
gh-pages
redirect
issue531
issue510
issue260
issue72
wip-fakessh-exit-status
v0.2.10rc1
v0.3.0rc1
v0.2.9
v0.2.8
v0.2.7
v0.2.6
v0.2.5
v0.2.4
v0.2.3
working-django
working
v0.2.0
v0.2.1
v0.2.10
v0.2.10-rc.0
v0.2.2
v0.3.0
v0.3.0-rc.0
v0.3.1
v0.3.10
v0.3.11
v0.3.12
v0.3.13
v0.3.14
v0.3.15
v0.3.16
v0.3.17
v0.3.18
v0.3.2
v0.3.3
v0.3.4
v0.3.5
v0.3.6
v0.3.7
v0.3.8
v0.3.9
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'fe54b0ac3f'
${ noResults }
Commit Graph

1042 Commits (fe54b0ac3fcd47af2a4242f6949b7de4d2674df6)

Author SHA1 Message Date
Alex Willmer 933477fcbe Begin 0.3.7dev 8 months ago
Alex Willmer 5d789faee5 Prepare 0.3.6 8 months ago
Alex Willmer d7979c3597 mitogen: Raise TypeError on `mitogen.utils.cast(custom_str)` failures 
If casting a string fails then raise a TypeError. This is potentially an API
breaking change; chosen as the lesser evil vs. allowing silent errors.

`cast()` relies on `bytes(obj)` & `str(obj)` returning the respective
supertype. That's no longer the case for `AnsibleUnsafeBytes` &
`AnsibleUnsafeText`; since fixes/mitigations for  CVE-2023-5764.

fixes #1046, refs #977

See also
- https://github.com/advisories/GHSA-7j69-qfc3-2fq9
- https://github.com/ansible/ansible/pull/82293
 8 months ago
Alex Willmer fc24b3f25e Start v0.3.6 development 8 months ago
Alex Willmer e97ab2f597 Prepare v0.3.5 8 months ago
Alex Willmer 5ad3d14ceb mitogen: Support PEP 451 ModuleSpec API, required for Python 3.12 
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4
under PEP 451. They replace the find_module() API of PEP 302, which was
deprecated from Python 3.4. They were removed in Python 3.12 along with the
imp module.

This change adds support for the PEP 451 APIs. Mitogen should no longer import
imp on Python versions that support ModuleSpec. Tests have been added to cover
the new APIs.

CI jobs have been added to cover Python 3.x on macOS.

Refs #1033
Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
 8 months ago
Alex Willmer 3a31a7d886 mitogen: Workaround CPython importlib PermissionError when cwd is unreadable 
On macOS when using a become plugin as an unprivileged user, to another
unprivileged user it is likely that the current working directory can't be
read. In this case os.cwd() raises PermissionError.

On versions of Python currently in the wild (March 2024, CPython <= 3.13) if
any non-builtin or non-frozen module (e.g. zlib, base64) is imported then
`importlib._bootstrap_external.PathFinder._path_importer_cache()` attempts to
call os.cwd() without catching PermissionError.

The previous comment about needing an extra .encode() appears to be wrong,
atleast for Python 3.x >= 3.6.

Command size increased by 54 bytes, bootstrap by 804 bytes. Changed from
codecs module to binascii & zlib because they're extensions, and importing
them triggers fewer supporting imports (e.g. encodings module).

Before

```
✗ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17078 (16.68KiB)

                              Original          Minimized           Compressed
mitogen.parent            97884 95.6KiB  50515 49.3KiB 51.6%  12727 12.4KiB
13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB
19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB
19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB
18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB
7.8%
mitogen.service           41699 40.7KiB  22477 22.0KiB 53.9%   5885  5.7KiB
14.1%
mitogen.fakessh           15577 15.2KiB   7989  7.8KiB 51.3%   2623  2.6KiB
16.8%
mitogen.master            51398 50.2KiB  25715 25.1KiB 50.0%   6886  6.7KiB
13.4%
```

After

```
✗ ./preamble_size.py
SSH command size: 759
Bootstrap (mitogen.core) size: 17882 (17.46KiB)

                              Original          Minimized           Compressed
mitogen.parent            98173 95.9KiB  50571 49.4KiB 51.5%  12747 12.4KiB
13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB
19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB
19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB
18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB
7.8%
mitogen.service           41699 40.7KiB  22477 22.0KiB 53.9%   5885  5.7KiB
14.1%
mitogen.fakessh           15577 15.2KiB   7989  7.8KiB 51.3%   2623  2.6KiB
16.8%
mitogen.master            56116 54.8KiB  29427 28.7KiB 52.4%   7627  7.4KiB
13.6%
```

Fixes #885
Refs https://github.com/python/cpython/issues/115911
 8 months ago
Alex Willmer e2f4d9275c tests: Fix ansible_python_interpreter & discovered_interpreter_python tests on macOS 
Should account for fiddling in mitogen.parent.Connection._first_stage() and
symlinks. I won't be surprised if it breaks again soon and often.
 9 months ago
Alex Willmer fc3e788cb4 non functional: Add comments about imp module removal in Python 3.12 10 months ago
Alex Willmer 63457b4866 docs: Update external URLs (e.g. dw/mitogen -> mitogen-hq/mitogen) 
Found with sphinx-build -b linkcheck. Not all flagged URLs have been changed,
e.g. Ansible plugins, deleted Github users.
 1 year ago
Alex Willmer 6aa4fd3573 docs: Fix generation of static website 
Bare minimum syntax errors and requirements constraints to work with Netlify
hosting.
 1 year ago
Alex Willmer 455fd2bcdf Bump version 1 year ago
Alex Willmer f18f5165cd Prep for v0.3.4 1 year ago
Alex Willmer ad4b686836 master.PkgutilMethod: Skip module loaders that raise ValueError 
e.g. in Ansible 6, ansible-core 2.13

```
[mux  2717] 23:39:11.342416 D mitogen: PkgutilMethod(): _AnsibleCollectionLoader(path=None).get_file_name('ansible.plugins') failed: ValueError('_AnsibleCollectionLoader(path=None) cannot find files for ansible.plugins, only ansible_collections.ansible.builtin.plugins')
```
 2 years ago
Alex Willmer d71fb672e8 Begin v0.3.4.dev0 3 years ago
Alex Willmer c0d3deeac5 Prepare v0.3.3 3 years ago
Alex Willmer e101cc4f44 mitogen.utils: Preserve docstring of functions decorated @with_router 
Co-authored-by: Rezart Qelibari <gast-kontakt+mitogen@astzweig.de>

Replaces #837
Fixes #836
 3 years ago
Alex Willmer 109feec6d5 Fix lints found by flake8 3 years ago
Alex Willmer 18c89de5a9 Remove unused module imports 3 years ago
Alex Willmer 65809a6f0f mitogen: Handle Python 3.10 threading depreactions 3 years ago
Alex Willmer d2ca8a9423 master.ParentEnumerationMethod: Require matching pkg.__name__ 
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>

When the requested module (e.g. ansible.module_utils.distro)
- is provided by another module *e.g. distro)
- that itself was a package (e.g. distro 1.7.0)

At runtime
- ansible/module_utils/distro/__init__.py executes
- if https://pypi.org/project/distro/ is present, it's loaded as
ansible.module_utils.distro
- otherwise ansible/module_utils/distro/_distro.py is loaded

ParentEnumerationMethod would wrongly use whatever was in
sys.modules['ansible.module_utils.distro]. Instead we should ascend to
the first parent that has fullname == sys.modules[fullname].__name__.
Then descend to the appropriate .py file on disk.

This bug didn't show up before because until distro 1.7.0 (Feb 2022) the
top-level distro module was a module (distro.py) not a package
(distro/__init__.py)

fixes #906
 3 years ago
Alex Willmer 47699e15aa master.SysModulesMethod: log rejection reasons 3 years ago
Alex Willmer 0fa0a93f55 master.PkgutilMethod: log rejection reasons 3 years ago
Alex Willmer 60c4ae5599
Add notes on imports and importlib 3 years ago
Alex Willmer 5b8f7dd1be
Start v0.3.3 development 3 years ago
Alex Willmer e8c3fe7881
Fix Trove classifier, bump version 
fixes #891

(cherry picked from commit 1a84184838)
 3 years ago
Alex Willmer 9a19f0ed16 mitogen.master: Fix stdlib discovery when sys.prefix contains symlinks 
This can be he case when Python has been installed with Homebrew.
 3 years ago
Alex Willmer 552819e765 mitogen.parent: Detect and avoid Python2.7 wrapper on macOS 11 & 12 
Without this errors such as the following occur
```
✗ MITOGEN_LOG_LEVEL=DEBUG python3 foo.py
Python: execv: (null): No such file or directory
Traceback (most recent call last):
  File "foo.py", line 16, in <module>
    target = router.local(python_path='/usr/bin/python2.7', debug=True)
  File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2486, in local
    return self.connect(u'local', **kwargs)
  File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2446, in connect
    return self._connect(klass, **mitogen.core.Kwargs(kwargs))
  File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2426, in _connect
    conn.connect(context=context)
  File "/Users/alex/src/mitogen2/mitogen/parent.py", line 1708, in connect
    raise self.exception
mitogen.parent.EofError: EOF on stream; last 100 lines received:
MITO000
MITO001
```

Before
```
$ ./preamble_size.py
SSH command size: 625
Bootstrap (mitogen.core) size: 17007 (16.61KiB)

                              Original          Minimized           Compressed
mitogen.parent            97496 95.2KiB  50355 49.2KiB 51.6%  12663 12.4KiB 13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB 19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41644 40.7KiB  22431 21.9KiB 53.9%   5886  5.7KiB 14.1%
mitogen.fakessh           15599 15.2KiB   8011  7.8KiB 51.4%   2624  2.6KiB 16.8%
mitogen.master            48732 47.6KiB  24569 24.0KiB 50.4%   6768  6.6KiB 13.9%
```

After
```
$ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17007 (16.61KiB)

                              Original          Minimized           Compressed
mitogen.parent            97885 95.6KiB  50516 49.3KiB 51.6%  12728 12.4KiB 13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB 19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41644 40.7KiB  22431 21.9KiB 53.9%   5886  5.7KiB 14.1%
mitogen.fakessh           15599 15.2KiB   8011  7.8KiB 51.4%   2624  2.6KiB 16.8%
mitogen.master            48733 47.6KiB  24570 24.0KiB 50.4%   6771  6.6KiB 13.9%
```
 3 years ago
Alex Willmer e6bc53ca3a mitogen.parent: Remove tabs from minimized first stage 
Saves 4 bytes

Before
```
$ python preamble_size.py
SSH command size: 629
Bootstrap (mitogen.core) size: 17007 (16.61KiB)

                              Original          Minimized           Compressed
mitogen.parent            97497 95.2KiB  50356 49.2KiB 51.6%  12665 12.4KiB 13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB 19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41644 40.7KiB  22431 21.9KiB 53.9%   5886  5.7KiB 14.1%
mitogen.fakessh           15599 15.2KiB   8011  7.8KiB 51.4%   2624  2.6KiB 16.8%
mitogen.master            48732 47.6KiB  24569 24.0KiB 50.4%   6768  6.6KiB 13.9%
```

After
```
$ python preamble_size.py
SSH command size: 625
Bootstrap (mitogen.core) size: 17007 (16.61KiB)

                              Original          Minimized           Compressed
mitogen.parent            97496 95.2KiB  50355 49.2KiB 51.6%  12663 12.4KiB 13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB 19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41644 40.7KiB  22431 21.9KiB 53.9%   5886  5.7KiB 14.1%
mitogen.fakessh           15599 15.2KiB   8011  7.8KiB 51.4%   2624  2.6KiB 16.8%
mitogen.master            48732 47.6KiB  24569 24.0KiB 50.4%   6768  6.6KiB 13.9%
```
 3 years ago
Alex Willmer dd6d73db37 mitogen.parent: Eliminate use of platform module in first stage 
This reduces the size of the initial SSH command by 204 bytes, & may fix errors
running Mitogen on  macOS. AFAICT platform was used but not imported.

Before
```
$ python ./preamble_size.py
SSH command size: 833
Bootstrap (mitogen.core) size: 17007 (16.61KiB)

                              Original          Minimized           Compressed
mitogen.parent            97565 95.3KiB  50427 49.2KiB 51.7%  12689 12.4KiB 13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB 19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41644 40.7KiB  22431 21.9KiB 53.9%   5886  5.7KiB 14.1%
mitogen.fakessh           15599 15.2KiB   8011  7.8KiB 51.4%   2624  2.6KiB 16.8%
mitogen.master            48732 47.6KiB  24569 24.0KiB 50.4%   6768  6.6KiB 13.9%
```

After
```
$ python preamble_size.py
SSH command size: 629
Bootstrap (mitogen.core) size: 17007 (16.61KiB)

                              Original          Minimized           Compressed
mitogen.parent            97543 95.3KiB  50357 49.2KiB 51.6%  12665 12.4KiB 13.0%
mitogen.fork               8436  8.2KiB   4130  4.0KiB 49.0%   1648  1.6KiB 19.5%
mitogen.ssh               10892 10.6KiB   6952  6.8KiB 63.8%   2113  2.1KiB 19.4%
mitogen.sudo              12089 11.8KiB   5924  5.8KiB 49.0%   2249  2.2KiB 18.6%
mitogen.select            12325 12.0KiB   2929  2.9KiB 23.8%    964  0.9KiB 7.8%
mitogen.service           41644 40.7KiB  22431 21.9KiB 53.9%   5886  5.7KiB 14.1%
mitogen.fakessh           15599 15.2KiB   8011  7.8KiB 51.4%   2624  2.6KiB 16.8%
mitogen.master            48732 47.6KiB  24569 24.0KiB 50.4%   6768  6.6KiB 13.9%
```
 3 years ago
Denis Zalevskiy 84c567e265
Add podman connection support 
Shameless copy of buildah connection with modifications of invocation to
fit podman CLI.

Signed-off-by: Denis Zalevskiy <dez@aiven.io>
 3 years ago
Alex Willmer da536e8ae1 Fix stdlib typos that would cause NameError or AttributeError exceptions 3 years ago
Alex Willmer da3772cbff Increment version to 0.3.1.dev0 3 years ago
Alex Willmer 9b1617f26f Bump version to 0.3.0rc1 4 years ago
Alex Willmer fd8c2d3702
Merge branch 'master' into master 4 years ago
Alex Willmer bce3bab3e8 Add the msvcrt moduleto the default module deny list 
Commit https://github.com/python/cpython/commit/880d42a3b247 (first
released in Python 3.8a0) moved an import of msvcrt from an if <win32>
block, into a try/except block. So now the import is tried even on Linux
or MacOS.

https://docs.python.org/3/library/msvcrt.html is a Windows specific
builtin.
 4 years ago
Alex Willmer ceb0a94467 Explain why the Python 2.x thread module is blacklisted 
Based on the original commit I believe it is only an optimization.
However I could be wrong. I intend to request review of this part.
 4 years ago
Michael Weigle 6b5a46ccd0
Fixes RuntimeWarning for improper bufferring on local connection 4 years ago
Steven Robertson 4657979210 adding clarifying comments 4 years ago
Steven Robertson f886293a6a Merge branch 'master' into collectionsSupport 4 years ago
Alex Willmer b0ce29dcfd ssh: Match newer ssh host key prompt that accepts the fingerprint 
This fixes an ERROR in test_accept_enforce_host_keys() while running the
test suite.

Fixes #756
 4 years ago
Steven Robertson 518324c371 fix regression in Darwin 19 (OSX 10.15+) ansible python interpreter detection 4 years ago
Steven Robertson 33e176d62e add support for ansible_collections site-package (from pip ansible==2.10.0 install) + switch to ansible 2.10.0 rather than github tag 4 years ago
Steven Robertson 1ffb319300 remove debugging 4 years ago
Steven Robertson 207e36194d try and get some visibility into test failures 4 years ago
Steven Robertson 5b3d90dac4 see if sys.path is being loaded properly on azure 4 years ago
Steven Robertson 4dea09a924 check sys.path issue 4 years ago
Steven Robertson ff8a276186 turn off failing Ansible-only tests for now, also raising errors to see what Azure is gonna do with collections 4 years ago
Steven Robertson 02aedcdacd fix issue with collections module_utils raised in alikins example 4 years ago
Steven Robertson 19f92a2435 fix inefficient basepath check 4 years ago