Alex Willmer
d2ca8a9423
master.ParentEnumerationMethod: Require matching pkg.__name__
...
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
When the requested module (e.g. ansible.module_utils.distro)
- is provided by another module *e.g. distro)
- that itself was a package (e.g. distro 1.7.0)
At runtime
- ansible/module_utils/distro/__init__.py executes
- if https://pypi.org/project/distro/ is present, it's loaded as
ansible.module_utils.distro
- otherwise ansible/module_utils/distro/_distro.py is loaded
ParentEnumerationMethod would wrongly use whatever was in
sys.modules['ansible.module_utils.distro]. Instead we should ascend to
the first parent that has fullname == sys.modules[fullname].__name__.
Then descend to the appropriate .py file on disk.
This bug didn't show up before because until distro 1.7.0 (Feb 2022) the
top-level distro module was a module (distro.py) not a package
(distro/__init__.py)
fixes #906
2 years ago
Alex Willmer
47699e15aa
master.SysModulesMethod: log rejection reasons
2 years ago
Alex Willmer
0fa0a93f55
master.PkgutilMethod: log rejection reasons
2 years ago
Alex Willmer
60c4ae5599
Add notes on imports and importlib
2 years ago
Alex Willmer
5b8f7dd1be
Start v0.3.3 development
2 years ago
Alex Willmer
e8c3fe7881
Fix Trove classifier, bump version
...
fixes #891
(cherry picked from commit 1a84184838
)
2 years ago
Alex Willmer
9a19f0ed16
mitogen.master: Fix stdlib discovery when sys.prefix contains symlinks
...
This can be he case when Python has been installed with Homebrew.
3 years ago
Alex Willmer
552819e765
mitogen.parent: Detect and avoid Python2.7 wrapper on macOS 11 & 12
...
Without this errors such as the following occur
```
✗ MITOGEN_LOG_LEVEL=DEBUG python3 foo.py
Python: execv: (null): No such file or directory
Traceback (most recent call last):
File "foo.py", line 16, in <module>
target = router.local(python_path='/usr/bin/python2.7', debug=True)
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2486, in local
return self.connect(u'local', **kwargs)
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2446, in connect
return self._connect(klass, **mitogen.core.Kwargs(kwargs))
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 2426, in _connect
conn.connect(context=context)
File "/Users/alex/src/mitogen2/mitogen/parent.py", line 1708, in connect
raise self.exception
mitogen.parent.EofError: EOF on stream; last 100 lines received:
MITO000
MITO001
```
Before
```
$ ./preamble_size.py
SSH command size: 625
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97496 95.2KiB 50355 49.2KiB 51.6% 12663 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
After
```
$ ./preamble_size.py
SSH command size: 705
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97885 95.6KiB 50516 49.3KiB 51.6% 12728 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48733 47.6KiB 24570 24.0KiB 50.4% 6771 6.6KiB 13.9%
```
3 years ago
Alex Willmer
e6bc53ca3a
mitogen.parent: Remove tabs from minimized first stage
...
Saves 4 bytes
Before
```
$ python preamble_size.py
SSH command size: 629
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97497 95.2KiB 50356 49.2KiB 51.6% 12665 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
After
```
$ python preamble_size.py
SSH command size: 625
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97496 95.2KiB 50355 49.2KiB 51.6% 12663 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
3 years ago
Alex Willmer
dd6d73db37
mitogen.parent: Eliminate use of platform module in first stage
...
This reduces the size of the initial SSH command by 204 bytes, & may fix errors
running Mitogen on macOS. AFAICT platform was used but not imported.
Before
```
$ python ./preamble_size.py
SSH command size: 833
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97565 95.3KiB 50427 49.2KiB 51.7% 12689 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
After
```
$ python preamble_size.py
SSH command size: 629
Bootstrap (mitogen.core) size: 17007 (16.61KiB)
Original Minimized Compressed
mitogen.parent 97543 95.3KiB 50357 49.2KiB 51.6% 12665 12.4KiB 13.0%
mitogen.fork 8436 8.2KiB 4130 4.0KiB 49.0% 1648 1.6KiB 19.5%
mitogen.ssh 10892 10.6KiB 6952 6.8KiB 63.8% 2113 2.1KiB 19.4%
mitogen.sudo 12089 11.8KiB 5924 5.8KiB 49.0% 2249 2.2KiB 18.6%
mitogen.select 12325 12.0KiB 2929 2.9KiB 23.8% 964 0.9KiB 7.8%
mitogen.service 41644 40.7KiB 22431 21.9KiB 53.9% 5886 5.7KiB 14.1%
mitogen.fakessh 15599 15.2KiB 8011 7.8KiB 51.4% 2624 2.6KiB 16.8%
mitogen.master 48732 47.6KiB 24569 24.0KiB 50.4% 6768 6.6KiB 13.9%
```
3 years ago
Denis Zalevskiy
84c567e265
Add podman connection support
...
Shameless copy of buildah connection with modifications of invocation to
fit podman CLI.
Signed-off-by: Denis Zalevskiy <dez@aiven.io>
3 years ago
Alex Willmer
da536e8ae1
Fix stdlib typos that would cause NameError or AttributeError exceptions
3 years ago
Alex Willmer
da3772cbff
Increment version to 0.3.1.dev0
3 years ago
Alex Willmer
9b1617f26f
Bump version to 0.3.0rc1
3 years ago
Alex Willmer
fd8c2d3702
Merge branch 'master' into master
3 years ago
Alex Willmer
bce3bab3e8
Add the msvcrt moduleto the default module deny list
...
Commit https://github.com/python/cpython/commit/880d42a3b247 (first
released in Python 3.8a0) moved an import of msvcrt from an if <win32>
block, into a try/except block. So now the import is tried even on Linux
or MacOS.
https://docs.python.org/3/library/msvcrt.html is a Windows specific
builtin.
3 years ago
Alex Willmer
ceb0a94467
Explain why the Python 2.x thread module is blacklisted
...
Based on the original commit I believe it is only an optimization.
However I could be wrong. I intend to request review of this part.
3 years ago
Michael Weigle
6b5a46ccd0
Fixes RuntimeWarning for improper bufferring on local connection
3 years ago
Steven Robertson
4657979210
adding clarifying comments
4 years ago
Steven Robertson
f886293a6a
Merge branch 'master' into collectionsSupport
4 years ago
Alex Willmer
b0ce29dcfd
ssh: Match newer ssh host key prompt that accepts the fingerprint
...
This fixes an ERROR in test_accept_enforce_host_keys() while running the
test suite.
Fixes #756
4 years ago
Steven Robertson
518324c371
fix regression in Darwin 19 (OSX 10.15+) ansible python interpreter detection
4 years ago
Steven Robertson
33e176d62e
add support for ansible_collections site-package (from pip ansible==2.10.0 install) + switch to ansible 2.10.0 rather than github tag
4 years ago
Steven Robertson
1ffb319300
remove debugging
4 years ago
Steven Robertson
207e36194d
try and get some visibility into test failures
4 years ago
Steven Robertson
5b3d90dac4
see if sys.path is being loaded properly on azure
4 years ago
Steven Robertson
4dea09a924
check sys.path issue
4 years ago
Steven Robertson
ff8a276186
turn off failing Ansible-only tests for now, also raising errors to see what Azure is gonna do with collections
4 years ago
Steven Robertson
02aedcdacd
fix issue with collections module_utils raised in alikins example
4 years ago
Steven Robertson
19f92a2435
fix inefficient basepath check
4 years ago
Steven Robertson
5215646c8a
code cleanup
4 years ago
Steven Robertson
0b421e0d3c
able to run docker_container installed via 'ansible-galaxy collection install community.general'
4 years ago
Steven Robertson
45797a0d34
able to send collections paths to master with very little change to core Mitogen, need to ensure imports work properly now though
4 years ago
Steven Robertson
583f540889
added comments
4 years ago
Steven Robertson
e34cf8667f
Merge branch 'master' into collectionsSupport
4 years ago
Steven Robertson
81076c9da8
fixes setup module relative import fail on some pythons
4 years ago
Steven Robertson
376d8d0fab
remove old hacks; ansible_collections is available at time of invoker but not later
4 years ago
Steven Robertson
1ef96b5827
normal search-jump-around-code-easily word exists in the code already so switching to jjj
4 years ago
Steven Robertson
afd8902d6b
hackily sorta got imp.find_module to load module_utils but it breaks further package discovery, needs fix
4 years ago
Steven Robertson
955e77c5db
WIP: able to load subdirs but now need to treat them as submodules properly
4 years ago
Steven Robertson
4b42dd1434
able to load things from sys.modules but not everything loads that way
4 years ago
Steven Robertson
3dbfe085ab
close-ish, getting 'ansible_collections has no submodule alikins' though because of empty submodules
4 years ago
Steven Robertson
acde13f9d6
handles a 'wait_for_connection' call right after a task caused a shutdown
4 years ago
Steven Robertson
05f74e3b30
since correctly detecting python3 now, makes 'def disconnect_all()' python3-safe
4 years ago
Steven Robertson
e728cee7cb
missed a linting spot
4 years ago
Steven Robertson
d64adb15d7
reverted autolinted code
4 years ago
Steven Robertson
99d0b08e2f
Merge branch 'master' into complexAnsiblePythonInterpreterArg
4 years ago
David Wilson
91f74a04ac
issue #667 : fix inverted test for Waker write() failure.
5 years ago
Steven Robertson
e8f3154cab
Merge branch 'master' into complexAnsiblePythonInterpreterArg
5 years ago
David Wilson
153d79b878
Bump version for release.
5 years ago