Commit Graph

14 Commits (b8d3f86b12cc67dc3be3b6703333096e4915131c)

Author SHA1 Message Date
Alex Willmer 0e5f47f145 mitogen: Refactor scan_code_imports() as mitogen.imports.codeobj_imports()
This replaces `mitogen.master.scan_code_imports()` with
`mitogen.imports.codeobj_imports()`. The Python 3.x implementation now uses
`str.find()`, relying on Python >= 3.6 "widecode" format. Behaviour and
semantics should be unchanged. Now implementations are approx
- 1.5 x faster on Python 2.x
- 2 - 3 x faster on Python 3.x

Before
```console
$ ./tests/bench/scan_code
scan_code_imports python2.7  100 loops, best of 3: 3.19 msec per loop
scan_code_imports python3.9  500 loops, best of 5: 685 usec per loop
scan_code_imports python3.10  500 loops, best of 5: 727 usec per loop
scan_code_imports python3.11  500 loops, best of 5: 601 usec per loop
scan_code_imports python3.12  500 loops, best of 5: 609 usec per loop
scan_code_imports python3.13  500 loops, best of 5: 586 usec per loop
```

After
```console
codeobj_imports python2.7  1000 loops, best of 3: 1.98 msec per loop
codeobj_imports python3.9  1000 loops, best of 5: 302 usec per loop
codeobj_imports python3.10  1000 loops, best of 5: 297 usec per loop
codeobj_imports python3.11  1000 loops, best of 5: 243 usec per loop
codeobj_imports python3.12  1000 loops, best of 5: 278 usec per loop
codeobj_imports python3.13  1000 loops, best of 5: 259 usec per loop
```
```console
$ uname -a
Darwin kintha 24.6.0 Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:29 PDT
2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6000 arm64
```
4 months ago
Alex Willmer 5ad3d14ceb mitogen: Support PEP 451 ModuleSpec API, required for Python 3.12
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4
under PEP 451. They replace the find_module() API of PEP 302, which was
deprecated from Python 3.4. They were removed in Python 3.12 along with the
imp module.

This change adds support for the PEP 451 APIs. Mitogen should no longer import
imp on Python versions that support ModuleSpec. Tests have been added to cover
the new APIs.

CI jobs have been added to cover Python 3.x on macOS.

Refs #1033
Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
2 years ago
Alex Willmer fc3e788cb4 non functional: Add comments about imp module removal in Python 3.12 2 years ago
Alex Willmer 31b3a4eb4a ansible_mitogen: Standardise __future__ imports to match Ansible
Some modules additionally enable unicode_literals (which Ansible doesn't do).
I've chosen not to change that, for now.
4 years ago
David Wilson 402dba4197 module_finder: pass raw file to compile()
Newer Ansibles have e.g. UTF-8 present in apt.py.
6 years ago
David Wilson 1f77d24bec Update copyright year everywhere. 7 years ago
David Wilson 3b7a1d4f36 ansible: fix 3.x dependency scanning on OS X
On OS X with case-insensitive filenames, resolving
'ansible.module_utils.facts.base.Hardware' finds
'ansible.module_utils.facts.hardware/__init__.py', because
module_finder's procedure is completely wrong for resolving child
modules. Patch over it for now since it otherwise works for Ansible.
8 years ago
David Wilson 410016ff47 Initial Python 3.x port work.
* ansible: use unicode_literals everywhere since it only needs to be
  compatible back to 2.6.
* compat/collections.py: delete this entirely and rip out the parts of
  functools that require it.
* Introduce serializable Kwargs dict subclass that translates keys to
  Unicode on instantiation.
* enable_debug_logging() must set _v/_vv globals.
* cStringIO does not exist in 3.x.
* Treat IOLogger and LogForwarder input as latin-1.
* Avoid ResourceWarnings in first stage by explicitly closing fps.
* Fix preamble_size.py syntax errors.
8 years ago
David Wilson e35694acd5 ansible: flake8 fixes. 8 years ago
David Wilson 088a7e5cff ansible: handle "from timeout import timeout" imports.
It's not simple without executing a module to determine whether the
above refers to a submodule of a package, or an object defined within a
module.

Therefore detect when resolution of a child module yields the same path
as the parent, and ignore the result.
8 years ago
David Wilson bb61745a1a issue #217: pass through non-custom module utils to regular importer.
This may come back to bite later, but in the meantime it avoids shipping
up to 12KiB of junk metadata for every single task invocation.

For detachment (aka. async), we must ensure the target has two types of
preloads completed (modules and module_utils files) before detaching.
8 years ago
David Wilson 30034877a5 issue #217: ansible: working, if extremely inefficient implementation 8 years ago
David Wilson 0f8190eff6 issue #217: ansible: working module_finder.py 8 years ago
David Wilson f96c552f87 issue #217: initial module scanner code.
This is sketch code, it's being done separately from mitogen.master.* to
begin with to avoid breaking what's there.
8 years ago