Commit Graph

1012 Commits (21e874e60c5cc8fb8c6a0532a98fcc447c1ae2ab)

Author SHA1 Message Date
Alex Willmer 123efa7510 mitogen: Support Python 3.12
Most of the necessary changes were made in recent PEP 451 commits. This bumps
the CI jobs, and declares the support. Test dependendancies are bumped to
latest supportted/available versions.

refs #1033
8 months ago
Alex Willmer 92c00d913e tests: Skip "discovered python matches invoked" on macOS 11/Python 2.7/Vanilla 8 months ago
Alex Willmer 5ad3d14ceb mitogen: Support PEP 451 ModuleSpec API, required for Python 3.12
importlib.machinery.ModuleSpec and find_spec() were introduced in Python 3.4
under PEP 451. They replace the find_module() API of PEP 302, which was
deprecated from Python 3.4. They were removed in Python 3.12 along with the
imp module.

This change adds support for the PEP 451 APIs. Mitogen should no longer import
imp on Python versions that support ModuleSpec. Tests have been added to cover
the new APIs.

CI jobs have been added to cover Python 3.x on macOS.

Refs #1033
Co-authored-by: Witold Baryluk <witold.baryluk@gmail.com>
8 months ago
Alex Willmer 1031551dd9 tests: Clarify transport config tests optimisation & correct value
The ini inventory parser doesn't support comments after a value, so the value
parsed was "python3000  # Not expected to exist".
9 months ago
Alex Willmer 2973d90670 tests: Enable su tests under vanilla Ansible >= 2.11
cwd_show was useful when debugging these tests, worth keeping around.
9 months ago
Alex Willmer e2f4d9275c tests: Fix ansible_python_interpreter & discovered_interpreter_python tests on macOS
Should account for fiddling in mitogen.parent.Connection._first_stage() and
symlinks. I won't be surprised if it breaks again soon and often.
9 months ago
Alex Willmer c2ad52e54e tests: Fix tests using get_url across Python versions
Using https:// requires certificate store management and additional parameter
passing that changed across Ansible and Python versions. Using http:// allows
the same tests to be used across wider spans of Python version on the
controller, and Python verison on the targets.

Python 3.12 on a target + get_uri needs Ansible >= 8 (ansible-core >= 2.15).
Python 3.12 removed deprecated httplib.HTTPSConnection() arguments.
https://github.com/ansible/ansible/pull/80751
9 months ago
Alex Willmer a6a5c5bb97 tests: Clarify status/purpose of Python 2.x era Ansible Module workaround 9 months ago
Alex Willmer 2839954559 tests: Account for /tmp symlink in virtualenv test on macOS 9 months ago
Alex Willmer adfd4e17f3 tests: Declare inventory file types to Visual Studio Code and Vim
Works with the VS Code modeline extension. Enables syntax highlighting.
9 months ago
Alex Willmer 591152bef0 tests: Avoid intermittant 2 hour timeout in new style Ansible module tests
This has been lurking for years, raising it's head at unpredictable times.
This change doesn't fix it, but it should make it a lot less mysterious.
9 months ago
Alex Willmer a6c89751f9 tests: Cleanup ansible-lint errors & warnings in user creation playbook
Task " Install slow profile for one account" removed because it duplicates
earlier work.
9 months ago
Alex Willmer 8b574f234d tests: Report Ansible controller parameters before image prep & user creation 9 months ago
Alex Willmer bde7f062b9 tests: Fix Ansible module shebangs
With https://github.com/ansible/ansible/pull/76677 Ansible
fixed shebang substitution for Ansible modules and tightened
up what shebang is allowed.

Changing these fixes the tests using them with vanilla Ansible.

https://docs.ansible.com/ansible/latest/dev_guide/testing/sanity/shebang.html
9 months ago
Nerijus Baliūnas 4089e875a9 Add Python 3.11 support
Co-authored-by: Alex Willmer <alex@moreati.org.uk>
1 year ago
Alex Willmer 49c54386b3 tests: Only use subprocess32 package on Python 2.x
This is how the package documentation recommends and it's less likely to
interfere with new features in stdlib subprocess module.
1 year ago
Alex Willmer 98d110ed16 tests: Bump Django ModuleFinder test cases
Preperation for Python 3.11 support
1 year ago
Alex Willmer 6258365df6 tests: Handle square bracket IPv6 in `docker port` output
Fixes
```
======================================================================
ERROR: setUpClass (ssh_test.BannerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vsts/work/1/s/tests/testlib.py", line 625, in setUpClass
    cls.dockerized_ssh = DockerizedSshDaemon(**daemon_args)
  File "/home/vsts/work/1/s/tests/testlib.py", line 553, in __init__
    self.start_container()
  File "/home/vsts/work/1/s/tests/testlib.py", line 533, in start_container
    self._get_container_port()
  File "/home/vsts/work/1/s/tests/testlib.py", line 510, in _get_container_port
    self.port = int(bport)
ValueError: invalid literal for int() with base 10: ':]:32770'
```
1 year ago
Alex Willmer 270c3a25de tests: Support Ubuntu 22.04 as test suite runner (controller)
To do so the test suite allows a weak cryptographic alogorithm (SHA1) to be
used, principally for CentOS 6 targets. This can be removed if/when support
for older (legacy) targets is dropped.

Only the test suite enables this known weak alogorithm. Mitogen as-shipped
doesn't enable or disable algorithms.
1 year ago
Alex Willmer 19b79f7ab5 CI: Fix tests on Linux, Ansible tests targetting Debian 9 & 11
Without Ubuntu 20.04 virtualenv package being installed pip was installing a
version of virtualenv that couldn't create the Tox environment for Python 2.7.

> Successfully installed distlib-0.3.6 filelock-3.12.2 platformdirs-3.8.0
> pluggy-1.2.0 py-1.11.0 tomli-2.0.1 tox-3.28.0 virtualenv-20.23.1
> Finishing: Install tooling
> ...
> py27-mode_mitogen-distro_centos6 create: /home/vsts/work/1/s/.tox/py27-
> mode_mitogen-distro_centos6
> ERROR: invocation failed (exit code 1), logfile: /home/vsts/work/1/s/.tox/
> py27-mode_mitogen-distro_centos6/log/py27-mode_mitogen-distro_centos6-0.log
> ================================== log start
> ===================================
> RuntimeError: failed to query /usr/bin/python2.7 with code 1 err:
> '  File "/home/vsts/.local/lib/python3.8/site-packages/virtualenv/discovery/
> py_info.py", line 24\n    return list(OrderedDict.fromkeys(["",
> *os.environ.get("PATHEXT", "").lower().split(os.pathsep)]))\n
> ^\nSyntaxError: invalid syntax\n'
1 year ago
Alex Willmer 21cb4a3472 CI: Remove faulthandler fallback requirement
faulthandler is a stdlib module in Python 3.3+. For a long time a PyPI package
of the same name was available for earlier Python releases. That package has
since been removed from PyPI, and the source respoitory archived. So we should
not rely on it.
fixes #983 refs #970
2 years ago
Alex Willmer 1871f2a9b1 Remove vendored mitogen.compat.simplejson
Python 2.6 added json to the stdlib. We no longer support Python <= 2.7 in
Mitogen 0.3.x, so this fallback is unneeded complexity. Fixes #659
2 years ago
Alex Willmer 03acf40315 tests: Speed up transport config tests by avoiding interpreter discovery
Reduced execution time of tests/ansible/integration/transport_config/all.yml
from 11 minutes to 49 seconds.
2 years ago
Alex Willmer a3a10cb32e tests: Upgrade coverage dependency 2 years ago
Alex Willmer 7d79c56cb6 tests: Clarify skipped Poller test reasons 2 years ago
Alex Willmer edd2868ef6 tests: Don't rely on facts when setting become
They won't be available if the play is first, and hence no facts have been
gather in previous play(s), e.g. due to --start-at-task
2 years ago
Alex Willmer 1ed932e8d5 tests: Eliminate MITOGEN_INVENTORY_FILE
Replaced with ansible_inventory_sources.
2 years ago
Alex Willmer 900760e913 tests: Increase Ansible timeout to reduce false positives
Was failing on my dsktop PC, with a spinning rust HDD
2 years ago
Alex Willmer 526422b74b tests: Name tasks
For easier grep, and easier identification in task_profiler summaries.
2 years ago
Alex Willmer 2e8bf73877 tests: Print filename of a failed task (Ansible >= 2.11) 2 years ago
Alex Willmer 99fe9d48e6 tests: Print task durations 2 years ago
Alex Willmer 39dfd2dfe8 ci: Upgrade VM Images to macOS 11 and Ubuntu 20.04 2 years ago
Alex Willmer f1503874de ansible_mitogen: Correct ansible_become_pass/ansible_become_password precendence
Until Ansible 2.9 it looks like ansible_become_password had higher priority.
From Ansible 2.10 ansible_become_pass has higher priority [1]. Mitogen was not
respecting this.

I may need to rework this further, instatiating the become plugin may have
slowed down execution.

[1] Based on testing with

```
[ubuntus]
become-pass-pass ansible_become_pass=1234
become-pass-password ansible_become_password=1234
become-pass-both ansible_become_password=wrong ansible_become_pass=1234

[ubuntus:vars]
ansible_host=ubuntu2004.local
ansible_user=ubuntu
```
```
- hosts: ubuntus
  gather_facts: false
  become: true
  tasks:
    - ping:
```
2 years ago
Alex Willmer e36bbde9ac tests: Replace uses of assertTrue() with specific methods 2 years ago
Alex Willmer eb4a7e0ad5 tests: cleanup subprocess file handles in create_child_test 2 years ago
Alex Willmer 64819ecb5f tests: Regression test for #776 (package/yum/dnf module called twice) 2 years ago
Alex Willmer 24c845379a tests: Remove redundant regression tags
The tag is applied by the playbook that imports this one.
2 years ago
Alex Willmer db0ffae352 tests: Enable stricter error handling, fix resulting failures 2 years ago
Alex Willmer c32577295a tests: Check and/or suppress stderr of subprocesses, reduce shell=True uses 2 years ago
Alex Willmer 216e7c9150 tests: Correct Ansible targets 2 years ago
Alex Willmer 8e79488768 tests: Mark or avoid sudo tasks on localhost 2 years ago
Alex Willmer f070767dad tests: Use meaningful play names 2 years ago
Felix Stupp b1e67cc7df tests/ansible/README: Replace reference with actual link
- working for GitHub and similar Markdown engines
3 years ago
Alex Willmer 25ea6dde02 ansible_mitogen: Allow mitogen_fetch to bypass slurp module
This reapplies an earlier change, when this plugin was first introduced to
Mitogen. The plugin was updated to fix

[DEPRECATION WARNING]: The '_remote_checksum()' method is deprecated.

I've elected to short-circuit the if statemtn logic, rather than
deleting/unindenting, to make the code delta much smaller. This should make it
easier to maintain/update.

Fixes #915
3 years ago
Alex Willmer e101cc4f44 mitogen.utils: Preserve docstring of functions decorated @with_router
Co-authored-by: Rezart Qelibari <gast-kontakt+mitogen@astzweig.de>

Replaces #837
Fixes #836
3 years ago
Alex Willmer 3dbb0e28ce tests: List leaked file descriptors 3 years ago
Alex Willmer 18c89de5a9 Remove unused module imports 3 years ago
Alex Willmer db9e52ce8e tests: Run containers on macOS with podman, instead of Docker 3 years ago
Alex Willmer 0417d4d73a Replace os.system() with subprocess.check_call()
Non-zero return codes should raise an exception, not pass silently.
3 years ago
Alex Willmer 1287d58a54 Use with open(): ... to ensure file objects get closed 3 years ago