Commit Graph

3560 Commits (ce6297b0e95d4e155a265cb5365d230f7bab53aa)
 

Author SHA1 Message Date
Alex Willmer ce6297b0e9 Begin v0.3.11 1 month ago
Alex Willmer cea2e7b98d Prepare v0.3.10 1 month ago
Gaige B Paulsen 2ba1b2b3f8
Fix: termios.error: (22, 'Invalid argument') during `become` on Solaris/Illumos/SmartOS (#1089)
This fixes compatibility with Solaris/Illumos/SmartOS, addressing an issue that shows up most frequently with become. The issue was mostly due to differences in how the TTY driver is handled and the pty driver not supporting echo on both sides of the pipe (as designed, from a Solaris point of view).

Fixes #950

Co-authored-by: Alex Willmer <alex@moreati.org.uk>
1 month ago
Alex Willmer b8b15580af
Merge pull request #1119 from moreati/ci-resourcewarnings
CI: Reliability, eliminate a race condition and some resource leaks
2 months ago
Alex Willmer d032c591c2 tests: Retry container process check during teardown
I'm about 75% sure the check is an unavoidable race condition, see
https://github.com/mitogen-hq/mitogen/issues/694#issuecomment-2338001694. If
it occurs again, then reopen the issue.

Fixes #694
2 months ago
Alex Willmer 315204271e tests: Don't suppress output while testing unix Listener
It's not noisy, and it has been hiding an error I wasn't aware of.
2 months ago
Alex Willmer a3192d2beb mitogen: close mitogen.unix.Listener socket in error conditions
To avoid ResourceWarning
2 months ago
Alex Willmer 598de81143 mitogen: Fix subprocess ResourceWarning
Python 3.x emits `ResourceWarning`s if certains resources aren't correctly
closed. Due to the way Mitogen has been terminating child processes this has
been occurring.

```
test_dev_tty_open_succeeds
(create_child_test.TtyCreateChildTest.test_dev_tty_open_succeeds) ...
/opt/hostedtoolcache/Python/3.12.5/x64/lib/python3.12/subprocess.py:1127:
ResourceWarning: subprocess 3313 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok
```

During garbage collection subprocess.Popen() objects emit
ResourceWarning("subprocess 123 is still running")
if proc.returncode hasn't been set. Typically calling proc.wait() does so,
once the sub-process has exited. Calling os.waitpid(proc.pid, 0) also waits
for the sub-process to exit, but it doesn't update proc.returncode, so the
ResourceWarning is still emitted.

This change exposes `subprocess.Popen` methods on
`mitogen.parent.PopenProcess`, so that the returncode can be set.

See https://gist.github.com/moreati/b8d157ff82cb15234bece4033accc5e5
2 months ago
Alex Willmer 7238403392 tests: Add missing logging import 2 months ago
Alex Willmer 7c92b8ef2b tests: Shutdown contexts on completion
This should terminate any child processes and connections.
2 months ago
Alex Willmer 7d9eebdb9a tests: Close file object in six_brokenpkg 2 months ago
Alex Willmer 16c602aaa2
Merge pull request #1117 from moreati/macos-no-py311
ci: Drop macOS Python 3.12 + Ansible 9 tests
2 months ago
Alex Willmer 4b4bfdc0f3 ci: Drop macOS Python 3.12 + Ansible 9 tests
They were meant to be replaced by Python 3.12 + ANsible 10, not supplemented.
2 months ago
Alex Willmer a8c0a414b8
Merge pull request #1071 from moreati/ansible-unpin
tests: Unpin versions of Ansible 2.10, 3, & 4
2 months ago
Alex Willmer f76ccbf8ed tests: Unpin versions of Ansible 2.10, 3, & 4 2 months ago
Alex Willmer fc7b7eaba1
Merge pull request #1115 from moreati/pr956
Initial support for templated `ansible_ssh_args`,   `ansible_ssh_common_args`, and `ansible_ssh_extra_args`
2 months ago
Alex Willmer 79ed797bad tests: Test templating of ansible_ssh_common_args et al
refs #905
2 months ago
Alex Willmer f3097b5743 ci: Template Ansible test-targets inventory with Jinja2 2 months ago
root be288ad398 patch #509 : ansible_ssh_common_args issues 2 months ago
Alex Willmer 979241e171
Merge pull request #1113 from moreati/ssh-password-tests
tests: Simplify Ansible ssh password tests, test priority
2 months ago
Alex Willmer 46c9f772d8 tests: Simplify Ansible ssh password tests, test priority
This
- Removes the indirection of calling ansible in a sub-shell
- Includes vanilla Ansible, which was previously skipped
- Tests whether ansible_ssh_pass overrides ansible_password, as it should

As a one off I've the new tests against vanilla Ansible 2.10 through Ansible
10, to confirm the baseline priorities have remained unchanged all releases
currently supported by Mitogen 0.3.x.
2 months ago
Alex Willmer 289a78040d
Merge pull request #1110 from bbc/unsafe-large-copy
Fix AnsibleUnsafeText when copying files larger than SMALL_FILE_LIMIT
2 months ago
Alex Willmer 5af6534a70 tests: Test AnsibleUnsafeText when copying files larger SMALL_FILE_LIMIT
The bug was fixed in a previous commit by Jonathan Rosser. This adds testing.
The bug is only triggered when the copy module is used inside a `with_items:`
loop and the destination filename has an extension. A `loop:` loop is not
sufficient.

refs #1110
2 months ago
Jonathan Rosser 0bd3c6cba5 Fix AnsibleUnsafeText when copying files larger than SMALL_FILE_LIMIT
Small files are carried in-band in the communication between
controller and remote, with larger files being copied by falling back
to a more traditional ansible put_file mechanism. This large
file code path was missed in b822f20.
2 months ago
Alex Willmer ce1accedbc tests: Refactor Ansible copy integration tests to be loop driven
This is in anticipation of #1110, which only exhibits inside a with_items:
loop. For this refactor `loop:` is used, to confirm the refactored tests are
still correct. A subsequent commit will change them to with_items.

The content of the files and their SHA1 checksums are unchanged.
2 months ago
Alex Willmer 84a4fcdf00
Merge pull request #1087 from bbc/unsafe-chmod
ansible_mitogen: Handle unsafe paths in _remote_chmod
2 months ago
Alex Willmer 5ab872f289 ansible_mitogen: Add regression test for ActionModuleMixin._remote_chmod()
Adapted from Jonathon's reproducer in #1087.
2 months ago
Jonathan Rosser 06617f8231 ansible_mitogen: Handle unsafe paths in _remote_chmod
This is missing from b822f20007
2 months ago
Alex Willmer c95d41128f
Merge pull request #1101 from moreati/prep-0.3.9
Prepare v0.3.9
3 months ago
Alex Willmer d15051b187 Begin v0.3.10dev 3 months ago
Alex Willmer 6fbad3ae7d Prepare v0.3.9 3 months ago
Alex Willmer c1c33297ac
Merge pull request #1098 from moreati/docs-untrack
docs: Remove dead references to Piwik and mailing list
3 months ago
Alex Willmer c948e6668a docs: Remove email form from website
https://networkgenomics.com is no longer served, so the form submission would
fail.
3 months ago
Alex Willmer 2edcb31996 docs: Remove piwik analytics hooks from website
https://networkgenomics.com is no longer served, so the javascript and other
attempts to beacon or redirect result in HTTP errors.
3 months ago
Alex Willmer 022f0c4b5f
Merge pull request #1095 from moreati/ansible-2.17
Ansible 10 support
3 months ago
Alex Willmer 357fe38766 Ansible 10 (ansible-core 2.17) support
Notably
- Python 2.7 and 3.6 are no longer supported by Ansible on targets
- The yum module has been removed, and redirected to dnf
- _INTERPRETER_PYTHON_DISTRO_MAP has been neutered. Interpreter discovery
  always favours specific `python3.<x>` interpreters in decending version
  order, then generic `python3` or `python`.
- Add the ability for an action plugin to call self._execute_module(*,
  ignore_unknown_opts=True) to execute a module with options that may not be
  supported for the version being called.

https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_10.html
https://github.com/ansible-community/ansible-build-data/blob/main/10/CHANGELOG-v10.md
https://github.com/ansible/ansible/blob/stable-2.17/changelogs/CHANGELOG-v2.17.rst

fixes #1074, refs #1082

Co-authored-by: Claude Becker <becker@phys.ethz.ch>
3 months ago
Alex Willmer 85b1b4070a tests: Respect configured or detected Python more often
Relying on the virtualenv default or hardcoding "python" results in a Python
2.x virtualenv on some targets (e.g. debian10-test). This caused a failure
when testing with Ansible >= 10 (ansible-core >= 2.17), which have dropped
Python 2.x support.

refs #1074
3 months ago
Alex Willmer 863f923f14 tests: Bypass interpreter discovery on non-existant connection delegation targets
By setting ansible_python_interpreter for these fictious hosts we avoid
Ansible trying and failing to connect to them in a attempt to populate
ansible_facts.discovered_interpreter_python. This speeds up these tests by
avoiding a timeout.

It is also a necessary pre-requisite for Ansible 10 (ansible-core 2.17). In
that release no hardcoded fallback is used, failure to determine a valid
Python interpreter is a fatal error.

refs #1074
3 months ago
Alex Willmer 40695f413b ansible_mitogen: Respect ansible_facts.discovered_interpreter_python more
fixes #1097
3 months ago
Alex Willmer 9185805bf2 ansible_mitogen: cast ansible_python_interpreter value
This was the last remaining use of `mitogen.utils.cast()`. I missed it in
#1046.
3 months ago
Alex Willmer 8613f685ab tests: Skip AWS ECR login outside of CI jobs
To avoid rate limiting errors, CI (currently Azure Devops) logs into the
container registry (currently AWS ECR). Outside CI this is unnnecessary and
makes it harder to run the tests, because very few people have access to a
suitable AWS secret token.

Following this change `aws ecr-public get-login-password` will only be run if
the environment variable $TF_BUILD==True. This is set by Azure Pipelines
jobs. If the CI platform is changed then another indicator should be used.

https://adamj.eu/tech/2020/03/09/detect-if-your-tests-are-running-on-ci/
3 months ago
Alex Willmer fe26b70902
Merge pull request #1092 from moreati/prep-v0.3.8
Prepare v0.3.8
3 months ago
Alex Willmer 62cde17150 Start v0.3.9 development 3 months ago
Alex Willmer e334b50d9d Prepare v0.3.8 3 months ago
Alex Willmer 8bec30d97c
Merge pull request #1091 from moreati/issue1090
ci: Summer 2024 test fixups
3 months ago
Alex Willmer fe435bb7d0 CI: Workaround "No module named 'setuptools.command.test'"
Pip 72 was released yesterday (2024-07-28), dropping `setup.py test` support.
hdrhistogram 0.6.1 requires it to install.

For now constrain Pip to earlier releases, so our tests can be run.

refs #1090
3 months ago
Alex Willmer 924dbd6f0c CI: Migrate macOS integration tests to macOS 12, drop Python 2.7 jobs
macOS 11 is not longer an available runner on Azure Devops. The minimum is now
macOS 12. This runner does not have Python 2.7 installed, so running them
would require a custom install - which I'm declaring too much effort for too
little gain.

refs #1090
3 months ago
Alex Willmer f5a8840668 CI: Use archived RPMs on CentOS 8
CentOS 8 has reached EOL. Packages are no longer mirrored or maintained. A
historic snapshot of the packages is kept on vault.centos.org.

refs #1088, #1090
3 months ago
Alex Willmer 23d9d0bc82
Merge pull request #1060 from moreati/issue1059
Speed up test suite
6 months ago
Alex Willmer 7079a07a15 tests: Fix duplicate local task executions
integration/ssh/timeouts.yml is noteworthy. It was an accidental N**2 in time
-  executing num hosts * num hosts tasks.
6 months ago