* ansible-test: Set max number of open files in containers to 10240 (#75498)
(cherry picked from commit e50ad6f118)
* ansible-test - Add RHEL 9 support. (#76217)
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
(cherry picked from commit 730194e2b6)
* fix support containers under podman (#76292)
* `podman inspect` falls back to a same-named image if the named container is not present; since eg `http-test-container` is both the name of the image and container, it wasn't working properly under podman in many instances. Switching to `docker|podman container inspect` limits the query to containers only for both podman and docker, allowing the support container detection/creation to work properly.
(cherry picked from commit abac141122)
* ansible-test - Code cleanup. (#76540)
* Add missing typing imports.
* Remove unnecessary non-capturing group.
* Whitespace.
* Add type hints.
* Ignore PyCharm false positives.
* Remove Python 2.x plugin loading logic.
* Remove Python 2.x exception handling.
* Remove Python 2.x display logic.
* Add changelog.
(cherry picked from commit 4e70156d7e)
* ansible-test - Fix type hints.
(cherry picked from commit d2daa67b00)
* ansible-test - Fix plugin loading.
This fixes a traceback when loading plugins that use dataclasses.
(cherry picked from commit 7e814dd4db)
* ansible-test - Remove AIX provisioning support. (#76747)
* ansible-test - Remove AIX provisioning support.
* Remove `skip/aix` integration test aliases.
* Remove build for AIX binary module.
(cherry picked from commit 3a891827b7)
* ansible-test: fix typo in validate-modules (#76765)
The correct error strategy for the to_text method is surrogate_or_strict
Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a0f6747f07)
* ansible-test - Improve help for unsupported cwd. (#76866)
* ansible-test - Improve help for unsupported cwd.
* The `--help` option is now available when an unsupported cwd is in use.
* The `--help` output now shows the same instructions about cwd as would be shown in error messages if the cwd is unsupported.
* Add `--version` support to show the ansible-core version.
* The explanation about cwd usage has been improved to explain more clearly what is required.
Resolves https://github.com/ansible/ansible/issues/64523
Resolves https://github.com/ansible/ansible/issues/67551
(cherry picked from commit de5f60e374)
* ansible-test - Validate collection ns and name.
Resolves https://github.com/ansible/ansible/issues/62079
(cherry picked from commit 26b43f425f)
* ansible-test - Add default for Windows remotes.
(cherry picked from commit 18ed2c64e5)
* ansible-test - Clean up future boilerplate. (#76874)
* ansible-test - Clarify need for empty __init__.py
* ansible-test - Update code-smell boilerplate.
* Update code-smell boilerplate for core.
* Update future boilerplate test for ansible-test.
All ansible-test code (except for targets) and core-specific sanity tests now use the same boilerplate.
The test also checks for unwanted `__future__` and `metaclass` boilerplate.
* Relocate target tools to the correct directory.
Several tools used on target Python versions were incorrectly placed in the controller directory.
(cherry picked from commit d19b506ce8)
* ansible-test - Remove obsolete display of MAXFD.
Use of ``subprocess.MAXFD`` only worked on Python 2.x, which is no longer supported.
(cherry picked from commit dc6b0d4857)
* ansible-test - Code cleanup and refactoring. (#77169)
* Remove unnecessary PyCharm ignores.
* Ignore intentional undefined attribute usage.
* Add missing type hints. Fix existing type hints.
* Fix docstrings and comments.
* Use function to register completion handler.
* Pass strings to display functions.
* Fix CompositeAction handling of dest argument.
* Use consistent types in expressions/assignments.
* Use custom function to keep linters happy.
* Add missing raise for custom exception.
* Clean up key/value type handling in cloud plugins.
* Use dataclass instead of dict for results.
* Add custom type_guard function to check lists.
* Ignore return type that can't be checked (yet).
* Avoid changing types on local variables.
(cherry picked from commit a06fa496d3)
* ansible-test - Simplify type hints.
(cherry picked from commit af2b1361ff)
* Support podman-remote in ansible-test (#75753)
(cherry picked from commit 7cb581ed2c)
* docker_util: Handle error in JSON parsing (#77298)
While getting hostname from container, podman command
fails to return JSON so wrap exception and return
hostname as 'None'
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 1100289a45)
* ansible-test - Remove RHEL 9.0b support.
Support can be restored once RHEL 9 has been released.
(cherry picked from commit 55f90ba31f)
* ansible-test - Don't fail if network cannot be disconnected (#77472)
* Don't fail if network cannot be disconnected
* add clog frag
(cherry picked from commit 66c5844978)
* ansible-test - Backport `InternalError`
NOTE: This is a partial backport, including only one new class.
(cherry picked from commit b960641759)
* ansible-test - Fix subprocess management. (#77641)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
(cherry picked from commit 5c2d830dea)
* ansible-test - Add support for remote Ubuntu VMs.
(cherry picked from commit 6513453310)
* ansible-test - Fix remote completion validation.
(cherry picked from commit e2200e8dfc)
* ansible-test - Add multi-arch remote support.
(cherry picked from commit 2cc74b04c4)
* ansible-test - Enhance the shell command. (#77734)
* ansible-test - Add shell --export option.
* ansible-test - Support cmd args for shell command.
Also allow shell to be used without a valid layout if no delegation is required.
* ansible-test - Improve stderr/stdout consistency.
By default all output goes to stdout only, with the exception of a fatal error.
When using any of the following, all output defaults to stderr instead:
* sanity with the `--lint` option -- sanity messages to stdout
* coverage analyze -- output to stdout if the output file is `/dev/stdout`
* shell -- shell output to stdout
This fixes issues two main issues:
* Unpredictable output order when using both info and error/warning messages.
* Mixing of lint/command/shell output with bootstrapping messages on stdout.
* ansible-test - Add changelog fragment.
(cherry picked from commit fe349a1ccd)
* ansible-test - Fix remote args restriction.
The platform-specific and global fallbacks were not working with the `--remote` option.
This regression was introduced by https://github.com/ansible/ansible/pull/77711
(cherry picked from commit 76ead1e768)
* ansible-test - Fix shell target options handling.
(cherry picked from commit f933314101)
* Refactor ansible-test integration tests. (#78168)
* Relocate update-ignore.py for easier re-use.
* Add script to ease collection testing.
* Skip ignore rewrite if file does not exist.
* Add integration test for the shebang sanity test.
* Fix ansible-test-no-tty integration test.
Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.
* Clean up ansible-test-docker integration test.
(cherry picked from commit f70cc2fb7e)
* ansible-test - Fix TTY and output handling. (#78350)
(cherry picked from commit a3c90dd0bc)
* ansible-test - Parse content config only once. (#78418)
(cherry picked from commit f2abfc4b3d)
* ansible-test - Fix self-test change handling. (#78521)
(cherry picked from commit 99016653a2)
* ansible-test - Fix integration test target filter. (#78862)
- Allow disabled, unsupported, unstable and destructive integration test targets to be selected using their respective prefixes.
- Allow unstable tests to run when targeted changes are made and the ``--allow-unstable-changed`` option is specified (resolves https://github.com/ansible/ansible/issues/74213).
(cherry picked from commit d3d7785472)
* [stable-2.13] ansible-test - Improve container management. (#79538)
* ansible-test - More flexible become support.
(cherry picked from commit 5666c6d6a3)
* ansible-test - Add support for more remotes.
(cherry picked from commit 24d91f552c)
* ansible-test - Enable ACLs on FreeBSD remotes.
This allows integration tests to switch from one unprivileged user
to another, sharing files between the users using ACLs.
(cherry picked from commit b63812bc08)
* ansible-test - Enable ACL support on more remotes. (#78299)
(cherry picked from commit 8bb20fe06b)
* ansible-test - Install `acl` on Alpine remotes. (#78303)
(cherry picked from commit 8565deaae3)
* ansible-test - Use sudo for all remotes. (#78310)
(cherry picked from commit f3f4ad9332)
* ansible-test - Fix Ubuntu 20.04 bootstrapping. (#78551)
(cherry picked from commit d683c5bd21)
* ansible-test - Verify executables are executable. (#78606)
(cherry picked from commit ece85abbc4)
* ansible-test - Use --tmpfs to run containers. (#78605)
(cherry picked from commit 4187707f03)
* ansible-test - Remove Fedora 35 support. (#78720)
(cherry picked from commit 38a82a5cc4)
* ansible-test - Fix container error handling. (#78739)
(cherry picked from commit 79f67ed561)
* ansible-test - Improve container management. (#78550)
See changelogs/fragments/ansible-test-container-management.yml for details.
(cherry picked from commit cda16cc5e9)
* ansible-test - Fix container detection. (#79530)
(cherry picked from commit 80d2f8da02)
* Update test matrix symlinks. (#78737)
* Add remote platform symlinks.
These were already supported by ansible-test.
This change adds the symlinks for use in the CI test matrix.
* Remove obsolete AIX test symlink.
AIX was already removed from ansible-test.
(cherry picked from commit 62221a3be2)
* Use patched containers without VOLUME instruction.
* Make test suite compatible with older ansible.
* Fix compatibility with Python 3.8.
(cherry picked from commit 1d7436935f)
* ansible-test - fix warning to include image name (#79560)
(cherry picked from commit 0a12d8a5bc)
* ansible-test - Fix target PyPI proxy config.
(cherry picked from commit dcc08eec35)
* ansible-test - Fix `--prime-containers` and `--explain` (#79581)
* Remove unused code.
* Fix explain errors.
* Fix `--prime-containers` with docker on cgroup v2.
(cherry picked from commit da3b1d3f50)
* ansible-test - Support RSA SHA-1 for SSH clients.
(cherry picked from commit 75b60b17ee)
* ansible-test - Limit paramiko to < 2.9.0
(cherry picked from commit d6588293fb)
* Add more retries to ansible-test-container test.
(cherry picked from commit f6c0e22f98)
* ansible-test - Improve container startup handling.
Also improve the ansible-test-container integration test:
- Add coverage for the no-probe code path.
- Add work-arounds for centos6 containers (to support backporting).
- Avoid systemd debug when the container doesn't use cgroup.
(cherry picked from commit 04fc98c794)
* Fix shebang test.
A false positive is generated by the test and a fix was never backported to stable-2.12.
Co-authored-by: Andrew Klychkov <aklychko@redhat.com>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Co-authored-by: Dimitri Savineau <dsavinea@redhat.com>
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* ansible-test - Create injector scripts at runtime.
* Set bootstrap.sh shebang at runtime.
* Remove shebang and execute bit from importer.
* Update shebang sanity test.
* Preserve line numbers.
* Fix get_available_python_versions calls.
* Make run_playbook vars optional.
* Use ansible_pipelining in inventory.
* Fix type hint.
* Fix order of conditional evaluation.
* Remove unused ibmi platform.
* Add changelog fragment.
* Emit warning when running on the controller with a Python older than 3.8
* Add spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* and more spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* s/Py/Python/
Co-authored-by: Matt Clay <matt@mystile.com>
* Add note to Control node requirements about Py3.8 requirement
* Add collection_name to deprecated call
* more spaces
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Clarify that we are only packaging for py3.8+
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* config: singular ANSIBLE_COLLECTIONS_PATH
Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories. Let's be consistent both internally and with POSIX
tradition.
* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular
* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
* Split out sanity test requirements.
* Run each --venv test separately.
This provides verification that the requirements for each test are properly specified.
* Use a separate requirements file per sanity test.
* Skip setuptools/cryptography setup for sanity.
* Eliminate pyyaml missing warning.
* Eliminate more pip noise.
* Fix conflicting generate_pip_install commands.
* Add changelog fragment.
The last task in a play should now properly report code coverage.
This change should also eliminate empty coverage files, as well as incomplete coverage files resulting from early worker termination.
* Support private test plugins in ansible-test.
This feature is exclusively used for testing Ansible itself and is not available to collections.
Content in the newly supported directories will be added in follow-up PRs.
* Add new directory to path classification.
* Support new paths in shebang test.
* Add new directory to manifest.
* Add warning when running devel. Addresses #67362
* Add config entry to disable devel warning, so it doesn't impact CI
* Update warning about using devel
Running from an installed version of ansible-test now results in tests using a dedicated directory for PYTHONPATH instead of using the site-packages directory where ansible is installed.
This provides consistency with tests running from source, which already used a dedicated directory.
Resolves https://github.com/ansible/ansible/issues/62716
* Remove .keep files from test/results/ dirs.
* Remove classification of test/results/ dir.
* Add results_relative to data context.
* Use variables in delegation paths.
* Standardize file writing and results paths.
* Fix issues reported by PyCharm.
* Clean up invocation of coverage command.
It now runs through the injector.
* Hack to allow intercept_command in cover.py.
* Simplify git ignore for test results.
* Use test result tmp dir instead of cache dir.
* Remove old .pytest_cache reference.
* Fix unit test docker delegation.
* Show HTML report link.
* Clean up more results references.
* Move import sanity test output to .tmp dir.
* Exclude test results dir from coverage.
* Fix import sanity test lib paths.
* Fix hard-coded import test paths.
* Fix most hard-coded integration test paths.
* Fix PyCharm warnings.
* Fix import placement.
* Fix integration test dir path.
* Fix Shippable scripts.
* Fix Shippable matrix check.
* Overhaul key pair management.
* Update ansible-test collection inventory handling.
- The `windows-integration` command now supports the `--inventory` option.
- The incomplete support for host_vars and group_vars directories has been removed.
- The incomplete support for an inventory directory has been removed.
- The inventory specified by `--inventory` can now reside outside the install and content roots.
- Using `ansible_ssh_private_key_file` with `--docker` or `--remote` results in a warning about the combination being unsupported and likely to fail.
* Fix config handling.
* Fix payload handling of ssh keys.
* Disable pylint no-self-use rule for ansible-test.
* De-duplicate payload paths.
* Move Ansible test config into env vars.
This allows ansible-test to use an empty Ansible config file, leaving open the option for users to customize the one used to run tests (although such usage is discouraged).
* Use config from content under test when present.
* Cache ansible version lookup.
* Fix method of determining ANSIBLE_ROOT.
* Clean up based on PyCharm inspections.
* Generate minimal PKG-INFO without setup.py.
* Use ANSIBLE_LIB_ROOT where possible.
* Use import instead of subprocess to get version.
* Fix install layout type.
* Correct required paths message for installs.
* Update list of files copied during delegation.
* Fix ansible-test entry point.
* Fix pylint issue.
* Fix version lookup on Python 2.x.
* Fix pylint issue.
* Remove unwanted print statement.
Matt Clay