Commit Graph

11649 Commits (f2abfc4b3d03a2baa078477d0ad2241263a00668)

Author SHA1 Message Date
Matt Clay f70cc2fb7e
Refactor ansible-test integration tests. (#78168)
* Relocate update-ignore.py for easier re-use.

* Add script to ease collection testing.

* Skip ignore rewrite if file does not exist.

* Add integration test for the shebang sanity test.

* Fix ansible-test-no-tty integration test.

Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.

* Clean up ansible-test-docker integration test.
2 years ago
Matt Martz bcdc2e167a
Support for Python 3.11+ tomllib for inventory (#77435) 2 years ago
Martin Krizek 43153c5831
`FieldAttribute`s as descriptors (#73908) 2 years ago
Matt Clay 4594c0c609 Fix HTTP tests on FreeBSD when running split. 2 years ago
Matt Clay b9d13d222c ansible-test - Improve pip bootstrap download. 2 years ago
Brian Coca fa35aa4865
Galaxy server config updates (#77106)
* ansible-galaxy configurable timeouts

  - also fixed issues with precedence,
    so --ignore-certs now overrides config
  - made galaxy_timeout generic setting,
    if set, it becomes default for server configs,
    but now specific servers can override
  - updated tests or added notes (some tests ignore/override precedence)

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Sviatoslav Sydorenko e481b35e23
Delete core-ci `query` param from ansible-test (#78136)
Ref: https://github.com/ansible/ansible-core-ci/pull/343
2 years ago
Matt Clay ca04dc28a2 ansible-test - Remove deprecated test platforms. 2 years ago
Brian Coca cea18bf60a
password lookup argument parsing fix (#78080)
fixes #78079
2 years ago
Matt Clay cf9fd2d7da ansible-test - Use shlex.join where appropriate. 2 years ago
Matt Clay 24d91f552c ansible-test - Add support for more remotes. 2 years ago
Matt Clay 5666c6d6a3 ansible-test - More flexible become support. 2 years ago
Chris James ea351f0ae2
user - Use -n instead of -N for luseradd on all distros (#75042)
* Use -n instead of -N for luseradd on all distros

Co-authored-by: Chris James <git@etcet.net>
2 years ago
Don Naro 3cd2c494bd
issue #72449 winrm script rm (#77931) 2 years ago
Matt Martz 18992b7947
Add new `loop_control.extended_allitems` option (#75760)
* Add new `loop_control.extended_allitems` option. Fixes #75216

* Add test for extended_allitems

* docs code block fix
2 years ago
Matt Martz a90f666ab3
Use ctypes to access shadow, instead of spwd which is deprecated (#78050)
* Use ctypes to access shadow, instead of spwd which is deprecated

* Remove sanity ignore

* cast to str to satisfy mypy

* Add typing.cast polyfill

* Appease mypy redefine

* Add clog frag
2 years ago
David Moreau Simard d81a519f25
Remove .travis.yml files from galaxy init skeletons (#75740)
* Remove .travis.yml files from galaxy init skeletons

These files have existed for many years but in light of the recent
circumstances around Travis, we should not include them by default in
the galaxy init skeletons.

* Add breaking_changes changelog fragment with recommendation to use a custom skeleton if the .travis.yml file is still needed.

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Matt Clay 95df5cb740 Fix invalid-ansiblemodule-schema sanity errors.
The validate-modules sanity test expects a list not a dict_keys iterable.
2 years ago
Matt Clay 2f0530396b ansible-test - Clean up type hints and comments. 2 years ago
Matt Clay 5f5c4ef2ef Remove incidental_inventory_foreman test. 2 years ago
Matt Clay a4e4413cde ansible-test - Update distro containers to 4.3.0.
The primary change in this update is removal of `/etc/ansible/hosts` from the containers.
2 years ago
Matt Clay 5b3557f8ba ansible-test - Allow docstring in docs-only module 2 years ago
Matt Clay c2b36ba04d ansible-test - Update base and default containers. 2 years ago
Matt Clay 23914d3f0b
ansible-test - Adjust unit test mock usage. (#77961) 2 years ago
Sloane Hertel e059dde9fb
Add missing inventory cache test (#77990)
Fix intentional coverage to replace incidental_inventory_foreman

ci_complete
ci_coverage
2 years ago
Sloane Hertel 9d6cc7b576
unarchive - fall back to unzip -Z if zipinfo is not available (#76971)
Add a new handler class ZipZArchive to use unzip -Z as an alternative to zipinfo

Run 'unzip -Z' in can_handle_archive so we fall back to the next handler if it's not available (failing in is_unarchived is too late)

* Add a test for unzip -Z when zipinfo is not available

* Update test for missing binary altogether by removing /usr/bin from the PATH
2 years ago
Sloane Hertel a43112290a
fix ansible-galaxy-collection test with revoked key (#77989)
* Remove unnecessary gpg passphrase
2 years ago
Sloane Hertel e89176caac
Pin PyYAML version compatible with Python 3.8+ (#77936)
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2 years ago
Matt Martz 5e369604e1
Forked display via queue (#77056)
* Forked Display via queue

* Docs and simple code cleanup

* Only proxy Display.display

* Remove unused import

* comment

* Update deadlock comment, remove py3 check

* Don't flush display, and don't lock from forks

* clog frag

* ci_complete ci_coverage

* Add units for queue proxying

* Cleanup flush

* ci_complete

* Only lock the write, switch to RLock

* Remove unused import
2 years ago
Sloane Hertel 143e7fb45e
ansible-galaxy - support resolvelib versions >= 0.5.3, < 0.9.0 (#77649)
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1

Test incompatibilities are removed for resolvelib >= 0.6.0

Test against the latest 0.8.x version and fix requirements

* Fix tests - use a venv for testing the range of resolvelib versions

* Update temporary hardcoded fallback for ansible-test

* Update hardcoded upperbound for sanity tests

* Make error check more flexible
2 years ago
Matt Clay df765c0ecf ansible-test - Update Alpine, openSUSE containers 2 years ago
Matt Clay fe2d8b7066 Skip libuser tests on openSUSE 15.4+
The libuser package is not available.
2 years ago
Matt Clay e31e918e04 Don't install lgroupadd for SUSE tests.
It's not used by the tests and is not available as of openSUSE 15.4.
2 years ago
Brian Coca 89c6547892
preserve add_host/group_by on refresh (#77944)
* preserve add_host/group_by on meta: refresh_inventory

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2 years ago
Matt Clay 52c8613a04 ansible-test - Skip mypy runs under Python 3.11. 2 years ago
Matt Clay 11b6032e73 ansible-test - Ignore vendored code in mypy test. 2 years ago
Sloane Hertel 6c6a7851ed
Add inventory cache integration tests (#77895)
* Add intentional coverage for incidental_inventory_foreman coverage cache tests

ci_complete
ci_coverage
2 years ago
Matt Clay 5391150b30 Integration tests for invalid group name. 2 years ago
Sloane Hertel 1706d35fc4
ansible-galaxy-collection - use --pinentry-mode loopback for gpg key revocation (#77967) 2 years ago
Sloane Hertel c908d782fb
get_url - permit to have a checksum only file (#77948)
checksum can also accept a checksum only file (no filename beside the checksum).
fixes #54390

Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
2 years ago
Sloane Hertel f270b4e224
Fix items2dict errors for incorrect list item type and missing keys (#77946)
* Fix items2dict errors for incorrect list item type and missing keys

* keep dict comprehension
2 years ago
Martin Krizek 058a69e6b0
Properly send a skipped message for empty templated loop (#77945)
Fixes #77934
2 years ago
Matt Clay 81351022d8 ansible-test - Prefer unittest.mock for core. 2 years ago
Matt Clay f933314101 ansible-test - Fix shell target options handling. 2 years ago
Matt Clay 86779cc903 ansible-test - Code cleanup.
This helps prepare for a future pylint upgrade.
2 years ago
Sloane Hertel e6075109d0
Remove incidental_inventory_aws_ec2 and supporting plugins (#77877)
* Remove aws/2.7 and aws/3.6 from incidental test matrix entries

* Remove aws.sh symlink
2 years ago
Matt Clay bd849b3076 Fix ansible-runner integration tests. 2 years ago
Sandra McCann f3c684564c
Remove platform dev guides from core docs (#77915) 2 years ago
Julien Lecomte 33beeace10
Fix lusermod using a group name and not id (#61965) (#77914)
* Fix lusermod using a group name and not id (#61965)
2 years ago
Matt Clay e259317c3e
Remove incidental_inventory_docker_swarm tests. (#77924)
ci_complete
ci_coverage
2 years ago
Sloane Hertel 1214b63f4f
Fix 'ansible-config dump --only-changed -t all' verbosity (#77898)
* Fix 'ansible-config dump --only-changed -t all' to only display headers if plugin options are changed

* changelog

* add a test
2 years ago
Matt Clay 43d650f924
Add RHEL 9.0 to CI. (#77853)
* Add RHEL 9.0 to CI.

* Restrict network manager inspection to RHEL8

* Skip module tests when astream_name is undefined, undefine it for RHEL9 until 9.1

* Remove redundant test.

Co-authored-by: Matt Martz <matt@sivel.net>
2 years ago
Wouter Schoot 711b51fad6
Remove double spaces in errormessage (#77839) 2 years ago
Matt Clay 289cba333b ansible-test - More multi-arch containers. 2 years ago
OscarBell 74eb8b779d
Blockinfile multiline search (#75090) 2 years ago
Matt Martz e7e1d592a6
Incidental coverage 1 (#77846)
* Add find test to assert patterns with regex

* Add test to assert that nested includes with relative path work

* Add test to assert symmetric_difference supports unhashable types

* ci_complete ci_coverage

* Add test to assert JSON values are converted to Python

* Add test to assert how unexpected module failures behave

* Add test to verify remote_is_local

* ci_complete ci_coverage

* Get real path of local tmp

* ci_complete ci_coverage

* Expanded symmetric_difference tests

* ci_complete ci_coverage

* I don't even know

* ci_complete ci_coverage

* Revert mathstuff changes

* ci_complete ci_coverage
2 years ago
Brian Coca f7d7604454
Add uri, urn and url test plugins (#77423)
and docs!

Co-authored-by: Tabah Baridule <dulemartins07@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
2 years ago
Ian Wienand e4c0bbf885
apt: fix virtual package install version detection (#76781)
* apt: fix virtual package install version detection

Change 4a62c4e3e4 introduced version
matching in installation.

The problem stems from

 if version_installable or version:
     pkg_list.append("'%s=%s'" % (name, version_installable or version))

When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.

For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with

 $ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
 ... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...

In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.

The tests are updated to install and remove this package.

Fixes: #76779
2 years ago
Sloane Hertel 82f3a57bee
Fix ansible-galaxy traceback when unexpected version of resolvelib is installed (#77630)
* Fix traceback when a supported version of resolvelib is not installed

Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).

* Add tests for unsupported resolvelib versions

* Resolve remaining import sanity test issues.

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
2 years ago
Sloane Hertel 621e782ed0
Add toggle to fix module_defaults with module-as-redirected-action on a per-module basis (#77265)
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults

Add a toggle for action plugins to prefer the resolved module when loading module_defaults

Allow moving away from modules intercepted as actions pattern

Fixes #77059
2 years ago
Matt Clay 2595b42aca
Prefer unittest.mock over mock. (#77886) 2 years ago
Matt Clay 3a9a23fb1a
Use unittest.mock instead of mock. (#77883)
This can only be done for controller tests.
2 years ago
Sloane Hertel 1eb7f71f0e
Replace incidental_inventory_aws_ec2 with intentional coverage (#77743)
* Add intentional test coverage for lib/ansible/plugins/inventory/__init__.py

* Add intentional (integration) test coverage for lib/ansible/module_utils/common/network.py

* Add ansible_release info test from https://github.com/ansible/ansible/pull/74673

ci_complete
ci_coverage

Co-authored-by: Rick Elrod <rick@elrod.me>
2 years ago
Sloane Hertel 400475acc0
validate meta/main.yml dependencies and meta/requirements.yml are both lists before concatenating them together (#77821) 2 years ago
Matt Clay 13d03c3c22
ansible-test - Add RHEL 9 / FreeBSD 13.1 remotes. (#77851) 2 years ago
Matt Clay 3d2c2e60aa
ansible-test - Update distro containers. (#77847)
* ansible-test - Add a Ubuntu 22.04 container.
* ansible-test - Add a Fedora 36 container.
* ansible-test - Update distro containers.
* Fix dnf test on Fedora 36.
* Work around scp test issues.
2 years ago
Matt Martz ef3ea4cdc3
Incidental docker chip away (#77830)
* Add new templating target, test splitter

* Add test for host:port parsing in parse_address via add_host

* Test already notified listening handler

* Add test for gathering bridge network facts

* Add veth to bridge

* ci_complete ci_coverage
2 years ago
Matt Clay 76ead1e768 ansible-test - Fix remote args restriction.
The platform-specific and global fallbacks were not working with the `--remote` option.

This regression was introduced by https://github.com/ansible/ansible/pull/77711
2 years ago
Matt Clay 185340e667 ansible-test - Update default containers.
These containers include Python 3.11 sanity test environments.
2 years ago
Matt Clay dfde4be444 Add Python 3.11 support.
ci_complete
ci_coverage
2 years ago
Matt Clay 8ca28acd0d Fix unit tests for Python 3.11. 2 years ago
Matt Clay 5f74350fd5 Avoid deprecated method in unit tests. 2 years ago
Matt Clay 9600e3064a
ansible-test - Improve requirements handling. (#77825)
The requirements for virtualenv and coverage are now kept in a requirements file for easier container builds.

The test-constraints sanity test has been updated to make sure the requirements file is kept up-to-date.
2 years ago
Sloane Hertel fecc78b5fb
remove extra validate-modules ignore entry (#77826) 2 years ago
Brian Coca 4706091a38
rename systemd module to services only (#77644)
* rename systemd module to services only

 disambiguates what it handles since systemd is now much more
 that a service manager, but the module is specific to services
2 years ago
Joseph Torcasso c9ce7d08a2
template/__init__.py - fix KeyError when wantlist=False with non-list 2 years ago
Brian Coca 5e50284693
Restrict role loading .. to role ... (#77683)
* Ansible will now error out if you try to use the `*_from` to load files from outside the role being loaded by `{import,include}_role`
2 years ago
Matt Clay 290fb5e0ca
ansible-test - Upgrade to coverage 6.3.3. (#77817) 2 years ago
Martin Krizek e9af6efee6
Raise a proper error when include/import_role is used as a handler (#77807) 2 years ago
Matt Clay 1e3cf9f732
ansible-test - Make distro containers multi-arch. (#77814) 2 years ago
Abhijeet Kasurde 524d30b8b0
distribution: add distribution_minor_version for Debian Distro (#74721)
* Added a fact about minor version details for Debian Distro

Fixes: #74481

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Matt Clay 449d82a521 ansible-test - Add RHEL 8.6 support. 2 years ago
Matt Clay fe349a1ccd
ansible-test - Enhance the shell command. (#77734)
* ansible-test - Add shell --export option.

* ansible-test - Support cmd args for shell command.

Also allow shell to be used without a valid layout if no delegation is required.

* ansible-test - Improve stderr/stdout consistency.

By default all output goes to stdout only, with the exception of a fatal error.

When using any of the following, all output defaults to stderr instead:

* sanity with the `--lint` option -- sanity messages to stdout
* coverage analyze -- output to stdout if the output file is `/dev/stdout`
* shell -- shell output to stdout

This fixes issues two main issues:

* Unpredictable output order when using both info and error/warning messages.
* Mixing of lint/command/shell output with bootstrapping messages on stdout.

* ansible-test - Add changelog fragment.
2 years ago
s-hertel fbc5b3f9c5 fix copy test for split controller/remote 2 years ago
s-hertel 3f4cacd355 Remove MacOSX-specific cleanup for unarchive test 2 years ago
Matt Clay cc5ac88c4c Update mypy test. Fix type hints for Python 3.9.
This also bumps the minimum controller version (for mypy only) to Python 3.9.
2 years ago
Matt Clay 2cc74b04c4 ansible-test - Add multi-arch remote support. 2 years ago
Matt Clay de5d6820f8 ansible-test - Remove obsolete code. 2 years ago
Matt Clay a115a7fce2 Install acl on Ubuntu for copy integration test. 2 years ago
Matt Clay e2200e8dfc ansible-test - Fix remote completion validation. 2 years ago
Matt Davis 4ff10fcd31 fix SELinux facts tests
* make the pre-flight sniffing more robust to different failure conditions (was failing on Ubuntu 22.04 VMs)
* remove skip aliases (the test needs to function everywhere to assert that the selinux facts bits behave properly when it's N/A)
2 years ago
Matt Clay 6513453310 ansible-test - Add support for remote Ubuntu VMs. 2 years ago
Matt Clay dcda1e7239 Trim message in split integration test. 2 years ago
Martin Krizek dbdbfe845a
Remove deprecated subclassing DefaultCallback without doc_fragment (#77677) 2 years ago
Brian Coca a3cc6a581e
dont rely on path to set config defs for plugins (#77659) 2 years ago
Martin Krizek ed30fc9a06
Remove deprecated loading cache plugins directly without cache_loader (#77655) 2 years ago
Martin Krizek 572bc1354a Fix lazy eval version in integration tests 2 years ago
Brian Coca 749a105956
ansible-doc stay 3.8 python compatible (#77682)
pathlib with_stem will wait till 3.9
  also add tests by listing filters
2 years ago
Matt Clay cc872a57f2 Reduce output from ansible-doc integration test.
This removes about 20K lines of output which are not evaluated as part of the test.
2 years ago
Matt Clay 5c2d830dea
ansible-test - Fix subprocess management. (#77641)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
2 years ago