Commit Graph

66 Commits (c8de45e5535e43d2872002eef8218aa7d0f461e9)

Author SHA1 Message Date
Sviatoslav Sydorenko 5c7d361b6e
[stable-2.13] Replace FreeBSD 12.{3->4} & 13.{0->1} @ testing (#81498)
* Replace FreeBSD 12.3 w/ 12.4 in CI & ansible-test

FreeBSD 12.3 bootstrap packages stopped being available.

* Replace FreeBSD 13.0 w/ 13.1 in CI & ansible-test
2 years ago
Matt Clay abab52a7b4
[stable-2.13] Docs removal and other backports (#81409)
* Remove straight.plugin dependency (#80084)

(cherry picked from commit f587856beb)

* Update package-data sanity test (#80344)

The test no longer relies on the Makefile.

(cherry picked from commit 46362bbd27)

* Remove obsolete release bits (#80347)

Releases are now built using the `packaging/release.py` tool.
This makes the `Makefile` and associated files in `packaging/release/` and `packaging/sdist/` obsolete.

(cherry picked from commit 6a66761e0e)

* Use --no-isolation for package-data sanity test (#80377)

The dependencies are already in the sanity test venv. This avoids use of unpinned dependencies and a dependency on a network connection.

(cherry picked from commit 7fcb9960e6)

* Set the minimum setuptools to 45.2.0 (#80649)

Also update the package-data sanity test to use the minimum setuptools version.

(cherry picked from commit 4d25e3d54f)

* Use package_data instead of include_package_data (#80652)

This resolves warnings generated by setuptools such as the following:

_Warning: Package 'ansible.galaxy.data' is absent from the `packages` configuration.

(cherry picked from commit 5ac292e12d)

* Fix os.walk issues in package-data sanity test (#80703)

(cherry picked from commit eea7137449)

* Remove `docs` and `examples` directories (#81011)

* Remove docs dir

* Updates to reflect docs removal

* Fix integration test

* Remove examples dir

* Updates to reflect examples removal

* Remove build_library and build-ansible.py

* Remove refs to build_library and build-ansible.py

* Remove obsolete template

* Remove obsolete template reference

* Remove the now obsolete rstcheck sanity test

(cherry picked from commit 72e038e823)

* Omit pre-built man pages from sdist (#81395)

Since man pages aren't accessible to users after a `pip install`, there's no need to include them in the sdist.
This change makes it trivial to build man pages from source, which makes them much easier to iterate on.
It also simplifies creation and testing of the sdist, since it no longer requires building man pages.

The new `packaging/cli-doc/build.py` script can generate both man pages and RST documentation.
This supports inclusion on the docs site without a dependency on `ansible-core` internals.
Having a single implementation for both simplifies keeping the two formats in sync.

(cherry picked from commit 691c8e8603)
2 years ago
Matt Clay 191cdf5ae5
[stable-2.13] ansible-test - Pre-build PyYAML wheels (#81300) (#81311)
* [stable-2.13] ansible-test - Pre-build PyYAML wheels (#81300)

This works around Cython failures when attempting to install PyYAML >= 5.4 <= 6.0..
(cherry picked from commit e964078a83)

Co-authored-by: Matt Clay <matt@mystile.com>

* Skip PEP 517 tests on FreeBSD
2 years ago
Matt Clay ab220107d5
ansible-test - Relax pylint config for backports (#80770) 3 years ago
Matt Clay 8fb7c10498
[stable-2.13] ansible-test - Use FreeBSD packaged setuptools (#80615) (#80618)
This will avoid issues during bootstrapping caused by breaking changes in setuptools.

(cherry picked from commit abc58c026b)
3 years ago
Matt Clay af1cbc9bb6
[stable-2.13] ansible-test - Add support for argcomplete 3 (#80482) (#80486)
(cherry picked from commit 0371ea08d6)
3 years ago
Matt Clay a35e8015e5 [stable-2.13] ansible-test - Update git diff handling (#80202)
This change allows ansible-test to work with newer versions of git on AZP.

(cherry picked from commit bad8843124)

Co-authored-by: Matt Clay <matt@mystile.com>
3 years ago
Jordan Borean 2e541a5450 ansible-test - fix ps argspec check inside cmdlet (#79699)
* ansible-test - fix ps argspec check inside cmdlet

* Added error condition test

* Fix sanity problem

(cherry picked from commit ee33be9484)
3 years ago
Matt Clay 1d7436935f
[stable-2.13] ansible-test - Improve container management. (#79538)
* ansible-test - More flexible become support.

(cherry picked from commit 5666c6d6a3)

* ansible-test - Add support for more remotes.

(cherry picked from commit 24d91f552c)

* ansible-test - Enable ACLs on FreeBSD remotes.

This allows integration tests to switch from one unprivileged user
to another, sharing files between the users using ACLs.

(cherry picked from commit b63812bc08)

* ansible-test - Enable ACL support on more remotes. (#78299)

(cherry picked from commit 8bb20fe06b)

* ansible-test - Install `acl` on Alpine remotes. (#78303)

(cherry picked from commit 8565deaae3)

* ansible-test - Use sudo for all remotes. (#78310)

(cherry picked from commit f3f4ad9332)

* ansible-test - Fix Ubuntu 20.04 bootstrapping. (#78551)

(cherry picked from commit d683c5bd21)

* ansible-test - Verify executables are executable. (#78606)

(cherry picked from commit ece85abbc4)

* ansible-test - Use --tmpfs to run containers. (#78605)

(cherry picked from commit 4187707f03)

* ansible-test - Remove Fedora 35 support. (#78720)

(cherry picked from commit 38a82a5cc4)

* ansible-test - Fix container error handling. (#78739)

(cherry picked from commit 79f67ed561)

* ansible-test - Improve container management. (#78550)

See changelogs/fragments/ansible-test-container-management.yml for details.

(cherry picked from commit cda16cc5e9)

* ansible-test - Fix container detection. (#79530)

(cherry picked from commit 80d2f8da02)

* Update test matrix symlinks. (#78737)

* Add remote platform symlinks.

These were already supported by ansible-test.
This change adds the symlinks for use in the CI test matrix.

* Remove obsolete AIX test symlink.

AIX was already removed from ansible-test.

(cherry picked from commit 62221a3be2)

* Use patched containers without VOLUME instruction.

* Make test suite compatible with older ansible.

* Fix compatibility with Python 3.8.
3 years ago
Matt Clay 99217ca2b6 [stable-2.13] ansible-test - Improve pip bootstrap download.
(cherry picked from commit b9d13d222c)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Sandra McCann f09e6b7769
Backportapalooza 06 16 (#78072)
* Docsite: Updated style guide resources (#78048)

(cherry picked from commit 561cf1956d)

* Create reusable role layout snippet for documentation (#78057)

* reuse role structure
* reuse role snippet in role docs

(cherry picked from commit f4ad1c771e)

* Docsite: update links to community docs (#78043)

(cherry picked from commit 46d86a882d)

* Windows - incorrect python-devel package in WinRM docs (#78037)

Co-authored-by: Emanuele Barbato <manu@Giuseppes-MacBook-Air.local>
(cherry picked from commit 681dc6eab9)

* issue #72449 winrm script rm (#77931)

(cherry picked from commit 3cd2c494bd)

* reword the regex note (#75393)

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
(cherry picked from commit 66c0882314)

* Document debconf interaction with dpkg-reconfigure (#74196)

Signed-off-by: David Greaves <david@dgreaves.com>
(cherry picked from commit 5b90601ca4)

Co-authored-by: Lewis Brogan <ilewisbrogan@gmail.com>
Co-authored-by: Andrew Klychkov <aklychko@redhat.com>
Co-authored-by: Emanuele Barbato <30394740+emanuelebarbato@users.noreply.github.com>
Co-authored-by: Don Naro <dnaro@redhat.com>
Co-authored-by: David Greaves <david@dgreaves.com>
4 years ago
Matt Clay ae380e3bef
[stable-2.13] ansible-test - Multiple backports (#77951)
* ansible-test - Backport `InternalError`

NOTE: This is a partial backport, including only one new class.

(cherry picked from commit b960641759)

* ansible-test - Fix subprocess management. (#77641)

* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.

(cherry picked from commit 5c2d830dea)

* ansible-test - Add support for remote Ubuntu VMs.

(cherry picked from commit 6513453310)

* ansible-test - Fix remote completion validation.

(cherry picked from commit e2200e8dfc)

* ansible-test - Add multi-arch remote support.

(cherry picked from commit 2cc74b04c4)

* ansible-test - Enhance the shell command. (#77734)

* ansible-test - Add shell --export option.

* ansible-test - Support cmd args for shell command.

Also allow shell to be used without a valid layout if no delegation is required.

* ansible-test - Improve stderr/stdout consistency.

By default all output goes to stdout only, with the exception of a fatal error.

When using any of the following, all output defaults to stderr instead:

* sanity with the `--lint` option -- sanity messages to stdout
* coverage analyze -- output to stdout if the output file is `/dev/stdout`
* shell -- shell output to stdout

This fixes issues two main issues:

* Unpredictable output order when using both info and error/warning messages.
* Mixing of lint/command/shell output with bootstrapping messages on stdout.

* ansible-test - Add changelog fragment.

(cherry picked from commit fe349a1ccd)

* ansible-test - Fix remote args restriction.

The platform-specific and global fallbacks were not working with the `--remote` option.

This regression was introduced by https://github.com/ansible/ansible/pull/77711

(cherry picked from commit 76ead1e768)
4 years ago
Matt Clay 3b20b02272
[stable-2.13] Fix sanity test crash. (#77475)
(cherry picked from commit 6216c9fc93)

Co-authored-by: Felix Fontein <felix@fontein.de>

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Sloane Hertel 2769f5621b
Add find_spec and exec_module to RestrictedModuleLoader (#76427)
* Add find_spec and exec_module to RestrictedModuleLoader

* Fix getting new loader with correct path

Fix pep8 errors

* Use convert_ansible_name_to_absolute_paths instead of the loader path

* Apply suggestions from code review

* Fix type hints and test them in CI

* Fix error message for ansible.module_utils.basic if it's missing

Add mypy ignored missing imports for controller sanity tests

* Add mypy attr-defined ignore entries for python 3.8, 3.9, 3.10 for vendored six

Add mypy attr-defined ignore for python 2.7 in lib/ansible/utils/collection_loader/_collection_finder.py

* Just test controller python versions to simplify ignoring mypy errors
4 years ago
Felix Fontein 4d984613f5
validate-modules for plugins: do some more schema validations so that some issues that are currently reported for modules are also reported for plugins (#77268)
* Add more sanity tests on schema level (so they also work for plugins).

* Fix various issues the sanity test reported.

* Add changelog fragment.

* Fix function name.
4 years ago
Matt Clay f68c66a3ef Remove collections compat from controller code. 4 years ago
Matt Clay 3d5637beec Initial mypy sanity test support for core. 4 years ago
Felix Fontein 0990c4ca7c
Extend validate-modules to also validate plugins (#71734)
* Let validate-modules also validate plugins.

* Support 'option' in 'cli'.

* Use DOCUMENTABLE_PLUGINS instead of UNDOCUMENTED_PLUGIN_TYPES.

* Support 'keyword', clean up error codes.

* Call settings.process_errors only once; remove __version__.

* Add changelog fragment.
4 years ago
Matt Clay a06fa496d3
ansible-test - Code cleanup and refactoring. (#77169)
* Remove unnecessary PyCharm ignores.
* Ignore intentional undefined attribute usage.
* Add missing type hints. Fix existing type hints.
* Fix docstrings and comments.
* Use function to register completion handler.
* Pass strings to display functions.
* Fix CompositeAction handling of dest argument.
* Use consistent types in expressions/assignments.
* Use custom function to keep linters happy.
* Add missing raise for custom exception.
* Clean up key/value type handling in cloud plugins.
* Use dataclass instead of dict for results.
* Add custom type_guard function to check lists.
* Ignore return type that can't be checked (yet).
* Avoid changing types on local variables.
4 years ago
Matt Martz bdf37336c8
Update pylint deprecated plugin to use ansible-invalid* symbols (#77086) 4 years ago
Matt Clay 4f9c7fd509 ansible-test - Remove validate-modules symlink. 4 years ago
Matt Clay 5a6e05affb More strict pylint checking of ansible-test code. 4 years ago
Matt Clay d3d5244912 ansible-test - Fail sanity test on distutils use. 4 years ago
Matt Clay d286c2e8b3 ansible-test - Fix CParser import in yamllint. 4 years ago
Matt Clay 68fb3bf90e
ansible-test - Fix consistency of managed venvs. (#77028) 4 years ago
Matt Clay ef4c5cd61b ansible-test - Use newer pip to bootstrap FreeBSD. 4 years ago
Matt Clay 0d40423f1c
ansible-test - Fix collection loader import. (#76986)
* ansible-test - Fix collection loader import.

Resolves https://github.com/ansible/ansible/issues/76960
4 years ago
Matt Clay 0d5401d950 ansible-test - Remove unused code.
This was overlooked in a previous commit that removed FreeBSD 12.2 support.
4 years ago
Matt Clay 00a2b7788e ansible-test - Remove cap on cryptography version. 4 years ago
Matt Clay 7e64c4fe55 ansible-test - Update bootstrap URL to not use S3. 4 years ago
Matt Clay d19b506ce8
ansible-test - Clean up future boilerplate. (#76874)
* ansible-test - Clarify need for empty __init__.py
* ansible-test - Update code-smell boilerplate.
* Update code-smell boilerplate for core.
* Update future boilerplate test for ansible-test.

All ansible-test code (except for targets) and core-specific sanity tests now use the same boilerplate.

The test also checks for unwanted `__future__` and `metaclass` boilerplate.

* Relocate target tools to the correct directory.

Several tools used on target Python versions were incorrectly placed in the controller directory.
4 years ago
Matt Clay 029c991b0d ansible-test - Update outdated links in comments. 4 years ago
Dimitri Savineau a0f6747f07
ansible-test: fix typo in validate-modules (#76765)
The correct error strategy for the to_text method is surrogate_or_strict

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
4 years ago
Matt Clay 3a891827b7
ansible-test - Remove AIX provisioning support. (#76747)
* ansible-test - Remove AIX provisioning support.
* Remove `skip/aix` integration test aliases.
* Remove build for AIX binary module.
4 years ago
Matt Clay bab315b780
ansible-test - Add support for FreeBSD 12.3. (#76745)
* ansible-test - Add support for FreeBSD 12.3.
* Use OS package for yaml if available with CLoader.
* Make sure libyaml is available.
4 years ago
Matt Clay aa67ff2cf2
Remove shebangs from core Python modules. (#76664)
* Remove module shebangs.
* Relax shebang check in validate-modules.
* Add changelog fragment.
4 years ago
Matt Clay 21b9389a02 ansible-test - Remove invalid type hint. 4 years ago
Matt Clay 16cdac66fe ansible-test - Use ansible-core for docs links. 4 years ago
Matt Clay e56e47faa7 ansible-test - Fix import test for collections. 4 years ago
Matt Clay 82f59d4843
ansible-test - Hide ansible._vendor in import test (#76503) 4 years ago
Matt Clay 97ef1d031c Revert "ansible-test - Fix import test when vendoring."
This reverts commit 33a8d06332.
4 years ago
Matt Clay 41ee4a5b12 ansible-test - Fix traceback in validate-modules test. 4 years ago
Matt Clay 33a8d06332 ansible-test - Fix import test when vendoring. 4 years ago
Sloane Hertel b50f16db91
Update collection loader for Python 3.10 (#76225)
* Implement find_spec and exec_module to remove reliance on deprecated methods in the collection loader

ci_complete

* Move module execution to exec_module

Remove extra sys.modules handling

Use default module initialization by returning None from loader.create_module

Refactor

ci_complete

* Remove ansible-test's copy of the collection loader

ci_complete

* Fix metaclass for Python 2.x

ci_complete

* Fix Py2/Py3 syntax compatibility

* Refactor

ci_complete

* update collection_loader comments

ci_complete

* simplify find_module

ci_complete

* Fix Py2 compatibility - don't get loader from nonexistent spec

Remove unnecessary PY3 checking

* Refactor common code in load_module and exec_module

ci_complete

* tidy diff

ci_complete

* Include collection_loader in target paths for 'compile' sanity test

* add changelog

* Add "return None" instead of doing it implicitly

Remove get_filename

short-circuit exec_module if it's a redirect

ci_complete
4 years ago
Felix Fontein fe77bc9e3c
Validate some markup in documentation in validate-modules (#76262)
* Validate some markup in documentation.

* Add changelog fragment.

* Use urlparse instead of URL regex.

* Document new error code.

Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
4 years ago
Jordan Borean 9985b8a975
ansible-test pssa update and new rules (#76256) 4 years ago
Matt Clay 730194e2b6
ansible-test - Add RHEL 9 support. (#76217)
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Matt Clay cd64e0b070
ansible-test - Fix import test on old Pythons. (#76137) 4 years ago
Matt Clay cae7d2a671
Overhaul ansible-test integration tests. (#76111)
* Overhaul ansible-test integration tests.
* ansible-test - Fix import test pyyaml usage.
* ansible-test - Remove unused import.
* ansible-test - Fix traceback when pip is unavailable.
* ansible-test - Fix typo in port forwarding message.
* ansible-test - Fix controller logic in requirements install.
* Fix unit tests in ansible-test integration test.

Unit tests are now run for available Python versions which
provide `virtualenv` (Python 2.x) or `venv` (Python 3.x).
4 years ago
Matt Clay b9694ce4fb ansible-test - Remove duplicate code. 4 years ago