Commit Graph

875 Commits (56e39b3e7289d09803b09b392678f06b42e7e3bb)

Author SHA1 Message Date
Matt Clay 0d40423f1c
ansible-test - Fix collection loader import. (#76986)
* ansible-test - Fix collection loader import.

Resolves https://github.com/ansible/ansible/issues/76960
3 years ago
Matt Clay 0d5401d950 ansible-test - Remove unused code.
This was overlooked in a previous commit that removed FreeBSD 12.2 support.
3 years ago
Matt Clay 00a2b7788e ansible-test - Remove cap on cryptography version. 3 years ago
Matt Clay e27b94c467 ansible-test - Update alpine3 container to 3.3.0.
This updates the base image from 3.14.2 to 3.15.0, which includes support for installing binary wheels using pip.
3 years ago
Matt Clay 17dd9ff924 ansible-test - Update default containers to 5.5.0. 3 years ago
Matt Clay b568e0db6f ansible-test - Remove outdated remote support. 3 years ago
Matt Clay 7e64c4fe55 ansible-test - Update bootstrap URL to not use S3. 3 years ago
Matt Clay 995d7e4db3 ansible-test - Limit installed pyopenssl version. 3 years ago
Matt Clay fbb5d56bd2
ansible-test - Use relative paths in junit output. (#76871)
* ansible-test - Use relative paths in junit output.

Also fix a traceback in the junit callback during automatic fact gathering.

* ansible-test - Handle out-of-tree JUnit paths.
3 years ago
Matt Clay d19b506ce8
ansible-test - Clean up future boilerplate. (#76874)
* ansible-test - Clarify need for empty __init__.py
* ansible-test - Update code-smell boilerplate.
* Update code-smell boilerplate for core.
* Update future boilerplate test for ansible-test.

All ansible-test code (except for targets) and core-specific sanity tests now use the same boilerplate.

The test also checks for unwanted `__future__` and `metaclass` boilerplate.

* Relocate target tools to the correct directory.

Several tools used on target Python versions were incorrectly placed in the controller directory.
3 years ago
Matt Clay 18ed2c64e5 ansible-test - Add default for Windows remotes. 3 years ago
Matt Clay 26b43f425f ansible-test - Validate collection ns and name.
Resolves https://github.com/ansible/ansible/issues/62079
3 years ago
Matt Clay de5f60e374
ansible-test - Improve help for unsupported cwd. (#76866)
* ansible-test - Improve help for unsupported cwd.

* The `--help` option is now available when an unsupported cwd is in use.
* The `--help` output now shows the same instructions about cwd as would be shown in error messages if the cwd is unsupported.
* Add `--version` support to show the ansible-core version.
* The explanation about cwd usage has been improved to explain more clearly what is required.

Resolves https://github.com/ansible/ansible/issues/64523
Resolves https://github.com/ansible/ansible/issues/67551
3 years ago
Matt Clay 029c991b0d ansible-test - Update outdated links in comments. 3 years ago
Matt Clay e9ffcf3c85
ansible-test - Defer loading of completion entries. (#76852)
* ansible-test - Defer loading of completion entries.

This avoids a traceback when running ansible-test outside of a supported directory.
3 years ago
Dimitri Savineau a0f6747f07
ansible-test: fix typo in validate-modules (#76765)
The correct error strategy for the to_text method is surrogate_or_strict

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
3 years ago
Matt Clay 7fff408652 ansible-test - Remove azure support for Sherlock. 3 years ago
Matt Clay 3a891827b7
ansible-test - Remove AIX provisioning support. (#76747)
* ansible-test - Remove AIX provisioning support.
* Remove `skip/aix` integration test aliases.
* Remove build for AIX binary module.
3 years ago
Matt Clay bab315b780
ansible-test - Add support for FreeBSD 12.3. (#76745)
* ansible-test - Add support for FreeBSD 12.3.
* Use OS package for yaml if available with CLoader.
* Make sure libyaml is available.
3 years ago
Matt Clay b1ebc192b5 ansible-test - Update default containers. 3 years ago
Matt Clay d6588293fb ansible-test - Limit paramiko to < 2.9.0 3 years ago
Matt Clay b30b8fb79c ansible-test - Set pytest rootdir option. 3 years ago
Matt Clay eaeec8a65c ansible-test - Use `--forked` instead of `--boxed`
The `--boxed` option is deprecated.
3 years ago
Matt Clay aa67ff2cf2
Remove shebangs from core Python modules. (#76664)
* Remove module shebangs.
* Relax shebang check in validate-modules.
* Add changelog fragment.
3 years ago
Matt Clay 7e814dd4db ansible-test - Fix plugin loading.
This fixes a traceback when loading plugins that use dataclasses.
3 years ago
Matt Clay d2daa67b00 ansible-test - Fix type hints. 3 years ago
Matt Clay 4e70156d7e
ansible-test - Code cleanup. (#76540)
* Add missing typing imports.
* Remove unnecessary non-capturing group.
* Whitespace.
* Add type hints.
* Ignore PyCharm false positives.
* Remove Python 2.x plugin loading logic.
* Remove Python 2.x exception handling.
* Remove Python 2.x display logic.
* Add changelog.
3 years ago
Matt Clay 21b9389a02 ansible-test - Remove invalid type hint. 3 years ago
Matt Clay 470c89518c ansible-test - Fix delegation from install. 3 years ago
Matt Clay 16cdac66fe ansible-test - Use ansible-core for docs links. 3 years ago
Matt Clay e56e47faa7 ansible-test - Fix import test for collections. 3 years ago
Matt Clay 82f59d4843
ansible-test - Hide ansible._vendor in import test (#76503) 3 years ago
Matt Clay c45989c76f ansible-test - Remove RHEL 8.4 support. 3 years ago
Matt Clay 97ef1d031c Revert "ansible-test - Fix import test when vendoring."
This reverts commit 33a8d06332.
3 years ago
Matt Clay 41ee4a5b12 ansible-test - Fix traceback in validate-modules test. 3 years ago
Matt Clay 33a8d06332 ansible-test - Fix import test when vendoring. 3 years ago
Matt Clay d22bdcfdc7 ansible-test - Update NIOS container to 1.4.0. 3 years ago
Sloane Hertel b50f16db91
Update collection loader for Python 3.10 (#76225)
* Implement find_spec and exec_module to remove reliance on deprecated methods in the collection loader

ci_complete

* Move module execution to exec_module

Remove extra sys.modules handling

Use default module initialization by returning None from loader.create_module

Refactor

ci_complete

* Remove ansible-test's copy of the collection loader

ci_complete

* Fix metaclass for Python 2.x

ci_complete

* Fix Py2/Py3 syntax compatibility

* Refactor

ci_complete

* update collection_loader comments

ci_complete

* simplify find_module

ci_complete

* Fix Py2 compatibility - don't get loader from nonexistent spec

Remove unnecessary PY3 checking

* Refactor common code in load_module and exec_module

ci_complete

* tidy diff

ci_complete

* Include collection_loader in target paths for 'compile' sanity test

* add changelog

* Add "return None" instead of doing it implicitly

Remove get_filename

short-circuit exec_module if it's a redirect

ci_complete
3 years ago
Sam Doran d8aeffc8b7
Add macOS 12 to CI (#76328)
* Disable lookup_url test of macOS 12
3 years ago
Matt Clay c8a14c6be8 ansible-test - Update base and default containers. 3 years ago
Matt Clay bb63c97c16
ansible-test - Update import test and sanity requirements. (#76308)
* Add script to freeze sanity requirements.
* Declare sanity test requirements and freeze
* Use pinned requirements for import.plugin test.
* Expand scope of import test for ansible-core.
* Add ignores for galaxy import errors.
* Update test-constraints sanity test.
3 years ago
Matt Clay 21ac52435b
ansible-test - Add support for RHEL 8.5 instances. (#76305)
* ansible-test - Add support for RHEL 8.5 instances.
* Clear libdnf checksum cache from unsigned package in dnf integration test.

Co-authored-by: Matt Martz <matt@sivel.net>
3 years ago
Matt Clay a71ba817b0 ansible-test - Remove unused test constraint. 3 years ago
Matt Clay 5e3d469738 ansible-test - Remove near-EOL test containers.
- fedora33
- centos8
3 years ago
Matt Davis abac141122
fix support containers under podman (#76292)
* `podman inspect` falls back to a same-named image if the named container is not present; since eg `http-test-container` is both the name of the image and container, it wasn't working properly under podman in many instances. Switching to `docker|podman container inspect` limits the query to containers only for both podman and docker, allowing the support container detection/creation to work properly.
3 years ago
Felix Fontein fe77bc9e3c
Validate some markup in documentation in validate-modules (#76262)
* Validate some markup in documentation.

* Add changelog fragment.

* Use urlparse instead of URL regex.

* Document new error code.

Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
3 years ago
Jordan Borean 9985b8a975
ansible-test pssa update and new rules (#76256) 3 years ago
Matt Clay d7fbb209b4 ansible-test - Fix collection import test priming.
The bug was introduced in https://github.com/ansible/ansible/pull/76137
3 years ago
Matt Clay f42ffe16d2
ansible-test - Add Fedora 35 test container. (#76228)
* ansible-test - Add Fedora 35 test container.
* Added Fedora 35 to CI and removed Fedora 33 from CI.
* Fix dnf integration test.
* Fix connection_paramiko_ssh test.
3 years ago
Matt Clay 730194e2b6
ansible-test - Add RHEL 9 support. (#76217)
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
3 years ago
Andrew Klychkov e50ad6f118
ansible-test: Set max number of open files in containers to 10240 (#75498) 3 years ago
Matt Clay 1ab2443e2d ansible-test - Update default containers. 3 years ago
Matt Clay cd64e0b070
ansible-test - Fix import test on old Pythons. (#76137) 3 years ago
Matt Clay cae7d2a671
Overhaul ansible-test integration tests. (#76111)
* Overhaul ansible-test integration tests.
* ansible-test - Fix import test pyyaml usage.
* ansible-test - Remove unused import.
* ansible-test - Fix traceback when pip is unavailable.
* ansible-test - Fix typo in port forwarding message.
* ansible-test - Fix controller logic in requirements install.
* Fix unit tests in ansible-test integration test.

Unit tests are now run for available Python versions which
provide `virtualenv` (Python 2.x) or `venv` (Python 3.x).
3 years ago
Matt Clay b9694ce4fb ansible-test - Remove duplicate code. 3 years ago
Matt Clay b4cbe1adcf ansible-test - Fix Python real prefix detection. 3 years ago
Martin Krizek 7621784b94
Require Jinja2 3.0.0 (#75881)
* Require Jinja2 3.0.0

ci_complete

* Fix sanity

* Remove Jinja min/max tests

* ansible-test changes

* ci_complete

* More cleanup

ci_complete

* Revert _count_newlines_from_end :( and other stuff

* Fix sanity

* It's using host_vars ...

* Unused import

* Remove overridden groupby filter

* environmentfilter -> pass_environment

* Explain preserve_trailing_newlines

* Add changelog

* ci_complete

* Deprecated ANSIBLE_JINJA2_NATIVE_WARNING

* native_helpers.py cleanup

* More cleanup in the find intgration test
3 years ago
Matt Martz 66a83314b9
Modernize install (#76021)
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
Felix Fontein 227f4398c2
Fix choices for return values. (#76070) 3 years ago
Matt Clay d099e19796 ansible-test - Move constants to remove symlink. 3 years ago
Felix Fontein 7a28246248
validate_modules: allow 'choices' for return values (#76009)
* Allow 'choices' for return values.
3 years ago
Matt Clay 5cb1210f82 ansible-test - Update base and default containers.
- The containers are now based on Ubuntu 20.04 instead of Ubuntu 18.04.
- The default Python version used for the containers is now Python 3.10.
- The included version of PowerShell was upgraded to version 7.1.4.
- The included version of shellcheck was upgraded to version 0.7.0.

ci_complete
3 years ago
Matt Clay 801eb73ccb
ansible-test - Update Python constraints. (#75959)
* Remove obsolete constraints.
* Simplify constraints.
* Add changelog fragment.
3 years ago
Jordan Borean 7e19957afa
ansible-test pslint - fix warning with nested objects (#75792)
* ansible-test pslint - fix warning with nested objects
3 years ago
Matt Clay 8cef40f839 ansible-test - Fix inventory path. 3 years ago
Matt Clay 2f531d73dd ansible-test - Update base and default containers.
The containers now include Python 3.10.0 instead of Python 3.10.0rc2.
3 years ago
Matt Clay f4d4391a1f ansible-test - Update distro test containers. 3 years ago
Matt Clay 1932f0008b
ansible-core - Remove support for Python 2.6. (#75853) 3 years ago
Matt Clay 099d80829f ansible-test - Fix inventory generation. 3 years ago
Matt Clay 68ab92e22a ansible-test - Remove support for Python 2.6.
Deferring removal of obsolete constraints until container updates are required.
3 years ago
Matt Clay 0d5a9f2138 ansible-test - More PEP 484 type hints. 3 years ago
Matt Clay e66e2e25c8 ansible-test - Update default containers. 3 years ago
Matt Clay 6a5d6896fd ansible-test - Warn about stale sanity venvs. 3 years ago
Matt Clay b70077364a ansible-test - More PEP 484 type hints. 3 years ago
Matt Clay 58b03be417
ansible-test - Code cleanup. (#75774)
* ansible-test - Remove unused code.
* ansible-test - More PEP 484 type hints.
3 years ago
Matt Clay 6c133da45e
ansible-test - Fix code coverage reporting. (#75771) 3 years ago
Jordan Borean 43e1cba8c1 Fix up validate-modules sanity check for PowerShell - ci_complete 3 years ago
Matt Clay b51e62f6e5 ansible-test - Add minimum constraints for pytest.
Use `--strict` for `pytest` on Python 2.6 since `--strict-markers` is not available.
3 years ago
Matt Clay e3fd9b0769 ansible-test - Add `--prime-containers` option.
Resolves https://github.com/ansible/ansible/issues/75320

The option `--prime-containers` was chosen over `--docker-pull-only` to match the recently added `--prime-venvs` option for sanity tests.
It would also fit well with a future `--prime-requirements` option for pre-installing requirements for unit and integration tests.
3 years ago
Matt Clay 28a094c5de ansible-test - Remove unused constraints.
ci_complete
3 years ago
Felix Fontein 67ca455829
attributes: support extra value 'not applicable' for support (#75751)
* Support two extra values 'unknown' and 'not applicable' for support.

* Simplify to only one new state: 'n/a'

* Adhere to the IBM style guide.
3 years ago
Brian Coca c5d8dc0e11
Add attributes 2/x (#75619)
* Added attribute information to more modules

 rearranged the shared data
 platform revamped
 use 'details' and avoid overriding description

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_core.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_core.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_core.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_core.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_core.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/fail.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/debug.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/command.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/assert.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_core.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/gather_facts.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/include_vars.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/meta.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/modules/meta.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_common_attributes.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_common_attributes.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/plugins/doc_fragments/action_common_attributes.py

Co-authored-by: Felix Fontein <felix@fontein.de>

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
3 years ago
Matt Clay 46c7b24188
ansible-test - Update distro test containers. (#75762) 3 years ago
Matt Clay 5cb54e8c58
ansible-test - Create injector scripts at runtime. (#75761)
* ansible-test - Create injector scripts at runtime.
* Set bootstrap.sh shebang at runtime.
* Remove shebang and execute bit from importer.
* Update shebang sanity test.
* Preserve line numbers.
3 years ago
Matt Clay 7a76130502 ansible-test - Update default containers.
ci_complete
3 years ago
Matt Martz 724800cd3f
Python 3.8 Controller Minimum (#74013) 3 years ago
Matt Clay 4652d6ee43 ansible-test - Remove deprecated environments. 3 years ago
Matt Clay 94410789d1 ansible-test - Add `--prime-venvs` option. 3 years ago
Matt Clay 178a67fd40
ansible-test - Add support for windows/2022 (#75743)
ci_complete
3 years ago
Matt Clay 4ea8d9a782
ansible-test - split controller/target testing (#75605) 3 years ago
Matt Clay 99a79e1969
ansible-test - Move code from _data to _util. (#75495)
* Update paths to match relocated files.
* Update ansible-test symlink paths.
* Update path classification.
* Update MANIFEST.in
* Update sanity test context paths.
* Update sanity ignores.
* Update shebang sanity test.
* Update configure-remoting-ps1 sanity test.
* Update BOTMETA.yml to reflect new paths.
* Update paths in collection loader comments.
* Update Makefile for ansible-test.
* Update docs.
* Add changelog fragment.
3 years ago
Matt Clay 2b463ef197
ansible-test - Upgrade pylint to 2.9.3. (#75480)
* ansible-test - Upgrade `pylint` to 2.9.3.
* Update pylint ignores due to rule name change.
* Disable pylint deprecated-class for compat code.
* Add pylint ignores for test support content.
* Add ignores for arguments-renamed in lib/ansible/
* Add pylint ignores for collection_loader.
* ansible-test - Ignore deprecations in legacy collection loader.
* ansible-test - Suppress pylint consider-using-with
* ansible-test - Suppress pylint false positive.
* ansible-test - Suppress pylint consider-using-with.
* ansible-test - Suppress pylint deprecated-module
* Disable some of the new pylint suggestions.
* Remove unnecessary six usage from string_format pylint plugin.
* Remove obsolete ignore entry.
3 years ago
Matt Clay 7450e87615
Fix up string splitting to prepare for pylint update. (#75479)
* Use rsplit with maxsplit in BaseCacheModule.
* Use maxsplit for accessing first item.
* ansible-test - More efficient splitting.
3 years ago
Matt Clay ca2d2c5f38
ansible-test - Cleanup to prepare for pylint update. (#75469)
* ansible-test - Fix use of abstractproperty
* ansible-test - Use dict.items() where possible.
* ansible-test - Remove unused code.
* ansible-test - Cleanup issues reported by pylint.
* ansible-test - Use dict.items() where possible.
* ansible-test - Use generator.
3 years ago
Sloane Hertel cea4886c7c
Update test container versions (#75447)
* Update test container versions to use the most recent which contain Python 3.10.0 RC1
3 years ago
Felix Fontein fae299708f
plugin attributes: make validate-modules strict again, improve version_added support (#74602)
* Add basic schema for attributes.

* Support attributes for version_added_collection adding mechanism.

* Add changelog fragment.

* Support new attribute properties from #74331.

* Require additional properties strictly.
3 years ago
Felix Fontein b8ebf32d85
Add sanity tests which ensures that _info and _facts modules set supports_check_mode=True. (#75324) 3 years ago
Matt Martz 2455d82c14
Make sure self doesn't end up in fake.args (#75403) 3 years ago
Sam Doran 1d2af0b89e
ansible-test - allow for the same listening port on multiple interfaces (#75387) 3 years ago
Sam Doran d8dcfe737a
ansible-test - add ssh debugging logging (#75374) 3 years ago
Sloane Hertel 83039a38ad
ansible-test - add RHEL 8.4 as a remote (#75362) 3 years ago
Sloane Hertel f6b27f6e35
Update test container versions (#75359)
* Update test container versions for default-test-container and ansible-core-test-container
3 years ago
Felix Fontein 7726b70d9a
Improve handling of AnsibleModule arguments in validate-modules (#75332)
* Make sure AnsibleModule positional arguments are validated.

* Extract ANSIBLE_MODULE_CONSTURCTOR_ARGS with inspection.

* Remove no longer necessary return value.

* Fix PR #.

* argument_spec might not have been specified, as in community.general.xenserver_facts.

* Fix typo.
3 years ago
Sergey 40ca87a963
Add packaging to requirement of ansible-test (#75356)
* Add packaging to requirement of ansible-test

Fix #75353

After requires_ansible field was added as mandatory to runtime.yml
file, ansible-test fails to check this field if it doesn't have
packaging module.

[1] https://github.com/ansible/galaxy-importer/pull/124
3 years ago
Sloane Hertel 3b861abce1
add action_groups support to collections (#74039)
* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play

* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list

* Add validation for action_group metadata and a toggle to disable the warnings

* Handle groups recursively referring to each other

* Remove special-casing for non-fqcn actions in module_defaults groups

* Error for actions and groups in module_defaults that can't be resolved

* Error for fully templated module_defaults

* Add integration tests for action_groups

* Changelog
3 years ago
Martin Krizek 8ab418f41b
Disable jinja2_native warning in CI (#75214) 3 years ago
Felix Fontein 67bc49e001
Restrict packaging to < 21.0 for Python < 3.6. (#75186) 3 years ago
Gonéri Le Bouder 5b8fb4dcd3 ansible-test: exposes tiny_prefix variable
ansible-test aws provider now creates and exposes a new tiny_prefix
variable to provide a shorter prefix for the AWS tests.
4 years ago
Matt Clay adc9e4a023 Updated ansible-test azure plugin help message. 4 years ago
Matt Clay 397d3763ea ansible-test - Do not downgrade cryptography. 4 years ago
Abhijeet Kasurde 870f943365
docs: Update Python 2 doc links (#74633)
* docs: Update Python 2 doc links

Update links from Python 2 to Python 3

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

* use docs.python.com/3/ everywhere, except onethat should remain 2.6

* refer to python 3 in module docs and comments

* format two python docs links as list

* updates links in unwanted.py test file

* per matt clay, this should link to python 2

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
4 years ago
Sam Doran c0cb353ce1
Remove PyCrypto (#74699)
* Remove PyCrypto from setup.py and packaging script
* Remove mention of pycrpto from installation docs
* Remove PyCrypto from vault
* Remove pycryto constraint and unit test requirement
* Remove PyCrypto tests from unit tests
* Add docs and fix warning message
* Remove section about cryptography library in Ansible Vault docs
4 years ago
Matt Clay 787f722d94 Fix combined coverage file detection.
Resolves an issue introduced with https://github.com/ansible/ansible/pull/74719
4 years ago
Matt Clay ce04056797 Expand delegation options for coverage commands. 4 years ago
Jordan Borean e875e91363
ansible-test - use pwsh to generate stubs (#74271) 4 years ago
Mark Mercado 1906d75907
Adding DigitalOcean cloud support to ansible-test (#74222) 4 years ago
Matt Martz d6e28e6859
Update vendored copy of `six` to 1.16.0 (#74680)
* Update to six 1.16.0

* Address linting issues

* Remove six find_spex/exec_module warning filters

* Remove unnecessary comment about Py2.6, 2.13 will not support Py2.6, and we're bumping this for 2.12

* ci_complete

* Add changelog fragment
4 years ago
Sam Doran f99d024851
CI and compat fixes for Jinja2 >= 3.0 (#74666)
* Add constraint for MarkupSafe

MarkupSafe >= 2.0.0 requires Python >= 3.6.0. Add a constraint for older Python versions
and fix the `groupby_filter` test.

* Fix template_jinja2_latest test.

* patch filter decorators on newer Jinja2

* Jinja2 >= 3.0 renames several filter decorators used by Ansible itself, as well as by filters in collections. This patch ensures that the old names are usable within Ansible and by collections without warnings or errors.

* Ignore docs-build issues.

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Matt Martz 8d1cf7f266
Vendor `distutils.version` (#74644)
* Vendor distutils.version

* Fix import order. ci_complete

* remove distutils warning filter

* Don't remove warnings filter from importer

* ci_complete

* Add pylint config for preventing distutils.version

* Add changelog fragment
4 years ago
Matt Clay 7b0c4845d9 Remove PyYAML warning filter in import test.
The filter is obsolete now that PyYAML imports go through our compat layer.
4 years ago
Matt Clay 51fd05e76b Add support for testing with Python 3.10. 4 years ago
Matt Clay 2e97240c67 Ignore more Python 3.10 warnings in import test. 4 years ago
Jordan Borean a277644301
Add python 3.10 constraint for pyspnego (#74612) 4 years ago
Matt Clay e6d7aecbe4
Reorganize more ansible-test code. (#74611)
* Split out shell command.
* Relocate ansible-test integration code.
4 years ago
Matt Clay 065fc3ca17 Remove check for legacy core and extras dirs. 4 years ago
Matt Clay 5487e9e4fd Update runtime-metadata sanity test for Python 3.10. 4 years ago
Matt Clay cbac26e500 Update import sanity test for Python 3.10. 4 years ago
Matt Clay 2c3a3bdd9c Update pylint sanity test to work on Python 3.10. 4 years ago
Matt Clay c2e15f45a6 Update ansible-test compile sanity test.
The test has been rewritten to improve error handling and add support for Python 3.10.
4 years ago
Matt Clay 07fb6f7547 Fix imports for the previous commit.
Also add a changelog entry.
4 years ago
Matt Clay 92c40f6365 Relocate change classification files. 4 years ago
Matt Clay 5dfc7ee078 Remove Shippable CI provider from ansible-test. 4 years ago
Matt Clay df2ae06d46 Fix ansible-test imports and paths after refactor.
Also include a changelog entry covering this commit and the previous.
4 years ago
Matt Clay bb3904d299 Reorganize command implementations.
This commit only renames existing files.

Changes to adapt imports will occur in a follow-up commit.
4 years ago
Matt Clay fb0d5609cb
Code cleanup in ansible-test. (#74529)
* Fix get_available_python_versions calls.
* Make run_playbook vars optional.
* Use ansible_pipelining in inventory.
* Fix type hint.
* Fix order of conditional evaluation.
* Remove unused ibmi platform.
* Add changelog fragment.
4 years ago
Rick Elrod d8c20a73a4
[ansible-test] Add Fedora 34, update Fedora 33 (#74506)
Change:
- Added Fedora 34 container image to docker.txt
- Dropped Fedora 32 from CI
- Added Fedora 34 to CI
- Updated Fedora 32 and 33 containers for ssh-related fixes
- Move cron fix from 698eae3f3d into
  cron_setup and make it more generic; it can affect modern distros too.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay a30c55f68a Remove deprecated `common.removed` module_util.
Tests have been updated to reflect its removal as well.
4 years ago
Matt Clay baa371e7b5 Add comments to keep collection loader in sync. 4 years ago
Matt Clay 877e5b3dfe Fix ansible-test AZP build lookup. 4 years ago
Matt Clay c4e76a7f80 Add collection config support to ansible-test. 4 years ago
Matt Clay 8223dfccbe ansible-test - Improve __main__ checks. 4 years ago
Matt Clay cb7f4f1971 Fix pypi-test-container port conflict. 4 years ago
Matt Clay ecc5a53288
Update default containers to 3.4.0. (#74415)
* Update default containers to 3.4.0.

The 3.4.0 containers use Python 3.6 (the system Python) for `/usr/bin/python3`.

Python 3.9 continues to be the default Python version selected by `ansible-test` for these containers.

* Fix shebang on build-ansible.py.

Using `python` instead of `python3` allows `ansible-test` python interception and requirements install to function.
4 years ago
Matt Clay 6cfe36fa1f
Set ansible-test min controller Python to 3.8. (#74395)
- Version neutral sanity tests now require Python 3.8 or later.
- Unit tests for controller-only code now require Python 3.8 or later.
4 years ago
Matt Clay f541c0af23
Update ansible-test default test container. (#74393)
* Update ansible-test default test container.
* Switch generic Python tests from 3.6 to 3.9.
4 years ago
Paul Arthur 7b5dad2321 validate-modules: don't error on valid Ansible YAML in EXAMPLES 4 years ago
Matt Clay 14ff5e213c Fix ansible-test docker container IP detection. 4 years ago
Matt Martz bf7d4ce260
Abstract away libyaml details into yaml utils (#72607)
* Add yaml utils file and use it

* Linting fix and missing import

* Abstract a few more details

* Parser imports

* Don't use CSafeDumper in AnsibleDumper

* Move and use convert_yaml_objects_to_native when libyaml is present

* yaml_load and yaml_dump, instead of safe_*

* re-use HAS_LIBYAML from utils.yaml

* add changelog fragment

* Address recent changes

* Use representer instead of recursive type converter

* Restore needed import

* move yaml utils to module_utils

* Properly guard imports

* Update from_yaml(_all)? to handle text wrappers with CSafeLoader

* Use yaml utils for legacy_collection_loader

* Add HAS_YAML, and ignore pylint issue

* oops

* GPL->BSD
4 years ago
Brian Coca 6418f368e3
Fix pb subs collapse (#74301)
* fix subdir parsing for plays

  fixes #74283

  Co-authored-by: Nikolaos Kakouros <nkak@kth.se>
4 years ago
Rick Elrod 738132da95
[ansible-test] add freebsd/13.0 remote (#74328)
Change:
- FreeBSD 13.0 was released on the 13th, add it to ansible-test.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay e6af2d6827 Fix ansible-test coverage traceback when no data.
If no coverage directory exists, ansible-test coverage would traceback.

Now it silently continues just as if the directory was present but empty.
4 years ago
Brian Coca 2296da95a2
fix module schema (#74295)
* fix module schema

  not out of date hardcoded list anymore, uses 'current + 4' to
  set valid deprecation targets

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay 7786dbbdc1 Split out module/module_utils unit test execution.
- Unit tests for `modules` and `module_utils` are now limited to importing only `ansible.module_utils` from the `ansible` module.
- Unit tests other than `modules` and `module_utils` are now run only on Python versions supported by the controller (Python 3.8+).
- Unit tests are now run in separate contexts (`controller`, `modules`, `module_utils`), each using separate invocations of `pytest`.
4 years ago
Matt Clay 277a06f641
Clean up ansible-test cloud plugins. (#74322)
- Improve code reuse.
- Add missing type hints, fix existing ones and convert them to PEP 484 style.
- Add missing imports and clean up existing ones.
- Add missing docstrings and clean up existing ones.
4 years ago
Brian Coca 07939b04f3
[2.12] 'attributes' (#73707)
* wip 'attributes'

* added version added tests

* syntzx

* not bile

* correztlys merges

* moved desc to frag

* simpler as dict

* unused

* clog

* Update lib/ansible/utils/plugin_docs.py

Co-authored-by: Jacob Floyd <cognifloyd@gmail.com>

* unnoted

* added action plugins

* longer list

* add sttri schema

* huh?

* itsdict

* dictit

* yolo

* gnore for now

* moar attribs

* allow extras

* positive

* added loop, documented 'imports'

* support is now none/partial/full

* import_playbook is outside host loop

Co-authored-by: Jacob Floyd <cognifloyd@gmail.com>
4 years ago
Brian Coca 0a7670d1f7
deprecated include (#71262)
* deprecated include

  Update lib/ansible/modules/_include.py
  updated version numbers in schema check (real fix in separate PR)

Co-authored-by: flowerysong <junk+github@flowerysong.com>
Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Rick Elrod f38d03f6fb [ansible-test] nuke virtualenv-isolated.sh
Change:
- This isn't used in our tests, so the suggestion was to just remove
  it.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay b752d07163 Overhaul ansible-test container management.
This brings ansible-test closer to being able to support split controller/remote testing.
4 years ago
Matt Clay becf941673 Add PyPI proxy container for tests on Python 2.6. 4 years ago
Sam Doran 4df9ea86c2 ansible-test - change diff mathching regexp so prefixes are optional
The `a/` and `b/` prefixes can be disabled in the `git diff` output by setting
`diff.noprefix` to `true`. The output is still a valid diff, but `ansible-test` would
raise an exception since without the prefixes, it thought the diff line was invalid.
4 years ago
Sam Doran 7b79b69369
ansible-test - ensure trailing separator is added for connection unit test target (#74176)
For connection units tests, if the `__init__.py` file is modified, the trailing separator was
not added, making the target invalid.
4 years ago
Matt Clay 459ea5a4a4
Update test containers in ansible-test. (#74166)
* Update distro test containers to version 2.0.2.

* Update the default test containers.
4 years ago
Matt Clay f436058fad Add ansible-test constraint for decorator. 4 years ago
Matt Clay 4e2da0f894 Add ansible-test six constraint for Python 2.6. 4 years ago
Matt Clay b75844af45 Fix ansible-test coverage exporting.
ci_complete
ci_coverage
4 years ago
Baptiste Mille-Mathias 3e63eb8418 fix typo 4 years ago
Paul Belanger 9ec4e08534
Don't match passive_interface for validate-modules (#73880)
This is a follow up to:

  https://github.com/ansible/ansible/pull/73508

To avoid adding no_log statements to passive_interface args.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
4 years ago
Felix Fontein 7c0af58449
validate-modules no_log test: extend search range, add more known false-positives (#73882)
* validate-modules no_log test: extend search range, add more known false-positives.

* Mark false-positives.
4 years ago
Mark Chappell ed18fcac3b
Ensure unit test paths for connection and inventory plugins are based on the context (#73877) 4 years ago
Felix Fontein b2015c98e2
validate-modules: make sure that options that potentially contain secret data have no_log set (#73508)
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.

Co-authored-by: John Barker <john@johnrbarker.com>
4 years ago
Matt Clay 15064c7a42 Update `resource_prefix` syntax for ansible-test. 4 years ago
Matt Clay f40dde44f4 Remove old ansible-test windows completion entries. 4 years ago
Matt Clay b6add939ff Remove outdated --docker and --remote entries. 4 years ago
Alexei Znamensky 920b68f5f2
Fixed/improved regular expresssion for collection names (#73577)
* added changelog fragment
* added a couple of tests to coll name validation
4 years ago
Felix Fontein b6566d26df
pylint sanity test: stop ignoring 'used-before-assignment' (#73639)
* Remove pylint ignore 'used-before-assignment'.

* Adjust changelog fragment.
4 years ago
Felix Fontein 29aef842d7
ansible-test: make sure tests are also run for the new plugin types (#73599)
* Make sure tests are also run for the new plugin types.
* New plugin types are not documentable.
4 years ago
Matt Clay d092356fc5 Fix ansible-test handling of egg-info.
Resolves https://github.com/ansible/ansible/issues/67990
4 years ago
Felix Fontein 1f3a90270b
Import sanity test for plugins (#72497) 4 years ago
Matt Clay 30a4ef4414 Update ansible-test default test containers. 4 years ago
Gonéri Le Bouder 0a8d5c0983 ansible-test: yamllint, check the assigment
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.

See: https://github.com/ansible/ansible/pull/73322
4 years ago
Matt Clay f533d46572
Fix ansible-test constraints issues. (#73578)
* Silence Python 3.5 EOL notice in pip.
* Skip import test on compat/selinux.py utils.
* Improve Python version skip warning.
* Use Python 3.6 as minimum Python for sanity tests.
* Improve min Python handling for code-smell tests.
* Overhaul test-constraints sanity test.
* Merge sanity test constraints with requirements.
* Remove legacy content specific constraints.
* Add changelog fragment.
4 years ago
Matt Clay 8a175f59c9
Upgrade pylint used by ansible-test. (#70155)
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
4 years ago
Matt Clay 1a2da990a4
Temporary fix for cryptography issues. (#73530) 4 years ago
Felix Fontein 82cdd7e735 Bump antsibull-changelog version for changelog sanity test. 4 years ago
Florian Heiderich 8f887b7adb
add self-signed https endpoint for ansible-test (#73463)
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
4 years ago
Gonéri Le Bouder 7cf80f50d1
validate_modules: fails with .id attribute not found (#73322)
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
4 years ago
Matt Clay a9b5bebab3
Overhaul ansible-test SSH key management. (#73451)
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
4 years ago
Rick Elrod bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) (#73446)
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 81cd8e46f2 Update ansible-test distro containers to 1.29.0. 4 years ago
Matt Clay c7cb944315 Always use python exec wrapper in ansible-test. 4 years ago
Matt Clay 902c402b6f Update ansible-test default containers. 4 years ago
Matt Clay ded2d0b90e Use `sh` for ansible-test raw remote shell. 4 years ago
Matt Clay b1344a1847 Avoid sphinx usage in changelog sanity test. 4 years ago
Matt Clay 418b9b781d No longer warn about missing pip/python pair.
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.

In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
4 years ago
Matt Clay 672941fb0d
Add test support for Python 3.8 on RHEL and FreeBSD. (#73393)
* Support Python 3.8 on RHEL 8.2+ in ansible-test.

* Support Python 3.8 on FreeBSD in ansible-test.

* Use libyaml with PyYAML on FreeBSD.
4 years ago
Matt Clay 1b157ef42f
Limit rstcheck sanity test to ansible-core. (#73391) 4 years ago
Sviatoslav Sydorenko 595413d113
Replace the inhouse collection dependency resolver with `resolvelib`
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
4 years ago
Matt Clay 13d08d232c Rename ansible-test functions to match constants. 4 years ago
Matt Clay 73fadc5e97
Clean up code in ansible-test. (#73379)
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
4 years ago
Matt Clay fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 4 years ago
Matt Clay 484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 4 years ago
Rick Elrod 1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test (#69161)
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes #69203

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 4357a78130
[ansible-test] First attempt at freebsd/11.4 (#72655)
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes #48782

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Alexander Sowitzki fe792fdcd2
Vanquish rare container bootstrap failure (#73288)
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Sandra McCann 88763c7347
update rstcheck and sphinx versions in contraints (#73176)
* update rstceck and sphinx versions in contraints
* fix test error
4 years ago
Matt Clay 4164cb26f0 Remove obsolete `tower` plugin from ansible-test. 4 years ago
Matt Clay 21f1811ddf
Cleanup provisioning code in ansible-test. (#73207)
* Remove unused code in ansible-test.

* Remove obsolete endpoint logic from ansible-test.

* Remove obsolete region selection in ansible-test.

* Remove obsolete port logic in ansible-test.

* Clean up ansible-test remote providers.
4 years ago
Sam Doran a7e834071c
Add macOS 11 to CI (#72622)
* Fix connection_paramiko_ssh test for macOS 11
* Update Azure Pipelines config
* Add changelog
4 years ago
Matt Clay 05ba49cbba ansible-test - No virtualenv install on Python 3
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.

On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
4 years ago
Matt Clay 77942acefc Refactor ansible-test cryptography install code. 4 years ago
Rick Elrod b96732fa6b [ansible-test] Add Fedora 33 support and drop F31
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Sam Doooran 5ecfb19cad
arg_spec - move validator lookup method to a function (#72667)
* arg_spec - move type checking lookup method to a function

* Change get_wanted_type name and behavior

Change the name to get_validator to bette describe what it is doing.

Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.

* Add param to check_type_str()

Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.

* Remove private attribute mapping types to validator

Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.

Update tests that were referencing the private attribute.

* Use private method for 'str' type

To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.

Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.

Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.

* Add changelog

* Change function name to better reflect its... function

* Change dict name to better reflect its contents

CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS

* Fix changelog
4 years ago
Sam Doooran 8e022ef00a
import_playbook - change additional params to deprecation (#72987)
* import_playbook - change additional params to deprecation

I incorrectly recommended this be set as a warning when it should have been a deprecation.

* Fix deprecation sanity test to not required a collection name when not inside a collection
4 years ago
Sam Doooran 850a77f639
ansible-test - prefer venv over virtualenv on Python 3 (#73000)
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).

On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
4 years ago
Matt Clay 37d09f2488
Update ansible-test pylint Python support. (#72972)
* Update ansible-test pylint Python support.

* Python 3.8 is now officially supported.
* Python 3.9 is now skipped with a warning.
4 years ago
Felix Fontein 0ba96d2be8
Compare removal and deprecation dates and versions in runtime metadata against current version and today (#72625)
* Check deprecation and removal versions w.r.t. current version.

* Check removal dates from tombstones w.r.t. today.

* Add changelog.
4 years ago
Matt Clay 36ab3d1189
Fix cryptography constraints in ansible-test. (#72914) 4 years ago
Matt Martz 6bc1e9f5dd
Address additional ansible_core rename issues (#72906)
* Update __requires__ to reference the correct ansible_core package name

* ansible-test updates to handle the correct egg_info for ansible_core
4 years ago
Felix Fontein 5226ac5778
The implicit default for type=bool options is not 'false', but 'none' (#72699)
* The implicit default for type=bool options is not 'false', but 'none'.

* Fix modules, resp. add ignore.txt entries.
4 years ago
Felix Fontein f94ba68d8f
ansible-test sanity: fix UnicodeDecodeError for `--python 2.7 --docker` (#72623)
* Store target paths as unicode.

* Add changelog fragment.
4 years ago
Matt Clay 221c50b57c Use AZP config for integration-aliases test.
No changelog entry since this test is limited to the ansible/ansible repo.
4 years ago
Matt Clay 08842cd6bb
Fix Azure Pipelines change detection. (#72824) 4 years ago
Rick Elrod 3b88058b72 Bump centos6 image for pinned junit-xml dep
Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod a59825128d [centos6] update container for EOL
Change:
- Reference:
  https://lists.centos.org/pipermail/centos-devel/2020-December/056208.html
- Bump centos6 container to 1.25.0

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Paul Belanger a95213d2f5
Switch to hashlib.sha256() for ansible-test (#72411)
When FIPs mode is enable on centos-8, we are not able to load md5
functions.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
4 years ago
Sam Doran 1db9588279
ansible-test - add cryptography constraint for cffi (#72761)
The recently released version of cffi fails to install on systems with an older version of gcc. In
our case, this in the CentOS 6 test image. There is a fix but it has not yet been released.

https://foss.heptapod.net/pypy/cffi/-/issues/480
4 years ago
Jordan Borean fb092a82a1
ansible-galaxy - source deps from all servers and not just parent (#72576)
* ansible-galaxy - source deps from all servers and not just parent

* Added integration tests for this scenario
4 years ago
Brian Coca 13bc35e13c
actual min changes needed for rhel8 (#72477)
* actual min changes needed for rhel8

 - specifiy releaserver as dnf/vars is ignored
4 years ago
Rick Elrod 6894ae7d1d
Rename to ansible-core (#72594)
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base

Test Plan:
- ci_complete
4 years ago
Jordan Borean 03a395cba4
Rollback pulp container to stabilise CI (#72580) 4 years ago
Brian Coca 48c08f410c
allow any type of unsafe data (#72547)
* allow any type of unsafe data

  dont limit to strings
4 years ago
Jordan Borean d451433e5d
Add RHEL 7.9 - ci_complete (#72558) 4 years ago
Matt Clay fa2be89cd4 Add coverage exporting to ansible-test
A new `--export` option for `ansible-test coverage combine` allows multi-step aggregation of code coverage for CI pipelines.
4 years ago
Matt Clay f022dedd0a Fix container discovery for the acme test plugin. 4 years ago
Rick Elrod e7bf0696ef
[ansible-test] use newer container images (#72126)
Change:
- Bump default, ansible-base, distro containers
- We do NOT add fedora33 yet, because it doesn't work right on Shippable
  due to an old kernel. This will be added post-AZP.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 3b379f372e [ansible-test] first attempt at freebsd 12.2
Change:
- Add freebsd/12.2 remote target

Test Plan:
- ci_complete

Tickets:
- Fixes #72366

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 96ad5b799e
Emit warning when running on the controller with a Python older than 3.8 (#72467)
* Emit warning when running on the controller with a Python older than 3.8

* Add spaces

Co-authored-by: Matt Clay <matt@mystile.com>

* and more spaces

Co-authored-by: Matt Clay <matt@mystile.com>

* s/Py/Python/

Co-authored-by: Matt Clay <matt@mystile.com>

* Add note to Control node requirements about Py3.8 requirement

* Add collection_name to deprecated call

* more spaces

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Clarify that we are only packaging for py3.8+

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Jordan Borean 1eae2bb890
Update Galaxy NG version used in CI (#72469) 4 years ago
Matt Clay 79021df2e4 Restore ansible-test coverage config.
Reverts a portion of the changes made in https://github.com/ansible/ansible/pull/70028
4 years ago
Felix Fontein 6b30efa454 Fix fallout from #70028. 4 years ago
Felix Fontein 64a809d2b6
Improve ansible-test classifications for collections (#72353) 4 years ago
Jordan Borean 809d5fc398
ansible-test - skip installing PowerShell sanity test reqs if they are already installed (#72423) 4 years ago
Brian Coca 4b673484f0
rethink wording (#70028)
* rethink wording
* removed unrequired requirement
* fix tests
* fixed versions
Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
Jordan Borean 4856ab0e68
Galaxy server update (#72286)
* Update galaxy server image used in CI

* Fix port and migration issue

* Add delete step, still need to deal with pagination

* Make cleanup more efficient

* Remove testing code
4 years ago
Felix Fontein 233e7beb5b
Fix processing of add_file_common_args=True when argument_spec is not specified as kwarg. (#72334) 4 years ago
Rick Elrod 2293b327c0
[ansible-test] Limit cryptography to <3.2 (#72342)
Change:
- Cryptography 3.2 drops support for OpenSSL 1.0.2. Some of our CI
  infrastructure still uses this version (FreeBSD, namely). For now,
  just add a constraint to use old cryptography.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 3c2e8b99be
Fix ansible-test handling of user-defined docker networks. (#72256)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry.
4 years ago
Matt Clay 2ef4b7e07e Fix ansible-test Azure Pipelines container auth. 4 years ago
Jordan Borean caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
4 years ago
Florian Heiderich e673b8e4ec
fix typo (foward -> forward) (#72182) 4 years ago
Matt Clay 5f76bd2af7 Support collection constraints in ansible-test.
This allows collections to specify requirements and constraints for packages that ansible-test has requirements or constraints for.
4 years ago
Rick Elrod 0332046699
[ansible-test] attempt to work around podman (#72096)
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.

Test Plan:
- Tested with podman 2.0.6 in Fedora 31.

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Jordan Borean 672df07a33
Bump httptester version (#72036) 4 years ago
Felix Fontein a077bca5d5
ansible-test: improve version number validation, validate some semantic versioning properties (#71679)
* Validate removal versions.
* Validate that removal collection versions and version_added collection versions conform to semver spec.
* Validate removal version numbers in meta/runtime.yml.
* Stricter validation for isodates (f.ex. YYYY-M-D is not allowed).
* Improve error reporting.
* Validate removal collection versions.

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
4 years ago
Alicia Cozine 27826827e9
updates intersphinx references for docs links (#71921)
* DOCS: updates intersphinx references for docs links
* TESTS: Raise the number of bytes scanned to determine if a file is binary. The newest ansible-2.10.inv file has its first null byte at position 2261. 4096 is still a cheap chunksize to read so it still makes sense to raise this.

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
4 years ago
Matt Clay 92b66e3e31
CI provider fixes for ansible-test. (#71929)
* Make Azure Pipelines resource_prefix lowercase.

* Make classification of CI files consistent.

* Update package-data sanity test for AZP.
4 years ago
René Moser 5348e7bf61
test: cloudstack: Use test-container 1.4.0 (#71905) 4 years ago
Tadej Borovšak 74a103d655 Remove em dash from the pytest configuration file
On some systems (tested with official Debian Buster-based Python 3.6
docker image), setting the LC_ALL environment variable to en_US.UTF-8
will trigger Python into switching its preferred encoding to ASCII.
If any python process tries to read a non-ASCII file in this scenario,
it will terminate with an error.

And this is exactly what happens to pytest when it tries to load its
configuration that ansible-test supplies because the configuration
contains an em dash.

In order to bypass this issue, we replaced the em dash with a regular
dash and things started working again.

An alternative solution would be to replace the en_US.UTF-8 locale with
something safer, but unfortunately, the en_US.UTF-8 is probably as safe
as it gets.
4 years ago
Rick Elrod db519bc17c
[tests] Bump container versions (#71518)
Change:
- This pulls in python 3.9.0rc1

Test Plan:
- CI, hopefully

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 98febab975 Use new endpoint for Parallels based instances. 4 years ago
Matt Clay 2bbcbe99fd Support macOS 10.15 for ansible-test --remote.
Use of this new version is experimental, so it is not enabled in CI yet.
4 years ago
Matt Clay f5b6df14ab
Fix ansible-test coverage traceback. (#71446)
* Add integration test for ansible-test coverage.

* Fix ansible-test coverage traceback.

* Fix coverage reporting on Python 2.6.
4 years ago
Matt Clay d099591964
Update ansible-test remote endpoint handling. (#71413)
* Request ansible-core-ci resources by provider.
* Remove obsolete us-east-2 CI endpoint.
* Add new --remote-endpoint option.
* Add warning for --remote-aws-region option.
* Update service endpoints.
* Allow non-standard remote stages.
* Add changelog fragment.
4 years ago
Toshio Kuratomi d816a5966e Force an upgrade to a newer version of antsibull 4 years ago
Matt Martz c59472715a
Add support for non-yaml EXAMPLES (#71184)
* Add support for non-yaml EXAMPLES

* Make pattern raw string

* fmt tag should be the first non-whitespace line

* Add changelog fragment
4 years ago
Matt Martz 7c60dadb9a
Updates to Integration tests to pass against Alpine (#70946)
* Start of alpine testing

* More updates

* Add forgotten file

* remove debug

* Add alpine3

* equal

* group 4

* group 4

* group 5

* Try to decrease test length

* libuser only available in testing

* Remove debug

* Make loops target work on hosts without gnu date

* Enable alpine testing

* ci_complete

* Don't specify uid for creating test user

* ci_complete

* Re-sort docker completion

* use newer container image

* ci_complete

* fix indentation

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Felix Fontein 050841324c
ansible-test: bump acme test container version to 2.0.0 (#71097) 4 years ago
Felix Fontein fbfc0f99eb
Make changelog tool be more strict about suffixes (#70909)
* Bump antsibull-changelog version.

* Flag all dotfiles, except .keep and .gitkeep.

* Enable ignoring other fragment extensions.
4 years ago
Abhijeet Kasurde 504ef607f3
Misc typo fixes (#71089) 4 years ago
Matt Davis 86b6c4bbb6
fix collection package root location under pytest >=6.0.0 (#70963) 4 years ago
Matt Clay b53e7f8720 Fix ansible-test self-test change classification.
Changes to sanity and unit tests now trigger the ansible-test self-test integration tests.

No changelog entry since this only affects tests for ansible itself and not collections.
4 years ago
Toshio Kuratomi e10902d744
Add the ansible_2_10 intersphinx reference. (#71028)
There are links from the scenario guides to collections but collections
docs aren't built in testing or locally when we're on the devel branch.
Due to that we need to make sure those references resolve to the
production docsite.  We can use intersphinx to make sure that happens.

* The test for binary files wasn't reading enough of the file.

Checking for null bytes in the first 1024 bytes failed to diagnose the
ansible_2_10.inv file as binary
4 years ago
Matt Clay 2e0097ada3 Fix ansible-test relative import analysis. 4 years ago
Matt Clay 3a8ac62596 Cap pytest version to avoid relative import issue. 4 years ago
Felix Fontein 7e2cc7db12
validate-modules: fix version_added validation for top-level, fix error codes (#70869)
* Also validate top-level version_added.

* Fix error code.

* Produce same version_added validation error in schema than in code (and stop returning it twice).

* Return correct error codes for invalid version_added for options and return values.

* Add changelog.

* Fix forgotten closing braket.

* Accept 'historical' for some top-level version_added.
4 years ago
Matt Clay 8b96caf712
Update ansible-base test container to 1.5.0. (#70856) 4 years ago
Abhijeet Kasurde c1402ddee0
validate_modules: added missing single quote (#70744)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Clay 1e02a201a6 Fix ansible-test virtualenv management. 4 years ago
Matt Clay 09f02980a4 Update ansible-test change detection for plugins. 4 years ago
Alan Rominger b0d9deeae3
Fix ansible-test error in community.aws (#70507)
* Fix ansible-test error in community.aws

* Add changelog entry for fix

* Change check from None to string_types

* Update changelogs/fragments/70507-validate-null-author.yaml

clarify wording "or a list of strings"

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py

clarify wording - single string or not specified valid

Co-authored-by: Felix Fontein <felix@fontein.de>

* Do not fail but return None when given outside list

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Matt Martz ce570691e7
Test against galaxy_ng (#70303)
* Test against galaxy_ng

* Switch container image

* Remove redundant |default

* Re-enable

* Update image

* Update wording

* Don't use pulp as the container name
4 years ago
Jordan Borean 40ce448657
ansible-test - do not validate blacklisted ps modules (#70376)
* ansible-test - do not validate blacklisted ps modules

* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay 74494a1908 Code cleanup for ansible-test coverage support. 5 years ago
Matt Clay 8d92df4537 Test netconf plugins in ansible-doc sanity test. 5 years ago
Matt Clay 262f93e22c Code cleanup in the import sanity test. 5 years ago
Matt Clay 70c59423fc Pin ansible-test requirements for RHEL.
The `packaging` and `pyparsing` packages are now installed by `ansible-test` during provisioning of RHEL instances to match the downstream vendored versions.
5 years ago
Matt Clay fa48678a08
Rename pylint plugin and add tests. (#70225)
* Renamed custom pylint plugin for unwanted names.
* Add integration tests for sanity test failures.
5 years ago
Matt Clay 1a0d8a51cd Fix yamllint sanity test line numbers.
Parse errors from libyaml now compensate for the offset of the documentation within a module.
5 years ago
Matt Clay feb163e0de
Improve consistency of ansible-test pylint results. (#70132)
* Disable too-many-ancestors pylint check.

This check was only enabled for ansible-test code.

In some cases this check counts 1 more ancestor in python 3.7+ than under python 3.6, making the check inconsistent and unsuitable for use in ansible-test.

* Disable additional pylint checks:

- import-error
- no-name-in-module

These checks were only enabled for ansible-test code.

These checks provide inconsistent results since they are dependent on available imports, which vary between environments.

It may be practical to enable these checks in the future if changes are made to ensure a consistent test environment for pylint.
5 years ago
Matt Clay e3f3b4002e
Clean up ansible-test pylint configs. (#70129)
* Sort disabled pylint codes in config.
* Fix comments in pylint config.
* Add comments to pylint config.
* Prepare to sort pylint config.
* Fix pylint config indentation.
* Sort pylint good/bad names in config.
* Add changelog fragment.
5 years ago
Matt Martz fa1fb2d13b
Test ansible-galaxy against pulp/pulp_ansible (#69605)
* Test galaxy cli against pulp

* linting fix

* Renames and small fixes

* Better handling for resetting pulp

* Clean up some things, add a comment

* I can't spell

* Bump fallaxy, use alternate pulp image

* Only reset pulp when we're are executing against pulp

* Update for updated pulp container

* Update some comments with correct URLs and typos

* Linting fix

* Pin pulp-fedora31 to a digest

* Address review comments for documentation
5 years ago
Matt Clay 8152d8bc1a Disabled inconsistent pylint checks. 5 years ago
Monty Taylor 0133757d44 Add an author exception for OpenStack Ansible SIG
The current author line wants to match a github author id. But
some people, including the OpenStack project, do not use github,
and additionally do not claim individual ownership but instead
group ownership.

Since there are already a couple of hard-coded examples in the
regex, just add one more. Alternately we could come up with some
mechanism to indicate that the author is purposely not listing
a github id, but that seems a bit heavywight.
5 years ago
Matt Clay 598786e16d Update ansible-test default test containers.
The main change is the upgrade to Python 3.9.0b3.
5 years ago
Matt Clay f69b1846f9 Add pyparsing constraint to ansible-test.
The upcoming pyparsing 3 release will require Python 3.5 or later, see:

https://github.com/pypa/packaging/issues/313

Unfortunately pip 8.x and earlier versions do not support python version requirements, which is why this constraint is needed.
5 years ago
Matt Clay 7641d32f8e
Fix ansible-test import sanity test issues. (#70084) 5 years ago
Felix Fontein 80410f292b Fix missing allow_date=False. 5 years ago
Felix Fontein 0617514176
Boolean error: fix deprecation date sanity check. (#70047) 5 years ago
Matt Clay 5b3719805e Update ansible-test test containers. 5 years ago
Felix Fontein 2fa65eb5e1
validate-modules: allow YAML dates in module documentation and meta/runtime.yml. (#70025) 5 years ago
John R Barker 37785255f2
validate ansible-base's and collections runtime.yml (#69742)
* Validate ansible-base & collection's runtime.yml

Add new test `runtime-metadata`

* Schema validation of file
* Error if a a legacy meta/routing.yml exist in a collection
* removal_date OR removal_version

* Add tombstone validation.

* Allow both ISO 8601 date strings and datetime.date objects (from YAML dates).

* Address review comments.

* Add metadata to test collection.

* Add requirements file.

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay 9f49db1f99 Avoid use of deprecated junit-xml method. 5 years ago
Matt Clay 26f318d277 Ignore return code from antsibull_changelog lint. 5 years ago
Matt Clay 8ffaed00f8 Add Azure Pipelines support to ansible-test. 5 years ago
Matt Martz 4a4a11d282
Add mccabe complexity testing (#64623)
* Add mccabe complexity testing

* Make mccabe complexity an optional error

* Add mccabe to new sanity pylint requirements

* Add a changelog fragment.

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay 2251b239a3 Fix changelog sanity test config detection. 5 years ago
James Cassell 47d14a33bd
config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
5 years ago
Felix Fontein a114da80ee
Use antsibull-changelog instead of packaged changelog generator (#69313)
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
5 years ago
Gonéri Le Bouder 4869874337
ansible-test: vcenter initialize group/vmware (#70011)
Ensure the vcenter provider initialize the `module_defaults` of all
the vmware modules, not just `vmware_guest`.
The VMware CI relies on this for the authentication of the different
VMware modules.

The commit adjust `incidental_vmware_prepare_tests/tasks/init_vcsim.yml`.
The test-suite uses a copy of `vmware_guest` that is not in the
`group/vmware` group. As a result, we need to manually pass the
authentification parameter.
5 years ago
Matt Clay 6c2be04b6f Intercept code-smell tests using subprocess.
Some code-smell sanity tests for ansible-base use subprocess to invoke ansible commands.
Intercept these commands to make sure the correct script and python version are used.
5 years ago
Jordan Borean a58fcde3a0
Use common ps sanity requirements file (#69992)
* Use common pssanity requirements file

* Fix up sanity ignore
5 years ago
Matt Clay a5679d4dd1
Additional ansible-test sanity adjustments. (#69974)
* Fix ansible-test check_pyyaml usage.
* Avoid unnecessary pip commands.
* Avoid pointless pip check.
* Only install ansible-test requirements as needed.
5 years ago
Matt Clay dc9c1d202b Fix ansible-test pip filter on Fedora 32.
The original version works on CentOS 8 but not Fedora 32.
5 years ago
Felix Fontein f1ab7cf0c6
Validate-modules: adjust test to fixes in #69977 (#69978) 5 years ago
Jordan Borean 15f07e7a6c
Fix up schema for collection deprecation (#69977)
* Fix up schema for collection deprecation

* Fix up modules using wrong name

* Another fix
5 years ago
Matt Clay a81dd4f06a
Update ansible-test default containers. (#69819)
* Move ansible-test completion code.

* Fix a few type hints.

* Change docker completion based on context.

Collections now use version 2.0 of the default-test-container.
This is an updated version of the earlier 1.x default-test-container with ansible-base and cloud specific requirements removed.

Testing of ansible-base now uses version 1.0 of a new ansible-base-test-container.
This container is similar to the earlier 1.x default-test-container, but with unnecessary duplication of requirements across Python versions removed.

Collections which have tests that depend on requirements no longer present in the default test container should specify them in their test requirements files:

* tests/integration/requirements.txt
* tests/unit/requirements.txt

* Bump test container versions

Co-authored-by: Jordan Borean <jborean93@gmail.com>
5 years ago
Sloane Hertel 51f6d129cb
support hard coded module_defaults.yml groups for collections (#69919)
* Only allow groups which were hardcoded in module_defaults.yml

only load action groups from the collection if module_defaults contains a potential group for the action

* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml

Third party modules should not be using group/ - use the action name instead

* add externalized module_defaults tests

add the missing group and collections

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

ci_complete

* Fix import in tests

ci_complete

* Update with requested changes

ci_complete

* don't traceback since we don't validate the contents of module_defaults

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>
5 years ago
Felix Fontein a862ff2d43
Deprecation revisited (#69926)
* Allow to specify collection_name separately for deprecation.

* Use new functionality in Ansible.

* Use new functionality in tests.

* Update tagging/untagging functions.

* Update pylint deprecated sanity test.

* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).

* Improve version validation. Re-add version_added validation.

* Make sure collection names are added to return docs before schema validation.

* Extra checks to avoid crashes on bad data.

* Make C# module utils code work, and update/extend tests.

* Add changelog fragment.

* Stop extracting collection name from potentially tagged versions/dates.

* Simplify C# code.

* Update Windows modules docs.

* Forgot semicolons.
5 years ago
Matt Clay 7bff3d312f
Split out and install sanity test requirements. (#69971)
* Split out sanity test requirements.

* Run each --venv test separately.

This provides verification that the requirements for each test are properly specified.

* Use a separate requirements file per sanity test.

* Skip setuptools/cryptography setup for sanity.

* Eliminate pyyaml missing warning.

* Eliminate more pip noise.

* Fix conflicting generate_pip_install commands.

* Add changelog fragment.
5 years ago
Sam Doran c41a160951
Update test images to version 1.16.0 (#66771) 5 years ago
Felix Fontein 3e566768c1 Fix broken validate-modules. 5 years ago
Matt Clay 06efc41fb0
Reduce pip output noise in ansible-test. (#69925) 5 years ago
Matt Clay f733517e6e Support dates in metadata in import sanity test.
No changelog entry since this feature and the associated bug have not been included in a release.
5 years ago
Jordan Borean 112251c7f4
Bump fallaxy version (#69917) 5 years ago
Matt Davis d79b23910a
fix sys.modules cleanup and blacklist behavior (#69898)
* fix sys.modules cleanup and blacklist behavior

* fix map-as-generator py2/py3 issue
* clear path_importer_cache between runs

* sanity fix

* don't be stupid with moving target generators
5 years ago
Brian Coca 062e780a68
starting metadata sunset (#69454)
* starting metadata sunset

 - purged metadata from any requirements
 - fix indent in generic handler for yaml content (whey metadata display was off)
 - make more resilient against bad formed docs
 - removed all metadata from docs template
 - remove metadata from schemas
 - removed mdata tests and from unrelated tests

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
5 years ago
Felix Fontein 2dbd5dc2ae
ansible-test: do not accept empty string as valid version number (#69816)
* Work around strange behavior of StrictVersion and SemanticVersion constructors that they accept an falsy value.

* Do not accept empty strings as versions.
5 years ago
Felix Fontein 0b82d4499e
ansible-test ansible-doc sanity test: also run with --json parameter (#69288) 5 years ago
Felix Fontein 4794b98f2a
Fix version_added test (#69756)
* Fix version_added test for changes in #69680.

* Fix error message.
5 years ago
Felix Fontein 40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
5 years ago
Jordan Borean f5f3ba7ab5
ansible-test - fix up relative util import for powershell validate-modules (#69753)
* ansible-test - fix up relative util import for powershell validate-modules

* Use different tactic for generic group

* Use python 2 and 3
5 years ago
Felix Fontein 31bf3a5622
Deprecate module in collection: allow removal date in documentation, make validate-modules ensure version and date match (#69727)
* Allow to deprecate module by date in documentation.

* Make sure deprecation date/version match between module docs and meta/runtime.yml.

* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.

* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.

* Linting.

* Allow to deprecate plugin options by date.

* Add changelog fragment for deprecation by date (also covers #68177).
5 years ago
Matt Clay 65c57a7f5b Update default-test-container to version 1.14. 5 years ago
Felix Fontein ea04e0048d
Allow to deprecate options and aliases by date (#68177)
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in #67684.

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
5 years ago
Matt Davis f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
5 years ago
Matt Martz b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
5 years ago
Felix Fontein 43acd61901
ansible-test local change detection: use --base-branch if specified (#69508) 5 years ago
Matt Clay c495c92a6e Code cleanup in ansible-test. 5 years ago
Matt Clay 159d79995c Freeze pycodestyle in ansible-test. 5 years ago
Matt Clay e7c2eb519b
Add Shippable request signing to ansible-test. (#69526) 5 years ago
Matt Clay 83381cd912 Update the default-test-container in ansible-test.
The updated container includes fewer requirements now that the collection migration has completed.

Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
5 years ago
Matt Clay d8e0aadc0d Update ansible-test support for CI providers.
Refactored CI provider code to simplify multiple provider support and addition of new providers.
5 years ago
Matt Martz 4c4406b2df
Flatten the directory hierarchy of modules (#68966)
* Flatten the directory hierarchy of modules

* Update ignore.txt, flatten units

* Update imports

* Completely flatten the modules directory

* Update sanity ignore

* Fix some sanity test ignores

* Fix relative import

* Fix docs builds without category

* ci_complete

* Clean up docs. ci_complete

* Adjust needs/file alias

* ci_complete

* fix hardcoded ping module paths

Co-authored-by: Matt Davis <mrd@redhat.com>
5 years ago
Felix Fontein 0e15375ffe
Add deprecated removed_in_version and deprecated_aliases version tests (#66920) 5 years ago
Matt Davis 776f4840fc
fix `ansible-test units` to work(ish) under podman (#69462)
* ignore the missing `Networks` key, issue a warning that network disconnection won't function
5 years ago
Rick Elrod de59b17c7f
Add Fedora 32 to CI (#69222)
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.

Test Plan:
CI

Tickets:
Fixes #69230

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
John R Barker 084587913a
validate-modules: deprecated modules in collections (#68646)
* validate-modules: deprecated modules in collections

In Collections a module is marked as deprecated via meta/routing.yml

Use this file, rather than the leading `_` as part of the deprecated
test.

* Correct variable

* review comments

* indentation

* Read routing.yml only once

* pep8

* Apply suggestions from code review

Co-authored-by: Matt Clay <matt@mystile.com>

* review: remove duplicated conditional

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Toshio Kuratomi 39b3942048 Remove left hand side slicing
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances.  There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code.  (The most likely case of
this is when it's a global variable and some other code might import
that variable name).

Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement.  At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
5 years ago
Felix Fontein 947fa3bad3
ansible-test validate-modules: don't allow arbitrary lists and dicts for 'default', 'sample' and 'example' (#69287)
* Don't allow arbitrary lists and dicts for 'default', 'sample' and 'example'.

* Add changelog.

* Make compile with Python 2.
5 years ago
Matt Clay 1cf26896c5 Default ansible-test provisioning to us-east-1.
Previously windows instances on Shippable would be automatically directed to us-east-2.
5 years ago