Commit Graph

287 Commits (3204d260ddb142a75453aa1a8ea8d89cc90a87a0)

Author SHA1 Message Date
Matt Clay 159d79995c Freeze pycodestyle in ansible-test. 5 years ago
Matt Clay e7c2eb519b
Add Shippable request signing to ansible-test. (#69526) 5 years ago
Matt Clay 83381cd912 Update the default-test-container in ansible-test.
The updated container includes fewer requirements now that the collection migration has completed.

Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
5 years ago
Matt Clay d8e0aadc0d Update ansible-test support for CI providers.
Refactored CI provider code to simplify multiple provider support and addition of new providers.
5 years ago
Matt Martz 4c4406b2df
Flatten the directory hierarchy of modules (#68966)
* Flatten the directory hierarchy of modules

* Update ignore.txt, flatten units

* Update imports

* Completely flatten the modules directory

* Update sanity ignore

* Fix some sanity test ignores

* Fix relative import

* Fix docs builds without category

* ci_complete

* Clean up docs. ci_complete

* Adjust needs/file alias

* ci_complete

* fix hardcoded ping module paths

Co-authored-by: Matt Davis <mrd@redhat.com>
5 years ago
Felix Fontein 0e15375ffe
Add deprecated removed_in_version and deprecated_aliases version tests (#66920) 5 years ago
Matt Davis 776f4840fc
fix `ansible-test units` to work(ish) under podman (#69462)
* ignore the missing `Networks` key, issue a warning that network disconnection won't function
5 years ago
Rick Elrod de59b17c7f
Add Fedora 32 to CI (#69222)
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.

Test Plan:
CI

Tickets:
Fixes #69230

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
John R Barker 084587913a
validate-modules: deprecated modules in collections (#68646)
* validate-modules: deprecated modules in collections

In Collections a module is marked as deprecated via meta/routing.yml

Use this file, rather than the leading `_` as part of the deprecated
test.

* Correct variable

* review comments

* indentation

* Read routing.yml only once

* pep8

* Apply suggestions from code review

Co-authored-by: Matt Clay <matt@mystile.com>

* review: remove duplicated conditional

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Toshio Kuratomi 39b3942048 Remove left hand side slicing
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances.  There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code.  (The most likely case of
this is when it's a global variable and some other code might import
that variable name).

Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement.  At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
5 years ago
Felix Fontein 947fa3bad3
ansible-test validate-modules: don't allow arbitrary lists and dicts for 'default', 'sample' and 'example' (#69287)
* Don't allow arbitrary lists and dicts for 'default', 'sample' and 'example'.

* Add changelog.

* Make compile with Python 2.
5 years ago
Matt Clay 1cf26896c5 Default ansible-test provisioning to us-east-1.
Previously windows instances on Shippable would be automatically directed to us-east-2.
5 years ago
Rick Elrod 1c5c89baa6 Add RHEL 8.2 to CI
Change:
RHEL 8.2 GA was released this week, test it in CI instead of 8.1.

Test Plan:
CI

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay a050d892d8 Remove obsolete vcenter setup from ansible-test. 5 years ago
Matt Clay d7da1d9bd0 Fix ansible-test coverage constraints. 5 years ago
Matt Clay 5a3ca0ae5f Fix ansible-test delegation from pip install. 5 years ago
Matt Clay 122743ae30
Update ansible-test handling of delegation paths. (#69056) 5 years ago
Yanis Guenane 46d82179d8
Testing: Add support for CentOS Linux On Power platform (#68130)
* Testing: Add CentOS Linux On Power platform

* Add arch designation to remotes.

This avoids overloading the provider with the arch.

Also add a changelog entry.

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay 7d51706278 Revert "Switch network incidental tests to VyOS 1.1.7-R2."
This reverts commit 7d5177d6a0.
5 years ago
Matt Clay 7d5177d6a0 Switch network incidental tests to VyOS 1.1.7-R2.
The previous version being used, VyOS 1.1.8, is currently unavailable in AWS.
5 years ago
Matt Clay a095172130 Fix ansible-test --docker without tests dir. 5 years ago
Felix Fontein bae5f18f5c
ansible-test: don't trigger full CI run for changes to changelogs/ and docs/ in collections (#68550)
* Don't trigger full CI run for changes to changelogs/ and docs/ in collections.

* Add changelog fragment.

* Update changelogs/fragments/68550-ansible-test-docs-changelogs.yml

Co-Authored-By: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Rick Elrod 38b7ceb75a
Update docker.txt to use the OpenSUSE 15.1 container image (devel edition) (#68785)
* Update docker.txt to use the OpenSUSE 15.1 container image

Signed-off-by: Rick Elrod <rick@elrod.me>

* handle installing mysql on suse

Signed-off-by: Rick Elrod <rick@elrod.me>

* add changelog fragment

Signed-off-by: Rick Elrod <rick@elrod.me>

* Update changelogs/fragments/ansible-test-opensuse-15.1.yml

Co-Authored-By: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay 04edd77c42
Update tests to use RHEL 7.8. (#68787)
* Update tests to use RHEL 7.8.

Keeping support for RHEL 7.6 since collections are still using it.

* Fix tests for RHEL 7.7+ due to extras repo name change.
5 years ago
Matt Clay 148e83f832
Fix ansible-test submodule handling. (#68759)
* Refactor ansible-test integration test.
* Add env --list-files option.
* Add tests for collection files tracked by git.
* Fix ansible-test submodule usage on older git.
* Fix submodule directory detection as files.
* Improve handling of nested source control.
5 years ago
Rick Elrod 13aef3c2e5
Add --docker-terminate flag to ansible-test (#68688)
* Add --docker-terminate flag to ansible-test

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Jordan Borean 6db66bcadd
Add relative module_util support for powershell (#68321)
* Add relative module_util support for powershell

* Added ansible-test classification support
5 years ago
Matt Clay 7323d5dd0d Fix references to old egg-info directory. 5 years ago
Matt Clay 735885d57c Fix ansible-test requirements installation. 5 years ago
Matt Clay c888035e02
Update Ubuntu 18.04 test container to 1.13. (#68534) 5 years ago
Matt Clay 53a3d1ffdb Fix ansible-test change detection traceback. 5 years ago
Matt Clay d8b5c11a63 Report error for coverage 5+ in ansible-test. 5 years ago
Matt Clay 27fc049993
Fix ansible-test code coverage collection. (#68502)
The last task in a play should now properly report code coverage.

This change should also eliminate empty coverage files, as well as incomplete coverage files resulting from early worker termination.
5 years ago
Jordan Borean e12cea2c7c
ansible-test - fix up powershell module_util analysis for collections (#68422) 5 years ago
Yanis Guenane fb2a7ce923 Testing: Move from ibmcloud to ibmps 5 years ago
Matt Clay f1b3e8364e Fix ansible-test PATH handling. 5 years ago
Matt Clay 9765a80bd4
Fix ansible-test coverage analysis option usage. (#68406)
* Fix ansible-test coverage analysis option usage.

The `--input-dir` option for `coverage analyze targets generate` was being ignored.

No changelog entry since this feature has not yet been released.

* Move coverage config to fix type annotations.

Declaring the types before referencing them makes sure they're recognized by tools such as PyCharm.
5 years ago
Jordan Borean 7c9889a72b validate-modules - fix ps module delegate type inspection 5 years ago
Matt Clay 70827c7923 Fix ansible-test import analysis warning.
Fix overlooked in https://github.com/ansible/ansible/pull/68372/
5 years ago
Matt Clay 10fe54de58 Fix ansible-test module_utils import analysis.
Now empty `*.py` files are ignored during module_utils import analysis for change detection.
This eliminates "No imports found" warnings for files which should have no imports.
5 years ago
Matt Clay cf75e3f526 Fix ansible-test import analysis for collections. (#68352)
* Fix ansible-test import analysis for collections.

* Ignore plugins/module_utils/__init__.py
5 years ago
Matt Clay 6fde9b3a23 Fix shebang sanity test module identification. 5 years ago
Jordan Borean aaf575f0b2 Internally redirect win modules to collection name (#68317)
* Internally redirect win modules to collection name

* Added comment for how this should be fixed in a subsystem plugin

* add collection plugins for base tests
5 years ago
Matt Clay 2c7f8bf586 Fix ansible-test windows-integration module refs. 5 years ago
Matt Clay 41f6c73be0 Update ansible-test network test provisioning. (#68220)
* Update network provisioning defaults.

* Update network test requirements.
5 years ago
Matt Clay 5159457ba1 Fix ansible-test unit test requirements.
Requirements were incorrectly added to ansible-test in https://github.com/ansible/ansible/pull/61813

These requirements should have been placed into `test/units/requirements.txt` instead.
Now that the relevant content has been migrated out of the repository, the requirements are no longer necessary there either.

No changelog entry for this change since the original changes were not included in any release and also lacked a changelog entry.
5 years ago
Matt Clay 8715bc400a Add coverage filtering to ansible-test. (#68158)
* Relocate expand_indexes so it can be reused.

* Add generate_indexes function.

* Simplify type annotations.

* Add `coverage analyze targets filter` command.

* Add changelog entry.
5 years ago
Matt Clay 27ce6dbc1f Remove test support option from ansible-test.
This option was added to facilitate migration and is no longer needed.
5 years ago
Matt Clay 50442f9fdd Fix ansible-test handling of no Python coverage. 5 years ago
Matt Clay 99e136e153 Remove out-of-date test constraint on paramiko.
ci_complete
5 years ago
Matt Clay 1a8fdaadc7 Fix powershell coverage path rewriting.
This applies the same rewrite logic to PowerShell coverage as is used for Python coverage.
5 years ago
Matt Clay 29978d027b Update default test container to version 1.12. 5 years ago
Matt Clay ebb255f0d3 Add Python 3.9 to ansible-test.
ci_complete
ci_coverage
5 years ago
Matt Clay 64a2864158 Fix ansible-test constraints for AWS. 5 years ago
Matt Clay c06f442409 Fix ansible-test import sanity test venv setup. 5 years ago
Matt Clay 99e657162f Fix ansible-test module_utils analysis. 5 years ago
Matt Clay 70fc69c852 Fix vcenter requirements. 5 years ago
Matt Clay 7c8b046b5f
Fourth batch of incidental integration tests. (#67873)
* Copy in incidental posix tests.

* Update incidental test aliases.

* Update target names.

* Add support plugins.

* Fix paths.

* Update ignores.

* Update integration-aliases sanity test.

* Add incidental tests to CI.
5 years ago
Jordan Borean 26129fcb80
ansible-galaxy - optimise some paths and use fake galaxy int tests (#67685)
* ansible-galaxy - optimise some paths and use fake galaxy int tests

* Added init, built, and publish tests

* Test against both mocked Galaxy and AH server

* Finish off writing the install tests

* Fix up broken tests

* Rename test target and add migrated tests

* Use cloud provider for Galaxy implementation

* Added blank static config

* Use correct alias group

* Set release version and fix copy typo

* Remove reset step as it is no longer needed

* Use sane env var names for test container name
5 years ago
Sam Doran 7a42354021
Switch to using FreeBSD 12.1 (#67659) 5 years ago
Abhijeet Kasurde b6c2056ea2
mongodb_user: fix doc formatting (#67763)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Matt Clay f735fd672a
Third batch of incidental integration tests. (#67830)
* Copy in incidental windows tests.

* Update incidental test aliases.

* Add support plugins.

* Update target references.

* Update sanity ignores.

* Update integration-aliases test.

* Add to CI.
5 years ago
Matt Clay b68e55da26
Remove `test/legacy/` directory. (#67786)
* Remove `test/legacy/` directory.

* Remove references to `test/legacy/` directory.

* Update the remaining valid legacy reference.

* Remove outdated legacy references.
5 years ago
Matt Clay e3591223a0
Second batch of incidental integration tests. (#67765)
* Update incidental test aliases.

* Rewrite target references for renamed targets.

* Add incidental tests to CI.

* Update sanity tests for incidental cloud tests.

* Initial copy of incidental tests.

* Copy contrib files into test.

* Update paths in test.

* Add support plugins.

* Update plugin to work around missing deps.

* Update sanity ignores.

* Fix matrix entries.

* Remove debug echo.
5 years ago
Jill R 96244c4cde
Make AWS test plugin to refer to the aws-permissions-for-integration-tests (#67665)
Make AWS test plugin to refer to the aws-permissions-for-integration-tests section of the AWS devel docs now that we have them, rather than suggesting to flag down specific users.
5 years ago
Matt Clay a64b562071
Fix up network testing on Shippable. (#67678)
This prepares for code coverage collection for initial setup of incidental test coverage.
5 years ago
Matt Clay 482885e0c8 Fix network setup for collections in ansible-test. 5 years ago
Matt Clay 542d1b98ff
Support private test plugins in ansible-test. (#67660)
* Support private test plugins in ansible-test.

This feature is exclusively used for testing Ansible itself and is not available to collections.

Content in the newly supported directories will be added in follow-up PRs.

* Add new directory to path classification.

* Support new paths in shebang test.

* Add new directory to manifest.
5 years ago
Yanis Guenane 34800fe027 Test: Bump Tower version to latest release 5 years ago
Jordan Borean 36def8bf03
Use Ansible to load ps module utils in arg spec checker (#67596)
* Use Ansible to load ps module utils in arg spec checker

* Added changelog and note
5 years ago
Matt Martz c61c0f7ad5
Add warning when running devel (#67502)
* Add warning when running devel. Addresses #67362

* Add config entry to disable devel warning, so it doesn't impact CI

* Update warning about using devel
5 years ago
Matt Clay 19f6fc68d4
Relocate Shippable tools. (#67556)
* Move Shippable tools to hacking directory.

These limits the `test/utils/shippable/` directory to scripts required for CI.

* Fix `test/utils/shippable/` file classification.

* Update package-data sanity test.
5 years ago
Felix Fontein 4373863f25
Module validation: sanity check mutually_exclusive, required_if, required_xxx ... (#66961)
* required_if checks should have three or four parts.

* Validate mutually_exclusive, required_together, required_one_of, required_if and required_by.

* Simplify code.

* Improve messages.

* Add changelog.

* Sanity check.

* Update docs.

* Update ignore.txt.

* Don't continue with tests when terms are not strings.

* Remove ignore.txt entry.

* Make sure validate-modules doesn't choke on things already flagged by schema test.

* Check required_if requirements list for strings.
5 years ago
Abhijeet Kasurde b6753b46a9
Replace 'message' in module parameters (#60051)
* 'message' parameter is replaced by 'commit_message' in grafana_dashboard
* 'message' parameter is replaced by 'notification_message' in datadog_monitor

This change is required since 'message' as parameter name is used internally by
Ansible core engine.

Fixes: #39295 #45362 #47132 #59617

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Yanis Guenane b9bd2c25ca
Test: Bump TOWER_VERSION to supported version (#67378)
* Revert "Remove Tower module tests from CI. (#67368)"

This reverts commit b041d96762.

* Test: Bump TOWER_VERSION to 3.6.2
5 years ago
Felix Fontein f6815040fd
add_file_common_arguments: fix defaults and tpyes, improve sanity checking (#67243)
* Make validate-modules stop ignore FILE_COMMON_ARGUMENTS.

* Add types to FILE_COMMON_ARGUMENTS and update document fragment to match it.

* Update ignore.txt.

* Add changelog.
5 years ago
Sam Doran 8f296a6533
ansible-test - add constraint for virtualenv (#67289)
* ansible-test - add constraint for virtualenv

* Limit virtualenv only on macOS.

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay 1110e9d80d Test AIX with smaller volumes. 5 years ago
Matt Clay 5e68bb3d93
Add code coverage target analysis to ansible-test. (#67141)
* Refactor coverage file enumeration.
* Relocate sanitize_filename function.
* Support sets when writing JSON files.
* Generalize setting of info_stderr mode.
* Split out coverage path checking.
* Split out collection regex logic.
* Improve sanitize_filename type hints and docs.
* Clean up coverage erase command.
* Fix docs and type hints for initialize_coverage.
* Update type hints on CoverageConfig.
* Split out logic for finding modules.
* Split out arc enumeration.
* Split out powershell coverage enumeration.
* Raise verbosity level of empty coverage warnings.
* Add code coverage target analysis to ansible-test.
5 years ago
Matt Clay f4a80bb600
Code cleanup and refactoring in ansible-test. (#67063)
* Code cleanup in ansible-test.
* Split out encoding functions.
* Consoldate loading of JSON files.
* Split out disk IO functions.
* Simplify file access.
* Add functions for opening files.
* Replace open calls with appropriate functions.
* Expose more types from typing module.
* Support writing compact JSON.
* Add verbosity argument to display.warning.
* Add changelog entry.
* Update files overlooked during rebase.
* Use `io.open` instead of `open`.
* Fix file opening for imp.load_module.
* Remove use of `r+` mode to access files.
* Add missing import.
* Fix httptester on Python 2.x.
* Clarify changelog fragment.
* Consolidate imports. Remove extra newlines.
* Fix indirect imports.
5 years ago
Matt Clay d584584474 Reorganize ansible-test coverage code.
This change moves all code for the `ansible-test coverage` command into the `coverage` directory.

Each subcommand is split into a separate file.

Only minor spelling changes were made aside from code relocation.
5 years ago
Mark Chappell 5d4d9d40af
ansible-test - Add 'get' to the list of bad choices for state (#66921)
* ansible-test - Add 'get' to the list of bad choices for state

* Changelog fragment
5 years ago
Jordan Borean 811c54a7a4
validate-modules - support deprecated_aliases (#66965)
* validate-modules - support deprecated_aliases

* Removed changelog fragment
5 years ago
Matt Clay 49d8d5ae33 Fix ansible-test tar format handling. 5 years ago
Yanis Guenane 4fd2dce7f3
Testing: Add support for AIX platform (#65802) 5 years ago
Sviatoslav Sydorenko eec6afd116 Explicitly set junit_family=xunit1 @ pytest config
This change hides the deprecation warning and protects from
failures in pytest==6.0.0

Ref:
* https://github.com/ansible/ansible/pull/66445#discussion_r372530176

Co-Authored-By: Matt Clay <matt@mystile.com>
5 years ago
Mark Chappell 1156962cde
Forbid `state=(list|info)` in modules via ansible-test sanity check
PR #66898

This change introduces a new sanity check with code
`parameter-state-invalid-choice` in the `ansible-test sanity`
validator. It enforces modules not to support `list` or `info`
as their `state`.

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-Authored-By: Felix Fontein <felix@fontein.de>
5 years ago
Matt Clay 3c3cf50e7a Fix ansible-test coverage of ansible-connection. 5 years ago
Matt Clay 21069c84d9 Fix ansible-test color `ls` logic. 5 years ago
Matt Clay 428aaf7e65 Keep SSH authorized keys in ansible-test setup. 5 years ago
Matt Clay efd2dd8929 Add more ansible-test args to delegation config. 5 years ago
Matt Clay ee50792416 Add a `--no-pip-check` option to ansible-test. 5 years ago
Mark Chappell 35652ca788 Add tests to make sure that the documented 'elements' matches that defined in argument_spec (#66385)
* Add tests to make sure that the documented 'elements' matches that defined in argument_spec

* Mass-add test/sanity/ignore.txt
5 years ago
Mark Chappell 5ff899662d Add sanity test to require elements entry when argument type=list (#66386)
* Require elements entry for lists

* Bulk add initial test/sanity/ignore.txt
5 years ago
Matt Clay 965854fbd2
Add constraints for Jinja2 on Python 2.6. (#66826)
* Add constraint for Jinja2 on Python 2.6.

* Fix constraint in inventory_aws_conformance test.

* Add constrraints for template_jinja2_latest test.
5 years ago
Felix Fontein 8cff585891
Schema validation for argument_spec (#65747)
* Start of schema for argument_spec

* Add changelog.

* Remove superfluous import.

* Update ignore.txt

Co-authored-by: Matt Martz <matt@sivel.net>
5 years ago
Matt Clay 8b63da7b43 Fix ansible-test egg-info directory detection. 5 years ago
Matt Clay ebf21bb48d Fix ansible-test inventory delegation. 5 years ago
Matt Clay 934d6b235b Add `--venv-system-site-packages` to ansible-test. 5 years ago
Felix Fontein c58d8ed1f5 Improve deprecate sanity check; remove deprecated features which should have been removed for 2.9 (#65745)
* Update deprecation check.

* Remove support for check_invalid_arguments.
5 years ago