Commit Graph

75 Commits (652bfc7e19f3bab086c1d0389c8d87933f261d54)

Author SHA1 Message Date
Matt Clay 1351962eee Move "code smell" sanity test config loading. 5 years ago
Matt Clay 5d7cc993dd
Clean up sanity test ignore handling. (#59749)
* Allow sanity tests to easily ignore themselves.

Useful for simple regex based sanity tests that error on their own script due to the regex or error message generated.

* Simplify no-smart-quotes sanity test.

Pruning of unversioned directories is handled by ansible-test.

* Remove directory pruning from boilerplate tests.

Files not passing these tests should be ignored instead.

* Add missing sanity ignore entries.
5 years ago
Matt Clay 66654475e1 Update ansible-test Python version handling.
Minimum version requirements for sanity tests have been standardized:

- All single version sanity tests now require Python 3.5 or later.
- All multiple version sanity tests continue to use all supported Python versions.
- All version neutral sanity tests continue to work on any supported Python version.

Previously some tests required 3.5 or later with most of the remaining tests requiring 2.7 or later.

When using the `--python` option to specify a Python version:

- Tests which do not support the specified Python version will be skipped with a warning.
- If the specified Python version is not available, any test attempting to use it will generate an error.

When not using the `--python` option to specify a Python version:

- Multiple version tests will attempt to run on all supported versions.
- Single version tests will use the current version if supported and available, or if no supported version is available.
- Single version tests will use the lowest available and supported version if the current version is not supported.
- Any versions which are not available or supported will be skipped with a warning.

Unit tests automatically skip unavailable Python versions unless `--python` was used to specify a version.
5 years ago
Matt Clay 2f6de3aea4 Add missing decorator to ansible-test sanity.
This fixes sanity test errors so they show the proper error code instead of "bound method".
5 years ago
Matt Clay db7e9aa051
Use consolidated ignore.txt in ansible-test. (#59618) 5 years ago
Matt Clay 7c728118e6
Fix import sanity test for collections. (#59484) 5 years ago
Matt Clay d65d8d6e13 Fix default pylint config for collections. 5 years ago
Matt Clay 44a262714c
Add common sanity ignore & skip to ansible-test. (#59416)
* Add common sanity ignore & skip to ansible-test.
* Switch ansible-doc sanity test to single version.
* Remove duplicate ignore entry.
5 years ago
Matt Clay 79eca9c8fb
Initial ansible-test support for collections. (#59197)
* Initial ansible-test support for collections.
* Include cloud config in delegation payload.
* Add missing types import and fix `t` shadowing.
* Fix plugin traceback when config_path not set.
* Fix encoding issues.
* Remove unused imports.
* More encoding fixes.
* Handle delegation outside exception handler.
* Inject ssh keys only if not already in place.
* More defensive approach to getting remote pwd.
* Add missing string format var.
* Correct PowerShell require regex.
* Rename `is_install` and `INSTALL_ROOT`.
5 years ago
Matt Clay 4f1d0f1b39 Skip using paths for ansible-doc sanity test. 5 years ago
Matt Clay cb0dfc9793 Prepare pylint test for collections support. 5 years ago
Matt Clay 4255ee8e5a Prepare ansible-test for collections support.
Another round of changes to prepare ansible-test for supporting collections to help keep later PRs a manageable size.
5 years ago
Matt Clay d910c971b4
Minor ansible-test code cleanup. (#59052)
* Minor ansible-test code cleanup.
* Fix type annotations.
5 years ago
Matt Clay 6645054329
Code cleanup for test infrastructure. (#59046)
* Remove useless object inheritance in test code.
* Remove unnecessary pass statements from test code.
* Comment on why certain pylint rules are ignored.
* Add more pylint test contexts.
* Fix import order.
* Remove unused variables.
* Remove unnecessary pass statement.
* Fix bad continuations.
* Remove unnecessary elif.
* Allow legitimate broad except statements.
* Allow legitimate protected access.
* Clean up names to make pylint pass.
5 years ago
Matt Clay 3f2b766d10 Add future and metaclass boilerplate to test code.
Continue to ignore:

- test/integration/
- test/legacy/
- test/units/
5 years ago
Matt Clay bc3a599678 Fix invalid type hints in ansible-test. 5 years ago
Matt Clay 86ad4c99ba Reorganize util code in ansible-test.
Code in util.py that depends on CommonConfig is now in util_common.py.
5 years ago
Matt Clay 520af5cb47 Sanity test fixes for future collections support. 5 years ago
Matt Clay 1e1463401d
Prepare ansible-test for supporting collections. (#58886)
This is a small but incomplete set of the initial changes for supporting testing of collections with ansible-test.
5 years ago
Sam Doran 8cfd0ba039 Allow for test groups with double digits (#58532) 6 years ago
Matt Clay b06f5634c5 Simplify ansible-test target expansion.
Targets are always expanded to full lists now instead of optimizing
for shorter lists by collapsing directories.

This change only affects unit tests and the ansible-doc sanity test,
as they were the only remaining tests using the old behavior.
6 years ago
Matt Clay 535e646a66 Add test for invalid ignore entries. 6 years ago
Matt Clay 817ec5ab48 Add PyYAML check for libyaml support. 6 years ago
Matt Clay d758d5bab1 Disable code-smell tests on Python 2.6. 6 years ago
Jordan Borean 04cae4134f
Bump deps for ntlm-auth and PSScriptAnalyzer (#55269) 6 years ago
Matt Clay ade6977d66 Fix typo in doc_fragments path. 6 years ago
Jordan Borean 0bf14d61d0
Add PSCustomUseLiteralPath and update PSSA for pslint (#54080) 6 years ago
Matt Clay 63ce588382 Update pylint and split runs more to avoid hangs. 6 years ago
Matt Martz 07ff2a6494 cliconf and httpapi are documentable plugins now 6 years ago
Matt Clay a8e328f474
Overhaul ansible-test code coverage and injector. (#53510) 6 years ago
Matt Martz a639dc6d88
Add a new check to ensure that module arguments are valid python identifiers (#53278)
* Add a new check to ensure that module arguments are valid python identifiers

* Move the check up higher, out of the if docs block

* Skip validate-modules on py2

* Remove errant alias
6 years ago
Matt Clay 06d83bae05 Support `ignore_changes` in code-smell tests. 6 years ago
Brian Coca 96b3ef5553
Doc fragments to plugins (#50172)
* promote doc_fragments into actual plugins

  change tests hardcoded path to doc fragments
  avoid sanity in fragments
  avoid improper testing of doc_fragments
  also change runner paths
 fix botmeta
 updated comment for fragments
 updated docs
6 years ago
Matt Clay 38eba60849 Fix ansible-test --exclude with delegation.
Previously the option worked with integration commands but not units or sanity.
6 years ago
Matt Clay 76bf861308
Upgrade pylint to version 2.1.1. (#47036)
* Fix issues reported by the latest pylint.
* Split pylint runs into more contexts.
* Upgrade pylint.
6 years ago
Matt Clay e2b6047514
Add symlinks sanity test. (#46467)
* Add symlinks sanity test.
* Replace legacy test symlinks with actual content.
* Remove dir symlink from template_jinja2_latest.
* Update import test to use generated library dir.
* Fix copy test symlink setup.
6 years ago
Matt Clay 33a8be9109 Add file exists check in integration-aliases test. 6 years ago
Matt Clay e53390b3b1
Fix ansible-test multi-group smoke test handling. (#46363)
* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.
6 years ago
Matt Clay ac492476e5
Bug fixes and cleanup for ansible-test. (#45991)
* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.
6 years ago
Matt Clay 2056c981ae Improve ansible-test match error handling. 6 years ago
Matt Clay 5a3000af19 Support comments in ansible-test flat files. 6 years ago
Matt Clay 998badbda5 Fix ansible-test virtualenv use in import test. 6 years ago
Matt Clay 4e489d1be8
Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.
6 years ago
Matt Clay dff662fa0f Add plugins to ansible-doc test and fix issues. 7 years ago
Matt Clay 15b6837daf Add yamllint for plugin docs and fix issues. 7 years ago
Matt Clay a7d7df1450 Make docs-build sanity test disabled by default. 7 years ago
Matt Clay 8a223009ca
Improve handling of integration test aliases. (#38698)
* Include change classification data in metadata.
* Add support for disabled tests.
* Add support for unstable tests.
* Add support for unsupported tests.
* Overhaul integration aliases sanity test.
* Update Shippable scripts to handle unstable tests.
* Mark unstable Azure tests.
* Mark unstable Windows tests.
* Mark disabled tests.
7 years ago
Matt Clay a5cbc0a2c8
Multiple ansible-test fixes. (#38247)
* Add ansible-test integration --allow-root option.
* Fix destructive target override.
* Fix bad type hint SanityResult -> TestResult.
* Fix skip/python3 filtering with --docker option.
7 years ago
Matt Clay 05220d693d
Complete updates of remaining code-smell tests. (#37743)
* Add text/binary file support to code smell tests.
* Enhance line-endings code smell test.
* Enhance no-smart-quotes code-smell test.
* Enhance shebang code-smell test.
7 years ago
Matt Clay a8487feb70 Fix ansible-test python and pip executable search. 7 years ago