Commit Graph

228 Commits (8a175f59c939ca29ad56f3fa9edbc37a8656879a)

Author SHA1 Message Date
Matt Clay 3a633b278d [devel] Azure Pipelines configuration. 4 years ago
Sam Doran 6984081111
Test changes to allow using "macos" test image (#71849)
* Add link to remote.sh from macos.sh
* Add skip/macos to tests that have skip/osx
4 years ago
Rick Elrod 4f54fb8d43
Remove incidental azure tests (#71164)
Change:
- All incidental coverage from these tests has moved to intentional
  coverage.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz fa1fb2d13b
Test ansible-galaxy against pulp/pulp_ansible (#69605)
* Test galaxy cli against pulp

* linting fix

* Renames and small fixes

* Better handling for resetting pulp

* Clean up some things, add a comment

* I can't spell

* Bump fallaxy, use alternate pulp image

* Only reset pulp when we're are executing against pulp

* Update for updated pulp container

* Update some comments with correct URLs and typos

* Linting fix

* Pin pulp-fedora31 to a digest

* Address review comments for documentation
4 years ago
Matt Clay 0faec62354 Remove temporary migration hack from CI scripts. 4 years ago
Felix Fontein a114da80ee
Use antsibull-changelog instead of packaged changelog generator (#69313)
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
4 years ago
Jordan Borean f5f3ba7ab5
ansible-test - fix up relative util import for powershell validate-modules (#69753)
* ansible-test - fix up relative util import for powershell validate-modules

* Use different tactic for generic group

* Use python 2 and 3
4 years ago
Matt Clay 5fc01c0a1d
Use a custom build container on Shippable. (#69660)
* Use a custom build container on Shippable.

This supports key generation before git_sync, to avoid issues with pre-migration PRs.

If the node pool is switched to another version, a matching build container should be built and used.

* Don't kill our own custom container.
4 years ago
Yanis Guenane 46d82179d8
Testing: Add support for CentOS Linux On Power platform (#68130)
* Testing: Add CentOS Linux On Power platform

* Add arch designation to remotes.

This avoids overloading the provider with the arch.

Also add a changelog entry.

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay 41f6c73be0 Update ansible-test network test provisioning. (#68220)
* Update network provisioning defaults.

* Update network test requirements.
4 years ago
Matt Clay 27ce6dbc1f Remove test support option from ansible-test.
This option was added to facilitate migration and is no longer needed.
4 years ago
Matt Clay fb3e6d6b00 Remove unused Shippable scripts. 4 years ago
Matt Clay 4cecacb915 Update Shippable test matrix. (#68117)
* Update Shippable test matrix.

* Remove unit test groups.

* Remove extra windows groups.
4 years ago
Matt Clay a51266ba85 Skip removed windows CI groups after migration. 4 years ago
Matt Clay 3508e1197e
Update test matrix to prepare for migration. (#67983)
* Split fallaxy tests into their own group.

This keeps expected pass/fail tests separate during migration prep.

* Move network tests down in matrix.

This will help keep pass/fail tests grouped together during migration prep.

* Group all incidental tests together.

This will keep pass/fail tests separate leading up to migration.
4 years ago
Matt Clay f4b923a29b Automatically enable test plugins after migration. 4 years ago
Matt Clay 64782dd815
Split unit tests into 3 groups. (#67917) 4 years ago
Matt Clay 42eba3ce25 Increase unit test timeout. 4 years ago
Matt Clay bcddf2bda9 Increase unit test coverage timeout. 4 years ago
Matt Clay 7c8b046b5f
Fourth batch of incidental integration tests. (#67873)
* Copy in incidental posix tests.

* Update incidental test aliases.

* Update target names.

* Add support plugins.

* Fix paths.

* Update ignores.

* Update integration-aliases sanity test.

* Add incidental tests to CI.
4 years ago
Matt Clay f735fd672a
Third batch of incidental integration tests. (#67830)
* Copy in incidental windows tests.

* Update incidental test aliases.

* Add support plugins.

* Update target references.

* Update sanity ignores.

* Update integration-aliases test.

* Add to CI.
4 years ago
Matt Clay e3591223a0
Second batch of incidental integration tests. (#67765)
* Update incidental test aliases.

* Rewrite target references for renamed targets.

* Add incidental tests to CI.

* Update sanity tests for incidental cloud tests.

* Initial copy of incidental tests.

* Copy contrib files into test.

* Update paths in test.

* Add support plugins.

* Update plugin to work around missing deps.

* Update sanity ignores.

* Fix matrix entries.

* Remove debug echo.
4 years ago
Matt Clay 4e571248a9
First batch of incidental integration tests. (#67717)
* Initial copy of incidental network tests.

* Update incidental test aliases.

* Add incidental tests to CI.

* Rewrite module references in tests.

This should not be necessary once module redirection is supported.

* Rewrite target references for renamed targets.

* Add support collections for incidental tests.

* Add ignores for test support code.

* Remove echo used for debugging.
4 years ago
Matt Clay 2dadc259d3
Split hcloud tests into two CI groups. (#67683)
* Split hcloud tests into two CI groups.

This will hopefully resolve CI failures when running with code coverage.

* Fix smoketest logic.
4 years ago
Matt Clay 4a885072af Split vyos tests to avoid timeouts with coverage. 4 years ago
Matt Clay a64b562071
Fix up network testing on Shippable. (#67678)
This prepares for code coverage collection for initial setup of incidental test coverage.
4 years ago
Matt Clay 19f6fc68d4
Relocate Shippable tools. (#67556)
* Move Shippable tools to hacking directory.

These limits the `test/utils/shippable/` directory to scripts required for CI.

* Fix `test/utils/shippable/` file classification.

* Update package-data sanity test.
4 years ago
Matt Clay 80561ccef0 Improve sanity test handling in CI after migration. 4 years ago
Matt Clay a0242b2173 Improve unit test handling in CI after migration. (#67217)
* Improve unit test handling in CI after migration.

* Fix undefined var in bash 4.3 and earlier.

See https://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES?id=3ba697465bc74fab513a26dea700cc82e9f4724e#n878
4 years ago
Matt Clay 5e68bb3d93
Add code coverage target analysis to ansible-test. (#67141)
* Refactor coverage file enumeration.
* Relocate sanitize_filename function.
* Support sets when writing JSON files.
* Generalize setting of info_stderr mode.
* Split out coverage path checking.
* Split out collection regex logic.
* Improve sanitize_filename type hints and docs.
* Clean up coverage erase command.
* Fix docs and type hints for initialize_coverage.
* Update type hints on CoverageConfig.
* Split out logic for finding modules.
* Split out arc enumeration.
* Split out powershell coverage enumeration.
* Raise verbosity level of empty coverage warnings.
* Add code coverage target analysis to ansible-test.
4 years ago
Yanis Guenane 4fd2dce7f3
Testing: Add support for AIX platform (#65802) 4 years ago
Matt Clay 0971a0e7bc Consolidate remote Shippable scripts. 4 years ago
Matt Clay 5c1fe78685
Move CI coverage stub to sanity/5. (#66827) 4 years ago
Matt Clay 452ade007e Clean up Shippable scripts in tools directory. 5 years ago
Matt Clay 78e476eb20 Increase unit test timeout in CI. 5 years ago
Matt Clay d2562cde1c Temporarily increase CI timeout. 5 years ago
Matt Clay ada02f1966 Limit Shippable matrix check to ansible repo. 5 years ago
Gonéri Le Bouder cc758ea574 Revert "shippable: temporary bump the timeout for Azure jobs"
The Azure timeout should not accure anymore since
173d47d1f4.

This reverts commit aaa8835311.
5 years ago
Sam Doran c9e7a2122d
Increase unit test timeout to account for long image download times (#61717) 5 years ago
Gonéri Le Bouder aaa8835311 shippable: temporary bump the timeout for Azure jobs 5 years ago
Matt Clay e4e5005640
Fix ansible-test coverage path handling. (#61528)
* Fix ansible-test coverage path handling.
* Split CI unit tests into two groups.
5 years ago
Sam Doran e5b24a68cc Add additional group for sanity tests
Put validate-modules in its own group
5 years ago
Matt Clay 20d9c34862 Fix CI test result placement. 5 years ago
Matt Clay f5d829392a
Overhaul ansible-test test path handling. (#61416)
* Remove .keep files from test/results/ dirs.

* Remove classification of test/results/ dir.

* Add results_relative to data context.

* Use variables in delegation paths.

* Standardize file writing and results paths.

* Fix issues reported by PyCharm.

* Clean up invocation of coverage command.

It now runs through the injector.

* Hack to allow intercept_command in cover.py.

* Simplify git ignore for test results.

* Use test result tmp dir instead of cache dir.

* Remove old .pytest_cache reference.

* Fix unit test docker delegation.

* Show HTML report link.

* Clean up more results references.

* Move import sanity test output to .tmp dir.

* Exclude test results dir from coverage.

* Fix import sanity test lib paths.

* Fix hard-coded import test paths.

* Fix most hard-coded integration test paths.

* Fix PyCharm warnings.

* Fix import placement.

* Fix integration test dir path.

* Fix Shippable scripts.

* Fix Shippable matrix check.

* Overhaul key pair management.
5 years ago
Jordan Borean faaa669764
PowerShell - Added coverage collector (#59009)
* Added coverage collection for PowerShell - ci_complete ci_coverage

* uncomment out coverage uploader call

* Generate XML for PowerShell coverage

* Use whitelist to exclude coverage run on non content plugins

* Remove uneeded ignore entry

* Try to reduce diff in cover.py

* Fix up coverage report package - ci_complete ci_coverage
5 years ago
Matt Clay 4d424d0830 Only run package-data sanity once in CI. 5 years ago
Jordan Borean 3a3727d200
Remove Python 2.6 from Windows tests (#60235) 5 years ago
Matt Clay b446ca100f
Prepare ansible-test code for relocation. (#60110)
* Fix imports in cli.py.

* Fix imports in executor.py.

* Remove old test/runner/ansible-test entry point.

Use the official bin/ansible-test entry point instead, which has been around since Ansible 2.5.

* Use bin/ansible-test on Shippable.
5 years ago
Matt Clay d3da8e4a5b
Replace old shippable.py with new check_matrix.py. (#60022)
This new script does not depend on ansible-test and provides much more robust job matrix testing.

It is also run on every job in the matrix now, to detect issues with jobs being re-run after matrix changes are made.
5 years ago
Matt Clay f944bd2358 Update ansible-test to properly skip unit tests.
Unit tests will no longer run on "remote only" Python versions (2.6) for tests which are not "remote" (modules and module_utils).
5 years ago