Commit Graph

393 Commits (0c514bcf54ee1c20841b905bd9290071bfc9de3a)

Author SHA1 Message Date
Matt Clay e4e5005640
Fix ansible-test coverage path handling. (#61528)
* Fix ansible-test coverage path handling.
* Split CI unit tests into two groups.
5 years ago
Sam Doran e5b24a68cc Add additional group for sanity tests
Put validate-modules in its own group
5 years ago
Matt Clay 20d9c34862 Fix CI test result placement. 5 years ago
Matt Clay f5d829392a
Overhaul ansible-test test path handling. (#61416)
* Remove .keep files from test/results/ dirs.

* Remove classification of test/results/ dir.

* Add results_relative to data context.

* Use variables in delegation paths.

* Standardize file writing and results paths.

* Fix issues reported by PyCharm.

* Clean up invocation of coverage command.

It now runs through the injector.

* Hack to allow intercept_command in cover.py.

* Simplify git ignore for test results.

* Use test result tmp dir instead of cache dir.

* Remove old .pytest_cache reference.

* Fix unit test docker delegation.

* Show HTML report link.

* Clean up more results references.

* Move import sanity test output to .tmp dir.

* Exclude test results dir from coverage.

* Fix import sanity test lib paths.

* Fix hard-coded import test paths.

* Fix most hard-coded integration test paths.

* Fix PyCharm warnings.

* Fix import placement.

* Fix integration test dir path.

* Fix Shippable scripts.

* Fix Shippable matrix check.

* Overhaul key pair management.
5 years ago
Jordan Borean faaa669764
PowerShell - Added coverage collector (#59009)
* Added coverage collection for PowerShell - ci_complete ci_coverage

* uncomment out coverage uploader call

* Generate XML for PowerShell coverage

* Use whitelist to exclude coverage run on non content plugins

* Remove uneeded ignore entry

* Try to reduce diff in cover.py

* Fix up coverage report package - ci_complete ci_coverage
5 years ago
Matt Clay 4d424d0830 Only run package-data sanity once in CI. 5 years ago
Jordan Borean 3a3727d200
Remove Python 2.6 from Windows tests (#60235) 5 years ago
Matt Clay b446ca100f
Prepare ansible-test code for relocation. (#60110)
* Fix imports in cli.py.

* Fix imports in executor.py.

* Remove old test/runner/ansible-test entry point.

Use the official bin/ansible-test entry point instead, which has been around since Ansible 2.5.

* Use bin/ansible-test on Shippable.
5 years ago
Matt Clay 31c7a61f9d
Clean up old ansible-test references and code. (#60103)
* Remove obsolete ansible-test retry.py.
* Use ansible-test from bin/ intead of test/runner/.
5 years ago
Matt Clay d3da8e4a5b
Replace old shippable.py with new check_matrix.py. (#60022)
This new script does not depend on ansible-test and provides much more robust job matrix testing.

It is also run on every job in the matrix now, to detect issues with jobs being re-run after matrix changes are made.
5 years ago
Matt Clay f944bd2358 Update ansible-test to properly skip unit tests.
Unit tests will no longer run on "remote only" Python versions (2.6) for tests which are not "remote" (modules and module_utils).
5 years ago
Matt Martz 95f4282c42
Verify package data in setup.py installs all files (#59537)
* Add sanity test to ensure all non-py files are installed

* Fix mode and regex

* Fix role skel inventory package_data

* Add docs

* Update package_data for inventory files

* Address pylint concerns

* Another tweak to package_data

* Address review feedback

* Change index to 1

* add to ansible-only.txt
5 years ago
Matt Clay 13bbfacb04 Show ansible-test --explain errors in CI. 5 years ago
Matt Clay 3f2b766d10 Add future and metaclass boilerplate to test code.
Continue to ignore:

- test/integration/
- test/legacy/
- test/units/
5 years ago
Matt Clay 79685a4a2d
Use Python 3.7 to run ansible-test coverage in CI. (#57555)
Generating XML for coverage reports on Python 3.7 uses far less
memory than on Python 2.7. This change should avoid the MemoryError
we've been seeing in the nightly coverage runs.
6 years ago
Matt Clay b1e6ad9754
Limit Python 2.6 units to modules & module_utils. (#57377) 6 years ago
Matt Clay 85ae8f5258 Add Hetzner Cloud to CI integration tests. 6 years ago
Matt Clay c846501829 Increase unit test timeout for --coverage. 6 years ago
Matt Clay 2a8252ceb9 Slightly increase non-coverage unit test timeout. 6 years ago
Matt Clay 8fa294a33a
Fix ansible-test coverage stub memory usage. (#53652)
Split coverage stub files into multiple parts based on the number
of lines in files included in each stub. This will keep memory usage
down to avoid hanging on Shippable.
6 years ago
Matt Clay 887ab35656
Add support for ansible-test --coverage-check. (#53573) 6 years ago
Matt Clay 9546761407 Fix coverage stub generation in CI. 6 years ago
Matt Clay a8d829d9c3
Add timeout support to ansible-test. (#53302)
* Add timeout support to ansible-test.
* Fix ansible-test tar filename filter bug.
* Update timeouts used on Shippable.
* Kill subprocesses when parent process terminates.
* Require explicit use of env --show option.
6 years ago
Matt Clay b2f354ce29 Ignore errors during container cleanup in CI. 6 years ago
Matt Clay a1fec259c1
Reduce console noise at start of CI session. (#52387)
* Reduce console noise at start of CI session.
* Use allowed shebang.
6 years ago
Matt Clay 6a82827ddd Add Python 3.8 to ansible-test and CI. 6 years ago
Matt Clay d2a9b16f7e Make Shippable timing script more resilient.
This will permit the script to pass through content which could
not be previously decoded or encoded. This could occur when running
some tests on macOS using file paths with non-ASCII characters.
6 years ago
John R Barker 721a28202e
Revert Debug: Size of coverage file #50380 (#50945) 6 years ago
Matt Clay 31c1bdd6a8 Fix run.py --branch argument.
The API docs state that both `branch` and `branchName` are valid,
but only `branchName` appears to work.
6 years ago
John R Barker e6ced9c73f
Debug: Size of coverage file (#50380) 6 years ago
John R Barker 7ec82cdfc9
Revert TEMP HACK: Upload all to codecov (#50279)
Codecov.io have made some improvements to results processing.
Upload all the coverage data to see is codecov can process this.

Reverts https://github.com/ansible/ansible/pull/47515
6 years ago
Matt Clay 01833b6fb1
Add `env` command to ansible-test and run in CI. (#50176)
* Add `env` command to ansible-test and run in CI.
* Avoid unnecessary docker pull.
6 years ago
Matt Clay 9979a32e5c Fix passing of env vars to Shippable. 6 years ago
John R Barker fb2e838a42
TEMP HACK: Avoid overloading codecov.io (#47515)
Without this patch we are overloading codecov.io by uploading 90+ tests.

As a workaround limit uploading to only "Group 1"

Will be removed/updated based on codecov.io's support team
6 years ago
Matt Clay 6a51b5ed26 Use 4 nodes for sanity tests in CI. 6 years ago
Matt Clay 2761fe8272 Revert "Use 4 nodes for sanity tests in CI."
This reverts commit b887333029.
6 years ago
Matt Clay b887333029 Use 4 nodes for sanity tests in CI. 6 years ago
Jordan Borean 25c627d256
shippable: fix py3 issues with downloader (#46522) 6 years ago
Matt Clay e53390b3b1
Fix ansible-test multi-group smoke test handling. (#46363)
* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.
6 years ago
Matt Clay ac492476e5
Bug fixes and cleanup for ansible-test. (#45991)
* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.
6 years ago
Matt Clay 771768966f Flush output from timing script. 6 years ago
Matt Clay a96a51b0c6 List quay.io images during Shippable run. 6 years ago
Matt Clay 78bd020429 Remove obsolete `test/utils/docker/` directory. 6 years ago
Matt Clay 5461bfbf31 Split sanity tests into 3 groups. 6 years ago
Matt Clay 31a5b874a1 Use runtime instead of datetime for test output. 6 years ago
Matt Clay 4e489d1be8
Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.
6 years ago
Matt Clay 528de5b82f Remove unwanted containers at CI job start. 6 years ago
Matt Clay a7d7df1450 Make docs-build sanity test disabled by default. 7 years ago
Matt Clay ee02d0c120 Remove obsolete test containers.
- opensuse42.1 - EOL
- opensuse42.2 - EOL
- ubuntu1204 - EOL
- cloudstack-simulator - Migrated to https://github.com/ansible/cloudstack-test-container
7 years ago
Matt Clay 26312b6557 Remove migrated httptester files.
The httptester has been migrated to:

https://github.com/ansible/http-test-container
7 years ago
Matt Clay 27def6c5dc Remove vcenter-simulator container files.
The simulator has moved to its own repository:

https://github.com/ansible/vcenter-test-container
7 years ago
Matt Clay 8a223009ca
Improve handling of integration test aliases. (#38698)
* Include change classification data in metadata.
* Add support for disabled tests.
* Add support for unstable tests.
* Add support for unsupported tests.
* Overhaul integration aliases sanity test.
* Update Shippable scripts to handle unstable tests.
* Mark unstable Azure tests.
* Mark unstable Windows tests.
* Mark disabled tests.
7 years ago
Martin Krizek cb5e594c13 Fix building from ubuntu1404 Dockerfile 7 years ago
Matt Martz 7c311ad615 Update httptester for httpbin==0.6.2 7 years ago
Matt Clay 11ad559010
Terminate Tower instances after CI ends. (#37265)
* Remove obsolete Tower support from manage_ci.
* Add missing remote settings to cloud tests.
7 years ago
Matt Clay 1d419c737d Run each Windows version independently in CI. 7 years ago
Matt Martz e3b6b40d88
Match recent Dockerfile change in httptester playbook file, to extend ca cert lifetime (#35734) 7 years ago
Matt Clay df3265a811 Extend httptester CA cert lifetime. 7 years ago
John R Barker 9728857e01
Remove network-test-runner (#35556)
Not been used for a while, replaced by DCI
7 years ago
Matt Clay 7abdab6c9e Convert ansible-test compile into a sanity test. 7 years ago
Matt Clay eae23d2dbd Skip coverage on routine CI runs. 7 years ago
Matt Clay 20824b8f3d Simplify Windows versions in ansible-test. 7 years ago
Matt Clay d8673ed72a Preserve `test/results/data/` in Shippable jobs. 7 years ago
Matt Davis fbce62795d enable Windows Server 2016 in CI (#34127)
* enable Windows Server 2016 in CI
* avoid use of Write-Host in config script
7 years ago
Matt Clay 82b5a6a0c9
Fix ansible-test network-integration command. (#34661)
* Fix ansible-test network platform init filter.
* Fix ansible-test network inventory generation.
* Remove ios/csr1000v from CI.
* Run network tests on Python 2.7 and 3.6.
7 years ago
Matt Clay d5eef2f8a9 Use `dnf upgrade` in Fedora Dockerfiles. 7 years ago
Matt Clay 61d155b315 Update Fedora Dockerfiles for Python 3 testing. 7 years ago
Matt Clay b7bf98cd3a Add fedora27py3 Dockerfile. 7 years ago
Matt Martz a0ad7ffedf
Update httptester.yml to match Dockerfile (#34508) 7 years ago
Matt Clay fba178d852 Fix httptester Dockerfile so it builds again. 7 years ago
Matt Clay 1b5c4b72bd Add Azure provider support to ansible-test and CI.
ci_complete
7 years ago
Matt Clay 887f227994 Use vyos/1.1.8 in CI. 7 years ago
Matt Clay 2ef438c962 Improve zypper refresh/clean for opensuse images. 7 years ago
Matt Clay 6472723ba8
Add missing ansible-test --remote-terminate support. (#32918)
* Expand ansible-test --remote-terminate support:

- windows-integration
- network-integration

These commands previously accepted the option, but did not support it.

* Terminate windows and network instances when done.
7 years ago
John R Barker ff7c349f11
Provide details via DCI Env (#32770) 7 years ago
John R Barker 897b31f249
Use same TEST seperater as Shippable (#32758) 7 years ago
John R Barker 4443da2d79
Remove debug (#32736)
* Make script executable
* Tidy up comments
7 years ago
John R Barker 4464635787
Specify branch details, add verbose, disable colour (#32731) 7 years ago
John R Barker 6ce3972f21
Script to generate DCI Coverage (#32550) 7 years ago
Matt Clay cf1337ca9a Update ansible-test sanity command. (#31958)
* Use correct pip version in ansible-test.
* Add git fallback for validate-modules.
* Run sanity tests in a docker container.
* Use correct python version for sanity tests.
* Pin docker completion images and add default.
* Split pylint execution into multiple contexts.
* Only test .py files in use-argspec-type-path test.
* Accept identical python interpeter name or binary.
* Switch cloud tests to default container.
* Remove unused extras from pip install.
* Filter out empty pip commands.
* Don't force running of pip list.
* Support delegation for windows and network tests.
* Fix ansible-test python version usage.
* Fix ansible-test python version skipping.
* Use absolute path for log in ansible-test.
* Run vyos_command test on python 3.
* Fix windows/network instance persistence.
* Add `test/cache` dir to classification.
* Enable more python versions for network tests.
* Fix cs_router test.
7 years ago
James Tanner cc287daa3b Enable tracing and log collection in the vcsim container 7 years ago
Matt Clay 03e18aa52e Remove cryptography install from CI other test. 7 years ago
Matt Clay 5a91ed0a3b Switch CI for OS X back to a single job. (#31265)
* Update Shippable scripts for group/non-group use.
* Switch CI for OS X back to a single job.
7 years ago
Matt Clay 72237b63e7 Run OS X and FreeBSD CI tests using groups. (#31254)
* Run OS X and FreeBSD CI tests using groups.
* Improve readability of Shippable jobs.
7 years ago
jctanner 6b6a145027 Add an option to spawn vcsim in esx mode (#31155) 7 years ago
Matt Clay 765718bed2 Run all Windows tests if change detection disabled (#30320)
* Run all Windows tests if change detection disabled
* Run all Windows versions if change detection disabled
7 years ago
Matt Clay 8f4c8844c4 Divide Windows integration tests into 2 groups. (#29029)
* Divide Windows integration tests into 2 groups.
* Support `none` for `--changed-all-target`.
* Run 2 separate Windows groups on Shippable.
* Only run smoketest and minimal for the group1 job.
7 years ago
Matt Clay 55fb18b85c Split cloud tests into two CI groups. 7 years ago
Matt Clay e513ec10da Add safety check for Shippable "Rebuild with SSH" (#28857)
* Add safety check for Shippable "Rebuild with SSH".
* Run shippable.py for the `other` test.
7 years ago
Matt Clay 1a98962e19 Switch default cloud tests to smoketest. 7 years ago
3onyc 554496c404 [passwordstore] Use builtin _random_password function instead of pwgen (#25843)
* [password] _random_password -> random_password and moved to util/encrypt.py
* [passwordstore] Use built-in random_password instead of pwgen utility
* [passwordstore] Add integration tests
7 years ago
Matt Clay e1cfcdbbc6 Add opensuse42.3 docker image. 7 years ago
Matt Clay c59e32469f Add Fedora Python 3 Docker images and fix tests. (#27794)
* Add new Fedora docker images with Python 3.
* Use consistent env var for lookup test.
* Fix testing of virtualenv with Python 3.
* Fix docker_secret tests on Fedora 26.
* Add Python 3 support to Fedora postgresql test.
* Add Python 3 support to Fedora mysql tests.
* Fix uri test server for Python 3 on Fedora.
* Fix iso_extract test for Python 3 on Fedora.
* Add Python 3 support for Fedora to openssl tests.
* Fix dnf group test for Python 3 on Fedora.
* Use force with user deletion in become test.
7 years ago
Matt Clay 2cafb934eb Update vcenter-simulator base to fedora:26 image.
This is required because govmomi now requires go 1.8+ to be installed.
7 years ago
Matt Clay e0e5b4cf64 Optimize running of CI cloud integration tests. 7 years ago
Matt Clay 7931e11437 Use subset of Windows tests for "all" tests. (#26830) 7 years ago
Matt Clay 0b784c65b1 Add sanity import test to ansible-test. (#26730)
* Add sanity import test to ansible-test.
* Run sanity import test on all python versions.
7 years ago
Matt Clay 4a013d6d69 Add RHEL 7.3 to ansible-test and CI. (#26550)
* Add support for RHEL to ansible-test.
* Enable RHEL 7.3 on Shippable.
8 years ago
Matt Clay 7ab4a889ad Update tox.ini to use ansible-test. 8 years ago
Abhijeet Kasurde bc4ff8aed9 Modify flask_control for vmware_host test suites (#26175)
Fix adds additional helper methods to get information about
host system from vcsim

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
8 years ago