Commit Graph

79 Commits (7086863c526e4870a76b22e34589842113c18086)

Author SHA1 Message Date
Matt Clay c1f9efabf4
Overhaul httptester support in ansible-test. (#39892)
- Works with the --remote option.
- Can be disabled with the --disable-httptester option.
- Change image with the --httptester option.
- Only load and run httptester for targets that require it.
7 years ago
Matt Clay 8a223009ca
Improve handling of integration test aliases. (#38698)
* Include change classification data in metadata.
* Add support for disabled tests.
* Add support for unstable tests.
* Add support for unsupported tests.
* Overhaul integration aliases sanity test.
* Update Shippable scripts to handle unstable tests.
* Mark unstable Azure tests.
* Mark unstable Windows tests.
* Mark disabled tests.
7 years ago
Matt Clay a5cbc0a2c8
Multiple ansible-test fixes. (#38247)
* Add ansible-test integration --allow-root option.
* Fix destructive target override.
* Fix bad type hint SanityResult -> TestResult.
* Fix skip/python3 filtering with --docker option.
7 years ago
Matt Clay a8487feb70 Fix ansible-test python and pip executable search. 7 years ago
Matt Clay b9b8081a87
Cleanup and enhancements for ansible-test. (#37142)
* Fix type hint typos.
* Add one-time cloud env setup after delegation.
* Add generate_password to util.
* Add username/password support to HttpClient.
* Avoid pip requirement for ansible-test shell.
* Support provisioning Tower instances.
7 years ago
David Newswanger 3f5caf659e added support for --testcase flag in ansible-test (#36134)
* added support for --testcase flag in ansible-test

* fixed command format

* added tab completion

* fixed sanity issues

* added documenation for --testcase

* don't autocomplete when multiple modules are selected
7 years ago
Matt Clay 7abdab6c9e Convert ansible-test compile into a sanity test. 7 years ago
Matt Clay 5fa1edc15d
Track ansible-test cloud and target overhead. (#34902) 7 years ago
Matt Clay f968776d86 Remove ansible-test obsolete `net_*` target logic. 7 years ago
Matt Clay 82b5a6a0c9
Fix ansible-test network-integration command. (#34661)
* Fix ansible-test network platform init filter.
* Fix ansible-test network inventory generation.
* Remove ios/csr1000v from CI.
* Run network tests on Python 2.7 and 3.6.
7 years ago
Matt Clay 1b5c4b72bd Add Azure provider support to ansible-test and CI.
ci_complete
7 years ago
Matt Clay 1ee511f82c Use an abspath for network inventory ssh key path. 7 years ago
Matt Clay f9cd50411f Enable Python 3.7 in ansible-test. 7 years ago
Matt Clay ef21038dd5 Fix ansible-test race calling get_coverage_path. 7 years ago
Matt Clay 6472723ba8
Add missing ansible-test --remote-terminate support. (#32918)
* Expand ansible-test --remote-terminate support:

- windows-integration
- network-integration

These commands previously accepted the option, but did not support it.

* Terminate windows and network instances when done.
7 years ago
Matt Clay cf1337ca9a Update ansible-test sanity command. (#31958)
* Use correct pip version in ansible-test.
* Add git fallback for validate-modules.
* Run sanity tests in a docker container.
* Use correct python version for sanity tests.
* Pin docker completion images and add default.
* Split pylint execution into multiple contexts.
* Only test .py files in use-argspec-type-path test.
* Accept identical python interpeter name or binary.
* Switch cloud tests to default container.
* Remove unused extras from pip install.
* Filter out empty pip commands.
* Don't force running of pip list.
* Support delegation for windows and network tests.
* Fix ansible-test python version usage.
* Fix ansible-test python version skipping.
* Use absolute path for log in ansible-test.
* Run vyos_command test on python 3.
* Fix windows/network instance persistence.
* Add `test/cache` dir to classification.
* Enable more python versions for network tests.
* Fix cs_router test.
7 years ago
Matt Clay 36b13e3e3d Fix ansible-test default image. (#31966)
* Add openssh-client to default docker container.
* Include Azure requirements in default container.

To do so, handling of pip requirements was updated to install each
set of requirements separately and then run a verification pass to
make sure there are no conflicts between requirements.

* Add missing --docker-no-pull option.
* Add documentation for the azure-requirements test.
7 years ago
Matt Clay a1453a7917 Improve handling of first CI run for new branch. (#29070) 7 years ago
Matt Clay f71d861563 Collect data from integration test runs. (#28650) 7 years ago
Matt Clay 5ea8a5e34b Add support for setup targets to ansible-test. (#28544)
* Add support for setup targets to ansible-test.
* Code cleanup.
7 years ago
Ricardo Carrillo Cruz 746b433c29 Remove 'net' hardcoding on ansible-test executor (#28334)
We need to run network-integration against platform agnostic modules,
which they all are named as net_* .
There is a hardcoding in executor where if prefix is 'net' the hosts is set
to all, removing these lines to have desired behaviour.
7 years ago
Matt Clay f6d7fc548e Fix ansible-test type hints. 7 years ago
Matt Clay e241e15899 Add support for `non_local` alias in ansible-test. 7 years ago
Matt Clay 27033915cc Add --list-targets and speed up --explain. (#26838)
* Add ansible-test --list-targets option.
* Speed up ansible-test integration --explain.
7 years ago
Matt Clay e63f69464f Add ansible-test options and docs. (#26823)
* All integration commands support --continue-on-error
* The network-integration command supports --inventory
* Add landing page for compile test docs.
* Add bot documentation links.
7 years ago
Matt Clay 7931e11437 Use subset of Windows tests for "all" tests. (#26830) 7 years ago
Matt Clay 0b784c65b1 Add sanity import test to ansible-test. (#26730)
* Add sanity import test to ansible-test.
* Run sanity import test on all python versions.
7 years ago
Matt Clay 20d49af475 Add ANSIBLE_TEST_CI env var to integration tests. 7 years ago
Matt Clay ab5dd24430 Handle pip/python errors in ansible-test better. 7 years ago
David Newswanger 294c0f91c0 added option to ansible-test to run test playbooks using the debug strategy (#26499) 7 years ago
Matt Clay 45e377566c Refactor ansible-test config classes. (#26505)
* Move Config classes from executor.py to config.py.
* Move Environment and Test config to config.py.
* Move Coverage/CoverageReport Config to config.py.
* Clean up type hints.
7 years ago
Matt Clay c26a556929 Add check for known_hosts changing during tests. (#26470)
* Add check for known_hosts changing during tests.
* Don't use known hosts file for delegate_to test.
7 years ago
David Newswanger 0a6dcf4169 changed ansible-test so that it replaces . with - instead of . with _ so that the hostname is valid 7 years ago
René Moser 77d0542985 test, integration: add diff support (#26296) 7 years ago
Pierre-Louis Bonicoli d08ff86169 ansible-test: pass --[skip-]tags to ansible-playbook
Thanks to Olaf Klischat for the idea.
7 years ago
Matt Clay 53180ebb57 Fix Shippable BASE_BRANCH lookup on merges. 8 years ago
Matt Clay 5babe2daea Increase python version coverage for tests. (#24762)
* Improve ansible-test inventory handling.
* Fix python 3 re-raise of exception from thread.
* Fix python 3 encoding for windows-integration.
* Run network tests on multiple python versions.
* Run windows tests on multiple python versions.
* Support Shippable delegation using --tox.
* Skip vyos_command on python 3 tests until fixed.
* Add python 3 filtering to local and tox.
* Fix tests to support back to back runs.
* Temporarily test networking with python 2.7 only.

Running the tests back to back causes intermittent test failures
which need to be addressed before we can test multiple versions
in a single test run.
8 years ago
Matt Clay 6a2a7a2392 Enable codecov.io and add coverage grouping. 8 years ago
Matt Clay fdcad0f44d Improve on-demand coverage for Shippable. 8 years ago
John R Barker e9e2d67035 Run vyos_command Network tests in Shippable (#24514)
* Run VyOS Network tests in Shippable

* net_command no longer exists

* network/ci

* Disable other network tests

* pep8 fix

* Fix PEP 8 issue.
8 years ago
Matt Clay dfd19a812f Miscellaneous bug fixes for ansible-test.
- Overhauled coverage injector to fix issues with non-local tests.
- Updated integration tests to work with the new coverage injector.
- Fix concurrency issue by using random temp files for delegation.
- Fix handling of coverage files from root user.
- Fix handling of coverage files without arcs.
- Make sure temp copy of injector is world readable and executable.
8 years ago
Matt Clay b8cb3f519b Detect and fix environment tampering in tests. 8 years ago
Matt Clay a07d42e16d Add support for cloud tests to ansible-test. (#24315)
* Split out ansible-test docker functions.
* Add cloud support to ansible-test.
8 years ago
Matt Clay 6522d703a1 Fix ansible-test pylint and pep8 issues. (#24245)
* Fix pylint useless-super-delegation issues.
* Fix miscellaneous pylint issues.
* Fix PEP 8 issues.
* Fix warnings reported by PyCharm.
8 years ago
Matt Clay 991bf0a1c3 Fix ansible-test base branch handling on merge. (#22759) 8 years ago
Matt Clay 47887a2edd Fix ansible-test base branch usage on Shippable. 8 years ago
Matt Clay 3a22467b38 Remove python 2.4 compile test. 8 years ago
Matt Clay 869449e288 Add test verification to ansible-test. (#22636)
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
8 years ago
Matt Clay 89559f78de Add `--failure-ok` option to `ansible-test`. (#22623) 8 years ago
Matt Clay 3ab344e8bd Use lint/junit output for compile test. (#22395)
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
8 years ago