123 Commits (945e35ada2c76b3c410fe75ef197db0a88c9de5d)

Author	SHA1	Message	Date
Matt Clay	7abdab6c9e	Convert ansible-test compile into a sanity test.	7 years ago
Matt Clay	5fa1edc15d	Track ansible-test cloud and target overhead. (#34902)	7 years ago
Matt Clay	f968776d86	Remove ansible-test obsolete `net_*` target logic.	7 years ago
Matt Clay	82b5a6a0c9	Fix ansible-test network-integration command. (#34661) ... * Fix ansible-test network platform init filter. * Fix ansible-test network inventory generation. * Remove ios/csr1000v from CI. * Run network tests on Python 2.7 and 3.6.	7 years ago
Matt Clay	1b5c4b72bd	Add Azure provider support to ansible-test and CI. ... ci_complete	7 years ago
Matt Clay	1ee511f82c	Use an abspath for network inventory ssh key path.	7 years ago
Matt Clay	f9cd50411f	Enable Python 3.7 in ansible-test.	7 years ago
Matt Clay	ef21038dd5	Fix ansible-test race calling get_coverage_path.	7 years ago
Matt Clay	6472723ba8	Add missing ansible-test --remote-terminate support. (#32918) ... * Expand ansible-test --remote-terminate support: - windows-integration - network-integration These commands previously accepted the option, but did not support it. * Terminate windows and network instances when done.	7 years ago
Matt Clay	cf1337ca9a	Update ansible-test sanity command. (#31958) ... * Use correct pip version in ansible-test. * Add git fallback for validate-modules. * Run sanity tests in a docker container. * Use correct python version for sanity tests. * Pin docker completion images and add default. * Split pylint execution into multiple contexts. * Only test .py files in use-argspec-type-path test. * Accept identical python interpeter name or binary. * Switch cloud tests to default container. * Remove unused extras from pip install. * Filter out empty pip commands. * Don't force running of pip list. * Support delegation for windows and network tests. * Fix ansible-test python version usage. * Fix ansible-test python version skipping. * Use absolute path for log in ansible-test. * Run vyos_command test on python 3. * Fix windows/network instance persistence. * Add `test/cache` dir to classification. * Enable more python versions for network tests. * Fix cs_router test.	7 years ago
Matt Clay	36b13e3e3d	Fix ansible-test default image. (#31966) ... * Add openssh-client to default docker container. * Include Azure requirements in default container. To do so, handling of pip requirements was updated to install each set of requirements separately and then run a verification pass to make sure there are no conflicts between requirements. * Add missing --docker-no-pull option. * Add documentation for the azure-requirements test.	7 years ago
Matt Clay	a1453a7917	Improve handling of first CI run for new branch. (#29070)	7 years ago
Matt Clay	f71d861563	Collect data from integration test runs. (#28650)	7 years ago
Matt Clay	5ea8a5e34b	Add support for setup targets to ansible-test. (#28544) ... * Add support for setup targets to ansible-test. * Code cleanup.	7 years ago
Ricardo Carrillo Cruz	746b433c29	Remove 'net' hardcoding on ansible-test executor (#28334) ... We need to run network-integration against platform agnostic modules, which they all are named as net_* . There is a hardcoding in executor where if prefix is 'net' the hosts is set to all, removing these lines to have desired behaviour.	7 years ago
Matt Clay	f6d7fc548e	Fix ansible-test type hints.	7 years ago
Matt Clay	e241e15899	Add support for `non_local` alias in ansible-test.	7 years ago
Matt Clay	27033915cc	Add --list-targets and speed up --explain. (#26838) ... * Add ansible-test --list-targets option. * Speed up ansible-test integration --explain.	7 years ago
Matt Clay	e63f69464f	Add ansible-test options and docs. (#26823) ... * All integration commands support --continue-on-error * The network-integration command supports --inventory * Add landing page for compile test docs. * Add bot documentation links.	7 years ago
Matt Clay	7931e11437	Use subset of Windows tests for "all" tests. (#26830)	7 years ago
Matt Clay	0b784c65b1	Add sanity import test to ansible-test. (#26730) ... * Add sanity import test to ansible-test. * Run sanity import test on all python versions.	7 years ago
Matt Clay	20d49af475	Add ANSIBLE_TEST_CI env var to integration tests.	7 years ago
Matt Clay	ab5dd24430	Handle pip/python errors in ansible-test better.	7 years ago
David Newswanger	294c0f91c0	added option to ansible-test to run test playbooks using the debug strategy (#26499)	7 years ago
Matt Clay	45e377566c	Refactor ansible-test config classes. (#26505) ... * Move Config classes from executor.py to config.py. * Move Environment and Test config to config.py. * Move Coverage/CoverageReport Config to config.py. * Clean up type hints.	7 years ago
Matt Clay	c26a556929	Add check for known_hosts changing during tests. (#26470) ... * Add check for known_hosts changing during tests. * Don't use known hosts file for delegate_to test.	7 years ago
David Newswanger	0a6dcf4169	changed ansible-test so that it replaces . with - instead of . with _ so that the hostname is valid	7 years ago
René Moser	77d0542985	test, integration: add diff support (#26296)	7 years ago
Pierre-Louis Bonicoli	d08ff86169	ansible-test: pass --[skip-]tags to ansible-playbook ... Thanks to Olaf Klischat for the idea.	7 years ago
Matt Clay	53180ebb57	Fix Shippable BASE_BRANCH lookup on merges.	8 years ago
Matt Clay	5babe2daea	Increase python version coverage for tests. (#24762) ... * Improve ansible-test inventory handling. * Fix python 3 re-raise of exception from thread. * Fix python 3 encoding for windows-integration. * Run network tests on multiple python versions. * Run windows tests on multiple python versions. * Support Shippable delegation using --tox. * Skip vyos_command on python 3 tests until fixed. * Add python 3 filtering to local and tox. * Fix tests to support back to back runs. * Temporarily test networking with python 2.7 only. Running the tests back to back causes intermittent test failures which need to be addressed before we can test multiple versions in a single test run.	8 years ago
Matt Clay	6a2a7a2392	Enable codecov.io and add coverage grouping.	8 years ago
Matt Clay	fdcad0f44d	Improve on-demand coverage for Shippable.	8 years ago
John R Barker	e9e2d67035	Run vyos_command Network tests in Shippable (#24514) ... * Run VyOS Network tests in Shippable * net_command no longer exists * network/ci * Disable other network tests * pep8 fix * Fix PEP 8 issue.	8 years ago
Matt Clay	dfd19a812f	Miscellaneous bug fixes for ansible-test. ... - Overhauled coverage injector to fix issues with non-local tests. - Updated integration tests to work with the new coverage injector. - Fix concurrency issue by using random temp files for delegation. - Fix handling of coverage files from root user. - Fix handling of coverage files without arcs. - Make sure temp copy of injector is world readable and executable.	8 years ago
Matt Clay	b8cb3f519b	Detect and fix environment tampering in tests.	8 years ago
Matt Clay	a07d42e16d	Add support for cloud tests to ansible-test. (#24315) ... * Split out ansible-test docker functions. * Add cloud support to ansible-test.	8 years ago
Matt Clay	6522d703a1	Fix ansible-test pylint and pep8 issues. (#24245) ... * Fix pylint useless-super-delegation issues. * Fix miscellaneous pylint issues. * Fix PEP 8 issues. * Fix warnings reported by PyCharm.	8 years ago
Matt Clay	991bf0a1c3	Fix ansible-test base branch handling on merge. (#22759)	8 years ago
Matt Clay	47887a2edd	Fix ansible-test base branch usage on Shippable.	8 years ago
Matt Clay	3a22467b38	Remove python 2.4 compile test.	8 years ago
Matt Clay	869449e288	Add test verification to ansible-test. (#22636) ... * Add unified git diff parser. * Add metadata and diff handling. * Add test confidence/verification to bot output.	8 years ago
Matt Clay	89559f78de	Add `--failure-ok` option to `ansible-test`. (#22623)	8 years ago
Matt Clay	3ab344e8bd	Use lint/junit output for compile test. (#22395) ... * Refactor sanity classes for use in all tests. * Use lint/junit output for compile test. * Add missing options for compile test. * Fix early bailout on requirements install.	8 years ago
Matt Clay	fbf55886ba	Use lint friendly output for python syntax checks. (#22390)	8 years ago
Matt Clay	d66ce40ecb	Overhaul `ansible-test sanity` implementation. (#22177) ... - Tests are run to completion instead of stopping on first failure. - Test results are now parsed instead of passing through to the console. - Test results can be saved in junit xml format. - Test results will show up on the Shippable "Tests" result tab. - Added an experimental --lint option for easier integration with other tools. - Code smell tests are now usable with the --list-tests, --test and --skip-test options. - Code split out from executor.py into sanity.py. - Rename download-logs to download.py and add support for test and coverage results. - Miscellaneous improvements.	8 years ago
Matt Clay	28b882e737	Support ansible commands in sanity tests. (#22172)	8 years ago
Matt Clay	45cec93e99	Disable newer shellcheck tests which don't pass.	8 years ago
Matt Clay	fcac261eef	Run unit tests in isolation w/ coverage support.	8 years ago
Matt Clay	2797cff773	Run destructive win tests when creating instances.	8 years ago
Matt Clay	bc8186543a	Pass base branch to module validator on delegate.	8 years ago
Matt Martz	9b1bd2eb7e	Re-enable module comparisons in validate-modules (#21582) ... * Re-enable module comparisons, specifically for new module detection and for finding new options/arguments * Only do new module checks in shippable, local will display warning	8 years ago
Matt Clay	2192c1eb02	Fix ansible-test selection of inventory file.	8 years ago
John R Barker	5e28d3a15c	network_cli shouldn't be specified anymore (#21458)	8 years ago
Matt Clay	d54bc09fae	Consider module_utils deps when running tests. (#21382) ... * Skip pep8 analysis when --explain is used. * Fix return type annotations. * Match line length requirement of PEP 8 config. * Consider module_utils deps when running tests.	8 years ago
Matt Clay	4189592701	Revert "Enable process isolation for unit tests." ... This reverts commit 91526cd9f2. Removing this feature primarily because it interferes with collecting proper code coverage results. I may restore the feature later if that can be resolved.	8 years ago
Matt Clay	91526cd9f2	Enable process isolation for unit tests.	8 years ago
Matt Clay	6ef1a6aeb6	Add `pep8` to `ansible-test`. (#20745) ... The rule sets and legacy file list are a first draft. It is likely that they will need to be revised.	8 years ago
Matt Clay	fa7f84217b	Add startup checks to ansible-test.	8 years ago
Matt Clay	61e6e7868c	Improve jinja2 test coverage. (#20533) ... * Run `template` test on latest jinja2 version. * Update jinja2 install for groupby_filter. * Clean test output dir for each test.	8 years ago
Matt Clay	4ba9f63afe	Use multiple AWS regions in ansible-test. (#20429) ... * Move ansible-test EnvironmentConfig to util. * Use multiple AWS regions in ansible-test.	8 years ago
Matt Clay	4673a9c9c2	Generate ssh key once for network tests.	8 years ago
Matt Clay	3c51ce56fd	Enable more network support in ansible-test. (#20372) ... * Only start platform instances with tests selected. * Enable ios on Shippable. * Show inventory in explain mode. * Fix indentation of generated network inventory. * Update classification of network module_utils.	8 years ago
Matt Clay	e2bd71ac02	Clean up inventory generation in ansible-test. (#20313)	8 years ago
Matt Clay	d8733a5455	Complete initial network-integration support.	8 years ago
Matt Clay	f340b8d14f	Add test support for python 3.6.	8 years ago
Matt Clay	dac226de4a	Improve ansible-test completion and messaging. (#19627) ... * Simplify type annotation of args.docker_util. * Improve error messaging for ansible-doc test. * Add completion for docker, remote and windows.	8 years ago
Matt Clay	422857166b	Remove legacy test alias migration script. (#19629) ... * Require code-smell tests to be files. * Add sanity check for integration test aliases. * Remove migration script for test aliases.	8 years ago
Matt Clay	9b5c782a0b	Use `docker pull` by default in ansible-test.	8 years ago
Matt Clay	f28b5a0ed8	Add --tox-sitepackages option to ansible-test.	8 years ago
Matt Clay	47083f145a	Don't error if pytest skips all unit tests.	8 years ago
Matt Clay	bee58fe890	Fix compile skip path handling for targeted tests.	8 years ago
Matt Clay	6bbd92e422	Initial ansible-test implementation. (#18556)	8 years ago