Commit Graph

350 Commits (1305fca414e5473b636e0f15388547b4cf34acc1)

Author SHA1 Message Date
Matt Clay b8cb3f519b Detect and fix environment tampering in tests. 8 years ago
Matt Clay e6934e42be Update ansible-test change handling and CS plugin. (#24395)
* Use CloudStack simulator from Ansible repo.
* Recognize cloud specific ansible-test changes.
* Hide ansible-test cs notice in explain mode.
8 years ago
Matt Clay 0573b8bcd2 Fix ansible-test cloud integration bugs. 8 years ago
Matt Clay ac72fd9d2c Add missing return statements. 8 years ago
Matt Clay 201242f302 Fix call to wrong method in ansible-test. 8 years ago
Matt Clay 1582e9bf7d Fix handling of Shippable env vars. 8 years ago
Matt Clay a07d42e16d Add support for cloud tests to ansible-test. (#24315)
* Split out ansible-test docker functions.
* Add cloud support to ansible-test.
8 years ago
Matt Clay 6522d703a1 Fix ansible-test pylint and pep8 issues. (#24245)
* Fix pylint useless-super-delegation issues.
* Fix miscellaneous pylint issues.
* Fix PEP 8 issues.
* Fix warnings reported by PyCharm.
8 years ago
Matt Clay d662f6f0db Fix bugs in ansible-test units command. (#24044)
* Handle old versions of coverage.
* Handle old versions of setuptools.
* Detect python version for docker/remote units.
* Add sanity override for test constraints.
8 years ago
Matt Clay cb1f57d4e5 Add rstcheck to ansible-test and correct issues. (#23550)
* Add rstcheck to ansible-test.
* Fix rst code-block languages and syntax errors.
* Fix rst inline literals.
* Update python 2 code block to pass tests on py 3.
8 years ago
Matt Clay 4ee96b20a3 Temporarily cap pylint version < 1.7.0 in tests. 8 years ago
Toshio Kuratomi 4e8e366739 Ryanpineo fix ipaddr netmask with 32 cidr (#23506)
* Fix bug in ipaddr netmask filter when cidr is 32

* Add requirement on netaddr to the units.txt file
8 years ago
Matt Clay 92ad9a19b6 Fix ansible-test empty diff handling on Shippable. 8 years ago
Matt Clay 0cf1e8e061 Fix ansible-test handling of import/syntax errors. 8 years ago
Matt Clay 1daa69d685 Initial pylint support for ansible-test sanity. 8 years ago
Ryan Brown 2196fa0e95 [cloud][tests] fixtures for placebo to test AWS modules using boto3 (#21253)
* [cloud][tests] Create fixtures for using placebo to test boto3-using modules

* Use pytest's importorskip instead of manually skipping on missing deps

* Fix imports in cloudformation module

* Delete unused code

* Add maybe_sleep fixtures to speed up recorded test runs

* Build basic placebo-CFN tests

* Commit placebo recordings of basic stack operations

* Add placebo to test-requires

* Allow unit tests to run regardless of environment by setting a default region

* Use explicit relative import for Python 3 compat

* Use __name__ attribute that works on Python 2 and 3
8 years ago
Matt Clay 4fdeade389 Overhaul ansible-test import analysis. (#22888)
- Add support for module_utils packages for better organization.
- Add support for "virtual" module_utils packages such as `six`.
8 years ago
Matt Clay 042e387f27 Fix metadata handling for shell command. (#22801) 8 years ago
Matt Clay 991bf0a1c3 Fix ansible-test base branch handling on merge. (#22759) 8 years ago
Matt Clay 47887a2edd Fix ansible-test base branch usage on Shippable. 8 years ago
Matt Clay f7c83ccdc0 Fix ansible-test PR diff handling.
Permits use of PRs on branches other than devel.
8 years ago
Matt Clay 3a22467b38 Remove python 2.4 compile test. 8 years ago
Matt Clay 869449e288 Add test verification to ansible-test. (#22636)
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
8 years ago
Matt Clay 5e9a2b8528 Verify SHA in Shippable change detection. 8 years ago
Matt Clay 89559f78de Add `--failure-ok` option to `ansible-test`. (#22623) 8 years ago
Matt Clay 771b14acb2 Sort test messages for deterministic output. (#22581) 8 years ago
Matt Clay cfd313bced Disable ansible color output on sanity tests. 8 years ago
Matt Clay 3ab344e8bd Use lint/junit output for compile test. (#22395)
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
8 years ago
Matt Clay fbf55886ba Use lint friendly output for python syntax checks. (#22390) 8 years ago
Matt Clay 8f463fcdd2 Create bot friendly sanity output. (#22381) 8 years ago
Matt Clay 3e4be156d7 Pre-compile regexes to speed up target processing.
Without this, changing a large number of files results in target
processing taking a very long time due to repeatedly compiling
the same patterns in a loop over many targets.
8 years ago
Matt Clay d66ce40ecb Overhaul `ansible-test sanity` implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
8 years ago
Matt Clay 28b882e737 Support ansible commands in sanity tests. (#22172) 8 years ago
Matt Clay 4f3dade266 Add retries for ansible-test docker run. 8 years ago
Matt Clay 45cec93e99 Disable newer shellcheck tests which don't pass. 8 years ago
Matt Clay a7bb04800d Add boto to units requirements for s3 test. 8 years ago
Matt Clay fcac261eef Run unit tests in isolation w/ coverage support. 8 years ago
Matt Clay c87ebb6215 Deduplicate arcs when combining coverage data. (#22022) 8 years ago
Matt Clay 2797cff773 Run destructive win tests when creating instances. 8 years ago
Matt Clay 2804d2e848 Split up unit tests on Shippable. (#21931)
* Add more verbose output to coverage combine.
* Run unit tests on CI separately by version.
8 years ago
Matt Clay bc8186543a Pass base branch to module validator on delegate. 8 years ago
Matt Martz 9b1bd2eb7e Re-enable module comparisons in validate-modules (#21582)
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
8 years ago
Matt Clay 1ce2ea0b6c Fix ansible-test handling of deleted test targets. 8 years ago
Matt Clay 2192c1eb02 Fix ansible-test selection of inventory file. 8 years ago
John R Barker 5e28d3a15c network_cli shouldn't be specified anymore (#21458) 8 years ago
Matt Clay d54bc09fae Consider module_utils deps when running tests. (#21382)
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
8 years ago
Peter Sprygada 7f1c43e597 updates ios modules to support persistent socket (#21258)
* updates all ios modules to support persistent socket
* adds ios action plugin to connect to device
* adds exec_command() to ios shared module
* fixes ios_config and ios_template local action
* update all unit test cases
* adds base test module for ios module testing
8 years ago
Matt Clay 4189592701 Revert "Enable process isolation for unit tests."
This reverts commit 91526cd9f2.

Removing this feature primarily because it interferes with
collecting proper code coverage results. I may restore the
feature later if that can be resolved.
8 years ago
Matt Clay 91526cd9f2 Enable process isolation for unit tests. 8 years ago
Matt Clay f16ee2841b Add retries to Shippable CI scripts. (#21108)
* Add retry.py for auto-retry in CI scripts.
* Add retries to other and units CI scripts.
8 years ago
Matt Clay bce5fc7d96 Use us-east-2 for Windows CI on Shippable. 8 years ago
Matt Clay 309a37de86 Add `--debug` option to `ansible-test`. 8 years ago
Toshio Kuratomi 1609afbd12 Unittests for some of module_common.py (#20812)
* Unittests for some of module_common.py
* Port test_run_command to use pytest-mock

The use of addCleanup(patch.stopall) from the unittest idiom was
conflicting with the pytest-mock idiom of closing all patches
automatically.  Switching to pytest-mock ensures that the patches are
closed and removing the stopall stops the conflict.
8 years ago
Matt Clay 6ef1a6aeb6 Add `pep8` to `ansible-test`. (#20745)
The rule sets and legacy file list are a first draft.
It is likely that they will need to be revised.
8 years ago
Dag Wieers 3a0a74dc18 Make test/runner less verbose when spawning instances
As discussed on IRC
8 years ago
Matt Clay 5503b89831 Run fewer tests for winrm config script changes. 8 years ago
Matt Clay 76d563af79 Temporarily switch Windows tests to us-east-1.
Will switch back to us-east-2 after AWS limit increase is done.
8 years ago
Matt Clay fa7f84217b Add startup checks to ansible-test. 8 years ago
Matt Clay 61e6e7868c Improve jinja2 test coverage. (#20533)
* Run `template` test on latest jinja2 version.
* Update jinja2 install for groupby_filter.
* Clean test output dir for each test.
8 years ago
Matt Clay 7ed67746cf Enable Windows 2016 on Shippable.
Also launch Windows instances on us-east-2 and all others on us-east-1.
8 years ago
Matt Clay 4ba9f63afe Use multiple AWS regions in ansible-test. (#20429)
* Move ansible-test EnvironmentConfig to util.
* Use multiple AWS regions in ansible-test.
8 years ago
Matt Clay 4673a9c9c2 Generate ssh key once for network tests. 8 years ago
Matt Clay 4c97bad28e Update ansible-test network change classification. 8 years ago
Matt Clay 89898ed734 Update coverage constraints for ansible-test. 8 years ago
Matt Clay 3c51ce56fd Enable more network support in ansible-test. (#20372)
* Only start platform instances with tests selected.
* Enable ios on Shippable.
* Show inventory in explain mode.
* Fix indentation of generated network inventory.
* Update classification of network module_utils.
8 years ago
Matt Clay dbb89549ff Improve ansible-test error handling and timeouts. (#20328)
* Eliminate warning/retry on old instance check.
* Increase instance start timeout for windows.
8 years ago
Matt Clay 55b9a11468 Require coverage < 4.3.2 to avoid bugs. 8 years ago
Matt Clay b5d5888b6e Add support for ios to ansible-test. 8 years ago
Matt Clay e2bd71ac02 Clean up inventory generation in ansible-test. (#20313) 8 years ago
Matt Clay a59ce3a7f9 Add retry on failure to get instance. 8 years ago
Matt Clay f3130898bb Retry instance start on error. 8 years ago
Matt Clay f534573dcf Enable first network tests on Shippable. (#20208) 8 years ago
Matt Clay d8733a5455 Complete initial network-integration support. 8 years ago
Matt Clay 300181cfd3 Send current winrm config to ansible-core-ci. 8 years ago
Matt Clay f340b8d14f Add test support for python 3.6. 8 years ago
Matt Clay cc3d131f50 Fix group_by test to work with jinja2 >= 2.9. 8 years ago
Matt Clay 7ba47bfd2d Correct test constraints and add sanity check. 8 years ago
Matt Clay 07c68571b6 Require jinja2 < 2.9 until tests pass on 2.9. 8 years ago
Matt Clay 3ec7243918 Run all win tests for winrm config script changes. 8 years ago
Brian Coca 57f8b791d6 consolidated docs
point to new doc locations
removed non existing dirs
8 years ago
Matt Clay b7780ca530 Pass DOCKER_* env vars to docker in ansible-test. 8 years ago
Robin Roth 9f2d22425b Run tests on opensuse42.2
* add to shippable
* add to completion/readme
* Fixes #18645
8 years ago
Matt Clay 68346b128f Update Fedora tests to use releases 24 and 25. 8 years ago
Matt Clay dac226de4a Improve ansible-test completion and messaging. (#19627)
* Simplify type annotation of args.docker_util.
* Improve error messaging for ansible-doc test.
* Add completion for docker, remote and windows.
8 years ago
Matt Clay 422857166b Remove legacy test alias migration script. (#19629)
* Require code-smell tests to be files.
* Add sanity check for integration test aliases.
* Remove migration script for test aliases.
8 years ago
Matt Clay 9b5c782a0b Use `docker pull` by default in ansible-test. 8 years ago
Matt Clay f28b5a0ed8 Add --tox-sitepackages option to ansible-test. 8 years ago
Matt Clay 45772b961f Add private fork CI support to ansible-test. 8 years ago
Matt Clay 1c9a58a885 Run all tests when no last successful commit.
(cherry picked from commit 2e56035fddfd04eb084bd977fc1ee2f35c2f5283)
8 years ago
Matt Clay 5628e260f2 Relocate module unit tests. (#18812)
* Relocate module unit tests.
* Fix classification of unit test changes.
8 years ago
Matt Clay 573078fc6b Fix unit test directory path target. 8 years ago
Matt Clay 47083f145a Don't error if pytest skips all unit tests. 8 years ago
Matt Clay 8e951b9f98 Remove legacy `test/samples` directory. 8 years ago
Matt Clay 7d3e2b2121 Verify path before assigning to sanity test. 8 years ago
Matt Clay 14a2757116 Correct change classification for network plugins. 8 years ago
Matt Clay d2af88ba8a Do not replace existing test skip lists. 8 years ago
Adrian Likins 47a7cb733a use log attribute %(process)d instead of os.getpid (#18691) 8 years ago
Adrian Likins ba56c87566 Add SSH_AUTH_SOCK to default env
So ssh-agent works when using locally.
8 years ago
Matt Clay bee58fe890 Fix compile skip path handling for targeted tests. 8 years ago
Matt Clay 6bbd92e422 Initial ansible-test implementation. (#18556) 8 years ago