Commit Graph

234 Commits (2f33c1a1a1d0c74d31445984fd85d877f0d8ab59)

Author SHA1 Message Date
René Moser d96195131f cloudstack: test: improve startup time, fix network dependency on run (#24876) 8 years ago
Matt Clay 5babe2daea Increase python version coverage for tests. (#24762)
* Improve ansible-test inventory handling.
* Fix python 3 re-raise of exception from thread.
* Fix python 3 encoding for windows-integration.
* Run network tests on multiple python versions.
* Run windows tests on multiple python versions.
* Support Shippable delegation using --tox.
* Skip vyos_command on python 3 tests until fixed.
* Add python 3 filtering to local and tox.
* Fix tests to support back to back runs.
* Temporarily test networking with python 2.7 only.

Running the tests back to back causes intermittent test failures
which need to be addressed before we can test multiple versions
in a single test run.
8 years ago
Matt Clay 036ba7eeec Clean up Shippable tools and update job handling.
- Tools are now in a tools subdirectory.
- Removed obsolete ansible-core-ci tool.
- Added run.py for starting new CI runs.
- Improved handling of run IDs and URLs.
- General code cleanup and docs updates.
- Nightly CI runs use complete coverage.
8 years ago
Matt Clay 6e160650e9 Fix run filter on Shippable downloader. 8 years ago
Matt Clay 4daa16f9ce Add --all and --stub coverage options.
ci_coverage
ci_complete
8 years ago
Matt Clay 6a2a7a2392 Enable codecov.io and add coverage grouping. 8 years ago
Matt Clay fdcad0f44d Improve on-demand coverage for Shippable. 8 years ago
Matt Clay 5cb5228cde Implement on-demand coverage on Shippable. (#24535) 8 years ago
Matt Clay af5db3e4a9 Revert "Add support for on-demand coverage on Shippable."
This reverts commit 7b3f0b8870.
8 years ago
Matt Clay 7b3f0b8870 Add support for on-demand coverage on Shippable.
Triggered by setting the COVERAGE environment var to a non-empty value.
Alternatively the string `ci_coverage` can be added to a commit message.
8 years ago
Matt Clay fb9efd3c7e Add `--remote-terminate` option to ansible-test.
Set `--remote-terminate success` for osx tests.
8 years ago
René Moser aaf4f04574 automated integration tests for cloudstack (#20552) 8 years ago
Matt Clay 17e07a27b2 Enable cloud tests for use with ansible-test. 8 years ago
John R Barker ecbf8e933a Docs how to test (2nd) (#24094)
* Big testing doc refactor
* Combine all the testing documentation in to one place to make it easier to find
* Convert everything to RST
* Create testing_network guide
* Create testing landing page
* For each section detail "how to run" and "how to extend testing"
* More examples
* Lots more detail
8 years ago
Matt Clay 0587d59266 Use Shippable image: drydock/u16pytall:master (#23997)
* Use Shippable image: drydock/u16pytall:master
* Do not install python 3.6 on Shippable.
8 years ago
Matt Clay e7bb508ad6 Fix var precedence check to support python 3. (#23552)
* Fix var precedence check to support python 3.
* Run CI sanity tests using python 3.5.
* Disable pylint non-iterator-returned test to pass on python 3.5.
8 years ago
Matt Clay eec01e2137 Set IO encoding for ansible-test on Shippable. 8 years ago
Matt Clay 9e1bf1c6f2 Update fedora24 Dockerfile to resolve dnf issue. 8 years ago
Matt Clay dfe8eca3d2 Update Shippable downloader to support new API.
- Download run/jobs by URL (old or new).
- Download run metadata.
- Download job metadata.
- Format downloaded JSON.
- Handle and save coverage/testresults download errors.
8 years ago
Matt Clay 12f6beaebd Remove python2.4 package install from other test. 8 years ago
Matt Clay 869449e288 Add test verification to ansible-test. (#22636)
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
8 years ago
Matt Clay 89559f78de Add `--failure-ok` option to `ansible-test`. (#22623) 8 years ago
Matt Clay a64665ae91 Fix error handling in CI other script. 8 years ago
Matt Clay 3ab344e8bd Use lint/junit output for compile test. (#22395)
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
8 years ago
Matt Clay 9acf551674 Run all `other` tests on Shippable before failing. 8 years ago
Matt Clay 8f463fcdd2 Create bot friendly sanity output. (#22381) 8 years ago
Matt Clay 72c96b3ac3 Fix encoding issue in shippable downloader. 8 years ago
Matt Clay d66ce40ecb Overhaul `ansible-test sanity` implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
8 years ago
Matt Clay ae2d2860f5 Reduce CI output processing code coverage. 8 years ago
Matt Clay 2804d2e848 Split up unit tests on Shippable. (#21931)
* Add more verbose output to coverage combine.
* Run unit tests on CI separately by version.
8 years ago
Matt Clay b2bd75a408 Restore 2008-SP2 and 2008-R2_SP1 CI testing. 8 years ago
Matt Davis 8527013fbe Complete rewrite of Windows exec wrapper (#21510)
* supports pipelining for faster execution
* supports become (runas), creates interactive subsession under WinRM batch logon
* supports usage of arbitrary module_utils files
* modular exec wrapper payload supports easier extension
* integrates async wrapper behavior for pipelined/become'd async
* module_utils are loaded as true Powershell modules, no more runtime modifications to module code
8 years ago
Matt Clay f16ee2841b Add retries to Shippable CI scripts. (#21108)
* Add retry.py for auto-retry in CI scripts.
* Add retries to other and units CI scripts.
8 years ago
Matt Clay f42fa8879b Temporarily remove Windows Server 2016 from CI. 8 years ago
Matt Clay 7ed67746cf Enable Windows 2016 on Shippable.
Also launch Windows instances on us-east-2 and all others on us-east-1.
8 years ago
Matt Clay 3c51ce56fd Enable more network support in ansible-test. (#20372)
* Only start platform instances with tests selected.
* Enable ios on Shippable.
* Show inventory in explain mode.
* Fix indentation of generated network inventory.
* Update classification of network module_utils.
8 years ago
Matt Clay 34bca6fb55 Use distro packaged jinja2 instead of pip version. 8 years ago
Matt Clay 203e375449 Fix python 3.6 install. 8 years ago
Matt Clay f534573dcf Enable first network tests on Shippable. (#20208) 8 years ago
Matt Clay cd3fdca540 Switch tests to pytest and ansible-test.
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
8 years ago
Matt Clay f340b8d14f Add test support for python 3.6. 8 years ago
Matt Martz 9048cf2758 Changes to the httptester docker image to support new client cert functionality in tests (#19912)
* Fix Dockerfile
* Add changes to docker image to support new client cert functionality
* Update repo:tag for docker
8 years ago
Robin Roth 61897fa4b7 Add Dockerfile for opensuse 42.2 (#19805) 8 years ago
Matt Clay 68346b128f Update Fedora tests to use releases 24 and 25. 8 years ago
Matt Clay 165e7c4e61 Add Dockerfile for fedora24 and fedora25. 8 years ago
Matt Clay 422857166b Remove legacy test alias migration script. (#19629)
* Require code-smell tests to be files.
* Add sanity check for integration test aliases.
* Remove migration script for test aliases.
8 years ago
Matt Clay f789a20f29 Update log download url. 8 years ago
Toshio Kuratomi c6b42028c4 Fixes for uri under python3 and local (non-httptester) testing 8 years ago
Matt Clay dab2bf7e13 Skip test_uri on python 3. 8 years ago
Lumír 'Frenzy' Balhar 84544ee8fd Fix AST nodes for Python 3 and enable dependent test_uri (#18597)
* Enable tests on python 3 for uri

* Added one more node type to SAFE_NODES into safe_eval module.

ast.USub represents unary operators. This is necessary for
parsing some unusual but still valid JSON files during testing
with Python 3.
8 years ago