Commit Graph

118 Commits (f18f480a3c3f6c2cce67d7a7e11e425c3794bc50)

Author SHA1 Message Date
Matt Clay 8715bc400a Add coverage filtering to ansible-test. (#68158)
* Relocate expand_indexes so it can be reused.

* Add generate_indexes function.

* Simplify type annotations.

* Add `coverage analyze targets filter` command.

* Add changelog entry.
5 years ago
Matt Clay 27ce6dbc1f Remove test support option from ansible-test.
This option was added to facilitate migration and is no longer needed.
5 years ago
Matt Clay 50442f9fdd Fix ansible-test handling of no Python coverage. 5 years ago
Matt Clay 1a8fdaadc7 Fix powershell coverage path rewriting.
This applies the same rewrite logic to PowerShell coverage as is used for Python coverage.
5 years ago
Matt Clay ebb255f0d3 Add Python 3.9 to ansible-test.
ci_complete
ci_coverage
5 years ago
Matt Clay c06f442409 Fix ansible-test import sanity test venv setup. 5 years ago
Matt Clay 99e657162f Fix ansible-test module_utils analysis. 5 years ago
Matt Clay 7c8b046b5f
Fourth batch of incidental integration tests. (#67873)
* Copy in incidental posix tests.

* Update incidental test aliases.

* Update target names.

* Add support plugins.

* Fix paths.

* Update ignores.

* Update integration-aliases sanity test.

* Add incidental tests to CI.
5 years ago
Jordan Borean 26129fcb80
ansible-galaxy - optimise some paths and use fake galaxy int tests (#67685)
* ansible-galaxy - optimise some paths and use fake galaxy int tests

* Added init, built, and publish tests

* Test against both mocked Galaxy and AH server

* Finish off writing the install tests

* Fix up broken tests

* Rename test target and add migrated tests

* Use cloud provider for Galaxy implementation

* Added blank static config

* Use correct alias group

* Set release version and fix copy typo

* Remove reset step as it is no longer needed

* Use sane env var names for test container name
5 years ago
Matt Clay f735fd672a
Third batch of incidental integration tests. (#67830)
* Copy in incidental windows tests.

* Update incidental test aliases.

* Add support plugins.

* Update target references.

* Update sanity ignores.

* Update integration-aliases test.

* Add to CI.
5 years ago
Matt Clay b68e55da26
Remove `test/legacy/` directory. (#67786)
* Remove `test/legacy/` directory.

* Remove references to `test/legacy/` directory.

* Update the remaining valid legacy reference.

* Remove outdated legacy references.
5 years ago
Matt Clay e3591223a0
Second batch of incidental integration tests. (#67765)
* Update incidental test aliases.

* Rewrite target references for renamed targets.

* Add incidental tests to CI.

* Update sanity tests for incidental cloud tests.

* Initial copy of incidental tests.

* Copy contrib files into test.

* Update paths in test.

* Add support plugins.

* Update plugin to work around missing deps.

* Update sanity ignores.

* Fix matrix entries.

* Remove debug echo.
5 years ago
Jill R 96244c4cde
Make AWS test plugin to refer to the aws-permissions-for-integration-tests (#67665)
Make AWS test plugin to refer to the aws-permissions-for-integration-tests section of the AWS devel docs now that we have them, rather than suggesting to flag down specific users.
5 years ago
Matt Clay 482885e0c8 Fix network setup for collections in ansible-test. 5 years ago
Matt Clay 542d1b98ff
Support private test plugins in ansible-test. (#67660)
* Support private test plugins in ansible-test.

This feature is exclusively used for testing Ansible itself and is not available to collections.

Content in the newly supported directories will be added in follow-up PRs.

* Add new directory to path classification.

* Support new paths in shebang test.

* Add new directory to manifest.
5 years ago
Yanis Guenane 34800fe027 Test: Bump Tower version to latest release 5 years ago
Matt Martz c61c0f7ad5
Add warning when running devel (#67502)
* Add warning when running devel. Addresses #67362

* Add config entry to disable devel warning, so it doesn't impact CI

* Update warning about using devel
5 years ago
Matt Clay 19f6fc68d4
Relocate Shippable tools. (#67556)
* Move Shippable tools to hacking directory.

These limits the `test/utils/shippable/` directory to scripts required for CI.

* Fix `test/utils/shippable/` file classification.

* Update package-data sanity test.
5 years ago
Yanis Guenane b9bd2c25ca
Test: Bump TOWER_VERSION to supported version (#67378)
* Revert "Remove Tower module tests from CI. (#67368)"

This reverts commit b041d96762.

* Test: Bump TOWER_VERSION to 3.6.2
5 years ago
Matt Clay 5e68bb3d93
Add code coverage target analysis to ansible-test. (#67141)
* Refactor coverage file enumeration.
* Relocate sanitize_filename function.
* Support sets when writing JSON files.
* Generalize setting of info_stderr mode.
* Split out coverage path checking.
* Split out collection regex logic.
* Improve sanitize_filename type hints and docs.
* Clean up coverage erase command.
* Fix docs and type hints for initialize_coverage.
* Update type hints on CoverageConfig.
* Split out logic for finding modules.
* Split out arc enumeration.
* Split out powershell coverage enumeration.
* Raise verbosity level of empty coverage warnings.
* Add code coverage target analysis to ansible-test.
5 years ago
Matt Clay f4a80bb600
Code cleanup and refactoring in ansible-test. (#67063)
* Code cleanup in ansible-test.
* Split out encoding functions.
* Consoldate loading of JSON files.
* Split out disk IO functions.
* Simplify file access.
* Add functions for opening files.
* Replace open calls with appropriate functions.
* Expose more types from typing module.
* Support writing compact JSON.
* Add verbosity argument to display.warning.
* Add changelog entry.
* Update files overlooked during rebase.
* Use `io.open` instead of `open`.
* Fix file opening for imp.load_module.
* Remove use of `r+` mode to access files.
* Add missing import.
* Fix httptester on Python 2.x.
* Clarify changelog fragment.
* Consolidate imports. Remove extra newlines.
* Fix indirect imports.
5 years ago
Matt Clay d584584474 Reorganize ansible-test coverage code.
This change moves all code for the `ansible-test coverage` command into the `coverage` directory.

Each subcommand is split into a separate file.

Only minor spelling changes were made aside from code relocation.
5 years ago
Matt Clay 49d8d5ae33 Fix ansible-test tar format handling. 5 years ago
Yanis Guenane 4fd2dce7f3
Testing: Add support for AIX platform (#65802) 5 years ago
Matt Clay 3c3cf50e7a Fix ansible-test coverage of ansible-connection. 5 years ago
Matt Clay efd2dd8929 Add more ansible-test args to delegation config. 5 years ago
Matt Clay ee50792416 Add a `--no-pip-check` option to ansible-test. 5 years ago
Matt Clay 8b63da7b43 Fix ansible-test egg-info directory detection. 5 years ago
Matt Clay ebf21bb48d Fix ansible-test inventory delegation. 5 years ago
Matt Clay 934d6b235b Add `--venv-system-site-packages` to ansible-test. 5 years ago
Gonéri Le Bouder 68a8cc2b1f vcenter: don't bind container on <1024 port
Bind the govcsim service on port 1443, this way we don't need root privilege
to start the container. This allow us to use ansible-test with podman
seamlessly.
The commit also avoid the mapping of port 80. We don't need it.
5 years ago
Gonéri Le Bouder 16d3f6bcbb vcenter provider: do not set env var
Some secrets are exposed because of the environment variables. The
Ansible variables are now enough, we don't need to expose the envvar
anymore.
5 years ago
Toshio Kuratomi 2b6d94c4c8 Change the check_pyyaml() function to return the features of pyyaml that we check 5 years ago
Toshio Kuratomi ece306b201 Add a sanity test for yaml to make sure we can parse all yaml
Check that all yaml we ship is parsable by the pyyaml c backend.  Since
Ansible uses Pyyaml for docs and playbooks, if the yaml files aren't
parsable, they will error out if they were used.

Warn and skip yamllint if libyaml backend is not present

Ignore new errors in examples until someone can fix them
5 years ago
Matt Clay fb69d68821
Fix plugin paths for ansible-test pylint test. (#65526)
* Fix plugin paths for ansible-test pylint test.
5 years ago
Matt Clay ba273c72d8
Fix ansible-test handling of git submodules. (#65027)
* Revert most of PR #61605 commit e218c9814c

This removes the git error handling that converted all git errors into warnings.

* Fix ansible-test handling of git submodules.
5 years ago
Matt Clay 4f51b7d323 Remove ansible-test --tox delegation option.
Use the --venv option instead.

This option was only available when running from source to test the ansible/ansible repository.

This will have no effect on testing collections or running from an installed version of Ansible.

Also update docs to reference the --venv option instead of the --tox option.
5 years ago
Felix Fontein f4ff45dcd4 ansible-test: bumping ACME test container version (#64424)
* Bumping ACME test container version.

* Add changelog.
5 years ago
Matt Clay e9f8a34dce
Fixes for validate-modules import handling. (#63932)
* Fix validate-modules support for collections.

- Relative imports now work correctly.
- The collection loader is now used.
- Modules are invoked as `__main__`.

* Remove obsolete validate-modules code ignores.

* Handle sys.exit in validate-modules.

* Add check for AnsibleModule initialization.

* Remove `missing-module-utils-import` check.

This check does not support relative imports or collections.

Instead of trying to overhaul the test, we can rely on the `ansible-module-not-initialized` test instead.

* Fix badly named error codes with `c#` in the name.

The `#` conflicts with comments in the sanity test ignore files.

* Add changelog entries.
5 years ago
Matt Clay 92ccdeac31 Fix various import sanity test issues.
- Relative imports are now properly recognized.
- Correct script invocation of Ansible modules is used.
- Warnings are now consistently reported as errors.
- Errors are now consistently reported with the file tested.

Resolves https://github.com/ansible/ansible/issues/62723
Resolves https://github.com/ansible/ansible/issues/61884
5 years ago
Matt Clay 6be4741f72 Fix ansible-test env var mixing running commands. 5 years ago
Gonéri Le Bouder 73febd4ea6 prepare_vmware_tests: use module_defaults (#63209) 5 years ago
Matt Davis 6d52bdf4db
fix default collection resolution in adhoc (#63298)
* fix default collection resolution in adhoc

* if an adhoc command is run with a playbook-dir under a configured collection, default collection resolution is used to resolve unqualified module/action names

* Set ANSIBLE_PLAYBOOK_DIR in integration tests.

* Fix config conflict in ansible integration test.

* add adhoc default collection test

* text-ify warning string
5 years ago
Matt Clay 022335669c
Work around ssh-keygen issue in ansible-test. (#63211)
Newer versions of ssh-keygen create PEM keys that are not recognized by Paramiko.

Now ansible-test compensates for this by updating they keys it generates so Paramiko will recognize them.
5 years ago
Matt Clay 4c79f1ec4d Fix ansible-test integration temp dir location.
Previously the temporary directory used to run integration tests resided under the user's home directory. This prevented ansible-playbook from detecting the default collection when running tests.

Now the temporary directory is created within the collection to facilitate default collection detection.
5 years ago
Matt Clay 32979430d0 Add missing var for format string in ansible-test. 5 years ago
Matt Clay b91f452f4f Fix ansible-test virtualenv real python search.
ansible-test now properly searches for `pythonX.Y` instead of `python` when looking for the real python that created a `virtualenv`.
5 years ago
Matt Clay 3f2380ccce Exclude tmp dir from ansible-test results copy.
This fixes test errors related to failures copying temporary test results files from a remote system back to the local system.

It also speeds up processing of test results and reduces network utilization by avoiding the temporary files.
5 years ago
Matt Clay 831e1bf2e0 Fix ansible-test PYTHONPATH handling.
Running from an installed version of ansible-test now results in tests using a dedicated directory for PYTHONPATH instead of using the site-packages directory where ansible is installed.

This provides consistency with tests running from source, which already used a dedicated directory.

Resolves https://github.com/ansible/ansible/issues/62716
5 years ago
Gonéri Le Bouder 5ecbe9cbbb vcenter provider: Only rely on VMWARE_TEST_PLATFORM
Until now, the vcenter provider was switching between `static` and
`govcsim` depending on the presence of the following configuration file:
`test/integration/cloud-config-vcenter.ini`.

This was not consistent with Worldstream, which we enable with the
`VMWARE_TEST_PLATFORM` environment variable.

We now only rely on `VMWARE_TEST_PLATFORM` to know which platform should be
used. `govcsim` is still the default, this to preserve the original
behaviour.

This commit also rename the following variables to be consistent with the rest
of the code base. It also ensures they are alway defined, even with `govcsim`:

- `VCENTER_HOSTNAME`
- `VCENTER_USERNAME`
- `VCENTER_PASSWORD`
5 years ago