This avoids displaying the credentials in CI when retrying tests at maximum verbosity.
(cherry picked from commit b73e7721df)
Co-authored-by: Matt Clay <matt@mystile.com>
The documentation links are now displayed when running from an install.
Previously the links were only displayed when running from source.
This was due to ansible-test checking for the presence of documentation files locally, which are only present when running from source.
The check is no longer necessary since there is a sanity test in place to enforce the presence of documentation for all sanity tests.
(cherry picked from commit 32d965e)
Co-authored-by: Matt Clay <matt@mystile.com>
The default behavior of the ansible-test vcenter plugin is to use the govcsim container to run tests.
However, unless the govcsim mode was specified using the VMWARE_TEST_PLATFORM environment variable, the filter code would skip the tests unless the tests ran on Shippable or the user had an ansible-core-ci key.
Now the filter correctly recognizes that govcsim is the default.
(cherry picked from commit cd4882e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix location of unit test requirements.
* Preserve ansible-test unit test requirements.
* Remove redundant unit test requirements.
* Fix location of network test requirements.
* Preserve ansible-test network test requirements.
* Remove redundant network test requirements.
* Add missing ordereddict requirements.
* Load collection requirements correctly.
* Add changelog fragment.
(cherry picked from commit cdc4926)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] Fix ansible-test pytest plugin loading. (#62119)
* Avoid assertion rewriting in pytest plugins.
Adding PYTEST_DONT_REWRITE to the ansible-test pytest plugin docstrings disables assertion rewriting in pytest for those plugins.
This avoids warnings during test execution if the plugins are loaded multiple times (such as being imported within tests).
* Run ansible-test pytest plugins early.
The ansible-test pytest plugins need to load and run earlier than conftest modules.
To facilitate this, the pytest_configure function is run during loading, which works since they are loaded (but not always run) before conftest modules are loaded.
A check has also been added to the pytest_configure functions to prevent them from running multiple times in the same process.
* Load pytest plugins using an env var.
The -p command line option loads plugins before conftest, but only during collection.
The PYTEST_PLUGINS environment variable loads plugins before confest, both during collection and test execution.
(cherry picked from commit aaa6d2e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add missing changelog entry for ansible-test fix.
PR https://github.com/ansible/ansible/pull/62119 was missing a changelog entry.
(cherry picked from commit 6c78f02121)
Creating a virtual environment using `venv` when running in a virtual environment created by `virtualenv` results in a copy of the original virtual environment instead of creation of a new one.
To work around this, `ansible-test` now identifies when it is running in a `virtualenv` created virtual environment and uses the real Python interpreter to create the `venv` virtual environment.
(cherry picked from commit a7bc11c)
Co-authored-by: Matt Clay <matt@mystile.com>
The `test/results/` directory for Ansible test output was already ignored when not using git.
When Ansible Collections were switched to `tests/output/` the ignore entry was previously overlooked.
(cherry picked from commit f110abb)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix ansible-test venv activation.
When using the ansible-test --venv option, an execv wrapper for each python interpreter is now used instead of a symbolic link.
* Fix ansible-test execv wrapper generation.
Use the currently running Python interpreter for the shebang in the execv wrapper instead of the selected interpreter.
This allows the wrapper to work when the selected interpreter is a script instead of a binary.
* Fix ansible-test sanity requirements install.
When running sanity tests on multiple Python versions, install requirements for all versions used instead of only the default version.
* Fix ansible-test --venv when installed.
When running ansible-test from an install, the --venv delegation option needs to make sure the ansible-test code is available in the created virtual environment.
Exposing system site packages does not work because the virtual environment may be for a different Python version than the one on which ansible-test is installed.
(cherry picked from commit c77ab11051)
Co-authored-by: Matt Clay <matt@mystile.com>
* ansible-test - Contiune if the git command returns an error
* Just return stdout
* Use to_text() when displaying exception
* Add a message property to SubprocessError
(cherry picked from commit e218c98)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* [stable-2.9] Fix ansible-test coverage path handling. (#61528)
* Fix ansible-test coverage path handling.
* Split CI unit tests into two groups.
(cherry picked from commit e4e5005640)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add changelog fragment.
The `git submodule status` command is relative to the current git repository by default.
When running from a repository subdirectory paths can be returned above the current directory.
Specifying the current directory with `git submodule status` avoids listing submodules above that directory.
This will fix issues when testing a collection that is rooted below the repository root when that repository uses submodules.
(cherry picked from commit 4063d58339)
Co-authored-by: Matt Clay <matt@mystile.com>
* Update collections integration targets path.
* Fix integration path handling.
* Add collections test target fallback.
Also add warnings and errors for common path mistakes.
* Improve role target detection.
This directory is currently a fixed location to make troubleshooting easier.
It is cleared before each test target runs, but is preserved when a test target finishes.
This allows the contents to be inspected when a test fails.
The previous location was `~/ansible_testing/`.
The new location is within the content root:
- `test/results/.tmp/output_dir` for Ansible
- `tests/output/.tmp/output_dir` for Ansible Collections
Moving the directory reduces the number of places on the filesystem where tests create output.
It also enables the results to be returned from delegated systems.
* Clean up layout paths for integration tests.
* Remove "special" integration test target type.
* Remove unnecessary role detection logic.
* Remove support for non-sh runme scripts.
* Simplify reading of aliases.
The directories used for collections tests are changing as follows:
`test/` -> `tests/`
`test/results/` -> `tests/output/`
This is a breaking change for collections tests executed by ansible-test.
All collections will need to be updated to use the new directory.
The `tests/output/` directory should be added to the `.gitignore` or equivalent in each collection.
This change is being made before the first pre-release of Ansible 2.9 since ansible-test has not yet been shipped.
Using the `tests/` directory matches the other collections directories `plugins/` and `roles/`.
This resolves https://github.com/ansible/ansible/issues/60218
* Add --venv delegation to ansible-test.
* Update import sanity test venv creation.
* Fix import test when using --venv on Python 2.x.
* Improve virtualenv setup overhead.
* Hide pip noise for import sanity test.
* Raise verbosity on venv info messages.
* Get rid of base branch noise for collections.
* Add missing --requirements check.
* Remove .keep files from test/results/ dirs.
* Remove classification of test/results/ dir.
* Add results_relative to data context.
* Use variables in delegation paths.
* Standardize file writing and results paths.
* Fix issues reported by PyCharm.
* Clean up invocation of coverage command.
It now runs through the injector.
* Hack to allow intercept_command in cover.py.
* Simplify git ignore for test results.
* Use test result tmp dir instead of cache dir.
* Remove old .pytest_cache reference.
* Fix unit test docker delegation.
* Show HTML report link.
* Clean up more results references.
* Move import sanity test output to .tmp dir.
* Exclude test results dir from coverage.
* Fix import sanity test lib paths.
* Fix hard-coded import test paths.
* Fix most hard-coded integration test paths.
* Fix PyCharm warnings.
* Fix import placement.
* Fix integration test dir path.
* Fix Shippable scripts.
* Fix Shippable matrix check.
* Overhaul key pair management.
* Change collection PS util import pattern
* Add changes for py2 compat
* fix up regex and doc errors
* fix up import analysis
* Sanity fix for 2.6 CI workers
* Get collection util path for coverage collection
* Added coverage collection for PowerShell - ci_complete ci_coverage
* uncomment out coverage uploader call
* Generate XML for PowerShell coverage
* Use whitelist to exclude coverage run on non content plugins
* Remove uneeded ignore entry
* Try to reduce diff in cover.py
* Fix up coverage report package - ci_complete ci_coverage
ansible-test: async deployment w/ vcenter provider
So far, `ansible-test`'s vcenter provider was trigger by a blocking POST
call. Since the depoyment take a lot of time (> 8m), we decided to use
an async call instead and poll the API server until the deployment
is ready.
* Relocate ansible-test self tests outside package.
We don't want to include the tests for verifying ansible-test within the ansible-test package.
* Add `test/ansible_test/` to classification.py.
* Fix test invocation.
* Relocate tests in MANIFEST.in.
* Improve package-data sanity test error checking.
* Only use includes for ansible-test in MANIFEST.in.
* Improve readability of MANIFEST.in.
* Run no-unwanted-files sanity test only on Ansible.
Since collections should be able to use binary modules there is not really any limit on what could exist in a collection `plugins` directory.
* Add support for symlinks in sanity target lists:
- Sanity tests that need to analyze symlinks can do so using the supplied target list.
- Tests that analyze directories will now only look at symlinks if requested.
- Directory symlinks will now be seen as directories instead of files.
* Enable symlinks on filename based sanity tests.
Sanity tests that evalulate filenames instead of content should include symlinks.
* Update symlinks sanity test.
Use the sanity test target list now that it can include symlinks.
DCI uses an inventory directory instead of a file, along with --no-temp-workdir.
This changes the inventory check to keep that scenario working until all DCI jobs can be migrated to Zuul.
* Update ansible-test collection inventory handling.
- The `windows-integration` command now supports the `--inventory` option.
- The incomplete support for host_vars and group_vars directories has been removed.
- The incomplete support for an inventory directory has been removed.
- The inventory specified by `--inventory` can now reside outside the install and content roots.
- Using `ansible_ssh_private_key_file` with `--docker` or `--remote` results in a warning about the combination being unsupported and likely to fail.
* Fix config handling.
* Fix payload handling of ssh keys.
* Disable pylint no-self-use rule for ansible-test.
* De-duplicate payload paths.
Host specific settings are created as part of the generated inventory instead of using a host_vars directory.
Also update connection tests to work properly with the generated inventory changes.
* Start of work to support collections
* remove version_added from base schema
* If a collection, pass that to validate-modules
* clean ups
* Allow version_added in a collection, just make it optional
* Don't traceback on missing doc_fragment
* Don't validate metadata in a collection
The `test/integration/integration_config.yml` vars file will now be loaded from the content under test and is now optional.
The `output_dir` and `win_output_dir` vars are now provided by ansible-test.
* Move Ansible test config into env vars.
This allows ansible-test to use an empty Ansible config file, leaving open the option for users to customize the one used to run tests (although such usage is discouraged).
* Use config from content under test when present.
The vcenter provider allow one to use the `VMWARE_TEST_PLATFORM` to
manually select a test platform. The test platforms are:
- `govcsim`
- `static`
- and `worldstream`.
Before this commit, the `govcsim` value was ignored.
Example:
```shell
$ VMWARE_TEST_PLATFORM=govcsim ansible-test integration --python 3.7 needs/target/prepare_vmware_tests/
```
Matt Clay