* [stable-2.9] Fix ansible-test pytest plugin loading. (#62119)
* Avoid assertion rewriting in pytest plugins.
Adding PYTEST_DONT_REWRITE to the ansible-test pytest plugin docstrings disables assertion rewriting in pytest for those plugins.
This avoids warnings during test execution if the plugins are loaded multiple times (such as being imported within tests).
* Run ansible-test pytest plugins early.
The ansible-test pytest plugins need to load and run earlier than conftest modules.
To facilitate this, the pytest_configure function is run during loading, which works since they are loaded (but not always run) before conftest modules are loaded.
A check has also been added to the pytest_configure functions to prevent them from running multiple times in the same process.
* Load pytest plugins using an env var.
The -p command line option loads plugins before conftest, but only during collection.
The PYTEST_PLUGINS environment variable loads plugins before confest, both during collection and test execution.
(cherry picked from commit aaa6d2e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add missing changelog entry for ansible-test fix.
PR https://github.com/ansible/ansible/pull/62119 was missing a changelog entry.
(cherry picked from commit 6c78f02121)
* bump hcloud version to 1.4.1
`hcloud`<=1.4.0 has requirement `requests==2.20.0`. This prevents the
installation of the Vcenter Automation SDK which depends on `requests>=2.22.0`.
`hcloud` 1.4.1 does not have the problem: 8bff356efb
Bumping the dependency will resolve the issue.
(cherry picked from commit 0f52b18)
Co-authored-by: Gonéri Le Bouder <goneri@lebouder.net>
Creating a virtual environment using `venv` when running in a virtual environment created by `virtualenv` results in a copy of the original virtual environment instead of creation of a new one.
To work around this, `ansible-test` now identifies when it is running in a `virtualenv` created virtual environment and uses the real Python interpreter to create the `venv` virtual environment.
(cherry picked from commit a7bc11c)
Co-authored-by: Matt Clay <matt@mystile.com>
The `test/results/` directory for Ansible test output was already ignored when not using git.
When Ansible Collections were switched to `tests/output/` the ignore entry was previously overlooked.
(cherry picked from commit f110abb)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix ansible-test venv activation.
When using the ansible-test --venv option, an execv wrapper for each python interpreter is now used instead of a symbolic link.
* Fix ansible-test execv wrapper generation.
Use the currently running Python interpreter for the shebang in the execv wrapper instead of the selected interpreter.
This allows the wrapper to work when the selected interpreter is a script instead of a binary.
* Fix ansible-test sanity requirements install.
When running sanity tests on multiple Python versions, install requirements for all versions used instead of only the default version.
* Fix ansible-test --venv when installed.
When running ansible-test from an install, the --venv delegation option needs to make sure the ansible-test code is available in the created virtual environment.
Exposing system site packages does not work because the virtual environment may be for a different Python version than the one on which ansible-test is installed.
(cherry picked from commit c77ab11051)
Co-authored-by: Matt Clay <matt@mystile.com>
* ansible-test - Contiune if the git command returns an error
* Just return stdout
* Use to_text() when displaying exception
* Add a message property to SubprocessError
(cherry picked from commit e218c98)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* [stable-2.9] Fix ansible-test coverage path handling. (#61528)
* Fix ansible-test coverage path handling.
* Split CI unit tests into two groups.
(cherry picked from commit e4e5005640)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add changelog fragment.
The `git submodule status` command is relative to the current git repository by default.
When running from a repository subdirectory paths can be returned above the current directory.
Specifying the current directory with `git submodule status` avoids listing submodules above that directory.
This will fix issues when testing a collection that is rooted below the repository root when that repository uses submodules.
(cherry picked from commit 4063d58339)
Co-authored-by: Matt Clay <matt@mystile.com>
* Update collections integration targets path.
* Fix integration path handling.
* Add collections test target fallback.
Also add warnings and errors for common path mistakes.
* Improve role target detection.
This directory is currently a fixed location to make troubleshooting easier.
It is cleared before each test target runs, but is preserved when a test target finishes.
This allows the contents to be inspected when a test fails.
The previous location was `~/ansible_testing/`.
The new location is within the content root:
- `test/results/.tmp/output_dir` for Ansible
- `tests/output/.tmp/output_dir` for Ansible Collections
Moving the directory reduces the number of places on the filesystem where tests create output.
It also enables the results to be returned from delegated systems.
* Clean up layout paths for integration tests.
* Remove "special" integration test target type.
* Remove unnecessary role detection logic.
* Remove support for non-sh runme scripts.
* Simplify reading of aliases.
The directories used for collections tests are changing as follows:
`test/` -> `tests/`
`test/results/` -> `tests/output/`
This is a breaking change for collections tests executed by ansible-test.
All collections will need to be updated to use the new directory.
The `tests/output/` directory should be added to the `.gitignore` or equivalent in each collection.
This change is being made before the first pre-release of Ansible 2.9 since ansible-test has not yet been shipped.
Using the `tests/` directory matches the other collections directories `plugins/` and `roles/`.
This resolves https://github.com/ansible/ansible/issues/60218
* Add --venv delegation to ansible-test.
* Update import sanity test venv creation.
* Fix import test when using --venv on Python 2.x.
* Improve virtualenv setup overhead.
* Hide pip noise for import sanity test.
* Raise verbosity on venv info messages.
* Get rid of base branch noise for collections.
* Add missing --requirements check.
* Remove .keep files from test/results/ dirs.
* Remove classification of test/results/ dir.
* Add results_relative to data context.
* Use variables in delegation paths.
* Standardize file writing and results paths.
* Fix issues reported by PyCharm.
* Clean up invocation of coverage command.
It now runs through the injector.
* Hack to allow intercept_command in cover.py.
* Simplify git ignore for test results.
* Use test result tmp dir instead of cache dir.
* Remove old .pytest_cache reference.
* Fix unit test docker delegation.
* Show HTML report link.
* Clean up more results references.
* Move import sanity test output to .tmp dir.
* Exclude test results dir from coverage.
* Fix import sanity test lib paths.
* Fix hard-coded import test paths.
* Fix most hard-coded integration test paths.
* Fix PyCharm warnings.
* Fix import placement.
* Fix integration test dir path.
* Fix Shippable scripts.
* Fix Shippable matrix check.
* Overhaul key pair management.
* Change collection PS util import pattern
* Add changes for py2 compat
* fix up regex and doc errors
* fix up import analysis
* Sanity fix for 2.6 CI workers
* Get collection util path for coverage collection
* Added coverage collection for PowerShell - ci_complete ci_coverage
* uncomment out coverage uploader call
* Generate XML for PowerShell coverage
* Use whitelist to exclude coverage run on non content plugins
* Remove uneeded ignore entry
* Try to reduce diff in cover.py
* Fix up coverage report package - ci_complete ci_coverage
ansible-test: async deployment w/ vcenter provider
So far, `ansible-test`'s vcenter provider was trigger by a blocking POST
call. Since the depoyment take a lot of time (> 8m), we decided to use
an async call instead and poll the API server until the deployment
is ready.
* Remove pointless sanity tests on `bin` dir.
* Update action-plugin-docs test for collections.
* Update no-main-display test for collections.
* Update empty-init test for collections.
* Update no-assert test for collections.
* Move required-and-default-attributes test.
This test only applies to Ansible itself.
* Update use-argspec-type-path test for collections.
* Relocate ansible-test self tests outside package.
We don't want to include the tests for verifying ansible-test within the ansible-test package.
* Add `test/ansible_test/` to classification.py.
* Fix test invocation.
* Relocate tests in MANIFEST.in.
* Improve package-data sanity test error checking.
* Only use includes for ansible-test in MANIFEST.in.
* Improve readability of MANIFEST.in.
* Run no-unwanted-files sanity test only on Ansible.
Since collections should be able to use binary modules there is not really any limit on what could exist in a collection `plugins` directory.
* Add support for symlinks in sanity target lists:
- Sanity tests that need to analyze symlinks can do so using the supplied target list.
- Tests that analyze directories will now only look at symlinks if requested.
- Directory symlinks will now be seen as directories instead of files.
* Enable symlinks on filename based sanity tests.
Sanity tests that evalulate filenames instead of content should include symlinks.
* Update symlinks sanity test.
Use the sanity test target list now that it can include symlinks.
DCI uses an inventory directory instead of a file, along with --no-temp-workdir.
This changes the inventory check to keep that scenario working until all DCI jobs can be migrated to Zuul.
* Update ansible-test collection inventory handling.
- The `windows-integration` command now supports the `--inventory` option.
- The incomplete support for host_vars and group_vars directories has been removed.
- The incomplete support for an inventory directory has been removed.
- The inventory specified by `--inventory` can now reside outside the install and content roots.
- Using `ansible_ssh_private_key_file` with `--docker` or `--remote` results in a warning about the combination being unsupported and likely to fail.
* Fix config handling.
* Fix payload handling of ssh keys.
* Disable pylint no-self-use rule for ansible-test.
* De-duplicate payload paths.
* add subdir support to collection loading
* collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py
* centralize parsing/validation in AnsibleCollectionRef class
* fix issues loading Jinja2 plugins from multiple sources
* resolves#59462, #59890,
* sanity test fixes
* string fixes
* add changelog entry
Host specific settings are created as part of the generated inventory instead of using a host_vars directory.
Also update connection tests to work properly with the generated inventory changes.
Matt Clay