This fixes test errors related to failures copying temporary test results files from a remote system back to the local system.
It also speeds up processing of test results and reduces network utilization by avoiding the temporary files.
(cherry picked from commit 3f2380ccce)
Co-authored-by: Matt Clay <matt@mystile.com>
Running from an installed version of ansible-test now results in tests using a dedicated directory for PYTHONPATH instead of using the site-packages directory where ansible is installed.
This provides consistency with tests running from source, which already used a dedicated directory.
Resolves https://github.com/ansible/ansible/issues/62716
(cherry picked from commit 831e1bf2e0)
Co-authored-by: Matt Clay <matt@mystile.com>
The `pids` module returns the list of the PID in a `pids` key.
This change ensures we correctly wait for the end of the previous mongod
instances before we start the next ones.
In addition, we remove an unnecessary `ignore_errors`.
See: #61938
Original PR: https://github.com/ansible/ansible/pull/62627
(cherry picked from commit cee55ab718)
* Add a representer for AnsibleUnsafeBytes
* changelog
* Add unit tests
Remove native string test until we have time to evaluate how this the function should work
Add non-ASCII characters to test cases
* Compare to the string on Python 2
Add a comment in the test about this behavior
(cherry picked from commit 4cc4c44dd0)
* Ensure k8s apply works with check mode
Update the new predicted object with fields from the previous object
before applying in check mode
Don't log output of `file` with `state: absent` on huge virtualenvs!
Fixes#60510
* Use openshift client fix to improve apply for check mode
Use new apply_object method to get a better approximation
of the expected object in check mode.
Requires released upgrade to openshift
* Add changelog fragment for k8s apply check mode fix
* Update changelogs/fragments/60510-k8s-apply-check-mode.yml
Co-Authored-By: Felix Fontein <felix@fontein.de>
(cherry picked from commit a684bb9f5b)
* Fix plugin names for collection plugins.
Add an integration test to verify plugin __name__ is correct for collection plugins.
* Fix collection loader PEP 302 compliance.
The `find_module` function now returns `None` if the module cannot be found. Previously it would return `self` for modules which did not exist.
Returning a loader from `find_module` which cannot find the module will result in import errors on Python 2.x when using implicit relative imports.
* add changelog
* sanity/units/merge fixes
(cherry picked from commit 1c64dba3c9)
* Update default test container with Python 3.8b4 (#62100)
* [stable-2.9] test: bump default-test-container
VMware VSphere SDK needs an up to date version of `pip` for the
installation step. With the current image, we face the following error:
```
(...)
02:27 Collecting git+https://github.com/vmware/vsphere-automation-sdk-python.git (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2))
02:27 Cloning https://github.com/vmware/vsphere-automation-sdk-python.git to /tmp/pip-req-build-pm27t16b
02:33 Requirement already satisfied: pyvmomi in /usr/local/lib/python3.6/dist-packages (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 1)) (6.7.1.2018.12)
02:33 Requirement already satisfied: lxml>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from vSphere-Automation-SDK==1.4.0->-r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2)) (4.4.0)
02:33 Processing ./\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl
02:33 Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/root/ansible/\\\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl'
```
Bump default-test-container to 1.9.3 to get an up to date release of
`pip` (was 19.0.2, is now 19.2.3)..
(cherry picked from commit b68f5b406a)
Co-authored-by: Gonéri Le Bouder <goneri@lebouder.net>
* Add missing changelog fragments. (#62471)
* Add missing default-test-container 1.9.2 fragment.
* Add missing default-test-container 1.9.3 fragment.
The os_keystone_endpoint module has two interface arguments -
'interface' for authentication, and 'endpoint_interface' for the
endpoint being registered. Currently the module documentation only
covers 'interface', when in fact it should cover 'endpoint_interface'
and pick up 'interface' from the standard OpenStack module parameter
documentation.
(cherry picked from commit 448a8c4e5f)
* Add integration tests for ansible-doc.
* Enable tests that now pass
* Cleanup processing of plugin docs
* Mostly separate the steps of processing plugin docs
1) Acquire source data
2) Transform and calculate additonal data
3) Format data for output
4) Output data
format_plugin_doc() is still mixing transformation and formatting but
that should be fixed in a devel-only change
* Raise exceptions in _get_plugin_doc() on errors.
* Remove check to exclude on blacklisted extensions. We already request
only .py files
* If there is no DOCUMENTATION entry in the plugin, raise an exception
from _get_plugin_doc(). Everywhere we use _get_plugin_doc(), this is
treated as an error
* If there is no ANSIBLE_METADATA raise an exception as well as
displaying of docs assumes that this has been set.
* If there is neither DOCUMENTATION nor ANSIBLE_METADATA, warn about the
lack of METADATA and error on the lack of DOCUMENTATION. Lack of
DOCUMENTATION is more important so it is what the user should see.
* Add a few special cases for backwards compat. These should probably
be made errors in 2.10:
* no docs but has metadata shows no documentation rather than an error
* empty plugin file shows no doumentation rather than an error
* Simplify backwards compatibility logic.
(cherry picked from commit 3b86dc3e12)
* Fix ansible-doc traceback for removed modules.
This avoids tracebacks with errors like the following when a module has been removed:
module module_name missing documentation (or could not parse documentation): 'NoneType' object does not support item assignment
* Fix ansible-doc sanity test warning handling.
Warnings about removed modules/plugins on stderr are now properly ignored.
Previously an ansible-doc error could result in unrelated errors going undetected because tests were stopped early and the underlying error was ignored.
(cherry picked from commit 064e8e1ef4)
Co-authored-by: Matt Clay <matt@mystile.com>
* routeros_facts: fix for error when there's more than 10 interfaces (#61376)
* fix: proper regex for preprocessing routeros output
* test: regression test
* test: fix nondeterministic unit test
* changelog
* Commands tests (#62322)
* commands tests
* add space in order to delete it and tun shipable tests again
* delete space in order to run shipable tests again
(cherry picked from commit 47cf4e6565)
* changelog
Fixes#62319
Change `enable` option to `enabled` in junos_interfaces
and junos_lldp_interfaces
data model to be in sync with other network platform
resource modules added in 2.9 version.
(cherry picked from commit a9a5f4e40d)
This avoids displaying the credentials in CI when retrying tests at maximum verbosity.
(cherry picked from commit b73e7721df)
Co-authored-by: Matt Clay <matt@mystile.com>
The documentation links are now displayed when running from an install.
Previously the links were only displayed when running from source.
This was due to ansible-test checking for the presence of documentation files locally, which are only present when running from source.
The check is no longer necessary since there is a sanity test in place to enforce the presence of documentation for all sanity tests.
(cherry picked from commit 32d965e)
Co-authored-by: Matt Clay <matt@mystile.com>
Fixes#61978
* moar tests for get_url fetch behavior with existing file
* add changelog fragment
(cherry picked from commit 7d51cac)
Co-authored-by: Matt Martz <matt@sivel.net>
* Fix for junos cli_config replace option
* For device that support replace option by loading
configuration from a file on device `config` option
is not required and value of `replace` option is the
path of configuration file on device. This fix allows
invoking run() function in cli_config if `config` option
is None and `replace` option is not boolean
* The command to replace running config on junos device
is `load override <filename>` and not `load replace <filename>`
This is fixed in the junos cliconf plugin.
* Add integration test
(cherry picked from commit 200ed25648)
* VMware: Fix issue with order of changes in vmware_vcenter_statistics
* [WIP] VMware: Fix fragile sort order in vmware_vcenter_statistics (#62288)
* vmware_vcenter_statistics: Fix fragile sort order
* vmware_vcenter_statistics: Python 2.6 compatibility
(cherry picked from commit 3e4d5aeee3)
The default behavior of the ansible-test vcenter plugin is to use the govcsim container to run tests.
However, unless the govcsim mode was specified using the VMWARE_TEST_PLATFORM environment variable, the filter code would skip the tests unless the tests ran on Shippable or the user had an ansible-core-ci key.
Now the filter correctly recognizes that govcsim is the default.
(cherry picked from commit cd4882e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix location of unit test requirements.
* Preserve ansible-test unit test requirements.
* Remove redundant unit test requirements.
* Fix location of network test requirements.
* Preserve ansible-test network test requirements.
* Remove redundant network test requirements.
* Add missing ordereddict requirements.
* Load collection requirements correctly.
* Add changelog fragment.
(cherry picked from commit cdc4926)
Co-authored-by: Matt Clay <matt@mystile.com>
* docs: Update apt_key.py, add requirements of gpg (#61552)
(cherry picked from commit 5d7d4a93c8)
* Proposals aren't really used much, delete reference from docs (#61800)
(cherry picked from commit 41055b5dc8)
* Update gitlab_hooks docs (#61837)
(cherry picked from commit 223dab99ea)
* Unify ios_logging and eos_logging documentation with accepted options (#61777)
(cherry picked from commit cc9adf7f1a)
* don't set default:None in documentation when there is no default (#61111), otherwise "None" will be rendered as a string on docsite
(cherry picked from commit 1aca1f86b6)
* Update mso_schema_template_deploy.py (#61849), correct module name in examples
(cherry picked from commit 394a05108d)
* Update hashi_vault.py documentation with kv V2 example (#61221)
(cherry picked from commit 70f501d5c8)
* [stable-2.9] Fix ansible-test pytest plugin loading. (#62119)
* Avoid assertion rewriting in pytest plugins.
Adding PYTEST_DONT_REWRITE to the ansible-test pytest plugin docstrings disables assertion rewriting in pytest for those plugins.
This avoids warnings during test execution if the plugins are loaded multiple times (such as being imported within tests).
* Run ansible-test pytest plugins early.
The ansible-test pytest plugins need to load and run earlier than conftest modules.
To facilitate this, the pytest_configure function is run during loading, which works since they are loaded (but not always run) before conftest modules are loaded.
A check has also been added to the pytest_configure functions to prevent them from running multiple times in the same process.
* Load pytest plugins using an env var.
The -p command line option loads plugins before conftest, but only during collection.
The PYTEST_PLUGINS environment variable loads plugins before confest, both during collection and test execution.
(cherry picked from commit aaa6d2e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add missing changelog entry for ansible-test fix.
PR https://github.com/ansible/ansible/pull/62119 was missing a changelog entry.
(cherry picked from commit 6c78f02121)
* bump hcloud version to 1.4.1
`hcloud`<=1.4.0 has requirement `requests==2.20.0`. This prevents the
installation of the Vcenter Automation SDK which depends on `requests>=2.22.0`.
`hcloud` 1.4.1 does not have the problem: 8bff356efb
Bumping the dependency will resolve the issue.
(cherry picked from commit 0f52b18)
Co-authored-by: Gonéri Le Bouder <goneri@lebouder.net>
Creating a virtual environment using `venv` when running in a virtual environment created by `virtualenv` results in a copy of the original virtual environment instead of creation of a new one.
To work around this, `ansible-test` now identifies when it is running in a `virtualenv` created virtual environment and uses the real Python interpreter to create the `venv` virtual environment.
(cherry picked from commit a7bc11c)
Co-authored-by: Matt Clay <matt@mystile.com>
The `test/results/` directory for Ansible test output was already ignored when not using git.
When Ansible Collections were switched to `tests/output/` the ignore entry was previously overlooked.
(cherry picked from commit f110abb)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix ansible-test venv activation.
When using the ansible-test --venv option, an execv wrapper for each python interpreter is now used instead of a symbolic link.
* Fix ansible-test execv wrapper generation.
Use the currently running Python interpreter for the shebang in the execv wrapper instead of the selected interpreter.
This allows the wrapper to work when the selected interpreter is a script instead of a binary.
* Fix ansible-test sanity requirements install.
When running sanity tests on multiple Python versions, install requirements for all versions used instead of only the default version.
* Fix ansible-test --venv when installed.
When running ansible-test from an install, the --venv delegation option needs to make sure the ansible-test code is available in the created virtual environment.
Exposing system site packages does not work because the virtual environment may be for a different Python version than the one on which ansible-test is installed.
(cherry picked from commit c77ab11051)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fixes to ecs_certificate cert chain for #61738
* Added changelog fragment
* Fixes to ecs_certificate for cleaner join, and better integration test
* Fix integration test formatting
* End cert chain with a \n
* Update changelogs/fragments/61738-ecs-certificate-invalid-chain.yaml
Co-Authored-By: Felix Fontein <felix@fontein.de>
* Update main.yml
(cherry picked from commit 943888b955)
Python < 2.7.9 does not have the ssl.SSLContext attribute.
ssl.SSLContext is only required when we want to validate the SSL
connection. If `validate_certs` is false, we don't initialize the
`ssl_context` variable.
Add unit-test coverage and a little refactoring:
- avoid the use of `mocker`, when we can push `monkeypatch` which is
`pytest`'s default.
- use `mock.Mocker()` when possible
closes: #57072
(cherry picked from commit 3ea8e0a144)
On OpenBSD, 13 asterisk characters as a password hash, marks the
account as disabled. Otherwise daily(8) script which executes
security(8) will email operator about not properly locked accounts.
Before the diff, we see following warning:
> [WARNING]: The input password appears not to have been hashed. The 'password' argument must be encrypted for this module to work properly.
After the diff, warning is gone.
(cherry picked from commit 1dea661ce8)
Co-authored-by: kucharskim <mikolaj@kucharski.name>
* [stable-2.9] aws_codebuild tests - add retries and delay instead of a pause task (#61731)
- Add retries instead of a pause task
- Shorten the IAM role name length
- Put the IAM role name in defaults/main.yml instead
- Fix the aws_codepipeline tests too
(cherry picked from commit 9ee0deea24)
Co-authored-by: Sloane Hertel <shertel@redhat.com>
* codebuild/codepipeline - fix regex for shippable resource_prefix (#61783)
* Fix bad assumption about shippable resource_prefix for codebuild and codepipeline tests
* Update test/integration/targets/aws_codepipeline/defaults/main.yml
(cherry picked from commit d5c8d325e4)
* ansible-test - Contiune if the git command returns an error
* Just return stdout
* Use to_text() when displaying exception
* Add a message property to SubprocessError
(cherry picked from commit e218c98)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* [stable-2.9] Fix ansible-test coverage path handling. (#61528)
* Fix ansible-test coverage path handling.
* Split CI unit tests into two groups.
(cherry picked from commit e4e5005640)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add changelog fragment.
The `git submodule status` command is relative to the current git repository by default.
When running from a repository subdirectory paths can be returned above the current directory.
Specifying the current directory with `git submodule status` avoids listing submodules above that directory.
This will fix issues when testing a collection that is rooted below the repository root when that repository uses submodules.
(cherry picked from commit 4063d58339)
Co-authored-by: Matt Clay <matt@mystile.com>
* Azure fix _info/_facts return values for some modules
* Further test fixes
* securitygroup fixes after the move to _info module
(cherry picked from commit 951dac7691)
* Refactor galaxy collection API for v3 support (#61510)
* Refactor galaxy collection API for v3 support
* Added unit tests for GalaxyAPI and starting to fix other failures
* finalise tests
* more unit test fixes
(cherry picked from commit a7fd6e99d9)
* Added changelog fragment
* Initial nxos_file_copy action plugin work
* Remove code from nxos_file_copy module
* Add file_push and file_pull support
* Additional refactoring and shipable updates
* Simplify outcomes and update doc header
* Add more error data information for easier debugging
* Reorder outcomes and add additional tests
* Capture more data for permission denied outcome
* Properly load module_prefix for collections
Now that we are using collections for tasks, we need to properly split
the name and load the prefix properly.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Add unit tests for task_executor
This commit adds missing unit tests for action handler in test_task_executor.
Signed-off-by: Daniel Mellado <dmellado@redhat.com>
Add support for create or re-configure VM with multiple CD-ROMs attaching to IDE controller now, will implement SATA controller support later.
parameters can be set as below:
cdrom:
- controller_type: ide
controller_number: 0
unit_number: 0
type: client
Mike Wiebe