Improve tests
- add more unit test cases
- add specific integration test with more cases
Testing shows no major downside to calling .strip() twice in a comprehension vs. using a regular for loop and only calling .strip() once. Going with the comprehension for ease of maintenance and because comprehensions are optimized in CPython.
(cherry picked from commit 987265a6ef)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Fix network plugin config option load from collection
Fixes#63975
* Update the complete sub-plugin name within network
connection plugins to handle sub-plugin in collection
scenario.
* Fix review comments
(cherry picked from commit 60276d3d8d)
* Master (#62626)
* Update win_package.ps1
Update Test-Path to use -LiteralPath instead of -Path to fix issue where powershell will not detect path with special characters such as '=' and '[]'.
* Update win_package.ps1
modified other instances of -Path and changed to -LiteralPath. All except line L243 since it is a different function.
* added literal path to get-itemproperty
(cherry picked from commit 153a322f54)
* add fragment
* Fix up role version pagination for Galaxy install
* Fix sanity issue
(cherry picked from commit 7acae62fa8)
Co-authored-by: Jordan Borean <jborean93@gmail.com>
This fixes a regression that was caused by switching from copy() to
deepcopy() when 'saving' variables before templating. Since HostVars
did not implement the __deepcopy__() method, deepcopy returned incorrect
results when host vars were present in the variables.
Fixes#63940
(cherry picked from commit cd8ce16d48)
update_resource and delete_resource takes and requires four argument.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 21c8dae83b)
This isn't used any place, lets remove it to fix lint checks on our
network collections.
(cherry picked from commit 119acc1afe)
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* [stable-2.9] Set alter_sys=True instead of False to address backwards incompat (#64670)
* Set alter_sys=True instead of False to address backwards incompat
* ci_complete
* Add integration test
* ci_complete
* sanity
* ci_complete
* Changelog fragment
* Update import test and validate-modules to match.
(cherry picked from commit b93d92ef9a)
Co-authored-by: Matt Martz <matt@sivel.net>
* Rebase and add alter_sys to validate-modules
wait_for_connection creates AnsiballZ_ping.py in temp directory,
which remains on remote machine even after playbook run.
Fixes: #62407
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 68428efc39)
* Move X25519, X448, Ed25519 and Ed448 feature tests to module_utils.
* Correctly sign with Ed25519 and Ed448 keys.
* Fix public key comparison. Ed25519 and Ed448 do not have public_numbers().
* Add tests.
* Add changelog.
* Give better errors for cryptography 2.6.x and 2.7.x.
* Test for new errors.
* Forgot one.
* Used wrong private key.
* Use private key password for CA key. Add more stuff to its certificate.
(cherry picked from commit fed267df03)
* docker_login: Use with statement for accessing files (#64382)
* Update changelogs/fragments/64382-docker_login-fix-invalid-json.yml
Co-Authored-By: Felix Fontein <felix@fontein.de>
(cherry picked from commit 52c4c1b00d)
The error is not always an invalid username/password. It could be a connection timeout or refusal.
(cherry picked from commit d5fbe6573b)
Co-authored-by: Sam Doran <sdoran@redhat.com>
Add integration test for copy: deep recursive with remote_src=True
(cherry picked from commit b7e38dfa52)
Co-authored-by: Alexander Korsunsky <A.Korsunsky@gmail.com>
* Fix#63919: don't run os.makedirs on empty dir path
* integration test for lineinfile create: yes without path (Sam Doran <sdoran@redhat.com>)
(cherry picked from commit 3c978a3225)
* check status code value lower boundary
Any HTTP code below 200 cannot be considered a success, should be
handled like a failure instead.
This is particularly true for below zero status codes.
Fixes#63139
* provide changelog fragment
* ensure connection errors are handled in Acme module
* add fetch_url check to ACME.send_signed_request
* remove module.fail_json
* move _assert_fetch_url_success out of ACMEAccount
* fix ansible-lint errors
* use simplified syntax status checking
(cherry picked from commit 0d905a0496)
* Make acl module to work with whitespaces in path
* Added a changelog fragment
* Add quotes to changelog fragment
(cherry picked from commit 504d76e956)
* Fix issue when setting an empty pass to no_log param (#62804)
* Fix issue when setting an empty pass to no_log param
* Fix typo
(cherry picked from commit 322e225830)
* Fix up actual get for older versions
This fixes ansible-test so it no longer tries to install sanity test dependencies on unsupported Python versions.
(cherry picked from commit 437e9b7063)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix validate-modules support for collections.
- Relative imports now work correctly.
- The collection loader is now used.
- Modules are invoked as `__main__`.
* Remove obsolete validate-modules code ignores.
* Handle sys.exit in validate-modules.
* Add check for AnsibleModule initialization.
* Remove `missing-module-utils-import` check.
This check does not support relative imports or collections.
Instead of trying to overhaul the test, we can rely on the `ansible-module-not-initialized` test instead.
* Fix badly named error codes with `c#` in the name.
The `#` conflicts with comments in the sanity test ignore files.
* Add changelog entries.
(cherry picked from commit e9f8a34dce)
Co-authored-by: Matt Clay <matt@mystile.com>
* clear configuration candidate when return to user-view.
* add a changelog fragment for the pr.
* Update 63513-ce_action_wait_prompt_trigger_time_out.yaml
* Update 63513-ce_action_wait_prompt_trigger_time_out.yaml
(cherry picked from commit 47c31c201b)
Use a separate variable for the boolean test rather than having the same variable sometimes be a boolean and sometimes be a regular expression match object
Add integration tests to cover this scenario
(cherry picked from commit 29d4d318a5)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* crypto modules: improve return value list documentation (#62929)
* Improve return value documentation by allowing entry for return values.
* Add docs formatting, adjust styling.
* Fix sample return value. (Taken from https://tools.ietf.org/html/rfc7517#appendix-A.1.)
* Work around abuse of .
(cherry picked from commit 054285c34c)
* Add changelog.
Initialize variables in case the shadow file is not found.
Handle IndexErrors if something goes wrong with file parsing.
(cherry picked from commit e9d10f94b7)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* ce_netstream_global: bugs fix(list index out of range) (#63332)
* update to fix bugs:index out of range
* list index out of range
* list index out of range
* update for bad-whitespace
* Update ce_netstream_aging.py
* Update ce_netstream_aging.py
* Update ce_netstream_global.py
* Update ce_netstream_global.py
* Update ce_netstream_global.py
* Update ce_netstream_template.py
* Update ce_netstream_global.py
* Update ce_netstream_global.py
* Update ce_netstream_template.py
* Update ce_netstream_aging.py
(cherry picked from commit b7f12f9ff3)
* add a changelog fragment.
* Update 63389_ce_netsream_list_index_out_of_range.yml
empty-lines too many blank lines (1 > 0)
* clean "changed" after it has been processed
without this change, a loop of `debug` tasks with `changed_when`
causes the "changed" status to get lost before output
* runme.sh tests for debug loop status
(cherry picked from commit bfd32c9b00)
* Specifying IP addresses needs API version 1.22 or newer.
* Simplify code.
* Use IPAMConfig.IPv*Address instead of IPAddress and GlobalIPv6Address.
* Add changelog.
* Fix syntax errors.
* Add integration test.
* Don't rely on netaddr.
* Normalize IPv6 addresses before comparison.
* Install netaddr, and use it.
(cherry picked from commit 62c0cae29a)
On python 3, if there is no explicit "return True", the
function call will be seen as "False", thus failling the module
(cherry picked from commit 75c4e9ec05)
* [stable-2.9] Fix various import sanity test issues.
- Relative imports are now properly recognized.
- Correct script invocation of Ansible modules is used.
- Warnings are now consistently reported as errors.
- Errors are now consistently reported with the file tested.
Resolves https://github.com/ansible/ansible/issues/62723
Resolves https://github.com/ansible/ansible/issues/61884
(cherry picked from commit 92ccdeac31)
Co-authored-by: Matt Clay <matt@mystile.com>
* Changelog entry for ansible-test sanity fixes.
(cherry picked from commit 0923ed56c7)
* AH servers include automation-hub as part of the server configuration.
So we don't need to add it here.
Fixes#63699
* Update unittests for the fix to galaxy wait_import_task
(cherry picked from commit cc1ff57)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
This brings in the final Python 3.8.0 release instead of a release candidate.
(cherry picked from commit 7448084858)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] Added changelog fragment
(cherry picked from commit 7da37e5)
Co-authored-by: Andrea Tartaglia <andrea@braingap.uk>
* Fixes --version in ansible-galaxy cli
(cherry picked from commit 202ad4f89a)
* Handle galaxy v2/v3 API diffs for artifact publish response
For publishing a collection artifact
(POST /v3/collections/artifacts/), the response
format is different between v2 and v3.
For v2 galaxy, the 'task' url returned is
a full url with scheme:
{"task": "https://galaxy-dev.ansible.com/api/v2/collection-imports/35573/"}
For v3 galaxy, the task url is relative:
{"task": "/api/automation-hub/v3/imports/collections/838d1308-a8f4-402c-95cb-7823f3806cd8/"}
So check which API we are using and update the task url approriately.
* Use full url for all wait_for_import messages
Update unit tests to parameterize the expected
responses and urls.
* update explanatory comment
* Rename n_url to full_url.
* Fix issue with overwrite of the complete path
* Fixes overwrite of the complete path in case there's extra path stored
in self.api_sever
* Normalizes the input to the wait_import_task function so it receives
the same value on both v2 and v3
Builds on #63523
* Update unittests for new call signature
* Add changelog for ansible-galaxy publish API fixes.
(cherry picked from commit 4cad7e4)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* Get no_log parameters from subspec
* Add changelog and unit tests
* Handle list of dicts in suboptions
Add fancy error message (this will probably haunt me)
* Update unit tests to test for list of dicts in suboptions
* Add integration tests
* Validate parameters in dict and list
In case it comes in as a string
* Make changes based on feedback, fix tests
* Simplify validators since we only need to validate dicts
Add test for suboptions passed in as strings to ensure they get validated properly and turned into a dictionary.
ci_complete
* Add a few more integration tests
(cherry picked from commit e9d29b1fe4)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* isa string should rewrap as unsafe in get_validated_value
* _is_unsafe shouldn't be concerned with underlying types
* Start with passwords as text, instead of bytes
* Remove unused imports
* Add changelog fragment
* Update changelog with CVE
* fix default collection resolution in adhoc
* if an adhoc command is run with a playbook-dir under a configured collection, default collection resolution is used to resolve unqualified module/action names
* Set ANSIBLE_PLAYBOOK_DIR in integration tests.
* Fix config conflict in ansible integration test.
* add adhoc default collection test
* text-ify warning string
(cherry picked from commit 6d52bdf4db)
* config encode errors should not be fatal (#63311)
* fixes#63310
* subset of fixes from #58638
* added warning on error
(cherry picked from commit 77de663879)
* bring back text-ification from #63349
* Stop appending '/api' to galaxy server url (#63238)
* Stop appending '/api' to configured galaxy server url
Since not all galaxy REST api server URLs live
at '/api', stop always appending it to the
'url' value loaded from config.
* Add note about manually migrated galaxy configs and /api
* Add '/api/' to galaxy url and guessing if galaxy API
* Fix most unit tests (update to expect /api/)
* Fix test_initialise_unknown unit test
Since we retry now with an added /api/, mock it as well.
* Update fallback default avail_ver to new format
(cherry picked from commit bad72693e4)
* Add changelog fragment galaxy_api_config
* Fix galaxy url use everywhere when url is set in config. (#63286)
In addition to trying the configured url (for ex, a migrated
'https://galaxy.ansible.com/') there is an attempt at that
URL with '/api' postpended.
If the extended URL works, update GalaxyAPI.api_server to
use the extended URL. Previously it only used it for finding
the API root info ('available_versions', etc)
(cherry picked from commit ed203c5902)
* add ANSIBLE_PLAYBOOK_DIR envvar support
* allows `ANSIBLE_PLAYBOOK_DIR` envvar as a fallback on CLI types that support `--playbook-dir`. This should have been implemented with #59464, but was missed due to an oversight.
* added basic integration test
* make first-class PLAYBOOK_DIR config entry
* update changelog
(cherry picked from commit fd229dcbb5)
ansible-test now properly searches for `pythonX.Y` instead of `python` when looking for the real python that created a `virtualenv`.
(cherry picked from commit b91f452f4f)
Co-authored-by: Matt Clay <matt@mystile.com>
Newer versions of ssh-keygen create PEM keys that are not recognized by Paramiko.
Now ansible-test compensates for this by updating they keys it generates so Paramiko will recognize them.
(cherry picked from commit 022335669c)
Co-authored-by: Matt Clay <matt@mystile.com>
Previously the temporary directory used to run integration tests resided under the user's home directory. This prevented ansible-playbook from detecting the default collection when running tests.
Now the temporary directory is created within the collection to facilitate default collection detection.
(cherry picked from commit 4c79f1ec4d)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix iosxr_lag_interfaces intermittent failures
* If the dictionary is read out of order from member
the current logic in `diff_list_of_dicts` returns
unwanted diff. Hence use `dict_diff` utils
function instead of sets.
Remove zip() to make existing tests happy
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
* Address review comments
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
(cherry picked from commit 69317a9d3e)
Add changelog for iosxr_lag_interfaces fix
This fixes test errors related to failures copying temporary test results files from a remote system back to the local system.
It also speeds up processing of test results and reduces network utilization by avoiding the temporary files.
(cherry picked from commit 3f2380ccce)
Co-authored-by: Matt Clay <matt@mystile.com>
Running from an installed version of ansible-test now results in tests using a dedicated directory for PYTHONPATH instead of using the site-packages directory where ansible is installed.
This provides consistency with tests running from source, which already used a dedicated directory.
Resolves https://github.com/ansible/ansible/issues/62716
(cherry picked from commit 831e1bf2e0)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] k8s: apply no longer the default behaviour (#62632)
There are too many lingering uncertainties about the correctness of
apply behaviour. All tests seem to suggest it works, and it's definitely
performed well in real world application, but it may be a breaking
and unexpected change to default to apply
We will let apply bed in during 2.9 and allow people to opt in, we
may default to it in future (or not)
(cherry picked from commit bb0fa0a)
Co-authored-by: Will Thames <will@thames.id.au>
* Add a representer for AnsibleUnsafeBytes
* changelog
* Add unit tests
Remove native string test until we have time to evaluate how this the function should work
Add non-ASCII characters to test cases
* Compare to the string on Python 2
Add a comment in the test about this behavior
(cherry picked from commit 4cc4c44dd0)
* Check module names in action plugin without collection attached (#60947)
* Check for eos_config in action plugin by module name, not entire fqmn
* Modify toher action plugins to find module name
* Restore missing `not`
* Cover netconf plugin as well
* Whoops
(cherry picked from commit e89048f68a)
* Add changelog entry
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Ensure k8s apply works with check mode
Update the new predicted object with fields from the previous object
before applying in check mode
Don't log output of `file` with `state: absent` on huge virtualenvs!
Fixes#60510
* Use openshift client fix to improve apply for check mode
Use new apply_object method to get a better approximation
of the expected object in check mode.
Requires released upgrade to openshift
* Add changelog fragment for k8s apply check mode fix
* Update changelogs/fragments/60510-k8s-apply-check-mode.yml
Co-Authored-By: Felix Fontein <felix@fontein.de>
(cherry picked from commit a684bb9f5b)
* Fix plugin names for collection plugins.
Add an integration test to verify plugin __name__ is correct for collection plugins.
* Fix collection loader PEP 302 compliance.
The `find_module` function now returns `None` if the module cannot be found. Previously it would return `self` for modules which did not exist.
Returning a loader from `find_module` which cannot find the module will result in import errors on Python 2.x when using implicit relative imports.
* add changelog
* sanity/units/merge fixes
(cherry picked from commit 1c64dba3c9)
In some remote environments, the `crontab` executable is
overloaded with a custom executable, which typically does
some pre/post processing before forwarding to crontab.
Instead of using the hardcoded `/usr/bin/crontab`, this uses
the `get_bin_path` utility to locate the default crontab executable.
(cherry picked from commit 951a80c8b0)
Co-authored-by: Jean-Frédéric <JeanFred@users.noreply.github.com>
* Remove unsed import for eos facts module (#61795)
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit f81b7dd10a)
* Remove unused import for cisco ios facts (#61790)
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit 44eaea9f80)
* Remove unsed import for junos facts (#61787)
This is no longer needed.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit 7be672e1c2)
* Remove unused import from iosxr facts (#61785)
This is no longer needed and can be removed.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit c723eb2f04)
* Remove unused import for vyos facts (#61784)
This is no longer needed and can be removed.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit 736938625b)
* Add changelog entry
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Update default test container with Python 3.8b4 (#62100)
* [stable-2.9] test: bump default-test-container
VMware VSphere SDK needs an up to date version of `pip` for the
installation step. With the current image, we face the following error:
```
(...)
02:27 Collecting git+https://github.com/vmware/vsphere-automation-sdk-python.git (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2))
02:27 Cloning https://github.com/vmware/vsphere-automation-sdk-python.git to /tmp/pip-req-build-pm27t16b
02:33 Requirement already satisfied: pyvmomi in /usr/local/lib/python3.6/dist-packages (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 1)) (6.7.1.2018.12)
02:33 Requirement already satisfied: lxml>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from vSphere-Automation-SDK==1.4.0->-r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2)) (4.4.0)
02:33 Processing ./\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl
02:33 Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/root/ansible/\\\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl'
```
Bump default-test-container to 1.9.3 to get an up to date release of
`pip` (was 19.0.2, is now 19.2.3)..
(cherry picked from commit b68f5b406a)
Co-authored-by: Gonéri Le Bouder <goneri@lebouder.net>
* Add missing changelog fragments. (#62471)
* Add missing default-test-container 1.9.2 fragment.
* Add missing default-test-container 1.9.3 fragment.
* Fix network_cli exec_command connection init
Fixes https://github.com/ansible/ansible/issues/61596
* If `exec_command` method is invoked from module side
on connection object to execute the command on target
host check if connection is created if not create the
connection.
* Fix review comment
(cherry picked from commit 74e4993628)
* Fix ansible-doc traceback for removed modules.
This avoids tracebacks with errors like the following when a module has been removed:
module module_name missing documentation (or could not parse documentation): 'NoneType' object does not support item assignment
* Fix ansible-doc sanity test warning handling.
Warnings about removed modules/plugins on stderr are now properly ignored.
Previously an ansible-doc error could result in unrelated errors going undetected because tests were stopped early and the underlying error was ignored.
(cherry picked from commit 064e8e1ef4)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] Fix "JSON object must be str, bytes or bytearray, not list" (#62350)
(cherry picked from commit 84d9b3e)
Co-authored-by: Nathaniel Case <ncase@redhat.com>
* Add changelog
* routeros_facts: fix for error when there's more than 10 interfaces (#61376)
* fix: proper regex for preprocessing routeros output
* test: regression test
* test: fix nondeterministic unit test
* changelog
* Commands tests (#62322)
* commands tests
* add space in order to delete it and tun shipable tests again
* delete space in order to run shipable tests again
(cherry picked from commit 47cf4e6565)
* changelog
Fixes#62319
Change `enable` option to `enabled` in junos_interfaces
and junos_lldp_interfaces
data model to be in sync with other network platform
resource modules added in 2.9 version.
(cherry picked from commit a9a5f4e40d)
The documentation links are now displayed when running from an install.
Previously the links were only displayed when running from source.
This was due to ansible-test checking for the presence of documentation files locally, which are only present when running from source.
The check is no longer necessary since there is a sanity test in place to enforce the presence of documentation for all sanity tests.
(cherry picked from commit 32d965e)
Co-authored-by: Matt Clay <matt@mystile.com>
Fixes#61978
* moar tests for get_url fetch behavior with existing file
* add changelog fragment
(cherry picked from commit 7d51cac)
Co-authored-by: Matt Martz <matt@sivel.net>
* Fix for junos cli_config replace option
* For device that support replace option by loading
configuration from a file on device `config` option
is not required and value of `replace` option is the
path of configuration file on device. This fix allows
invoking run() function in cli_config if `config` option
is None and `replace` option is not boolean
* The command to replace running config on junos device
is `load override <filename>` and not `load replace <filename>`
This is fixed in the junos cliconf plugin.
* Add integration test
(cherry picked from commit 200ed25648)
* VMware: Fix issue with order of changes in vmware_vcenter_statistics
* [WIP] VMware: Fix fragile sort order in vmware_vcenter_statistics (#62288)
* vmware_vcenter_statistics: Fix fragile sort order
* vmware_vcenter_statistics: Python 2.6 compatibility
(cherry picked from commit 3e4d5aeee3)
* remove choices from gather_network_resources facts and allow negating subset without needing to add a new subset specific for negation
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* negated all, min should not return any fact
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
(cherry picked from commit c1e02d5c7a)
* openssh_keypair: make sure public key has same permissions as private key (#61658)
* Make sure public key has same permissions as private key.
* Add changelog.
* Text, not binary.
(cherry picked from commit c19cea9b03)
* openssh_keypair file permissions/ownership: add porting guide entry (#62176)
* Add porting guide entry for 2.9.
(cherry picked from commit 0e72cbd451)
* In pika v1.0.0 BlockingChannel.is_closing was removed. Updating
plugin accordingly.
Ref: https://github.com/pika/pika/pull/1034
* Adding change fragment for is_closing bug.
* Updated change fragment description.
(cherry picked from commit 9b149917a6)
* Pika v1.0.0 and above were causing issues for publish_message. Updated
to ensure publish_message works with pika 0.13.1 and 1.0.0 and above.
* Adding changelog fragment for rabbitmq_publish fix.
* Updating return value.
(cherry picked from commit 1b2fd2cb5f)
* Always specify header of connection keep-alive regardless of python version.
* Add chgangelog fragment
* Fixes to changelog fragment
(cherry picked from commit 606e13919e)
Pilou