* Specifying IP addresses needs API version 1.22 or newer.
* Simplify code.
* Use IPAMConfig.IPv*Address instead of IPAddress and GlobalIPv6Address.
* Add changelog.
* Fix syntax errors.
* Add integration test.
* Don't rely on netaddr.
* Normalize IPv6 addresses before comparison.
* Install netaddr, and use it.
(cherry picked from commit 62c0cae29a)
On python 3, if there is no explicit "return True", the
function call will be seen as "False", thus failling the module
(cherry picked from commit 75c4e9ec05)
* [stable-2.9] Fix various import sanity test issues.
- Relative imports are now properly recognized.
- Correct script invocation of Ansible modules is used.
- Warnings are now consistently reported as errors.
- Errors are now consistently reported with the file tested.
Resolves https://github.com/ansible/ansible/issues/62723
Resolves https://github.com/ansible/ansible/issues/61884
(cherry picked from commit 92ccdeac31)
Co-authored-by: Matt Clay <matt@mystile.com>
* Changelog entry for ansible-test sanity fixes.
(cherry picked from commit 0923ed56c7)
* AH servers include automation-hub as part of the server configuration.
So we don't need to add it here.
Fixes#63699
* Update unittests for the fix to galaxy wait_import_task
(cherry picked from commit cc1ff57)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
This brings in the final Python 3.8.0 release instead of a release candidate.
(cherry picked from commit 7448084858)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] Added changelog fragment
(cherry picked from commit 7da37e5)
Co-authored-by: Andrea Tartaglia <andrea@braingap.uk>
* Fixes --version in ansible-galaxy cli
(cherry picked from commit 202ad4f89a)
* Handle galaxy v2/v3 API diffs for artifact publish response
For publishing a collection artifact
(POST /v3/collections/artifacts/), the response
format is different between v2 and v3.
For v2 galaxy, the 'task' url returned is
a full url with scheme:
{"task": "https://galaxy-dev.ansible.com/api/v2/collection-imports/35573/"}
For v3 galaxy, the task url is relative:
{"task": "/api/automation-hub/v3/imports/collections/838d1308-a8f4-402c-95cb-7823f3806cd8/"}
So check which API we are using and update the task url approriately.
* Use full url for all wait_for_import messages
Update unit tests to parameterize the expected
responses and urls.
* update explanatory comment
* Rename n_url to full_url.
* Fix issue with overwrite of the complete path
* Fixes overwrite of the complete path in case there's extra path stored
in self.api_sever
* Normalizes the input to the wait_import_task function so it receives
the same value on both v2 and v3
Builds on #63523
* Update unittests for new call signature
* Add changelog for ansible-galaxy publish API fixes.
(cherry picked from commit 4cad7e4)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* Get no_log parameters from subspec
* Add changelog and unit tests
* Handle list of dicts in suboptions
Add fancy error message (this will probably haunt me)
* Update unit tests to test for list of dicts in suboptions
* Add integration tests
* Validate parameters in dict and list
In case it comes in as a string
* Make changes based on feedback, fix tests
* Simplify validators since we only need to validate dicts
Add test for suboptions passed in as strings to ensure they get validated properly and turned into a dictionary.
ci_complete
* Add a few more integration tests
(cherry picked from commit e9d29b1fe4)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* isa string should rewrap as unsafe in get_validated_value
* _is_unsafe shouldn't be concerned with underlying types
* Start with passwords as text, instead of bytes
* Remove unused imports
* Add changelog fragment
* Update changelog with CVE
* fix default collection resolution in adhoc
* if an adhoc command is run with a playbook-dir under a configured collection, default collection resolution is used to resolve unqualified module/action names
* Set ANSIBLE_PLAYBOOK_DIR in integration tests.
* Fix config conflict in ansible integration test.
* add adhoc default collection test
* text-ify warning string
(cherry picked from commit 6d52bdf4db)
* config encode errors should not be fatal (#63311)
* fixes#63310
* subset of fixes from #58638
* added warning on error
(cherry picked from commit 77de663879)
* bring back text-ification from #63349
* Stop appending '/api' to galaxy server url (#63238)
* Stop appending '/api' to configured galaxy server url
Since not all galaxy REST api server URLs live
at '/api', stop always appending it to the
'url' value loaded from config.
* Add note about manually migrated galaxy configs and /api
* Add '/api/' to galaxy url and guessing if galaxy API
* Fix most unit tests (update to expect /api/)
* Fix test_initialise_unknown unit test
Since we retry now with an added /api/, mock it as well.
* Update fallback default avail_ver to new format
(cherry picked from commit bad72693e4)
* Add changelog fragment galaxy_api_config
* Fix galaxy url use everywhere when url is set in config. (#63286)
In addition to trying the configured url (for ex, a migrated
'https://galaxy.ansible.com/') there is an attempt at that
URL with '/api' postpended.
If the extended URL works, update GalaxyAPI.api_server to
use the extended URL. Previously it only used it for finding
the API root info ('available_versions', etc)
(cherry picked from commit ed203c5902)
* add ANSIBLE_PLAYBOOK_DIR envvar support
* allows `ANSIBLE_PLAYBOOK_DIR` envvar as a fallback on CLI types that support `--playbook-dir`. This should have been implemented with #59464, but was missed due to an oversight.
* added basic integration test
* make first-class PLAYBOOK_DIR config entry
* update changelog
(cherry picked from commit fd229dcbb5)
ansible-test now properly searches for `pythonX.Y` instead of `python` when looking for the real python that created a `virtualenv`.
(cherry picked from commit b91f452f4f)
Co-authored-by: Matt Clay <matt@mystile.com>
Newer versions of ssh-keygen create PEM keys that are not recognized by Paramiko.
Now ansible-test compensates for this by updating they keys it generates so Paramiko will recognize them.
(cherry picked from commit 022335669c)
Co-authored-by: Matt Clay <matt@mystile.com>
Previously the temporary directory used to run integration tests resided under the user's home directory. This prevented ansible-playbook from detecting the default collection when running tests.
Now the temporary directory is created within the collection to facilitate default collection detection.
(cherry picked from commit 4c79f1ec4d)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix iosxr_lag_interfaces intermittent failures
* If the dictionary is read out of order from member
the current logic in `diff_list_of_dicts` returns
unwanted diff. Hence use `dict_diff` utils
function instead of sets.
Remove zip() to make existing tests happy
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
* Address review comments
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
(cherry picked from commit 69317a9d3e)
Add changelog for iosxr_lag_interfaces fix
This fixes test errors related to failures copying temporary test results files from a remote system back to the local system.
It also speeds up processing of test results and reduces network utilization by avoiding the temporary files.
(cherry picked from commit 3f2380ccce)
Co-authored-by: Matt Clay <matt@mystile.com>
Running from an installed version of ansible-test now results in tests using a dedicated directory for PYTHONPATH instead of using the site-packages directory where ansible is installed.
This provides consistency with tests running from source, which already used a dedicated directory.
Resolves https://github.com/ansible/ansible/issues/62716
(cherry picked from commit 831e1bf2e0)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] k8s: apply no longer the default behaviour (#62632)
There are too many lingering uncertainties about the correctness of
apply behaviour. All tests seem to suggest it works, and it's definitely
performed well in real world application, but it may be a breaking
and unexpected change to default to apply
We will let apply bed in during 2.9 and allow people to opt in, we
may default to it in future (or not)
(cherry picked from commit bb0fa0a)
Co-authored-by: Will Thames <will@thames.id.au>
* Add a representer for AnsibleUnsafeBytes
* changelog
* Add unit tests
Remove native string test until we have time to evaluate how this the function should work
Add non-ASCII characters to test cases
* Compare to the string on Python 2
Add a comment in the test about this behavior
(cherry picked from commit 4cc4c44dd0)
* Check module names in action plugin without collection attached (#60947)
* Check for eos_config in action plugin by module name, not entire fqmn
* Modify toher action plugins to find module name
* Restore missing `not`
* Cover netconf plugin as well
* Whoops
(cherry picked from commit e89048f68a)
* Add changelog entry
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Ensure k8s apply works with check mode
Update the new predicted object with fields from the previous object
before applying in check mode
Don't log output of `file` with `state: absent` on huge virtualenvs!
Fixes#60510
* Use openshift client fix to improve apply for check mode
Use new apply_object method to get a better approximation
of the expected object in check mode.
Requires released upgrade to openshift
* Add changelog fragment for k8s apply check mode fix
* Update changelogs/fragments/60510-k8s-apply-check-mode.yml
Co-Authored-By: Felix Fontein <felix@fontein.de>
(cherry picked from commit a684bb9f5b)
* Fix plugin names for collection plugins.
Add an integration test to verify plugin __name__ is correct for collection plugins.
* Fix collection loader PEP 302 compliance.
The `find_module` function now returns `None` if the module cannot be found. Previously it would return `self` for modules which did not exist.
Returning a loader from `find_module` which cannot find the module will result in import errors on Python 2.x when using implicit relative imports.
* add changelog
* sanity/units/merge fixes
(cherry picked from commit 1c64dba3c9)
In some remote environments, the `crontab` executable is
overloaded with a custom executable, which typically does
some pre/post processing before forwarding to crontab.
Instead of using the hardcoded `/usr/bin/crontab`, this uses
the `get_bin_path` utility to locate the default crontab executable.
(cherry picked from commit 951a80c8b0)
Co-authored-by: Jean-Frédéric <JeanFred@users.noreply.github.com>
* Remove unsed import for eos facts module (#61795)
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit f81b7dd10a)
* Remove unused import for cisco ios facts (#61790)
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit 44eaea9f80)
* Remove unsed import for junos facts (#61787)
This is no longer needed.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit 7be672e1c2)
* Remove unused import from iosxr facts (#61785)
This is no longer needed and can be removed.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit c723eb2f04)
* Remove unused import for vyos facts (#61784)
This is no longer needed and can be removed.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
(cherry picked from commit 736938625b)
* Add changelog entry
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Update default test container with Python 3.8b4 (#62100)
* [stable-2.9] test: bump default-test-container
VMware VSphere SDK needs an up to date version of `pip` for the
installation step. With the current image, we face the following error:
```
(...)
02:27 Collecting git+https://github.com/vmware/vsphere-automation-sdk-python.git (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2))
02:27 Cloning https://github.com/vmware/vsphere-automation-sdk-python.git to /tmp/pip-req-build-pm27t16b
02:33 Requirement already satisfied: pyvmomi in /usr/local/lib/python3.6/dist-packages (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 1)) (6.7.1.2018.12)
02:33 Requirement already satisfied: lxml>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from vSphere-Automation-SDK==1.4.0->-r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2)) (4.4.0)
02:33 Processing ./\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl
02:33 Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/root/ansible/\\\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl'
```
Bump default-test-container to 1.9.3 to get an up to date release of
`pip` (was 19.0.2, is now 19.2.3)..
(cherry picked from commit b68f5b406a)
Co-authored-by: Gonéri Le Bouder <goneri@lebouder.net>
* Add missing changelog fragments. (#62471)
* Add missing default-test-container 1.9.2 fragment.
* Add missing default-test-container 1.9.3 fragment.
* Fix network_cli exec_command connection init
Fixes https://github.com/ansible/ansible/issues/61596
* If `exec_command` method is invoked from module side
on connection object to execute the command on target
host check if connection is created if not create the
connection.
* Fix review comment
(cherry picked from commit 74e4993628)
* Fix ansible-doc traceback for removed modules.
This avoids tracebacks with errors like the following when a module has been removed:
module module_name missing documentation (or could not parse documentation): 'NoneType' object does not support item assignment
* Fix ansible-doc sanity test warning handling.
Warnings about removed modules/plugins on stderr are now properly ignored.
Previously an ansible-doc error could result in unrelated errors going undetected because tests were stopped early and the underlying error was ignored.
(cherry picked from commit 064e8e1ef4)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] Fix "JSON object must be str, bytes or bytearray, not list" (#62350)
(cherry picked from commit 84d9b3e)
Co-authored-by: Nathaniel Case <ncase@redhat.com>
* Add changelog
* routeros_facts: fix for error when there's more than 10 interfaces (#61376)
* fix: proper regex for preprocessing routeros output
* test: regression test
* test: fix nondeterministic unit test
* changelog
* Commands tests (#62322)
* commands tests
* add space in order to delete it and tun shipable tests again
* delete space in order to run shipable tests again
(cherry picked from commit 47cf4e6565)
* changelog
Fixes#62319
Change `enable` option to `enabled` in junos_interfaces
and junos_lldp_interfaces
data model to be in sync with other network platform
resource modules added in 2.9 version.
(cherry picked from commit a9a5f4e40d)
The documentation links are now displayed when running from an install.
Previously the links were only displayed when running from source.
This was due to ansible-test checking for the presence of documentation files locally, which are only present when running from source.
The check is no longer necessary since there is a sanity test in place to enforce the presence of documentation for all sanity tests.
(cherry picked from commit 32d965e)
Co-authored-by: Matt Clay <matt@mystile.com>
Fixes#61978
* moar tests for get_url fetch behavior with existing file
* add changelog fragment
(cherry picked from commit 7d51cac)
Co-authored-by: Matt Martz <matt@sivel.net>
* Fix for junos cli_config replace option
* For device that support replace option by loading
configuration from a file on device `config` option
is not required and value of `replace` option is the
path of configuration file on device. This fix allows
invoking run() function in cli_config if `config` option
is None and `replace` option is not boolean
* The command to replace running config on junos device
is `load override <filename>` and not `load replace <filename>`
This is fixed in the junos cliconf plugin.
* Add integration test
(cherry picked from commit 200ed25648)
* VMware: Fix issue with order of changes in vmware_vcenter_statistics
* [WIP] VMware: Fix fragile sort order in vmware_vcenter_statistics (#62288)
* vmware_vcenter_statistics: Fix fragile sort order
* vmware_vcenter_statistics: Python 2.6 compatibility
(cherry picked from commit 3e4d5aeee3)
* remove choices from gather_network_resources facts and allow negating subset without needing to add a new subset specific for negation
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* negated all, min should not return any fact
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
(cherry picked from commit c1e02d5c7a)
* openssh_keypair: make sure public key has same permissions as private key (#61658)
* Make sure public key has same permissions as private key.
* Add changelog.
* Text, not binary.
(cherry picked from commit c19cea9b03)
* openssh_keypair file permissions/ownership: add porting guide entry (#62176)
* Add porting guide entry for 2.9.
(cherry picked from commit 0e72cbd451)
* In pika v1.0.0 BlockingChannel.is_closing was removed. Updating
plugin accordingly.
Ref: https://github.com/pika/pika/pull/1034
* Adding change fragment for is_closing bug.
* Updated change fragment description.
(cherry picked from commit 9b149917a6)
* Pika v1.0.0 and above were causing issues for publish_message. Updated
to ensure publish_message works with pika 0.13.1 and 1.0.0 and above.
* Adding changelog fragment for rabbitmq_publish fix.
* Updating return value.
(cherry picked from commit 1b2fd2cb5f)
* Always specify header of connection keep-alive regardless of python version.
* Add chgangelog fragment
* Fixes to changelog fragment
(cherry picked from commit 606e13919e)
The default behavior of the ansible-test vcenter plugin is to use the govcsim container to run tests.
However, unless the govcsim mode was specified using the VMWARE_TEST_PLATFORM environment variable, the filter code would skip the tests unless the tests ran on Shippable or the user had an ansible-core-ci key.
Now the filter correctly recognizes that govcsim is the default.
(cherry picked from commit cd4882e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix location of unit test requirements.
* Preserve ansible-test unit test requirements.
* Remove redundant unit test requirements.
* Fix location of network test requirements.
* Preserve ansible-test network test requirements.
* Remove redundant network test requirements.
* Add missing ordereddict requirements.
* Load collection requirements correctly.
* Add changelog fragment.
(cherry picked from commit cdc4926)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.9] Fix ansible-test pytest plugin loading. (#62119)
* Avoid assertion rewriting in pytest plugins.
Adding PYTEST_DONT_REWRITE to the ansible-test pytest plugin docstrings disables assertion rewriting in pytest for those plugins.
This avoids warnings during test execution if the plugins are loaded multiple times (such as being imported within tests).
* Run ansible-test pytest plugins early.
The ansible-test pytest plugins need to load and run earlier than conftest modules.
To facilitate this, the pytest_configure function is run during loading, which works since they are loaded (but not always run) before conftest modules are loaded.
A check has also been added to the pytest_configure functions to prevent them from running multiple times in the same process.
* Load pytest plugins using an env var.
The -p command line option loads plugins before conftest, but only during collection.
The PYTEST_PLUGINS environment variable loads plugins before confest, both during collection and test execution.
(cherry picked from commit aaa6d2e)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add missing changelog entry for ansible-test fix.
PR https://github.com/ansible/ansible/pull/62119 was missing a changelog entry.
(cherry picked from commit 6c78f02121)
Creating a virtual environment using `venv` when running in a virtual environment created by `virtualenv` results in a copy of the original virtual environment instead of creation of a new one.
To work around this, `ansible-test` now identifies when it is running in a `virtualenv` created virtual environment and uses the real Python interpreter to create the `venv` virtual environment.
(cherry picked from commit a7bc11c)
Co-authored-by: Matt Clay <matt@mystile.com>
The `test/results/` directory for Ansible test output was already ignored when not using git.
When Ansible Collections were switched to `tests/output/` the ignore entry was previously overlooked.
(cherry picked from commit f110abb)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix ansible-test venv activation.
When using the ansible-test --venv option, an execv wrapper for each python interpreter is now used instead of a symbolic link.
* Fix ansible-test execv wrapper generation.
Use the currently running Python interpreter for the shebang in the execv wrapper instead of the selected interpreter.
This allows the wrapper to work when the selected interpreter is a script instead of a binary.
* Fix ansible-test sanity requirements install.
When running sanity tests on multiple Python versions, install requirements for all versions used instead of only the default version.
* Fix ansible-test --venv when installed.
When running ansible-test from an install, the --venv delegation option needs to make sure the ansible-test code is available in the created virtual environment.
Exposing system site packages does not work because the virtual environment may be for a different Python version than the one on which ansible-test is installed.
(cherry picked from commit c77ab11051)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fixes to ecs_certificate cert chain for #61738
* Added changelog fragment
* Fixes to ecs_certificate for cleaner join, and better integration test
* Fix integration test formatting
* End cert chain with a \n
* Update changelogs/fragments/61738-ecs-certificate-invalid-chain.yaml
Co-Authored-By: Felix Fontein <felix@fontein.de>
* Update main.yml
(cherry picked from commit 943888b955)
Andrey Klychkov