* galaxy - preserve symlinks on build/install (#69959)
* galaxy - preserve symlinks on build/install
* Handle directory symlinks
* py2 compat change
* Updated changelog fragment
(cherry picked from commit d30fc6c0b3)
* Fix integration test
* ansible-galaxy - fix collection installation with trailing slashes (#70016)
If we fail to find a member when extracting a directory, try adding a trailing
slash to the member name. In certain cases, the member in the tarfile will
contain a trailing slash but the file name in FILES.json will never contain
the trailing slash.
If unable to find the member, handle the KeyError and print a nicer error.
Also check if a directory exists before creating it since it may have been
extracted from the archive.
Fixes#70009
* Add unit tests
* Use loop for trying to get members
(cherry picked from commit d45cb01b84)
Co-authored-by: Sam Doran <sdoran@redhat.com>
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from. Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.
This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.
- hosts: all
strategy: free
gather_facts: false
tasks:
- include_role:
name: random_sleep
- block:
- name: set a fact (1)
include_role:
name: set_a_fact
tasks_from: fact1.yml
- name: set a fact (2)
include_role:
name: set_a_fact
tasks_from: fact2.yml
- name: include didn't run
fail:
msg: >
set_a_fact didn't run
fact1: {{ fact1 | default('not defined')}}
fact2: {{ fact2 | default('not defined') }}"
when: (fact1 is not defined or fact2 is not defined)
Closes#69521
(cherry picked from commit 247e43b252)
* [stable-2.9] Allow the use of _paramiko_conn even if the connection hasn't been started. (#61570)
* Allow the use of _paramiko_conn even if the connection hasn't been started.
I'm not sure what the benefit is of Noneing paramiko_conn on close, but will keep for now
* Fix test
* Try to fix up net_put & net_get
* Add changelog.
(cherry picked from commit 50e09be14f)
Co-authored-by: Nathaniel Case <ncase@redhat.com>
* Restore check_prompt and task_uuid
* Fix filedescriptor out of range in select() when running commands
* Simplify the run_command() code
Now that we're using selectors in run_command(), we can simplify some of
the code.
* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
instead.
Also use a simpler conditional check of whether the selector map is
empty
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>.
(cherry picked from commit f200487414)
Co-authored-by: Bob Weinand <bobwei9@hotmail.com>
Co-authored-by: Bob Weinand <bobwei9@hotmail.com>
* Prevent Ansible 2.9 to choke on collections using deprecation by date or collection_name for deprecation calls.
* Add changelog fragment.
* Fix YAML.
* Improve C# compatibility.
* Add tests for AnsibleModule.
* Fix var name.
* Fix type.
* Update C# code.
* Show deprecation warning if removed_at_date is used for Python modules.
* Update changelogs/fragments/69935-2.10-deprecation-support.yml
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Prevent crash of validate-modules if 'removed_in' is not in 'deprecated'.
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* fix delegation vars usage (debug still shows inventory_hostname) (#69244)
* fix delegation vars usage and reporting
- just pass delegated host vars + task vars to plugins
and avoid poluting with original host vars
- updated tests
(cherry picked from commit 2165f9ac40)
* fix delegated interpreter discovery (#69604)
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
as it stands we rerun discovery for the delegated host
unless its saving facts to itself
* fixed test lacking delegate_to mock
(cherry picked from commit de3f7c7739)
* Fix `ansible -K` become_pass regression (#69629)
* Fix `ansible -K` become_pass regression
Change:
- This fixes a breaking change introduced in
2165f9ac40
Test Plan:
- Local VM for now, with plans to add an integration test for -K going
forward.
Tickets:
Refs #69244
(cherry picked from commit fe9696be52)
* fix discovery on loop with delegation (#70013)
* fix discovery on loop with delegation
fixes#69963
(cherry picked from commit 4c9d9dbb56)
Co-authored-by: Rick Elrod <rick@elrod.me>
* fix get_data on case_insensitive fs
* implement case-sensitive-forcing versions of various os.path methods that just pass through on case-sensitive systems.
* catch broader IOError for py2/py3 compat
* optimization: factor out case-insensitive comparison
* implement case-sensitive open
* ansible-galaxy - Fix tar path traversal issue during install - CVE-2020-10691 (#68596)
(cherry picked from commit a20a527014)
* Remove extra tests missing from rebase
* Fix colorization to not extend across newline boundary
* Fix unit test to look for the newline outside the coloration
* Add changelog fragment
(cherry picked from commit 2068131589)
* galaxy - preserve mode properly on artifact (#68418)
* galaxy - preserve mode properly on artifact
* Fix py2 encoding issue
* Update lib/ansible/galaxy/collection.py
Co-Authored-By: Matt Clay <matt@mystile.com>
* Use sane defaults instead of sourcing from tarfile
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 127d54b363)
* added mode to dir creation that was missed in backport
* Fix implicit string - only looked right because of the vault __repr__
* Add tests for strings and implicit strings
(cherry picked from commit 8eb00dd14c)
* ansible-galaxy - optimise some paths and use fake galaxy int tests (#67685)
* ansible-galaxy - optimise some paths and use fake galaxy int tests
* Added init, built, and publish tests
* Test against both mocked Galaxy and AH server
* Finish off writing the install tests
* Fix up broken tests
* Rename test target and add migrated tests
* Use cloud provider for Galaxy implementation
* Added blank static config
* Use correct alias group
* Set release version and fix copy typo
* Remove reset step as it is no longer needed
* Use sane env var names for test container name
(cherry picked from commit 26129fcb80)
* Use --api-key and not --token
* Set fallaxy tests as a smoketest
(cherry picked from commit b241c021b7)
As AnsibleModule._log_invocation is currently implemented, any parameter
with a name that matches PASSWORD_MATCH triggers the no_log warning as a
precaution against parameters that may contain sensitive data, but have not
been marked as sensitive by the module author.
This patch would allow module authors to explicitly mark the aforementioned
parameters as not sensitive thereby bypassing an erroneous warning message,
while still catching parameters which have not been marked at all by the
author.
Adds tests for various no_log states including True, False, and None (as
extracted by AnsibleModule._log_invocation) when applied to an argument with
a name that matches PASSWORD_MATCH.
Fixes: #49465#64656
(cherry picked from commit 3ca4580cb4)
* nxos_interfaces: RMB state fixes
* shippable fixes
* Add add'l comments per review
* fix long line
* Fix mode/enabled system defaults handling
* fix N3L test skips
* lint
* test updates for titanium images
* doc fix
(cherry picked from commit d72025be75)
Add changelog for nxos_interfaces fix
Co-authored-by: Chris Van Heuveln <cvanheuv@cisco.com>
* Fixes#66478
* When no quantity is set, then candlepin server usually uses
default value 1. When more quantities are required, then
candlepin server can automatically choose correct minimal
value.
(cherry picked from commit 6f1bb37feb)
* Make banner detection non-greedy in ios_banner (#63092)
* Make banner detection non-greedy in ios_banner
* Added ios_banner unit test to detect bug #63091
* Corrected PEP8 errors
* Added integration test
* Corrected typo in integration test
(cherry picked from commit 01a92f0191)
* Added changelog fragment
* Update 66274-ios_banner_nongreedy.yml
Co-authored-by: kalimsshar <34369784+kalimsshar@users.noreply.github.com>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
* throttle tests: fix detection of parallel execution
The test wasn't able to detect if too many workers were running.
On my laptop:
- without this change, the 'throttle' target takes ~20 seconds
- with this change, the 'throttle' target takes ~70 seconds
- 1 second isn't long enough to encounter the issue
* Fix throttle test when strategy is 'free' based
'free' strategy allows multiple tasks to be executed in parallel: use
one 'throttledir' per task.
Use 'linear' strategy with a dedicated play for cleanup/setup tasks
* throttle: reset worker idx before queuing a new task
* TestStrategyBase: define task.throttle
otherwise '1' will be used instead of the default value due to the
following expression being equal to '1':
int(templar.template(task_mock.throttle))
Co-authored-by: James Cammarata <jimi@sngx.net>
(cherry picked from commit bbbdc1c25c)
* Fix fact gathering for sensor-groups (#66439)
(cherry picked from commit 5c158755fc)
Add changelog for nxos_telemetry fix
* Update 66439_fix_nxos_telemetry_facts.yaml
Co-authored-by: Mike Wiebe <mwiebe@cisco.com>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
On Python 2, leave all fds open since there is no mechanism to close specific fds with subprocess.Popen() on Python 2
Add unit tests.
Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 617fbad743)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Fix network action plugin load in collection
Fixes https://github.com/ansible/ansible/issues/65071
* Load network action plugin that matches the module
prefix name from list of collections.
* Update changelog
* Fix unit test
(cherry picked from commit 74e9b1e219)
This causes multiple unneeded http redirects.
Since each http redirect takes a few hundred
milliseconds, handling the redirects starts to
take up a significant amount of wall time.
Fixes: #63281
(cherry picked from commit d1c39b9068)
Co-authored-by: Adrian Likins <alikins@redhat.com>
* Fix multiple subnet (of same IP version) idempotence for docker_network.
* Add changelog.
* Unit tests no longer make sense, since the part of the code they test has been removed.
* Re-add CIDR validation. Move it to better position (module setup instead of idempotence check).
* Update changelog.
* Only run new tests on VM test images.
* Actually do what is documented. Especially since an empty object is a valid value for aux_addresses.
(cherry picked from commit 17ef253ad1)
* fix all modules to be able to get parameter with value false (very important fix) (#65040)
* add import of future and __metaclass__
* fix all modules to be able to get parameter with value false (very importent fix)
* Update ignore.txt
(cherry picked from commit b1e6667664)
* Update ignore.txt
* Create 65043-enable-parameter-value-to-be-false.yaml
* Update 65043-enable-parameter-value-to-be-false.yaml
* [stable-2.9] Do not treat AnsibleUndefined as being unsafe (#65202)
* Do not treat AnsibleUndefined as being unsafe. Fixes#65198
* fix yaml formatting.
(cherry picked from commit b08e7daf46)
Co-authored-by: Matt Martz <matt@sivel.net>
* Linting fix
* additional linting fix
* docker_swarm_service: Sort lists when checking for changes
When two lists are checked for changes in this module, the lists are
reported changed when the order of the items is different. This PR
resolves this issue.
* docker_swarm_service: Minor typo fix
* docker_swarm_service: Another minor typo
* docker_swarm_service: Should use sorted(), not sort()
* docker_swarm_service: Sort lists of dictionaries
* docker_swarm_service: Fix style issues in tests
* docker_swarm_service: Updates to integration tests
* docker_swarm_service: Casting string types within lists when comparing
* docker_swarm_service: Special handling of unordered networks with ordered aliases
* docker_swarm_service: Sorting network lists
* docker_swarm_serivce: Better unit test code coverage for lists and networks
* docker_swarm_service: Fixed coding style for sanity tests
* docker_swarm_service: More coding style fixes
* docker_swarm_service: Ignoring test for Python < 3
* docker_swarm_service: Update to version info check for backwards compatibility
* docker_swarm_service: Added change fragment #63887
* docker_swarm_service: Better handling of missing sort key for dictionary of lists
* docker_swarm_service: Preventing sorts from modifying in-place
Co-Authored-By: Felix Fontein <felix@fontein.de>
* docker_swarm_service: Removed spurious import in test
* docker_swarm_service: Preventing sorts from modifying more data in-place
Co-Authored-By: Felix Fontein <felix@fontein.de>
(cherry picked from commit a096cd08c5)
Improve tests
- add more unit test cases
- add specific integration test with more cases
Testing shows no major downside to calling .strip() twice in a comprehension vs. using a regular for loop and only calling .strip() once. Going with the comprehension for ease of maintenance and because comprehensions are optimized in CPython.
(cherry picked from commit 987265a6ef)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Fix up role version pagination for Galaxy install
* Fix sanity issue
(cherry picked from commit 7acae62fa8)
Co-authored-by: Jordan Borean <jborean93@gmail.com>
- xenserver module_util: XenAPI lib import error message handling is now
ported to missing_required_lib(). Updated unit tests.
- xenserver_guest, xenserver_guest_info, xenserver_guest_powerstate modules:
docs are updated to reflect recent changes in availability of XenAPI lib
on PyPI.
- xenserver_guest, xenserver_guest_info, xenserver_guest_powerstate modules:
docs are updated to reflect recent Citrix rebranding of XenServer. Broken
URLs to external resources are fixed.
- xenserver_guest, xenserver_guest_info, xenserver_guest_powerstate modules:
more tested platforms are mentioned in docs.
(cherry picked from commit 7afba9420a)
* AH servers include automation-hub as part of the server configuration.
So we don't need to add it here.
Fixes#63699
* Update unittests for the fix to galaxy wait_import_task
(cherry picked from commit cc1ff57)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* Handle galaxy v2/v3 API diffs for artifact publish response
For publishing a collection artifact
(POST /v3/collections/artifacts/), the response
format is different between v2 and v3.
For v2 galaxy, the 'task' url returned is
a full url with scheme:
{"task": "https://galaxy-dev.ansible.com/api/v2/collection-imports/35573/"}
For v3 galaxy, the task url is relative:
{"task": "/api/automation-hub/v3/imports/collections/838d1308-a8f4-402c-95cb-7823f3806cd8/"}
So check which API we are using and update the task url approriately.
* Use full url for all wait_for_import messages
Update unit tests to parameterize the expected
responses and urls.
* update explanatory comment
* Rename n_url to full_url.
* Fix issue with overwrite of the complete path
* Fixes overwrite of the complete path in case there's extra path stored
in self.api_sever
* Normalizes the input to the wait_import_task function so it receives
the same value on both v2 and v3
Builds on #63523
* Update unittests for new call signature
* Add changelog for ansible-galaxy publish API fixes.
(cherry picked from commit 4cad7e4)
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* Get no_log parameters from subspec
* Add changelog and unit tests
* Handle list of dicts in suboptions
Add fancy error message (this will probably haunt me)
* Update unit tests to test for list of dicts in suboptions
* Add integration tests
* Validate parameters in dict and list
In case it comes in as a string
* Make changes based on feedback, fix tests
* Simplify validators since we only need to validate dicts
Add test for suboptions passed in as strings to ensure they get validated properly and turned into a dictionary.
ci_complete
* Add a few more integration tests
(cherry picked from commit e9d29b1fe4)
Co-authored-by: Sam Doran <sdoran@redhat.com>
* isa string should rewrap as unsafe in get_validated_value
* _is_unsafe shouldn't be concerned with underlying types
* Start with passwords as text, instead of bytes
* Remove unused imports
* Add changelog fragment
* Update changelog with CVE
* Stop appending '/api' to galaxy server url (#63238)
* Stop appending '/api' to configured galaxy server url
Since not all galaxy REST api server URLs live
at '/api', stop always appending it to the
'url' value loaded from config.
* Add note about manually migrated galaxy configs and /api
* Add '/api/' to galaxy url and guessing if galaxy API
* Fix most unit tests (update to expect /api/)
* Fix test_initialise_unknown unit test
Since we retry now with an added /api/, mock it as well.
* Update fallback default avail_ver to new format
(cherry picked from commit bad72693e4)
* Add changelog fragment galaxy_api_config
* Fix galaxy url use everywhere when url is set in config. (#63286)
In addition to trying the configured url (for ex, a migrated
'https://galaxy.ansible.com/') there is an attempt at that
URL with '/api' postpended.
If the extended URL works, update GalaxyAPI.api_server to
use the extended URL. Previously it only used it for finding
the API root info ('available_versions', etc)
(cherry picked from commit ed203c5902)
Jordan Borean