Commit Graph

5800 Commits (9b77492e96106b616f2c23e147357b323b97b265)

Author SHA1 Message Date
Felix Fontein 9b77492e96
blockinfile: do not crash when filename has no path (#81638) (#82869)
* Do not crash when filename has no path.

* Clean up file after test.

(cherry picked from commit e659c23bf2)
8 months ago
Jordan Borean daf71c51e0
winrm - Handle task timeout (#82784) (#82864)
When using winrm over HTTP with message encryption enabled and a task
has timed out the connection plugin will fail to cleanup the WinRM
command. This will change that exception into a warning as a timeout is
already an exception event and a failure to clean the operation should
not override the timeout error shown.

(cherry picked from commit 8aecd1f9b2)
8 months ago
Sloane Hertel 85697beee0
fix handling allow_duplicates with the role cache (#82691) (#82927)
allow_duplicates is not part of the role uniqueness, so the value on the cached role may not match the current role.

* remove the allow_duplicates check from Role.has_run() which operates on the deduplicated role
* check the current role's allow_duplicates value in the strategy

(cherry picked from commit b3d8cdde5d)

Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
8 months ago
Sloane Hertel 6ac75a20d0
Fix traceback inheriting from NetworkConnectionBase and add integration tests (#82954) (#82956)
(cherry picked from commit 4bddbe69d5)

Co-authored-by: Jeroen van Bemmel <jvb127@gmail.com>
8 months ago
Matt Martz bdf45d1fbc
[stable-2.16] dnf: obey the keepcache setting (#82735) (#82964)
Fixes #81954
(cherry picked from commit 77ab7af)

Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
8 months ago
Jordan Borean 5a3bcd4bac
fetch - add error check on calculated dest (#82970) (#82990)
Add explicit error when the calculated dest path for fetch becomes a
local directory. The existing behaviour will not be checked unlike when
the path did not end with a trailing slash.

(cherry picked from commit 179bc1dabd)
8 months ago
Jordan Borean 00d6f00a43
Re-enable psrp tests that were disabled (#82785) (#82788)
(cherry picked from commit bb030db546)
9 months ago
Martin Krizek 627a92e34a
Fix error when templating an unsafe string leading to a type error in Python (#82675) (#82745)
Fixes #82600

(cherry picked from commit 79ea21a39f)

Co-authored-by: Davide Sbetti <davide.sbetti@gmail.com>
9 months ago
Matt Martz eb73cc488b
[stable-2.16] Install crun from Alpine 3.19 for known musl compatibility (#82812) (#82814)
(cherry picked from commit 86f48a5)
9 months ago
Martin Krizek 56f31126ad
Prevent failures due to unsafe plugin name (#82759) 9 months ago
Matt Martz 6ad778c6b5
[stable-2.16] Disable ansible-test podman container tests on Ubuntu 22.04 (#82748) (#82750)
(cherry picked from commit 9a8be1e)
9 months ago
Mark Goddard 23d055e835
Fix issues with ansible-playbook-callbacks test (#82407) (#82630)
The timing of the async tasks was a little unpredictable, meaning that
sometimes we would get an unexpected number of v2_runner_on_async_poll
callbacks, and fail the test. This change fixes the issue by increasing
the poll interval to 2 seconds and the sleep duration to 3 seconds, such
that on a reasonably responsive system we will poll twice per task, with
the sleep ending in the middle of the two polls.

The include_me.yml file does not exist in this integration test. It has
been added.

The remote_tmp_dir.path expression is invalid - the setup_remote_tmp_dir
role uses set_fact to set remote_tmp_dir to remote_tmp_dir.path.

The integration tests run with ANSIBLE_HOST_PATTERN_MISMATCH=error,
meaning that the final play was never reached. Set
ANSIBLE_HOST_PATTERN_MISMATCH=warning to continue past the play and
trigger the v2_playbook_on_no_hosts_matched callback.

(cherry picked from commit 4a2de764ec)
10 months ago
Sloane Hertel 74f99dc36c
Fix ansible.builtin.include_vars - depth (#80995) (#82610)
* Changes as suggested by sivel

* Add changelog fragment and tests

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit 48bed1e15a)

Co-authored-by: tachyontec <92679798+tachyontec@users.noreply.github.com>
10 months ago
Martin Krizek c93234304b
Do not ignore SyntaxError from jinja2.Environment.from_string (#82607) (#82614)
Jinja may generate an invalid Python source code from a template. Trying
to compile such source code into a Python code object results in
SyntaxError being thrown. An example of such a template is providing the
same keyword argument into a lookup twice, resulting in:
`SyntaxError: keyword argument repeated`.

Since `jinja2.exceptions.TemplateSyntaxError` does not cover such a
case, as it is not a Jinja parsing error, we need to catch SyntaxError
explicitly ourselves.

Fixes #82606

(cherry picked from commit 6d34eb88d9)
10 months ago
Sloane Hertel add76f36f5
fix loading vars_plugins in roles (#82273) (#82609)
* Fix loading legacy vars plugins when the plugin loader cache is reset

* Remove extra cache layer by ensuring vars plugin names are cached (stateless or not) so that the plugin loader cache can double as the load order

(cherry picked from commit 13e6d8487a)
10 months ago
Sloane Hertel 9a07ab72b4
[2.16] expect - fix argument spec error with timeout=null (#82522) (#82608)
* expect - fix argument spec error with timeout=null (#82522)

* Fix using timeout=null to wait indefinitely

* fix error message

(cherry picked from commit da9edd7760)

* python2-ify
10 months ago
Nilashish Chakraborty d817f5e87f
Support `action_plugin` in plugin_routing_schema (#82562) (#82581)
now validation schema matches reality

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit b01f1f207c)
11 months ago
Brian Coca 611d0e4dcf Better errors for delegate_to (#82319)
Handle empty result of templating
  Also skip work when we omit

(cherry picked from commit 6ebefaceb6)
11 months ago
Brian Coca 50736c45ba lookups, make file searching use better is_role (#82290)
* lookups, make file searching use better is_role

 The dwim function will internally try by detecting tasks/main['','.yml','.yaml]
 but this is far from optimial, the existince of role path in vars is much better
 indicator that we can use to pass a hint

* updated test to avoid main.yml

(cherry picked from commit a9919dd7f6)
11 months ago
snipfoo c3b4b3ebe3 Run all handlers with the same `listen` topic when notified from another handler (#82364)
Fixes #82363

(cherry picked from commit 8328153121)
11 months ago
Brian Coca cfa8caff39
[stable-2.16] Role fixes (#82339) (#82452)
* Role fixes (#82339)

* Various fixes to roles

  - static property is now properly set
  - role_names and other magic vars now have full list
  - role public/private var loading is now done when adding to play.roles instead of on each var query
  - added tests

Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 55065c0042)

* import_role does not get public until next version
11 months ago
Brian Coca 46d9d4b17c ansible-config dedupe ini plugin entries (#82498)
added test for ini file integrity, also ensuring no dupes

(cherry picked from commit 6c2895fd88)
11 months ago
Matt Martz b9a03bbf5a
[stable-2.16] Ensure ANSIBLE_NO_LOG is respected (CVE-2024-0690) (#82565) (#82566)
(cherry picked from commit 6935c8e)
11 months ago
Sloane Hertel a25fe10056
Targeted fix for installing roles with symlinks containing '..' (#82165) (#82323)
Set the tarfile attribute to a normalized value from unfrackpath instead
of validating path parts and omiting potentially invald parts

Allow tarfile paths/links containing '..', '$', '~' as long as the
normalized realpath is in the tarfile's role directory

(cherry picked from commit 3a42a00368)
11 months ago
Matt Martz afe3fc184f
Additional Unsafe fixes (#82376)
* Allow older pickle protocols to pickle unsafe classes. Fixes #82356

* Address issues when iterating or getting single index from AnsibleUnsafeBytes. Fixes #82375

* clog frag
1 year ago
Sviatoslav Sydorenko d949af5093
[backport][stable-2.16] 🧪 Replace GitHub SVN integration test with local TLS (#82368)
* [backport][stable-2.16] 🧪 Replace GitHub SVN integration test with local TLS

PR #82334

* Run svn integration test locally with TLS

This patch uses a `trustme` to make an ephemeral CA, and server, and
client TLS artifacts for testing. These are integrated into the Apache
web server via it's `mod_ssl`.

Resolves #82207

* Replace GitHub SVN HTTPS URL w/ localhost over TLS

This change gets rid of the need to use GitHub, which is just about to
drop support for SVN [[1]]. Moreover, it eliminates the need to use
external network for any SVN commands in the test.

[1]: https://github.blog/2023-01-20-sunsetting-subversion-support/

(cherry picked from commit dd0138ba21)

* Implement compat w/ opensuse15, centos7 & rhel7/8
1 year ago
Matt Martz 270b39f6ff
Ensure that unsafe is more difficult to lose [stable-2.16] (#82293)
* Ensure that unsafe is more difficult to lose

* Add Task.untemplated_args, and switch assert over to use it
* Don't use re in first_found, switch to using native string methods
* If nested templating results in unsafe, just error, don't continue

* ci_complete
1 year ago
Martin Krizek f302b2f592
Allow include_tasks handlers for searching role subdirs (#82248) (#82268)
Fixes #82241

(cherry picked from commit d664f13b4a)
1 year ago
Brian Coca bb787c119d
ansible-pull: expand destinantion directoy to avoid purgin in / (#82030) (#82221)
* expand destinantion directoy to avoid purgin in /

  bad things could happen and help alone is not enough

(cherry picked from commit 8825e60add)
1 year ago
Brian Coca 8a87e1c5d3
no_log avoid masking booleans (#82217) (#82235)
* no_log avoid masking booleans (#82217)

* no_log avoid masking booleans

* clog

* fix issues

(cherry picked from commit 6e448edc63)

* unused boil is hot
1 year ago
Brian Coca fb5d254a79
wait_for, fallback to read for non mmapable files (#82064) (#82233)
* wait_for, fallback to read for non mmapable files (#82064)

* also handle oserror, added debug jic

(cherry picked from commit 8b102dca4a)

* skip problem versions
1 year ago
Martin Krizek 2f7376ce06
flush_handlers: handle a failure in a nested block with force_handlers (#81572) (#82197)
Fixes #81532

ci_complete

(cherry picked from commit a8b6ef7e7c)
1 year ago
Brian Coca 71b00cefac
restore role param precedence (#82106) (#82138)
* add test for setfact/param override

(cherry picked from commit 20a54eb236)
1 year ago
Brian Coca e13569d0e2
ansible-pull now handles all secret files CLI options (#82009) (#82070)
* ansible-pull added missing pasthrough for secrets

 Both become and connection password file options were missing.
 Also added test

(cherry picked from commit 99e0d25857)
1 year ago
Brian Coca 6cfa8ec021
Fix Jinja plugin deduplication (#82002) (#82053)
for j2 plugins dedupe on path and  not basename
for j2 this is a container file , for other plugins file name == plugin name

(cherry picked from commit b4566c18b3)
1 year ago
Abhijeet Kasurde 859f2876f0
[bp-2.16] Update Python3 (#82203)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

(cherry picked from commit c5b68ef16d)
1 year ago
Matt Davis a13df22c02
remove args passthru on role runme tests that use grep (#82060) (#82061)
* varying verbosity was masking some first-pass test failures

(cherry picked from commit 40263992df)
1 year ago
Matt Clay 00f74e03b2 [stable-2.16] Fix subversion integration test (#82029)
- Remove dependency on the htpasswd module (and thus passlib)
- Fix setup/teardown of the httpd process
- Fix cleanup of temporary directories.
(cherry picked from commit 09d943445c)

Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Martin Krizek 9c91e578d0
[stable-2.16] run_once: unnotify hosts on handlers that are not run (#81667) (#81920)
Fixes #81666
(cherry picked from commit 2d5861c)
1 year ago
Martin Krizek f3f0e6a0f8
[stable-2.16] Properly template tags in parent blocks (#81624) (#81921)
When templating tags (which happens outside of standard `post_validate`) we
need to template each object in the inheritance chain and set the templated
values on those objects individually. That way when `task.tags` is called the
`extend` functionality properly picks up the templated values of all
parents into one flatten list.

Fixes #81053
(cherry picked from commit 9b3ed5e)
1 year ago
Sloane Hertel 889248bcf7
optimize host_group_vars and vars plugin loading (#79945) (#81878)
* Improve host_group_vars efficiency:

* normalize the basedir with `os.path.realpath()` once and cache it
* cache missing paths/files
* reduce the calls to `isinstance`

Add a couple more general improvements in vars/plugins.py get_vars_from_path():

* call `PluginLoader.all()` once for vars plugins and reload specific
  plugins subsequently
* don't reload legacy/builtin vars plugins that are not enabled

Add a test for host_group_vars and legacy plugin loading

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

* Add a new is_stateless attribute to the vars plugin baseclass

update integration tests to be quieter and use the same test pattern

Fix deprecation and adjust test that didn't catch the issue (deprecation only occured when the value was False)

move realpath cache to host_group_vars (do not smuggle call state as instance data)

refactor under a single 'if cache:' statement

Call os.path.isdir instead of always calling os.path.exists first. Just call os.path.exists to differentiate between missing and non-directory.

remove call to super(VarsModule, self).get_vars()

use the entity name as the cache key instead of variable location

Remove isinstance checks and use a class attribute just in case any plugins are subclassing Host/Group

Replace startswith by checking index 0 of the name instead, since host/group names are required

* rename is_stateless to cache_instance to make it more clear what it does

* add plugin instance cache using the path to plugin loader

reduce loading stage option if a new instance isn't created

don't require a known subdir on PluginLoader instantiation for backwards
compatibility

rename attribute again

contain reading from/initializing cached instances to a plugin loader method

* Deprecate v2 vars plugins

* Refactor to use the cache in existing plugin loader methods

Rename the attribute again

Refactor host_group_vars with requested changes

Make changelog a bugfixes fragment

Add a deprecation fragment for v2 vars plugins.

Add type hints

* unbreak group_vars

* Apply suggestions from code review

* misc tweaks

* always cache instance by both requested and resolved FQ name
* add lru_cache to stage calculation to avoid repeated config consultation

* handle KeyError from missing stage option

---------

Co-authored-by: Matt Davis <mrd@redhat.com>
(cherry picked from commit debf2be913)
1 year ago
Jordan Borean 5665eca5e9
win_fetch - improve test time by not scanning Win dir (#81884) (#81888)
(cherry picked from commit dfc62589f6)
1 year ago
Brian Coca 9a4bc7e7b3
Restore import_role variable exporting behavior (#81840)
* Import role public (#81772)

revert to previous behavior to push vars to play at compile time
add `public` parameter to allow per import control of exporting (vs just the global config)

Co-authored-by: tchernomax <maxime.deroucy@gmail.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit ab6a544e86)

* adapted to prev version

 - removed new functionality
 - restored global config functioning overriding specific public option

* remove typoe

* quote it
1 year ago
Abhijeet Kasurde a8e59b2547
[backport/2.16] Bump Pylint sanity test requirements for 3.12 (#81850)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Martz 5008c56c50
[stable-2.16] Install upgraded crun from edge (#81833) (#81839)
(cherry picked from commit e78cc1e)
1 year ago
Matt Martz 831dc6e444
Add compat function for parsing Content-Disposition header (#81807)
* py2 compat for get_param

* Add tests, and handle ValueError

* Add clog frag
1 year ago
Matt Martz fffb3c403f
[stable-2.16] Prevent roles from using symlinks to overwrite files outside of the installation directory (#81780) (#81783)
* Sanitize linkname during role installs

* Add tests

* add clog frag
(cherry picked from commit ddf0311)
1 year ago
Martin Krizek 9033002a86
Allow for searching handler subdir for included task via include_role (#81733) (#81759)
Fixes #81722

(cherry picked from commit 1e7f7875c6)
1 year ago
Sviatoslav Sydorenko e470954985
Always allow "no-other-choice" pre-release dependencies when resolving collection dependency tree (#81746)
PR #81606.

Prior to this patch, when `--pre` CLI flag was not passed, the
dependency resolver would treat concrete collection dependency
candidates (Git repositories, subdirs, tarball URLs, or local dirs or
files etc) as not meeting the requirements.

This patch makes it so pre-releases in any concrete artifact
references, and the ones being specifically pinned dependencies or
user requests, met anywhere in the dependency tree, are allowed
unconditionally.

This is achieved by moving the pre-release check from
`is_satisfied_by()` to the `find_matches()` hook, following the
Pip's example.

As a bonus, this change also fixes the situation when a collection
pre-releases weren't considered if it didn't have any stable releases.
This now works even if `--pre` wasn't requested explicitly.

Finally, this patch partially reverts commit
6f4b4c345b, except for the tests. And it
also improves the `--pre` hint warning to explain that it mostly
affects Galaxy/Automation Hub-hosted collection releases.

Ref #73416
Ref #79112
Fixes #79168
Fixes #80048
Resolves #81605

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit 7662a05085)
1 year ago
Matt Clay e3eecf096f
[stable-2.16] ansible-test - Skip pylint test on Python 3.12 (#81706) (#81757)
(cherry picked from commit 3794612832)
1 year ago