Commit Graph

6967 Commits (2a53b851fee8ebaa07c1341122dd905354659237)

Author SHA1 Message Date
Abhijeet Kasurde 7a0c321054
Remove redundant undefined error message (#81867)
Fixes: #78703

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
10 months ago
Brian Coca bec27fb4c0
prettyfy ansible-doc (#75116)
* prettify ansibile-doc output

delimiters when no color
avoid triggering color mode for existing tests
all use _format to observe nocolor
more v more info
imporoved conditional display
updated version on -v
normalize role errors
expand role data, dedupe code, fix formatting on warning
fix bug with galaxy info
role list improvements:
 lists all roles (even w/o meta/argspec)
 specs now indented under role
 role only listed once
Updated tests to reflect format changes

Co-authored-by: Felix Fontein <felix@fontein.de>
10 months ago
Martin Krizek 6d34eb88d9
Do not ignore SyntaxError from jinja2.Environment.from_string (#82607)
Jinja may generate an invalid Python source code from a template. Trying
to compile such source code into a Python code object results in
SyntaxError being thrown. An example of such a template is providing the
same keyword argument into a lookup twice, resulting in:
`SyntaxError: keyword argument repeated`.

Since `jinja2.exceptions.TemplateSyntaxError` does not cover such a
case, as it is not a Jinja parsing error, we need to catch SyntaxError
explicitly ourselves.

Fixes #82606
10 months ago
Sloane Hertel 13e6d8487a
fix loading vars_plugins in roles (#82273)
* Fix loading legacy vars plugins when the plugin loader cache is reset

* Remove extra cache layer by ensuring vars plugin names are cached (stateless or not) so that the plugin loader cache can double as the load order
10 months ago
Brian Coca 8b2dd5fdd3
updated looups to use set_option in custom parse (#82425)
limit password params
  updated test error message catching
  make sure we reset params for each term
  ensure we only update options if we have em
10 months ago
Abhijeet Kasurde f73d72e830
find: do not fail on PermissionError (#82282)
* Log and skip permission errors on files and directories

Fixes: #82027

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
10 months ago
Martin Krizek f024cf35d7
Remove the yum module, redirect it to dnf (#81895)
Fixes #81728
10 months ago
Yacht Shaver c7334ea92c
Fix url encoded credentials in netloc (#82552)
Prior to this commit, it was impossible to use a module like dnf with a
URL that contains a username with an @ such as an email address
username, because:

  dnf:
    name: https://foo@example.com:bar@example.com/some.rpm

Would cause netloc parsing to fail. However, the following:

  dnf:
    name: https://foo%40example.com:bar@example.com/some.rpm

Would also fail because ansible would *not* URL-decode the credentials,
causing the following to be base64 encoded in the Authorization header:

  Zm9vJTQwZXhhbXBsZS5jb206YmFyCg==

Which decodes to:

  foo%40example.com:foo

Which is *not* the authorized username, and as such, *won't* pass basic
auth.

With this commit, Ansible's url lib behaves like curl, chromium, wget,
etc, and encodes the above to:

  Zm9vQGV4YW1wbGUuY29tOmJhcgo=

Which decodes to:

  foo@example.com:bar

Which will actually pass the HTTP Basic Auth, and is the same behaviour
that you will find ie. with:

  curl -vvI https://foo%40bar:test@example.com 2>&1 |grep Auth | awk '{ print $4 }'
10 months ago
Martin Krizek 06cd285901
Play recap stats and callbacks for include_role, consolidate with include_tasks (#79260)
This moves handling of callbacks and play recap stats from
_load_included_file to individual strategies so include_role tasks are
accounted for, not just include_tasks.

Fixes #77336
10 months ago
Mike Bonnet 9c5d3060e5
uri: Handle the "force" parameter properly (#82187)
* uri: Two tests that demonstrate missing handling of the "force" parameter

Add unit and integration tests that demonstrate that the uri module is not
handling the "force" parameter.

The unit test demonstrates that when "force" is present in the module parameters,
it is not being passed through to fetch_url().

The integration test demonstrates that "force" does not disable caching as
documented, and calls with a "dest" parameter that points to an existing file
can result in a "304 Not Modified" response.

* uri: Handle the "force" parameter properly

The uri module documents a "force" parameter that can be used to disable caching.
The module accepted the parameter but didn't pass it through to the fetch_url() method
which implements the logic to handle setting the appropriate headers for disabling
caching. This change passes the "force" parameter through as expected, allowing caching
to be disabled when requested by the module caller.
10 months ago
ShIRannx aa40167f40
find - add encoding option to use in conjunction with contains (#82284)
*  when doing a 'contains' search, determine the encoding of the files to be searched

* set default encoding to None for backwards compatibility

* changelog, error handling, tests added

* add sanity ignore.txt for non-utf-8 test
10 months ago
Nilashish Chakraborty b01f1f207c
Support `action_plugin` in plugin_routing_schema (#82562)
now validation schema matches reality

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
10 months ago
Matt Martz 6935c8e303
Ensure ANSIBLE_NO_LOG is respected (CVE-2024-0690) (#82565) 10 months ago
tachyontec 48bed1e15a
Fix ansible.builtin.include_vars - depth (#80995)
* Changes as suggested by sivel

* Add changelog fragment and tests

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
10 months ago
Martin Krizek 3f74bc08ce
Remove crypt support from ansible.utils.encrypt (#81721)
Fixes #81717

Co-authored-by: Matt Clay <matt@mystile.com>
10 months ago
Matt Clay 30169c76de Replace Unicode no-break spaces with ASCII spaces 10 months ago
Matt Davis 4aa109897f
default svn URL to 127.0.0.1 for subversion integration tests (#82551)
* default svn URL to 127.0.0.1 for subversion integration tests

* svn client gives up before trying IPv4 addresses when localhost->(::1,127.0.0.1) in dual-stack envs (eg podman, most real hosts)
* svn client also requires legacy CN match on cert
* IPv6 works, but setup playbook would need a bunch more templating exceptions to conditionally manage `[::1]`

* explain IPv4 defaults
10 months ago
Mark Goddard 4a2de764ec
Fix issues with ansible-playbook-callbacks test (#82407)
The timing of the async tasks was a little unpredictable, meaning that
sometimes we would get an unexpected number of v2_runner_on_async_poll
callbacks, and fail the test. This change fixes the issue by increasing
the poll interval to 2 seconds and the sleep duration to 3 seconds, such
that on a reasonably responsive system we will poll twice per task, with
the sleep ending in the middle of the two polls.

The include_me.yml file does not exist in this integration test. It has
been added.

The remote_tmp_dir.path expression is invalid - the setup_remote_tmp_dir
role uses set_fact to set remote_tmp_dir to remote_tmp_dir.path.

The integration tests run with ANSIBLE_HOST_PATTERN_MISMATCH=error,
meaning that the final play was never reached. Set
ANSIBLE_HOST_PATTERN_MISMATCH=warning to continue past the play and
trigger the v2_playbook_on_no_hosts_matched callback.
10 months ago
Matt Clay 5f1b3898e8
Avoid delattr in test callback plugin (#82550)
This prevents the test plugin from tampering with the base callback plugin,
which causes issues with all other callback plugins running in the test.
10 months ago
Sloane Hertel da9edd7760
expect - fix argument spec error with timeout=null (#82522)
* Fix using timeout=null to wait indefinitely

* fix error message
11 months ago
Matt Clay 78c9090986 Don't hard-code Alpine version in container test 11 months ago
Brian Coca 596c75c2bc
uri action plugin check_mode support update (#82484)
Also updated base action messages to be more specific
11 months ago
Brian Coca 6c2895fd88
ansible-config dedupe ini plugin entries (#82498)
added test for ini file integrity, also ensuring no dupes
11 months ago
Abhijeet Kasurde 53ddc1648e
deb822_repository: handle idempotency (#82483)
* deb822_repository: handle idempotency

Sort the parameters in order to handle idempotency

Fixes: #82454

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
11 months ago
Felix Fontein 6c647aa263
Interpret double newlines as paragraph breaks in documentation strings. (#82465) 11 months ago
Sloane Hertel caa86cc4df
Add the task info for tombstoned module/action plugins (#82451)
* Add the task info for tombstoned plugins

* Fix deprecation for 'include' by removing it from BUILTIN_TASKS which skip the plugin loader lookup

* changelog

remove obsolete unit test using 'include'

* Update changelogs/fragments/improve-tombstone-error.yml
11 months ago
Sloane Hertel b5649f9686
Remove more Python 2 handling from tests (#82449) 11 months ago
Matt Clay a1958eefed Fix plugin_loader integration test error handling 11 months ago
Matt Clay 630e5dda25 Clean up shellcheck issues in tests 11 months ago
Brian Coca 55065c0042
Role fixes (#82339)
* Various fixes to roles

  - static property is now properly set
  - role_names and other magic vars now have full list
  - role public/private var loading is now done when adding to play.roles instead of on each var query
  - added tests

Co-authored-by: Felix Fontein <felix@fontein.de>
11 months ago
Abhijeet Kasurde fa92228b50
Replace unwelcoming words (#82413)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
11 months ago
Brian Coca 4479c9df13
Fix plugin set option (#82300)
* plugin config, ensure set_option correctly casts

  until now we relied on plugin author getting the right type, now
  the config system itself will process as it would for set_options
11 months ago
Mark Chappell 5346009d2c
ansible-test - Allow multiple documents in the YAML stream for EXAMPLES (#82355)
* Allow multiple documents in the YAML stream for EXAMPLES.

* Add integration test

* linting
12 months ago
snipfoo 8328153121
Run all handlers with the same `listen` topic when notified from another handler (#82364)
Fixes #82363
12 months ago
Abhijeet Kasurde 7f2ad7eea6
assemble: fixed missing parameter error (#82360)
* content is an optional parameter for _get_diff_data API

Fixes: #82359

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
12 months ago
Brian Coca a9919dd7f6
lookups, make file searching use better is_role (#82290)
* lookups, make file searching use better is_role

 The dwim function will internally try by detecting tasks/main['','.yml','.yaml]
 but this is far from optimial, the existince of role path in vars is much better
 indicator that we can use to pass a hint


* updated test to avoid main.yml
12 months ago
Danny Luwel 9b002d2e63
check if there attributes to set (#78707)
Fixes: #76727
12 months ago
Sviatoslav Sydorenko dd0138ba21
🧪 Replace GitHub SVN integration test with local TLS (#82334)
* Run svn integration test locally with TLS

This patch uses a `trustme` to make an ephemeral CA, and server, and
client TLS artifacts for testing. These are integrated into the Apache
web server via it's `mod_ssl`.

Resolves #82207

* Replace GitHub SVN HTTPS URL w/ localhost over TLS

This change gets rid of the need to use GitHub, which is just about to
drop support for SVN [[1]]. Moreover, it eliminates the need to use
external network for any SVN commands in the test.

[1]: https://github.blog/2023-01-20-sunsetting-subversion-support/
12 months ago
Brian Coca 6ebefaceb6
Better errors for delegate_to (#82319)
Handle empty result of templating
  Also skip work when we omit
12 months ago
Sloane Hertel 3a42a00368
Targeted fix for installing roles with symlinks containing '..' (#82165)
Set the tarfile attribute to a normalized value from unfrackpath instead
of validating path parts and omiting potentially invald parts

Allow tarfile paths/links containing '..', '$', '~' as long as the
normalized realpath is in the tarfile's role directory
12 months ago
Brian Coca 1dd0d6fad7
remote log info (#81711)
Both posix and powershell avenues updated to allow for customized string logging on remote

Co-authored-by: Jordan Borean <jborean93@gmail.com>
12 months ago
Abhijeet Kasurde 265f5e724c
apt_repository: Do not convert symlink repo to a normal file (#82271)
* apt_repository: Do not convert symlink repo to a normal file

* module manually writes to repo files. If the file is a symlink
  module used to re-write and convert a symlink to a normal file.
  This fix tracks the symlink and keeps the changes intact.

Fixes: #49809

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Abhijeet Kasurde e6e19e37f7
validate-modules: Check for correct values in choice (#82266)
* validate-modules: Check for correct values in choice

* validate-modules test now checks the length of choices and
  correct option values inside the choice.

Fixes: #82179

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

* Fix sanity tests

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

---------

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Sebbo94BY e0bf76e3db
Issue #80267: Remove user not found warning (#80291)
Co-authored-by: Sebi94nbg <sebastian.kraetzig@4g-server.eu>
1 year ago
Martin Krizek d664f13b4a
Allow include_tasks handlers for searching role subdirs (#82248)
Fixes #82241
1 year ago
Chris Francy a4b00793be
Update known_hosts module to better handle @cert-authority keys (#70340)
Signed-off-by: Chris Francy <zoredache@gmail.com>
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Abhijeet Kasurde ce9d268ab8
unarchive: force unarchive if the symlink target changes (#82253)
Fixes: #30420

Original PR from nrwahl2 (https://github.com/ansible/ansible/pull/30463)

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Herman van Rink d6a8df3218
csvfile - add a keycol parameter to specify in which column to search. (#82242) 1 year ago
Martin Krizek c35e0e8fd8
Re-enable "group removal" integration test for dnf5 (#82250)
Fixed in https://github.com/rpm-software-management/dnf5/pull/1013
1 year ago
Abhijeet Kasurde 1a759659fe
Enable vendored copy of distutils in entry_point test (#82232)
* Enable use of vendored copy of distutils by removing
  the environment variable SETUPTOOLS_USE_DISTUTILS

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Brian Coca 8b102dca4a
wait_for, fallback to read for non mmapable files (#82064)
* also handle oserror, added debug jic
1 year ago
Brian Coca f6d7dd0840
displaly. log, handle when caplevel is already taken care of (#82227)
reneables ansible logging
1 year ago
Abhijeet Kasurde f8cdec6324
ansible-test: Downgrade containerd for Fedora 39 (#82231)
* containerd 1.6.23 on Fedora 39 causes issue. Downgrading
  to containered to 1.6.19-2 works.
* See: https://bugzilla.redhat.com/show_bug.cgi?id=2237396

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Abhijeet Kasurde a45dc10b8f
ansible-test: Skip modularity test for Fedora 39 and up (#82230)
* Modularity is deprecated in Fedora 39 and onwards,
  Skip modularity tests

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Brian Coca 6e448edc63
no_log avoid masking booleans (#82217)
* no_log avoid masking booleans

* clog

* fix issues
1 year ago
Matt Martz f42984eeb3
Fix ManifestControl dataclass to work with future annotations (#82215)
* Fix ManifestControl dataclass to work with future annotations

* Add tests
1 year ago
Brian Coca 8825e60add
ansible-pull: expand destinantion directoy to avoid purgin in / (#82030)
* expand destinantion directoy to avoid purgin in /

  bad things could happen and help alone is not enough
1 year ago
Brian Coca 4a84a9b3db
copy keep suffix on temp file for validation (#82158)
Also updated test to require suffix

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
1 year ago
Martin Krizek a8b6ef7e7c
flush_handlers: handle a failure in a nested block with force_handlers (#81572)
Fixes #81532

ci_complete
1 year ago
Dellucifer f2b5c33349
fixed typos in the documentation (#82077) 1 year ago
Matt Clay fd009a073a Remove sefcontext support plugin
It was only used by the subversion test, but is no longer required.
1 year ago
Abhijeet Kasurde c5b68ef16d Update Python3
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Martin Krizek 7538668731
Check strict before failing to respect skip_broken (#80795)
Fixes #80590
1 year ago
Brian Coca 20a54eb236
restore role param precedence (#82106)
* add test for setfact/param override
1 year ago
Matt Clay 89cda0bcae
pip - Add break_system_packages option (#82097) 1 year ago
Jordan Borean f5a0c0dfc8
Ignore testing data for gitleaks (#82083) 1 year ago
Thomas Sjögren fb8ede22e1
don't warn about using a yescrypt hash as user password (#82071)
* dont warn about using a yescrypt hash as password

Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>

* add changelog

Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>

* add yescrypt test

Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>

---------

Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
1 year ago
Martin Krizek fe94a99aa2
any_errors_fatal fixes (#78680)
Fixes #31543
Fixes #36308
Fixes #73246
Fixes #80981
Fixes #81533

ci_complete
1 year ago
Matt Clay c827dc0dab Remove obsolete requirements from prepare_http_tests 1 year ago
Brian Coca 99e0d25857
ansible-pull now handles all secret files CLI options (#82009)
* ansible-pull added missing pasthrough for secrets

 Both become and connection password file options were missing.
 Also added test
1 year ago
Matt Clay 0b7387d46c Improve filter_encryption test
- Update `data2_vaulted_string_with_id` to match the documented plaintext.
- Add a comment explaining how `data2_vaulted_string_with_id` was derived.
- Add assertions for unvaulted values to ensure they match their plaintext.
- Add round-trip tests for vault+unvault when no salt is used.
1 year ago
Matt Clay 2d2211e4e1
Simplify filter_encryption test (#82062) 1 year ago
Matt Davis 40263992df
remove args passthru on role runme tests that use grep (#82060)
* varying verbosity was masking some first-pass test failures
1 year ago
Brian Coca b4566c18b3
Fix Jinja plugin deduplication (#82002)
for j2 plugins dedupe on path and  not basename
for j2 this is a container file , for other plugins file name == plugin name
1 year ago
Matt Clay 09d943445c
Fix subversion integration test (#82029)
- Remove dependency on the htpasswd module (and thus passlib)
- Fix setup/teardown of the httpd process
- Fix cleanup of temporary directories
1 year ago
Jordan Borean 22568305d6
Add ignore invalid options override for mod wrapper (#81899)
Adds an option that can have an action plugin tell the module to ignore
options that do not fit its arg spec. This is to enable support for core
running modules that exist outside of the collection that may not be new
enough to support some of the options supplied to it.
1 year ago
Matt Clay 4f04df75cc
Fix outdated S3 URLs (#81964)
We should always use the CloudFront backed endpoint (ci-files) not the direct S3 bucket reference (ansible-ci-files).
1 year ago
Matt Clay 3f607359de
Remove obsolete compat code from integration tests (#81961)
Also fix up boilerplate for non .py extension files that contain Python code.
1 year ago
Matt Martz 92d2c66db2
Remove py2 support from urls.py (#81880) 1 year ago
Matt Clay 9f899f9492
Require `from __future__ import annotations` (#81902) 1 year ago
Jordan Borean dfc62589f6
win_fetch - improve test time by not scanning Win dir (#81884) 1 year ago
Sloane Hertel debf2be913
optimize host_group_vars and vars plugin loading (#79945)
* Improve host_group_vars efficiency:

* normalize the basedir with `os.path.realpath()` once and cache it
* cache missing paths/files
* reduce the calls to `isinstance`

Add a couple more general improvements in vars/plugins.py get_vars_from_path():

* call `PluginLoader.all()` once for vars plugins and reload specific
  plugins subsequently
* don't reload legacy/builtin vars plugins that are not enabled

Add a test for host_group_vars and legacy plugin loading

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

* Add a new is_stateless attribute to the vars plugin baseclass

update integration tests to be quieter and use the same test pattern

Fix deprecation and adjust test that didn't catch the issue (deprecation only occured when the value was False)

move realpath cache to host_group_vars (do not smuggle call state as instance data)

refactor under a single 'if cache:' statement

Call os.path.isdir instead of always calling os.path.exists first. Just call os.path.exists to differentiate between missing and non-directory.

remove call to super(VarsModule, self).get_vars()

use the entity name as the cache key instead of variable location

Remove isinstance checks and use a class attribute just in case any plugins are subclassing Host/Group

Replace startswith by checking index 0 of the name instead, since host/group names are required

* rename is_stateless to cache_instance to make it more clear what it does

* add plugin instance cache using the path to plugin loader

reduce loading stage option if a new instance isn't created

don't require a known subdir on PluginLoader instantiation for backwards
compatibility

rename attribute again

contain reading from/initializing cached instances to a plugin loader method

* Deprecate v2 vars plugins

* Refactor to use the cache in existing plugin loader methods

Rename the attribute again

Refactor host_group_vars with requested changes

Make changelog a bugfixes fragment

Add a deprecation fragment for v2 vars plugins.

Add type hints

* unbreak group_vars

* Apply suggestions from code review

* misc tweaks

* always cache instance by both requested and resolved FQ name
* add lru_cache to stage calculation to avoid repeated config consultation

* handle KeyError from missing stage option

---------

Co-authored-by: Matt Davis <mrd@redhat.com>
1 year ago
Martin Krizek 9b3ed5ec68
Properly template tags in parent blocks (#81624)
When templating tags (which happens outside of standard `post_validate`) we
need to template each object in the inheritance chain and set the templated
values on those objects individually. That way when `task.tags` is called the
`extend` functionality properly picks up the templated values of all
parents into one flatten list.

Fixes #81053
1 year ago
Martin Krizek 2d5861c185
run_once: unnotify hosts on handlers that are not run (#81667)
Fixes #81666
1 year ago
Sloane Hertel b3408ab80b
Fix dnf module crash for non-existent url (#81801)
* ci_complete ci_coverage
1 year ago
Martin Krizek 116766ac3e
Add intentional tests for dnf group remove (#81739)
* Add intentional tests for dnf group remove

* Copy tests from yum
1 year ago
Abhijeet Kasurde d8f791d88c
Bump sanity test requirements for 3.12 (#81846) 1 year ago
Brian Coca ab6a544e86
Import role public (#81772)
revert to previous behavior to push vars to play at compile time
add `public` parameter to allow per import control of exporting (vs just the global config)

Co-authored-by: tchernomax <maxime.deroucy@gmail.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
1 year ago
Matt Martz e78cc1e602
Install upgraded crun from edge (#81833) 1 year ago
Abhijeet Kasurde 4d4c50f856
Provide more info about plugin in deprecation message (#81719)
Provide more information about plugin usage which needs to be
used instead of the deprecated plugin in the deprecation message.

Fixes: #80561

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
lukaslihotzki-f 27bbff7c22
Add count and mandatory_count parameters in regex_replace filter (#81775) 1 year ago
Martin Krizek 01d190f3d2
Add intentional tests for module_utils/yumdnf.py (#81776) 1 year ago
Martin Krizek 60c9660952
Add intentional tests for dnf install rpm when higher version is installed (#81740)
* Add intentional tests for dnf install rpm when higher version is installed

* ci_complete ci_coverage
1 year ago
Matt Martz 68ab02a504
Test heuristic_log_sanitize (#81730)
* Test heuristic_log_sanitize. See #81689

* Add note about what this test is doing

* grammar

Co-authored-by: Matt Clay <matt@mystile.com>

---------

Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Matt Martz ddf0311c63
Prevent roles from using symlinks to overwrite files outside of the installation directory (#81780)
* Sanitize linkname during role installs

* Add tests

* add clog frag
1 year ago
Sloane Hertel 230f956e25
Add lineinfile integration test for removing a line that has already been removed (#81762)
ci_complete ci_coverage
1 year ago
Sloane Hertel 85adaa76f1
Add integration test for copy with force=false when dest exists (#81756)
ci_complete ci_coverage
1 year ago
Matt Clay 3794612832
ansible-test - Skip pylint test on Python 3.12 (#81706) 1 year ago
Martin Krizek 1e7f7875c6
Allow for searching handler subdir for included task via include_role (#81733)
Fixes #81722
1 year ago
Martin Krizek 62c10199d1
Add intentional tests for dnf list (#81738)
ci_complete ci_coverage
1 year ago
Sviatoslav Sydorenko 7662a05085
Always allow "no-other-choice" pre-release dependencies when resolving collection dependency tree
PR #81606.

Prior to this patch, when `--pre` CLI flag was not passed, the
dependency resolver would treat concrete collection dependency
candidates (Git repositories, subdirs, tarball URLs, or local dirs or
files etc) as not meeting the requirements.

This patch makes it so pre-releases in any concrete artifact
references, and the ones being specifically pinned dependencies or
user requests, met anywhere in the dependency tree, are allowed
unconditionally.

This is achieved by moving the pre-release check from
`is_satisfied_by()` to the `find_matches()` hook, following the
Pip's example.

As a bonus, this change also fixes the situation when a collection
pre-releases weren't considered if it didn't have any stable releases.
This now works even if `--pre` wasn't requested explicitly.

Finally, this patch partially reverts commit
6f4b4c345b, except for the tests. And it
also improves the `--pre` hint warning to explain that it mostly
affects Galaxy/Automation Hub-hosted collection releases.

Ref #73416
Ref #79112
Fixes #79168
Fixes #80048
Resolves #81605

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
1 year ago
Abhijeet Kasurde a1a6550daf
inventory_ini: Handle SyntaxWarning in ini parsing (#81707)
* handle SyntaxWarning ini inventory parsing

Fixes: #81328

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Abhijeet Kasurde 6f65397871
galaxy: check if the target for symlink exists (#81586)
* Symlinks in the collection might be pointing to non-existent
  targets. Check and report the failure to the user.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Felix Fontein 7f0baabbe0
blockinfile: avoid crash on Python 3 when creating directory fails (#81662)
* Avoid crash on Python 3.

* Add a test for the crash on Python 3
1 year ago
Abhijeet Kasurde e4468dc944
copy: print correct dest path when content + diff is used (#81678)
* when --diff is used with content parameter, print destination
  path instead of temporary file path.

Fixes: #79749

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Clay 7d3d4572ed
Fix set filters to use set operations (#81639)
* Fix set filters to use set operations

* Fix integration tests

* Update filter documentation
1 year ago
Felix Fontein b1b029c6b5
ansible-doc: allow to filter by more than one collection (#81450)
Make collection filters more flexible for listing collections.
Co-authored-by: Maxwell G <maxwell@gtmx.me>
1 year ago
Sloane Hertel 8034651cd2
Only mark a role as complete once a task in it executes for the target host (#81565)
* If all tasks in the role are skipped or unreachable, the role is not marked as complete for the host.

* Only mark the role as complete if a task in the role succeeds or fails for the host.
1 year ago
Abhijeet Kasurde 6177888cf6
ansible-vault: Check if the destination is writable (#81660)
* Before performing shredding the original file, check if the
  destination file location is writable or not. This will
  prevent corruption of original file.

Fixes: #81455

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Martz 3ec0850df9
Support packaging and importlib.metadata for pip module (#80881) 1 year ago
Matt Davis dd79c49a4d
fix various Jinja plugin caching issues (#79781)
* fix various Jinja plugin caching issues

* consolidate the wrapper plugin cache
* remove redundant cache in J2 filter/test interceptor

* intra-template loader bypass

* fix early exits swallowing some exception detail

* misc comment cleanup
1 year ago
Abhijeet Kasurde 1cc5efa77b
tarfile: Handle deprecation warning for extract and extractall (#81545)
* Python 3.11.4 introduces a new parameter 'filter' in extract and
extractall in tarfile. Handle deprecation warning message emitted
in Python 3.12.
* added probing mechanism in ansible-galaxy code to detect broken
data filter implementation in tarfile.

Fixes: #80832

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
John HU 65a96daaf4
Fix typos in url lookup plugin options (#81587) 1 year ago
Matt Martz 509be19357
Fix nested jinja statements in integration tests (#81591) 1 year ago
Aleksey Tsalolikhin bdaa091b33
Add warnings for illegal file names in role (#81555)
Co-authored-by: Aleksey Tsalolikhin <atsaloli.tech@gmail.com>
1 year ago
Brent Barbachem 4ab5ecbe81
dnf - fix for a package from URI and update_only (#81568)
Fixes #81376
1 year ago
Abhijeet Kasurde da63f32d59
script: add argument validation (#81469)
partially fixes: #81349

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Jordan Borean 7865b198d2
PowerShell - Improve error checking (#80984)
Improves the error checking when running PowerShell modules using
Ansible.ModuleUtils.Legacy. It will only return an rc of 1 if both the
PowerShell module runner signalled an error occurred and those error
records were present in the output. This should reduce some false
positive errors when using the older module style.
1 year ago
Jordan Borean f3976117fb
PowerShell - remove uneeded dotnet code for future compatibility (#81472) 1 year ago
Sam Doran 116948cd14
user - set current expiration correctly when no shadow entry exists (#75194) 1 year ago
Abhijeet Kasurde 2e6d849bdb
apt: ignore fail_on_autoremove and allow_downgrade when using aptitude (#81445)
* apt: ignore fail_on_autoremove and allow_downgrade when using aptitude

* fail_on_autoremove (--no-remove) and allow_downgrade (--allow-downgrades)
  parameters are only valid for apt-get and not for aptitude. Ignore them when
  aptitude is detected and used.

Fixes: #77868
1 year ago
Abhijeet Kasurde 863e2571db
debconf: idempotency for password question (#81484)
* Gather value of password using debconf-get-selections command,
  use this information for idempotency.

Fixes: #47676

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Caesarovich c69c83c962
Fix macos defaults (#79999) 1 year ago
Matt Clay 390e508d27
ansible-test - Always use managed entry points (#81537) 1 year ago
Martin Krizek a48feb4cfc
"retries" without "until": retry until success (#81531)
Fixes #20802
1 year ago
Martin Krizek 98f1627817
include_role: expose vars from parent roles to role's handlers (#81524)
* include_role: expose vars from parent roles to role's handlers

Fixes #80459
1 year ago
Matt Martz ca08261f08
Add ability to filter find on mode (#81485) 1 year ago
Martin Krizek 0cba3b7504
Last handler with same name wins for listen too (#81358)
Fixes #49371
Fixes #81013
1 year ago
Nikita Korolev bd3ffbe109
fix incorrect ansible_managed formatting (#79129)
Ansible breaks configs if file name could be parsed as jinja template
or contains `%` chars.
Especially `%n`[ame%] which is a line break pattern according to [strftime (3)](https://manpages.debian.org/bullseye/manpages-dev/strftime.3.en.html)
1 year ago
Matt Martz 553f51e728
Revert logic to use Popen.communicate (#80874)
* Back out use of communicate, add better comments, add bufsize, and align with subprocess._communicate

* tests

* re-order logic slightly

* more comments

* loopty loop

* yet another comment

* Revert "yet another comment"

This reverts commit 96cd8ada5fa0441b92f2298bdaa6cb40594847d2.

* Revert "loopty loop"

This reverts commit 96ea066f6a7d18902c04a14f18dd79b38e56f5e7.

* ci_complete

* Copy in comment too

* Wording updates

Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>

* Back out bufsize

---------

Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
1 year ago
Matt Davis 1c765a6afc
restore conditional lookup nerfing (#81460)
* a recent optimization lost the unsafe lookup disable behavior when templating conditionals with inline templates that referred to untrusted values
* added regression test to catch this case
1 year ago
Brian Coca f3a15a4a95
filters vault/unvault fix vault_id parameter usage (#81422)
* vault/unvault filters, fixed id field to match documented.

fixes #81420

Co-authored-by: Felix Fontein <felix@fontein.de>
1 year ago
Kristopher Newsome a1569ea4ca
iptables chain creation does not populate with a rule (#80257)
* iptables chain creation does not populate with a rule

fixes #80256

* Add changelog fragment

* Add rules and flush chain during integration tests

* Check chain rule on comment

* Update test/integration/targets/iptables/tasks/chain_management.yml
1 year ago
Abhijeet Kasurde f10d11bcdc
dpkg_selections: Check if package exists before selection operation (#81406)
* dpkg_selections: Check if the package exists before the selection operation

Fixes: #81404

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Sloane Hertel f5431321a2
password_hash - fix bcrypt algorithm when passlib is not installed (#81385) 1 year ago
Sloane Hertel 2ebeb43d6f
ansible-galaxy - improve ignoring multiple signature status codes (#77610)
* ansible-galaxy - improve ignoring multiple signature status codes when using --ignore-signature-status-code

* fix backwards compatibility by adding a new plural option instead, and hide the singular from --help
1 year ago
Matt Martz d1cd06073f
Extend setup_collections timeout to 3 minutes (#81408) 1 year ago
Matt Clay 691c8e8603
Omit pre-built man pages from sdist (#81395)
Since man pages aren't accessible to users after a `pip install`, there's no need to include them in the sdist.
This change makes it trivial to build man pages from source, which makes them much easier to iterate on.
It also simplifies creation and testing of the sdist, since it no longer requires building man pages.

The new `packaging/cli-doc/build.py` script can generate both man pages and RST documentation.
This supports inclusion on the docs site without a dependency on `ansible-core` internals.
Having a single implementation for both simplifies keeping the two formats in sync.
1 year ago
Davide Sbetti d5d1c27bf3
Add option to blockinfile to wrap inserted block with blank lines (#81083)
* Add new blockinfile options 'prepend_newline' and 'append_newline' to wrap inserted block with blank lines
1 year ago
Matt Davis 6a8c51bb9c
fix redis cache setup (#81329)
* use https repo
* increase default download timeout of 10s for slow/lagged connections
1 year ago
Matt Clay c8bc393871
Raise the minimum setuptools version to 66.1.0 (#81341)
* Raise the minimum setuptools version to 66.1.0

This is the first version to support Python 3.12.

While Python 3.10 and 3.11 could use an older version, a consistent minimum is easier to work with and test against.

* Fix PEP 517 integration test
1 year ago
Sloane Hertel efbc00b6e4
Consider all configured collection paths when installing collections (#81243)
* Only install collections which can't be satisfied by a collection in any of the configured paths.

* Improve warning for unexpected collection install path

Fix warning when path is configured, but is a pip-managed path

Normalize the path before validating to fix warning consistency
1 year ago
Sloane Hertel 26d855c87e
update ansible-galaxy testserver to use SSLContext.load_cert_chain (#80961)
* update ansible-galaxy testserver to use SSLContext.load_cert_chain for Python 3.12 compatibility
1 year ago
Matt Clay e964078a83
ansible-test - Pre-build PyYAML wheels (#81300)
This works around Cython failures when attempting to install PyYAML >= 5.4 <= 6.0.
1 year ago
Matt Clay 261a12b8a9
Disable cron integration test on Alpine (#81301)
The tests are now failing due to the lack of `libfaketime` in the Alpine repos.
1 year ago
Matt Clay 765b2bd783
ansible-test - Narrow the scope of some sanity tests (#81273)
* ansible-test - Limit scope of replace-urlopen test

Only ansible-core code and plugins in collections can be expected to rely on module_utils.

* ansible-test - Limit scope of use-compat-six test

Only ansible-core code and plugins in collections can be expected to rely on module_utils.

* ansible-test - Limit scope of no-get-exception test

Only ansible-core code and plugins in collections should be checked for usage of outdated module_utils functions.

* Add integration tests
1 year ago
Matt Clay 3f7bf0bcd4
ansible-test - Fix sanity traceback with `-e` opt (#81271)
Also remove redundant warning about missing programs.

Includes integration tests to verify `-e` does not traceback.
1 year ago
Abhijeet Kasurde c3015c5eb1
replace: handle exception while parsing escape char (#81244)
* replace: handle exception while parsing escape char

* Fail early when bad escape character is provided in replace module

Fixes: #79364

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

* Run tests in Python 3.6 or greater env

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

---------

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Sviatoslav Sydorenko b6ebb9d41a
Replace `reboot_command` integration test with unit tests (#78956)
Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Matt Clay 876be11f8c
Add pymarkdown sanity test for core (#81220)
* Clean up markdown

* Add pymarkdown sanity test for core

* Update unit test SHA
1 year ago
Matt Clay 72e038e823
Remove `docs` and `examples` directories (#81011)
* Remove docs dir

* Updates to reflect docs removal

* Fix integration test

* Remove examples dir

* Updates to reflect examples removal

* Remove build_library and build-ansible.py

* Remove refs to build_library and build-ansible.py

* Remove obsolete template

* Remove obsolete template reference

* Remove the now obsolete rstcheck sanity test
1 year ago
Brian Coca c7eca0e5c0
first_found lookup, let lookup handle templating errors (#81178)
* first_found lookup, let lookup handle templating errors

Avoids case in which TE was not sending valid and templatable entries to the lookup
The lookup already handles the case TE was attempting to itself, so no need for this code anymore.

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
1 year ago
Sloane Hertel 67b78a17c4
Remove Python 3.9 support for the controller (#80973)
* Remove obsolete Python <=3.9 controller code
* Remove Python 3.9 test controller bootstrapping
* Update test requirements

Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Martin Krizek eb19692f48
Properly disable modularity tests for dnf5 only (#81195) 1 year ago
Martin Krizek 7b2647f3b2
Re-introduce RHEL 9 modularity testing in CI (#81160) 1 year ago
Sloane Hertel 73dde45f0f
ansible-galaxy - add pre-release hint to dependency resolution error (#81147)
* add pre-release hint to dependency resolution error if --pre wasn't provided

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
1 year ago
Norman Ziegner bcdb82992a
user - add parameter for password expiration warning days (#79884)
* user - add parameter to set number of warning days before password expires

Signed-off-by: Norman Ziegner <norman.ziegner@ufz.de>
1 year ago
Martin Krizek 0cc50e0673
dnf5: enable now implemented cacheonly functionality (#81141)
Note that this also fixes a traceback caused by backwards incompatible
change in dnf5 where cacheonly was changed from bool to string:
https://github.com/rpm-software-management/dnf5/pull/665/files#diff-ab65249ff7fccadfb2864b6826f6559f7f16fad43fd3bf2da0b4fe8db790d59aR179
1 year ago
Brian Coca e22fe9be56
ansible-test add alpine 3.18 as remote (#81109)
* add crun to avoid runc
1 year ago
Jordan Borean 98d1cf7aa2
Deprecate plural collections_paths option (#81063)
* Deprecate plural collections_paths option

* Fix test assumption
1 year ago
Martin Krizek cde15f3c81
Add test support for Python 3.11 on RHEL 8.8/9.2 (#80990)
Fixes #80412
Fixes #80413
1 year ago
James Cammarata 660f1726c8
Register handlers immediately if currently iterating handlers (#80898)
This fixes the issue where handlers notifying other handlers are
not properly run because the notification is not registered unless
another flush_handlers occurs. Instead, if the current host state
is iterating handlers we immediately register the handler to be
run so the notification is not lost.

Fixes #80880
1 year ago
Matt Martz 73e04ef2d6
Don't mutate templar.environment, only overlay on local myenv (#81005) 1 year ago
Martin Krizek f13e35cbf9
dnf5 - enable env groups testing in CI (#81032)
Implemented in https://github.com/rpm-software-management/dnf5/pull/585
1 year ago
Matt Clay 6ac0ea3567
Use *.md instead of *.rst for test READMEs (#81009) 1 year ago
Holger Dörner 25b3d3a6f7
Check if skeleton is /dev/null while creating home folder (#75948)
* Check if skeleton is /dev/null while creating home folder

* Add test for linux

Co-authored-by: Holger Dörner <h.doerner@bitexpert.de>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
1 year ago
Matt Martz 92d5ffda82
Update galaxy_ng container for tests (#80721) 1 year ago
Matt Martz ef3b806dd3
Fix remote_tmp tests so that they actually use the remote_tmp (#80969)
* Fix remote_tmp tests so that they actually use the remote_tmp

* Isolate remote_tmp dir during remote_tmp tests. Fixes #80965

* Do everything without become
1 year ago
Matt Martz aa67d544fc
Handle unreachable errors in fetch properly. Fixes #27816 (#80952) 1 year ago
Abhijeet Kasurde e88ff32b10
deb822_repository: use http-agent with open_url (#80876)
* Use http-agent in open_url API while getting
  cloudflare content

Fixes: #80809

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Abhijeet Kasurde 9f4dfff69b
Pylint fixes (#80855)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Abhijeet Kasurde 56f2e65ee7
Remove unused test files (#80879)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Sloane Hertel 09b4cae4fb
unarchive - ensure src is an absolute path before picking handler (#80738) 2 years ago
Brent Barbachem 0c7361d9ac
URI Module find json sub type (#80745)
* uri: fixed search for json types to include strings in the format xxx/yyy+json
2 years ago
Matt Martz 61157f6a9e
Switch from multiprocessing.Queue to SimpleQueue (#80838) 2 years ago
Jordan Borean 8db9bd7574
Remove deprecated include (#80752)
* Remove deprecated include

* Add tombstone entry for include

* Use string for date

* Use todays date

* Remove uneeded if statement
2 years ago
Jordan Borean 2253837453
command - Add option to disable argument var expansion (#80512)
* command - Add option to disable argument var expansion

* Use test char that isn't an escape sequence
2 years ago
Brian Coca 0830b69059
config lookup, new option to show origin of value (#80651) 2 years ago
PRASOON KARUNAN V b576f0cda7
fix error if path had wildcard in it (#74723)
* fix error if path had wildcard in it

* add test

* add changelog fragment
2 years ago
Brian Coca 5a84ff26df
first_found fix no terms option (#76550)
also fix bug with multiple items clobbering previous settings

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Martin Krizek 8cd95a8e66
Account for overlays when interacting with Jinja envs (#80705)
Instead of using Templar.environment in Templar.do_template for
accessing/mutating the environment, myenv local variable should be used
because it is the environment used for actual templating. It can either
point to Templar.environment or newly created environment overlay.

Fixes #80605
2 years ago
Matt Clay 2cd1744be3
Use ansible.module_utils.common.text.converters (#80704)
Replace use of old `ansible.module_utils._text` and add a unit test to maintain backwards compatibility.
2 years ago
Dmitriy Rabotyagov 7f48fa0129
Better parse virtualenv_command option for pip (#80624)
Atthe moment if a users wants to protect  virtualenv_command by using
quotes around 'venv', module will fail out as literal parsing is used
at the moment. In order to imrpove things, argparse is leveraged to
parse out passed value to the virtualenv_command

Closes-Bug: #76372

Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>
2 years ago
Sloane Hertel d18d4f84ec
_symbolic_mode_to_octal- fix raising ValueError for invalid symbolic modes (#80449)
validate the full user and perm strings instead of just first character

fixes unhelpful unarchive error for some invalid modes
2 years ago
Brian Coca b2c0095722
Fix gather parallel (#80597)
* fix meaning of parallel in gather_facts
 * Update docs with note about parallel not always being faster
 * add 'smarter' usage of gahter_timeout for parallel tasks
 * restore async when needed, not always
 * added typing
 * parallelism tests
2 years ago
Martin Krizek 989a7d6951
Fix calling v2_playbook_on_notify callback (#80585)
Co-authored-by: Matt Martz <matt@sivel.net>
2 years ago
Matt Clay 7b50571588
ansible-test - Remove deprecated features (#80558) 2 years ago
Sloane Hertel 694c11d5bd
argspec - fix validating type for required options that are None (#79677)
* Only bypass type validation for null parameters if the default is None. A default is mutually exclusive with required.

* Prevent coercing None to str type. Fail the type check instead.
2 years ago
Sloane Hertel 964e678a7f
ansible-galaxy - fix installing dir sources with a trailing path separator (#79110)
* strip trailing path separator from type=dir sources
2 years ago
Matt Clay 676b731e6f
ansible-test - Replace pytest-forked (#80525)
- Unit tests now report warnings generated during test runs.
- Python 3.12 warnings about `os.fork` usage with threads (due to `pytest-xdist`) are suppressed.
- Added integration tests to verify forked test behavior.
2 years ago
kloud-byun 2e62724a8a
feature: add support for symbolic links when passing "dir" (#80460)
* update the include_vars action plugin to always follow symbolic links when traversing directories
2 years ago
Martin Krizek 09dd80b4ec
Last handler defined runs, fix for roles (#79558)
Fixes #73643
* clear_notification method and simplify ifs
* Deduplicate code
* Limit number of Templar creations
* Fix sanity
* Preserve handler callbacks order as they were notified
2 years ago
Matt Clay babdec80cc
Fix the parsing integration tests (#80454)
Most of the bad_parsing tests were no longer running, with several of them no longer being valid.
The invalid tests have been removed and the valid ones rewritten.
2 years ago
Felix Fontein 92c694372b
Replace validate-modules's semantic markup parser with antsibull-docs-parser (#80406) 2 years ago
Felix Fontein 96f7fd5539
Extend ansible-doc test with more classic markup. (#80407) 2 years ago
Sloane Hertel 460abe0cef
Fetch signatures from galaxy after the dependency resolver runs (#80334)
Reduce the number of Galaxy API calls made during dependency resolution by fetching remote signatures afterwards, since these are not used in backtracking.

Reduce the verbosity to `-vvvv` (to match other Galaxy API calls) to see this activity.

Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
2 years ago
Felix Fontein 2f647e9617
Implement semantic markup support for Ansible documentation in validate-modules. (#80243) 2 years ago
tu1h 043a0f3ee8
Fix run_once by instantly tiny post_validate (#78492) (#80051)
Signed-off-by: tu1h <lihai.tu@daocloud.io>
2 years ago
Felix Fontein 0e509ecf25
Support role extension for semantic markup. (#80305) 2 years ago
Martin Krizek a81b787a05
Add new dnf5 module (#80272) 2 years ago
Sloane Hertel 666188892e
fix using templated values for include/import role FA (#80320)
* fix using templated values for include/import role options 'public', 'allow_duplicates', and 'rolespec_validate'

* pass templated values without changing the instance

* Fix templating by setting always_post_validate to True and calling IncludeRole.post_validate() instead

ci_complete

* add changelog
2 years ago
Matt Clay 5d64024f1a
ansible-test - Fix packaging change classification (#80360) 2 years ago
Matt Clay 052da3c89e
Use `skip/macos` instead of `skip/macos/13.2` (#80343)
The reason for the skip won't be going away with future versions of macOS.
2 years ago
Matt Clay 1d6f019ce6
Remove obsolete integration test aliases (#80342)
* Remove obsolete integration test aliases

* Remove obsolete `skip/osx` aliases

* Use `skip/macos` instead of `skip/osx` in docs
2 years ago
Matt Martz d2f6ea4179
New deb822_repository module (#80018)
Fixes #77073
2 years ago
Felix Fontein 086ae42209
Fix URL processing. (#80295) 2 years ago
Jordan Borean ba4505f5cb
Ansible.Basic - Improve temporary file cleanup process (#80293)
* Ansible.Basic - Improve temporary file cleanup process

* Add comment on struct value used
2 years ago
Wong Hoi Sing Edison 016b7f71b1
ansible-galaxy - support `resolvelib >= 0.5.3, < 1.1.0` (#80196)
* ansible-galaxy - support `resolvelib >= 0.5.3, < 1.1.0`

<https://pypi.org/project/resolvelib/1.0.1> released on 2023-03-09:

-   <https://github.com/sarugaku/resolvelib/blob/main/CHANGELOG.rst#101-2023-03-09>
-   <https://github.com/sarugaku/resolvelib/releases/tag/1.0.1>

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>

* Trigger CI by pinning resolvelib with latest version

Since resolvelib is pre-installed in our test containers, we should temporarily pin the latest version allowed to force the tests to run with that version. Once the tests have passed that commit can be reverted.

Please make those changes without force pushing, so that we keep the reference to the passing CI run. We can squash the commits when merging the PR so the temporary commits won't be in the final commit merged to the devel branch.

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>

* https://github.com/ansible/ansible/pull/80196#discussion_r1136003637

Also test resolvelib with multiple supported versions.

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>

* Revert "Trigger CI by pinning resolvelib with latest version"

This reverts commit 5518e5dbca.

---------

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
2 years ago
Felix Fontein b398613120
Add support for plugin field in seealso. (#80212) 2 years ago
Matt Clay 054aa92158
Fix url lookup test to use test container (#80284) 2 years ago
Matt Martz ed4b1afc6c
Don't double loop, ensuring that machine formats don't dupe collections onto all paths (#80277)
* Don't double loop, ensuring that machine formats don't dupe collections onto all paths

* Add tests
2 years ago
Matt Martz 42355d181a
Do not double calculate loops and `delegate_to` (#80171) 2 years ago
Felix Fontein fafb23094e
ansible-doc: fix broken seealso links in text output (#80280)
* Fix broken URLs.

* Also remove auto-generated description for modules outside ansible.builtin.
2 years ago
Felix Fontein 4ea50cef23
validate-modules: check_mode attribute and compare with supports_check_mode parameter (#80090)
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
2 years ago
Felix Fontein 5388f6020a
validate-modules: support plugin see-also (#80244) 2 years ago
Felix Fontein a2dc5fcc7d
Implement semantic markup support for Ansible documentation in ansible-doc. (#80242) 2 years ago
Sviatoslav Sydorenko eebfd71a6d
Make PEP 517 mutation tests use pinned old setuptools (#80262)
* Pin setuptools to lowest supported @ PEP 517 test

This allows catching the behavior of builds under old setuptools.

* Stop invoking `setup.py install` in tests

This is not the part we care about since it involves dealing with the
external runtime dependencies rather than building our source
distribution.
2 years ago
Sviatoslav Sydorenko 7097df3eed
📦 Switch sdist build-system to pure setuptools (#80255)
This patch modifies the in-tree build backend to build sdists that swap
out pointers to it in the `pyproject.toml`'s `[build-system]` section.

The effect of this is that the first build from source (for example,
from a Git checkout) uses our PEP 517 in-tree build backend. But the
produced tarball has `build-backend` set to `setuptools.build_meta`
which is the native build backend of `setuptools`. So any following
builds from that sdist will skip using the in-tree build backend,
calling the setuptools' one.
The good news is that if the first build generated the manpages, they
will be included and won't go anywhere even though, a different build
system is in place.

Combined with #80253, this will make sure not to modify the current
source checkout on that first build.

Co-authored-by: Matt Clay <matt@mystile.com>
2 years ago
Sloane Hertel b7a0e0d792
copy - fix check mode with remote_src=True (#78624)
* Don't create dest directory in check mode

uncomment existing test

Fix checking for file attribute changes in check mode and add a test
2 years ago
Sviatoslav Sydorenko 701f8852ec
Revert "🧪 Switch macOS 13.2 to 12.0 in CI (#80145)" (#80156)
This reverts commit 326e70c02a.
2 years ago
Brian Coca a1bff416ed
paramiko connection now uses the correct source of data (#79704)
* paramiko connection now uses the correct source of data

* keep orig name

* added missing port to docs

* Update changelogs/fragments/paramiko_config.yml

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>

* test ansible_host templating

* added missing private_key_file

* removed all play_context fallbacks

* Update lib/ansible/plugins/connection/paramiko_ssh.py

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Update lib/ansible/plugins/connection/paramiko_ssh.py

Co-authored-by: Jordan Borean <jborean93@gmail.com>

---------

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
2 years ago
Sviatoslav Sydorenko 326e70c02a
🧪 Switch macOS 13.2 to 12.0 in CI (#80145)
* 🧪 Switch macOS 13.2 to 12.0 in CI

The former revealed unexpected flakiness while the latter is the
previous value that was used to be stable. This is a temporary revert.

* Skip lookup_url integration test under macOS 12.0
2 years ago
Brian Coca ff3ee9c4bd
ansible-inventory prevent duplicating host entries (#80059)
this happened after implementing limits we introduced a bug that a host would be duplicated if it existed in a group's children
2 years ago
Sviatoslav Sydorenko 8fec2d2a82
Add macOS 13 to CI and ansible-test (#79508)
*  Add macOS 13.2 to `ansible-test`

* 🧪 Replace macOS 12.0 with 13.2 in the CI matrix

* Skip `lookup_url` under macOS 13.2

This is due to https://wefearchange.org/2018/11/forkmacos.rst.html
that manifests itself as follows:

    TASK [lookup_url : Test that retrieving a url works] ***************************
    objc[15394]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
    objc[15394]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in t
    he fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
    ERROR! A worker was found in a dead state

* 📝 Extend ansible-test change note w/ macOS 13.2
2 years ago
Sviatoslav Sydorenko c0e0550bce
🐛 Make integration tests compatible w/ modern Git (#80122)
* 🐛 Make integration tests compatible w/ modern Git

This patch makes use of the `init.defaultBranch` setting to unify
the test across new and old Git versions since one defaults to
`master` and the other uses `main` for the default branch.

Where possible, it uses the `HEAD` committish to avoid having to
normalize the branch name.

The change fixes the following integration tests:

  * `ansible-galaxy`

  * `ansible-galaxy-collection-scm` (recursive collection)

  * `git`

* 🐛Replace `git-symbolic-ref` with a repo template

This custom Git repository template emulates the `init.defaultBranch` setting
on Git versions below 2.28. Ref: https://superuser.com/a/1559582.
Other workarounds mentioned there, like invoking
`git symbolic-ref HEAD refs/heads/main` after each `git init` turned
out to have mysterious side effects that break the tests in surprising ways.

* 🎨 Make Git integration test non-destructive

This patch makes use of the `$HOME` environment variable to trick Git
into using a user-global config generated in the temporary directory.
2 years ago
Sam Doran a56428de11
unarchive - properly handle relative path for `dest` (#75267)
* unarchive - properly handle relative path for dest

* Add integration test

* Return output of underlying commands with increased verbosity

* Revert "Return output of underlying commands with increased verbosity"

This reverts commit a2790c8275cdc5697b65670a0beffdc74b741bf6.

* Warn when a relative destination path was provided
2 years ago
Sloane Hertel b981a9dfcd
add a worker queue to get updates from the main results thread (#79886)
* Create a queue per WorkerProcess to receive intra-task updates
* Update `pause` action to use the worker queue
* Deprecate ConnectionBase()._new_stdin
* Add new `Display`  convenience method `prompt_until` to manage both controller- and worker-sourced prompting without cross-fork stdin sharing, in-worker mechanism to handle request-response over new worker queue.
2 years ago
Matt Martz 060a27f28f
Do not use hardcoded httpbin.org in uri tests (#80101) 2 years ago
Mark Chappell f64e79be41
sanity tests - ignore pre-release versions for deprecation comparisons (#79185) 2 years ago
Matt Clay f587856beb
Remove straight.plugin dependency (#80084) 2 years ago
Matt Clay 6bfe6b899a
ansible-test - Fix vendoring support (#80074)
- Support loading of vendored Python packages.
- Exclude vendored Python packages from payloads.
2 years ago
Sviatoslav Sydorenko 56036013cd
📦 Integrate manpage builds into PEP 517 build backend
This patch creates a thin wrapper around the `setuptools`' PEP 517
build backend in-tree. It features an ability to request generating
the manpage files in the process of building a source distribution.
This toggle is implemented using the `config_settings` mechanism of
PEP 517.
One must explicitly pass it a CLI option to the build front-end to
trigger said behavior. The packagers are expected to use the
following call:

    python -m build --config-setting=--build-manpages

This option has no effect on building wheels.

🧪 The change includes integration tests

This test runs building and re-building sdists and wheels with and
without the `--build-manpages` config setting under the
oldest-supported and new `setuptools` pinned.

It is intended to preserve the interoperability of the packaging setup
across Python runtimes.

An extra smoke test also verifies that non PEP 517 interfaces remain functional.

PR #79606


Co-authored-by: Matt Clay <matt@mystile.com>
2 years ago
Martin Krizek 6c3d2a4e51
Use FAs with inheritance only when applicable (#80026)
... and set default value of an attribute on an object
only in NonInheritableFA.

Fixes #79777

ci_complete
2 years ago
Matt Clay 5f6ebebce9
Update ansible-test openshift plugin tests (#80064)
Update ansible-test openshift plugin tests to use config to get server endpoint.

Also update the comment explaining why the tests are disabled.
2 years ago
Matt Clay bca55d945b
ansible-test - Adjust vcenter test plugin behavior (#80063)
The previous change to the default mode of operation for the plugin is now limited to collections.

This enables easier testing of the plugin in core, so the previously removed tests have been restored.
2 years ago
Sviatoslav Sydorenko c05d813661
Fix #80020 typos confusing `pylint` for `pytest` (#80056) 2 years ago
Matt Clay e6cffce0eb
ansible-test - Allow target prefixes to use `_` (#80021)
Integration test target prefixes defined in a `tests/integration/target-prefixes.{group}` file can now contain an underscore (`_`) character.
2 years ago
Matt Clay fe2732b91e
ansible-test - Support pylint assertion rewriting (#80020)
Add support for `pylint` assertion rewriting when running unit tests on Python 3.5 and later.
2 years ago
devon-mar 2f8f7fba4c
Update lookup to use positional only arg for name to avoid conflicts (#79839) 2 years ago
Matt Clay 21fc699b77
Remove trailing spaces from lines in YAML (#80032) 2 years ago
Matt Clay a4248a63aa
Add missing newlines to YAML files (#80031) 2 years ago
Jordan Borean 1e6b8249e7
Add condition that causes a when to skip a task to output msg (#78918)
* Add condition that causes a when to skip a task

* Fix up tests

* Use false_condition instead of failed_condition

* Remove formatting accidentially added

* Fix sanity
2 years ago
Martin Krizek bd329dc543
Make using blocks as handlers a parser error (#79993)
Fixes #79968
2 years ago
Sloane Hertel 556dadba6d
user - fix comparing existing group names to group IDs (#79981) 2 years ago
Brian Coca e2f147bcec
ansible-inventory, add --limit option (#79596)
* ansible-inventory, add --limit option

* also graph

* optimize the when not providing limit

* added tests and clog

* avoid empty groups, fix tests

* i swear there as an ignore_errors there already!!?!?!?

* Fix stdout test fallback

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>

* simpler approach using subset against inventory directly

* tyupose

* renamed funciton to what it actuall does
got yaml in line with others about removing empty groups

* have graph ignore limts, also note same for --host

* fixed typo long line

* better test

* cause hosvars are not a thing

---------

Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Mark Chappell ceafb5411a
Add an integration test to ensure that defaults don't count towards mutually exclusive (#79949) 2 years ago
Martin Krizek 09e0fb3516
Change ansible_job_id format (#79951)
* Change ansible_job_id format

...to be something that does not resemble float or other type which
value could be changed by literal_eval that jinja2_native uses.

Specifically the format of '%d.%d' is converted from str to float and
then back to float which may result in truncating the number resulting
in the job not being found because the job id does not exist.
2 years ago
Matt Davis e1d298ed61
don't ignore templated _raw_params that k=v parser failed to parse (#79913)
fixes #79862
2 years ago
Sloane Hertel fcdd7493ae
improve password_hash warning for unsupported algorithms (#79872)
* password_hash - give a warning for unsupported algorithms (that raise a TypeError)

* add suggested changes, a test and changelog
2 years ago
Matt Clay 9d65e122ff
Fix detection of available hashlib algorithms (#79946)
* Fix detection of available hashlib algorithms

Detection of hashlib algorithms now works on Python 3.x.
The new implementation works on Python 2.7 and later.

Test coverage is provided by both integration and unit tests.

* Add additional details about hashlib in docs
2 years ago