Commit Graph

3878 Commits (92d2c66db2eecd0f5e26e6f49589dbcdb4612618)

Author SHA1 Message Date
Matt Martz 92d2c66db2
Remove py2 support from urls.py (#81880) 1 year ago
Jordan Borean 8a5ccc9d63
ansible-galaxy - fix traceback error for invalid req file (#81917)
Provide a better error message when encountering a YAML requirements file that is not a dictionary or list.

Fixes: #81901
1 year ago
Matt Clay 921efce43a
ansible-test - Remove more Python 2.7 / 3.6 support (#81898) 1 year ago
Abhijeet Kasurde c0eefa955a
docs: update error message (#81554)
* Reword the error message when the module fails to parse parameters
  in JSON format
* misc typo fixes

Fixes: #81188

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Davis c1343cc304
switch available_hosts to `frozenset` from `list` in ansible-inventory (#81870)
* major speedup large inventories, since it's consulted iteratively
1 year ago
Sloane Hertel debf2be913
optimize host_group_vars and vars plugin loading (#79945)
* Improve host_group_vars efficiency:

* normalize the basedir with `os.path.realpath()` once and cache it
* cache missing paths/files
* reduce the calls to `isinstance`

Add a couple more general improvements in vars/plugins.py get_vars_from_path():

* call `PluginLoader.all()` once for vars plugins and reload specific
  plugins subsequently
* don't reload legacy/builtin vars plugins that are not enabled

Add a test for host_group_vars and legacy plugin loading

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

* Add a new is_stateless attribute to the vars plugin baseclass

update integration tests to be quieter and use the same test pattern

Fix deprecation and adjust test that didn't catch the issue (deprecation only occured when the value was False)

move realpath cache to host_group_vars (do not smuggle call state as instance data)

refactor under a single 'if cache:' statement

Call os.path.isdir instead of always calling os.path.exists first. Just call os.path.exists to differentiate between missing and non-directory.

remove call to super(VarsModule, self).get_vars()

use the entity name as the cache key instead of variable location

Remove isinstance checks and use a class attribute just in case any plugins are subclassing Host/Group

Replace startswith by checking index 0 of the name instead, since host/group names are required

* rename is_stateless to cache_instance to make it more clear what it does

* add plugin instance cache using the path to plugin loader

reduce loading stage option if a new instance isn't created

don't require a known subdir on PluginLoader instantiation for backwards
compatibility

rename attribute again

contain reading from/initializing cached instances to a plugin loader method

* Deprecate v2 vars plugins

* Refactor to use the cache in existing plugin loader methods

Rename the attribute again

Refactor host_group_vars with requested changes

Make changelog a bugfixes fragment

Add a deprecation fragment for v2 vars plugins.

Add type hints

* unbreak group_vars

* Apply suggestions from code review

* misc tweaks

* always cache instance by both requested and resolved FQ name
* add lru_cache to stage calculation to avoid repeated config consultation

* handle KeyError from missing stage option

---------

Co-authored-by: Matt Davis <mrd@redhat.com>
1 year ago
Martin Krizek 9b3ed5ec68
Properly template tags in parent blocks (#81624)
When templating tags (which happens outside of standard `post_validate`) we
need to template each object in the inheritance chain and set the templated
values on those objects individually. That way when `task.tags` is called the
`extend` functionality properly picks up the templated values of all
parents into one flatten list.

Fixes #81053
1 year ago
Abhijeet Kasurde 304e63d76e
Add family fallback for Amzn to register as RedHat (#81755)
* Add family fallback for Amzn to register as RedHat

Fixes: #80882

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Martin Krizek 2d5861c185
run_once: unnotify hosts on handlers that are not run (#81667)
Fixes #81666
1 year ago
Sloane Hertel b3408ab80b
Fix dnf module crash for non-existent url (#81801)
* ci_complete ci_coverage
1 year ago
Matt Clay b7903669b4
ansible-test - Update distro containers (#81851)
* ansible-test - Update distro containers

* Update the alpine3 container to Python 3.11

* Update changelog
1 year ago
Matt Clay 20f17687da
ansible-test - Invoke container sleep through env (#81853) 1 year ago
Sviatoslav Sydorenko ec265ec08a
Bump `coverage.py` to v7.3.2 in `ansible-test` (#81844) 1 year ago
Martin Krizek e756e359e0
Remove deprecated JINJA2_NATIVE_WARNING env var (#81720)
Fixes #81714
1 year ago
Brian Coca ab6a544e86
Import role public (#81772)
revert to previous behavior to push vars to play at compile time
add `public` parameter to allow per import control of exporting (vs just the global config)

Co-authored-by: tchernomax <maxime.deroucy@gmail.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
1 year ago
Sloane Hertel 7fab552563
Fix reporting role as not found when remote_data is None (#81829) 1 year ago
Abhijeet Kasurde 4d4c50f856
Provide more info about plugin in deprecation message (#81719)
Provide more information about plugin usage which needs to be
used instead of the deprecated plugin in the deprecation message.

Fixes: #80561

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
lukaslihotzki-f 27bbff7c22
Add count and mandatory_count parameters in regex_replace filter (#81775) 1 year ago
Jordan Borean f22231de20
winrm - make command input more resiliant (#81538)
* winrm - make command input more resiliant

* Expand warning message
1 year ago
Martin Krizek 60c9660952
Add intentional tests for dnf install rpm when higher version is installed (#81740)
* Add intentional tests for dnf install rpm when higher version is installed

* ci_complete ci_coverage
1 year ago
Matt Martz ddf0311c63
Prevent roles from using symlinks to overwrite files outside of the installation directory (#81780)
* Sanitize linkname during role installs

* Add tests

* add clog frag
1 year ago
Hungtsetse 51f2ddd445
Extend mount info (#81768) 1 year ago
Val 91f94fb59d
Daemonize follow-up fixes (#81584)
* Ensure binary data transmission in daemonize

* Add changelog fragment
1 year ago
Matt Clay 3794612832
ansible-test - Skip pylint test on Python 3.12 (#81706) 1 year ago
Brian Coca 2aef0406d4
ansible-galaxy fix scm dependency error (#81599)
* ansible-galaxy fix scm dependency error

  also changed usage of 'virtual colleciton' to actual type
  avoid error by filtering out virtual collections that dont have
expected properties

simplified as per webknjaz

* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
1 year ago
Matt Martz 7c7385521e
Harden the ansiballz and respawn python templates (#81753)
Harden our python templates for respawn and ansiballz around str literal quoting
1 year ago
Brian Coca 9edf64eaeb
Deprecating paramiko options in base config (#81593)
* Deprecating paramiko options in base config

 they exist in the plugin itself since 2.14

* clog
1 year ago
Martin Krizek 1e7f7875c6
Allow for searching handler subdir for included task via include_role (#81733)
Fixes #81722
1 year ago
Abhijeet Kasurde 86fd7026a8
Handle exceptions in interpreter discovery (#81745)
* Handle exceptions like AnsibleError, AnsibleConnectionFailure
  raise while interpreter discovery.

Fixes: #78264

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Carlos Camacho 9244b2bff8
Make sure paths are treated correctly when building collection files manifest (#81619)
* Make sure paths are correct when building collection files manifest

This commit makes sure the path of the files part of
the collection build manifest are correct.

This commit uses os.path.commonprefix instead of
dealing with strings.

Signed-off-by: Carlos Camacho <ccamacho@redhat.com>

Bugfix Pull Request
Fixes: #81618

* Revert the change note type to `minor_changes`

* Clarify the change note with user-oriented details

---------

Signed-off-by: Carlos Camacho <ccamacho@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
1 year ago
René Moser e4b9f9c6ae
tests: update CloudStack test container to version 1.7.0. (#81732) 1 year ago
Sviatoslav Sydorenko 7662a05085
Always allow "no-other-choice" pre-release dependencies when resolving collection dependency tree
PR #81606.

Prior to this patch, when `--pre` CLI flag was not passed, the
dependency resolver would treat concrete collection dependency
candidates (Git repositories, subdirs, tarball URLs, or local dirs or
files etc) as not meeting the requirements.

This patch makes it so pre-releases in any concrete artifact
references, and the ones being specifically pinned dependencies or
user requests, met anywhere in the dependency tree, are allowed
unconditionally.

This is achieved by moving the pre-release check from
`is_satisfied_by()` to the `find_matches()` hook, following the
Pip's example.

As a bonus, this change also fixes the situation when a collection
pre-releases weren't considered if it didn't have any stable releases.
This now works even if `--pre` wasn't requested explicitly.

Finally, this patch partially reverts commit
6f4b4c345b, except for the tests. And it
also improves the `--pre` hint warning to explain that it mostly
affects Galaxy/Automation Hub-hosted collection releases.

Ref #73416
Ref #79112
Fixes #79168
Fixes #80048
Resolves #81605

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
1 year ago
Abhijeet Kasurde 4b7705b07a
Remove deprecated APIs from ansible-doc (#81729)
* Removed add_collection_plugins
* Removed find_plugins

Fixes: #81716

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Martz 3ec7a6e0db
Fix ansible-test issues with detecting docker host and forwarding ports (#81726)
* Properly detect docker host using ssh

* Disable ControlPath when creating SSH port forwards

* typo fix

Co-authored-by: Matt Clay <matt@mystile.com>

* Add clog frag

---------

Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Abhijeet Kasurde 0ea40e09d1
vars: handle exception in combine_vars (#81700)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Martz f7234968d2
bump devel to 2.17.0.dev0 (#81712) 1 year ago
Abhijeet Kasurde a1a6550daf
inventory_ini: Handle SyntaxWarning in ini parsing (#81707)
* handle SyntaxWarning ini inventory parsing

Fixes: #81328

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Abhijeet Kasurde 2793dfa594
reboot: show last error message in verbose log (#81578)
* The last error message is now presented in a verbose log

Fixes: #81574

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Abhijeet Kasurde 6f65397871
galaxy: check if the target for symlink exists (#81586)
* Symlinks in the collection might be pointing to non-existent
  targets. Check and report the failure to the user.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Felix Fontein 7f0baabbe0
blockinfile: avoid crash on Python 3 when creating directory fails (#81662)
* Avoid crash on Python 3.

* Add a test for the crash on Python 3
1 year ago
Abhijeet Kasurde e4468dc944
copy: print correct dest path when content + diff is used (#81678)
* when --diff is used with content parameter, print destination
  path instead of temporary file path.

Fixes: #79749

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Matt Clay 7d3d4572ed
Fix set filters to use set operations (#81639)
* Fix set filters to use set operations

* Fix integration tests

* Update filter documentation
1 year ago
Felix Fontein b1b029c6b5
ansible-doc: allow to filter by more than one collection (#81450)
Make collection filters more flexible for listing collections.
Co-authored-by: Maxwell G <maxwell@gtmx.me>
1 year ago
Sloane Hertel 8034651cd2
Only mark a role as complete once a task in it executes for the target host (#81565)
* If all tasks in the role are skipped or unreachable, the role is not marked as complete for the host.

* Only mark the role as complete if a task in the role succeeds or fails for the host.
1 year ago
Martin Krizek 3eb96f2c68
Lazy load Templar for [changed,failed]_when (#81650)
The `handler_templar` in `StrategyBase._process_pending_results` was
used for templating handler names which is no longer the case. Now it is
only used for templating `changed_when`/`failed_when`. To prevent
re-creating templars each time results are processed, just create it
on-demand when whens are processed.
1 year ago
Abhijeet Kasurde cd0aa35c19
encrypt: deprecate passlib_or_crypt (#81571)
* deprecate passlib_or_crypt in favor of do_encrypt

Fixes: #55839
1 year ago
Abhijeet Kasurde 6177888cf6
ansible-vault: Check if the destination is writable (#81660)
* Before performing shredding the original file, check if the
  destination file location is writable or not. This will
  prevent corruption of original file.

Fixes: #81455

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Martin Krizek a861b1adba
Replace ConfigParser.readfp() with read_file() (#81657)
* Replace ConfigParser.readfp() with read_file()

readfp was removed in Python 3.12.

Fixes #81656

* py2?

* Use PY3 for the python check
1 year ago
Matt Martz 3ec0850df9
Support packaging and importlib.metadata for pip module (#80881) 1 year ago
Matt Davis dd79c49a4d
fix various Jinja plugin caching issues (#79781)
* fix various Jinja plugin caching issues

* consolidate the wrapper plugin cache
* remove redundant cache in J2 filter/test interceptor

* intra-template loader bypass

* fix early exits swallowing some exception detail

* misc comment cleanup
1 year ago