* apply owner/group permissions to top folder
* remove unused var unarchive30
* fix permissions for top folders if the tarball include multiple top folders
* added test for top folder ownership
* fix facts delegation loop overwrite
partial revert of change to allow facts to be present in each loop iteration
was not needed in final results as result processing alreayd had the disctiontion
and ended up breaking the assumptions in the calling code.
fixes#76676
* play_context, compensate for existing plugins
some connection plugins are not fully using the correct configuration,
but this was previously hidden from them as play_context was providing
the info instead, now play_context provides the 'correct' info, but hitting
these bad configurations.
* ansible-galaxy collection install|verify:
- Support verifying the origin of the MANIFEST.json when the Galaxy server has provided signatures.
- Allow supplemental signatures to use during verification on the CLI/requirements file.
* ansible-galaxy collection install:
- Support disabling signature verification. This silences the warning provided by ansible-galaxy if the Galaxy server provided signatures it cannot use because no keyring is configured.
- Store Galaxy server metadata alongside installed collections for provenance. This is used by 'ansible-galaxy collection verify --offline'.
* Add unit tests for method that gets signatures from a Galaxy server
* Add integration tests for user-provided signature sources
- Test CLI option combinations
- Test installing collections with valid/invalid signature sources
- Test disabling GPG verification when installing collections
- Test verifying collections with valid/invalid signature sources
* Make signature verification advisory-by-default if signatures are provided by the Galaxy server
- Make the default keyring None
- Warn if the keyring is None but the Galaxy server provided signatures
- Error if the keyring is None but the user supplied signatures
- Error if the keyring is not None but is invalid
* changelog
* add ansible-galaxy user documentation for new options
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Andy Mott <amott@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
* Return rc=0 on success.
Error handling in playbooks generally expects `rc` to be set to 0 when a module has not failed. Playbook authors should not have to check for the existence of `rc` first.
* Use single definition and added changelog
* Fix up tests with new return value
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* add DebianStrategy tests
* ensure hostname can be changed by using become
* use Systemd strat for debian and Base for generic.
* add test to ensure all strategies are available
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
avoid polluting vars with incorrect settings
simplify variables and templars
- variables, original, only updated with final results, reset are copies of this
- tempvars used for preliminary templating
- cvars used for connection/shell/become plugins, delegation aware
- vars_copy ignore tempvars updates and use connection plugin to get
finalized version per loop item/delegation, also used to store temp results
till we are ready to update 'variables'
- fine tune nolog just cause we are here
- also fix inventory_hostname_short for IP addresses
git module now uses env vars exclusively
- updated docs to clarify usage
- now env vars append instead of overwrite to allow existing custom setups to keep working
fixes#38104, #64673, #64674
- added note for hostkeychecking more securely
fixes#69846
- keep script cause old versions still choke on env
- env var cannot hold more than 'command' for older versions
- all ssh_opts in one place
ssh plugin, use 'correct' information source in all cases
* still fallback to pc
* added inventory to new test
* undef var can still show as parser error on pc
now task_exectuer has a more accurate error handling
* ansible-test - Use relative paths in junit output.
Also fix a traceback in the junit callback during automatic fact gathering.
* ansible-test - Handle out-of-tree JUnit paths.
* updated tests to conform to new block inheritance
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* minor refactor in other options by pushing common code into functions
* consolidate coll_filter
* more normalizing loader
* dont pass plugin_loader, its global import
* Also dump roles and collections
* adjusted tests to new err msg
* disable namespace filter (unused)
* Make include_role/include_tasks work with any_errors_fatal v2
Fixes#50897
Co-authored-by: Matt Martz <matt@sivel.net>
* Add failed to results in free strategy too
* Fix
* Avoid duplicating results
* ci_complete
Co-authored-by: Matt Martz <matt@sivel.net>
* allow inputting 0 for password_expire_{min|max}
0 is meaningful for min days (any time)
0 is technically valid for max_days
* add test for setting both min and max expiry
* [0] return result of execute_command from set_password_expire*
* [1] better return for set_password_expire
* [2] handle returns from set_password_expire*
* only set password expiry if user exists
* collect return-handling code
* combine password min and max into one execution
* handle case where spwd is not present like on macOS and FreeBSD
Co-authored-by: Sam Doran <sdoran@redhat.com>
* config, moved 'choices' from wall of text, to keys
* updated to ensure all pythons handle the 2 modes correctly
* Update lib/ansible/config/base.yml
Co-authored-by: Sandra McCann <samccann@redhat.com>
* go back to keyword being explicit for plugin configs instead of using option name as implicit match
* switch spec to singular, no changes needed since no one used
* Don't send full task with async callback events. Fixes#76729
* Use args for async_status task, instead of k=v
* Make sure we send back the async task attrs for polling
* Add clog frag
* load is a staticmethod
* include_vars initialize failed
if source dir is given, but not present and traversal is empty you
can end up trying to access failed w/o it ever being defined.
also future proof for more corner cases in decision tree
dont display _terms or _intput on only changed
those always change and it expected for the plugins that support them
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
- Avoids false positives on become strings being echoed back
by ssh cli itself
- added test for debug lines
- also simplified some of existing test code
modules with python were always normalized to /usr/bin/python,
while other interpreters could have specific versions.
* now shebang is always constructed by get_shebang and args are preserved
* only update shebang if interpreter changed
* updated test expectation
* added python shebang test
* ansible-galaxy - fix the --ignore-certs flag for the implicit galaxy server
* changelog
* Add a test without the server config
* Fix respecting --ignore-certs for individual --server URLs also
* Update changelogs/fragments/76735-ansible-galaxy-fix-ignore-certs.yaml
This adds the ability to specify a package version using >=. This will ensure the package is at the specified version or above.
* If the package is not installed, the latest version will be installed.
* If the package is installed and less than the specified version, it will be upgraded.
* If the package is installed and greater than or equal to the specified version, it will be left alone.
The version selection is handled by Apt itself, so things like the system policy, pinning, etc, are considered.
* Add additional tests for check_mode with shell/command/script
* update check_mode documentation: if a module/action is not skipped in check mode and accurately reflects whether a change is made to the remote support should be 'full'
* Make reporting skipped in check mode mutually exclusive with 'changed: True'
* Add missing documented attributes
* Fix tests to expect skipped=True and changed=True do not occur together
* Fix script check_mode support documentation
* Fix earlier changelog
* document platforms attribute
* Use tasks's check mode since the value from PlayContext does not reflect loop items
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
'dscl -list' returns 0 even if the user does not exists. This
leads to errorenous condition in user module.
Using 'dscl -read UniqueID' can return if user exists or not.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure the version is valid for directories and collections in git repos before installing
Fix the error message for invalid semantic versions
* Make requested changes
* Add a test case for unhandled ValueError exception
* Add changelog
* Update lib/ansible/galaxy/collection/galaxy_api_proxy.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Reword error message
Include link to learn how to compose a semver version
* Move version validation into the caller, find_matches
* Add tests for more invalid version types
* Remove unused import
Fix raising unexpected error
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* fixes#28977
* Include the original error message when pick_handler fails
* Add a test to unarchive a tar file with an invalid extra option
* add a changelog
* check if service is indirect status / add integration test
* Update changelogs/fragments/76453-indirect-systemd-status.yml
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify indirect check / remove integration test restriction on rhel9
* add dummy indirect service for integration tests
* unarchive: fix non-english locales
For GNU Gettext, the LANGUAGE environment variable takes precedence over LANG or LC_ALL. On systems where LANGUAGE was set to a non-english locale, the output of the tar command therefore not understood and the module failed silently ("changed": false, but the archive was not extracted).
* add tests
* changelog
* check finder type before passing path
ci_complete
* Reduce nesting
* Test find_module does not cause a traceback with Python 3 FileFinder
* Update lib/ansible/utils/collection_loader/_collection_finder.py
Filter out the /usr/bin/pkg because on Altlinux it is actually the
perl-Package (not Solaris package manager).
Since the pkg5 takes precedence over apt, this workaround is
required to select the suitable package manager on Altlinux.
* catch the case that cowsay is broken
fixes https://github.com/ansible/ansible/issues/72582
add changelog
raise Exception for broken cowsay
add test for broken cowsay
Co-authored-by: Matthias Bernt <m.bernt@ufz.de>
* reset defaults for json filters
these had change on unification of json parsing/dumping
now they behave like before, but are still controllable by user.
* Implement find_spec and exec_module to remove reliance on deprecated methods in the collection loader
ci_complete
* Move module execution to exec_module
Remove extra sys.modules handling
Use default module initialization by returning None from loader.create_module
Refactor
ci_complete
* Remove ansible-test's copy of the collection loader
ci_complete
* Fix metaclass for Python 2.x
ci_complete
* Fix Py2/Py3 syntax compatibility
* Refactor
ci_complete
* update collection_loader comments
ci_complete
* simplify find_module
ci_complete
* Fix Py2 compatibility - don't get loader from nonexistent spec
Remove unnecessary PY3 checking
* Refactor common code in load_module and exec_module
ci_complete
* tidy diff
ci_complete
* Include collection_loader in target paths for 'compile' sanity test
* add changelog
* Add "return None" instead of doing it implicitly
Remove get_filename
short-circuit exec_module if it's a redirect
ci_complete
* Add changelog fragment and integration tests
* Break up function signature into multiple lines
It passed the 160 character width. Change it to multi-line for easier reading
and better diffs in the future.
* Remove unused import
* Update version added for new feature
* Move changelog fragment to correct location
Co-authored-by: Thomas Stähle <thomas.staehle@check24.de>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* First pass of adding yaml result format output to default callback
* Add clog frag
* flow style always false
* Further normalization of results across distros
* no lossy, now pretty
* Fix env var in runme.sh
* Rename variable to better self document
* include NativeJinjaUnsafeText
* Linting cleanup
* Add tests specific to the prior yaml callback
* Make data munging more exact, following the checks in libyaml/pyyaml
* Remove unused import
* Extend comment
* more correct and some comments
* Fix consistency of tab vs space in this section of the file
* Add str representer
* be cooler
* faster character filtering
* Clarify None as a sentinel for default behavior
* Flip filtering logic to avoid CRLF issues with 'space break' filtering
* Py3 change and comment about the use of __call__
Co-authored-by: Matt Davis <mrd@redhat.com>
- On Python 3, decode bytes to str
- Tolerate whitespace around the line.
- Report a missing HOSTNAME line as an error.
- Don't clobber line separators when setting a new hostname.
- Add some tests for this strategy to verify correct operation.
* Skip recursive suboption validation if sub_parameters is not a dictionary
* Ensure sub parameter elements is a sequence to prevent iterating over string characters and causing duplicate error messages for the same param
* Try to find any valid vault id before raising an exception
* Apply the same logic during vault password file loading
* Also. Improve the warning msg and tests
When we introduced an include parameter to the unarchive module, we
inadvertenly flipped the exclusion logic. This flip meant that the
unarchive module started rejecting files that should be extracted.
This commit flips the bad logic and adds some tests that will make
sure things do not go bad again.
* Use NativeEnvironment for all templating
ci_complete
* Keep Templar.copy_with_new_env for backwards compat
* Mention that AnsibleUndefined.__repr__ changed in the porting guide
* Templar.copy_with_new_env backwards compat
* ci_complete
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Prepend the task name to the warning "The loop variable 'item' is
already in use", so that you can get some context even if stdout and
stderr go to separate places.
* Use the task dependency chain to compile the role instead of the parent role's _parents list
* keep existing role inheritance, but limit it to roles in the current task dependency chain
* Test getting role parents recursively by using the current task dependency chain
* changelog
* download role from api response location
* include changelog fragment
* add unit test for role download url
Co-authored-by: Sam Doran <sdoran@redhat.com>
* ansible-galaxy: Clone git collections using shallow clones
This ensures the collection obtained via git url is a result of a
shallow git clone (git clone --depth=1). The git history of the
collection is not used by ansible, and as such, cloning the entire
history of the repo is unnecessary.
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* ansible-galaxy: Only perform shallow clones on non-versioned git urls
In general, the version can be anything we can checkout (branch, tag,
but also a commit hash). In particular for commit hashes we cannot
perform a shallow clone.
Err on the safe side and only perform shallow clones for non-versioned
git urls (cloning HEAD).
* galaxy-install: Make shallow cloning compatible with older Python versions
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* Pass args as a tuple
Add tests for the git command created from different repo and version combinations
* changelog
Co-authored-by: Tomas Babej <tomas@tbabej.com>
* add tests for fail filter
also tests that fail does not block inspectability
* add fail filter
fallback message is a bit clunky,
since you can't invoke a filter without specifying an input.
That is, "{{ fail }}" doesn't work,
so you have to do "{{ None | fail }}"
* document 'fail' filter
* add changelog fragment
* fail filter uses default message on Undefined or emptystring
makes it slightly easier to use the default message:
```diff
- "{{ None | fail }}"
+ "{{ '' | fail }}"
```
and the user sees a slightly more relevant message
if the message itself is undefined:
```diff
- The error was: {{ failmsg | fail }}: 'failmsg' is undefined
+ The error was: {{ failmsg | fail }}: Mandatory variable has not been overridden
```
* rebuild as the builtin `Undefined`
* harmonise `hint` parameter for make_undefined with jinja
* use code block for documentation item
[ref](https://github.com/ansible/ansible/pull/75435#discussion_r707661035)
* rename to `undef` to expose less Python into the Jinja
[ref](https://github.com/ansible/ansible/pull/75435#pullrequestreview-757799031)
* explicitly instantiate undefined value now that it's possible
see I knew we would break something with reflection
* preserve test coverage of undefined variable
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Change:
- Remove only user-facing use of ANSIBLE_ASYNC_DIR.
- Remove two comments saying to change things that, apparently, we
aren't going to change...
Test Plan:
- ci_complete
Tickets:
- Fixes#74139
- Fixes#74138
- Refs #74226
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable ansible-galaxy to specify client id override with Keycloak Token
* Specify ability to provide override of client_id
* Test client_id can be configured for individual servers
* Add issue link to changelog
* Document client_id as a config option and add an example
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* apt module: add option to allow package downgrades
* Add new option to module so users don't have to force downgrades which
is insecure and dangerous
* Add integration tests similar to upgrade integration tests
* Changelog
* Update changelog fragment
* Update changelogs/fragments/74852-apt-allow-downgrade.yaml
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
Co-authored-by: Amin Vakil <info@aminvakil.com>
* ensure dump produces json
* clog and tests
* remove library additions since they are not usable
* avoid any/all dirs
* ensure we dont use local dir as playbookdir
* If an exception occurs when getting a collection's metadata, continue to the next in the server list.
* Warn for unknown exceptions when finding versions of a collection
* Test that an invalid server is no longer fatal if a subsequent server has the collection
* Fix server for verify tests - compare checksums against the server from which it was installed
* Add tests for verify and fix that code path to mirror install/download behavior for server errors
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* pip - Use pip from the current Python interpreter.
If `executable` and `virtualenv` were not specified, and
the `pip` Python module is available for the current interpreter,
use that `pip` module instead of searching for a `pip` command.
* Add comment about needing `__main__` to run `pip`.
* Fix unit test.
* Add porting guide entry.
* Update changelog to match porting guide description.
ci_complete
* Make individual dirs that should exist instead of using shutil.copytree
* Test build_ignore when installing collections in git repos
* changelog
* Fix assertion
Fix git directory
* Fix nested content while building the collection dir
Test installing a collection from a git repo and artifact have the same result
Refactor to use variables
* Support templating _from_files with --extra-vars for imported roles
* Add tests for templating the filenames for import_role, import_playbook, and import_tasks
* Add documentation
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Test a passlib wrapped algo with a password lookup
* Fix error when passlib is used with a wrapped algo
The exception was:
An unhandled exception occurred while running the lookup plugin 'password'.
Error was a <class 'TypeError'>, original message: issubclass() arg 1 must be a class
and can be reproduced using the following command:
ansible localhost -i localhost, -mdebug -amsg="{{ lookup('password', '/dev/null encrypt=ldap_sha512_crypt') }}"
The concerned algo are: bsd_nthash, django_argon2, django_bcrypt, ldap_bcrypt,
ldap_bsdi_crypt, ldap_des_crypt, ldap_hex_md5, ldap_hex_sha1, ldap_md5_crypt,
ldap_pbkdf2_sha1, ldap_pbkdf2_sha256, ldap_pbkdf2_sha512, ldap_sha1_crypt,
ldap_sha256_crypt, ldap_sha512_crypt, roundup_plaintext
* allow vars_files to use dwim (vars/) and vaults
* also fixed bug with temp file cleanup, some pythons dont like it when you alter loop
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Rather than adding further complexity to the regex, preprocess the output to
remove any empty lines. Now the only purpose of the regex is to fix
wrapped lines.
Fixes#70949
* avoid getting delegated vars w/o inventory host
fixes#75512
In the case of imports, we don't have a host, so getting host vars for
the delegated host makes no sense and should be avoided.
* also avoid error on vars_files with per host vars
* test
* testing given case
* oops
* add 'file options' for become and connection pass
* implemented getting passwords from file or script
* added config entry
* fixed env var name and noted executable behaviuor
Co-authored-by: Rick Elrod <rick@elrod.me>
* Add new comment attribute to template plugin
Add comment_start_string and comment_end_string attribute to template
plugin
Co-authored-by: Hossein Zolfi <h.zolfi@inside.sahab.ir>
* user - Remove unused code.
* Replace deprecated abstractproperty decorator.
* Fix __all__ to be a tuple.
* Use a generator in subelements lookup.
* Use from import in basic.py
* Add changelog fragment.
* Fix selinux unit test.
* Improve error for InconsistentCandidate exceptions
* Add test case for installing a collection with an inconsistent version
* Add test case for installing a collection that has a dependency with an inconsistent version
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Add basic schema for attributes.
* Support attributes for version_added_collection adding mechanism.
* Add changelog fragment.
* Support new attribute properties from #74331.
* Require additional properties strictly.
* Fix a bug with the dnf module not using all components of a package name when filtering to determine if it's installed
* changelog
* Simplify splitting on the last '.'
* Update lib/ansible/modules/dnf.py
Push /usr/bin/python to almost the bottom of the barrel. This makes the strategy to prefer
specific versions of the "mystery meat" version.
* Change INTERPRETER_PYTHON default to 'auto'
Change description to match change in behavior.
Change deprecation message to a warning.
* Update docs
* Add porting guide entry
* Update unit tests
* Update integration test
* Allow INTERPRETER_PYTHON_FALLBACK to be configure using a variable
* Prefer platform-python above other Python 2 interpreters
* Add Python 3.10 to the list of interpreters
Matt Martz