* Add additional tests for check_mode with shell/command/script
* update check_mode documentation: if a module/action is not skipped in check mode and accurately reflects whether a change is made to the remote support should be 'full'
* Make reporting skipped in check mode mutually exclusive with 'changed: True'
* Add missing documented attributes
* Fix tests to expect skipped=True and changed=True do not occur together
* Fix script check_mode support documentation
* Fix earlier changelog
* document platforms attribute
* Use tasks's check mode since the value from PlayContext does not reflect loop items
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
'dscl -list' returns 0 even if the user does not exists. This
leads to errorenous condition in user module.
Using 'dscl -read UniqueID' can return if user exists or not.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure the version is valid for directories and collections in git repos before installing
Fix the error message for invalid semantic versions
* Make requested changes
* Add a test case for unhandled ValueError exception
* Add changelog
* Update lib/ansible/galaxy/collection/galaxy_api_proxy.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Reword error message
Include link to learn how to compose a semver version
* Move version validation into the caller, find_matches
* Add tests for more invalid version types
* Remove unused import
Fix raising unexpected error
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* fixes#28977
* Include the original error message when pick_handler fails
* Add a test to unarchive a tar file with an invalid extra option
* add a changelog
* check if service is indirect status / add integration test
* Update changelogs/fragments/76453-indirect-systemd-status.yml
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify indirect check / remove integration test restriction on rhel9
* add dummy indirect service for integration tests
* unarchive: fix non-english locales
For GNU Gettext, the LANGUAGE environment variable takes precedence over LANG or LC_ALL. On systems where LANGUAGE was set to a non-english locale, the output of the tar command therefore not understood and the module failed silently ("changed": false, but the archive was not extracted).
* add tests
* changelog
* catch the case that cowsay is broken
fixes https://github.com/ansible/ansible/issues/72582
add changelog
raise Exception for broken cowsay
add test for broken cowsay
Co-authored-by: Matthias Bernt <m.bernt@ufz.de>
* reset defaults for json filters
these had change on unification of json parsing/dumping
now they behave like before, but are still controllable by user.
* Implement find_spec and exec_module to remove reliance on deprecated methods in the collection loader
ci_complete
* Move module execution to exec_module
Remove extra sys.modules handling
Use default module initialization by returning None from loader.create_module
Refactor
ci_complete
* Remove ansible-test's copy of the collection loader
ci_complete
* Fix metaclass for Python 2.x
ci_complete
* Fix Py2/Py3 syntax compatibility
* Refactor
ci_complete
* update collection_loader comments
ci_complete
* simplify find_module
ci_complete
* Fix Py2 compatibility - don't get loader from nonexistent spec
Remove unnecessary PY3 checking
* Refactor common code in load_module and exec_module
ci_complete
* tidy diff
ci_complete
* Include collection_loader in target paths for 'compile' sanity test
* add changelog
* Add "return None" instead of doing it implicitly
Remove get_filename
short-circuit exec_module if it's a redirect
ci_complete
* Add changelog fragment and integration tests
* Break up function signature into multiple lines
It passed the 160 character width. Change it to multi-line for easier reading
and better diffs in the future.
* Remove unused import
* Update version added for new feature
* Move changelog fragment to correct location
Co-authored-by: Thomas Stähle <thomas.staehle@check24.de>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add script to freeze sanity requirements.
* Declare sanity test requirements and freeze
* Use pinned requirements for import.plugin test.
* Expand scope of import test for ansible-core.
* Add ignores for galaxy import errors.
* Update test-constraints sanity test.
* ansible-test - Add support for RHEL 8.5 instances.
* Clear libdnf checksum cache from unsigned package in dnf integration test.
Co-authored-by: Matt Martz <matt@sivel.net>
* First pass of adding yaml result format output to default callback
* Add clog frag
* flow style always false
* Further normalization of results across distros
* no lossy, now pretty
* Fix env var in runme.sh
* Rename variable to better self document
* include NativeJinjaUnsafeText
* Linting cleanup
* Add tests specific to the prior yaml callback
* Make data munging more exact, following the checks in libyaml/pyyaml
* Remove unused import
* Extend comment
* more correct and some comments
* Fix consistency of tab vs space in this section of the file
* Add str representer
* be cooler
* faster character filtering
* Clarify None as a sentinel for default behavior
* Flip filtering logic to avoid CRLF issues with 'space break' filtering
* Py3 change and comment about the use of __call__
Co-authored-by: Matt Davis <mrd@redhat.com>
* `podman inspect` falls back to a same-named image if the named container is not present; since eg `http-test-container` is both the name of the image and container, it wasn't working properly under podman in many instances. Switching to `docker|podman container inspect` limits the query to containers only for both podman and docker, allowing the support container detection/creation to work properly.
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
* Skip recursive suboption validation if sub_parameters is not a dictionary
* Ensure sub parameter elements is a sequence to prevent iterating over string characters and causing duplicate error messages for the same param
* Try to find any valid vault id before raising an exception
* Apply the same logic during vault password file loading
* Also. Improve the warning msg and tests
When we introduced an include parameter to the unarchive module, we
inadvertenly flipped the exclusion logic. This flip meant that the
unarchive module started rejecting files that should be extracted.
This commit flips the bad logic and adds some tests that will make
sure things do not go bad again.
* Use NativeEnvironment for all templating
ci_complete
* Keep Templar.copy_with_new_env for backwards compat
* Mention that AnsibleUndefined.__repr__ changed in the porting guide
* Templar.copy_with_new_env backwards compat
* ci_complete
* Overhaul ansible-test integration tests.
* ansible-test - Fix import test pyyaml usage.
* ansible-test - Remove unused import.
* ansible-test - Fix traceback when pip is unavailable.
* ansible-test - Fix typo in port forwarding message.
* ansible-test - Fix controller logic in requirements install.
* Fix unit tests in ansible-test integration test.
Unit tests are now run for available Python versions which
provide `virtualenv` (Python 2.x) or `venv` (Python 3.x).
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Prepend the task name to the warning "The loop variable 'item' is
already in use", so that you can get some context even if stdout and
stderr go to separate places.
- The containers are now based on Ubuntu 20.04 instead of Ubuntu 18.04.
- The default Python version used for the containers is now Python 3.10.
- The included version of PowerShell was upgraded to version 7.1.4.
- The included version of shellcheck was upgraded to version 0.7.0.
ci_complete
Resolves https://github.com/ansible/ansible/issues/75320
The option `--prime-containers` was chosen over `--docker-pull-only` to match the recently added `--prime-venvs` option for sanity tests.
It would also fit well with a future `--prime-requirements` option for pre-installing requirements for unit and integration tests.
* ansible-test - Create injector scripts at runtime.
* Set bootstrap.sh shebang at runtime.
* Remove shebang and execute bit from importer.
* Update shebang sanity test.
* Preserve line numbers.
* Use the task dependency chain to compile the role instead of the parent role's _parents list
* keep existing role inheritance, but limit it to roles in the current task dependency chain
* Test getting role parents recursively by using the current task dependency chain
* changelog
* download role from api response location
* include changelog fragment
* add unit test for role download url
Co-authored-by: Sam Doran <sdoran@redhat.com>
* ansible-galaxy: Clone git collections using shallow clones
This ensures the collection obtained via git url is a result of a
shallow git clone (git clone --depth=1). The git history of the
collection is not used by ansible, and as such, cloning the entire
history of the repo is unnecessary.
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* ansible-galaxy: Only perform shallow clones on non-versioned git urls
In general, the version can be anything we can checkout (branch, tag,
but also a commit hash). In particular for commit hashes we cannot
perform a shallow clone.
Err on the safe side and only perform shallow clones for non-versioned
git urls (cloning HEAD).
* galaxy-install: Make shallow cloning compatible with older Python versions
Signed-off-by: Tomas Babej <tomas@tbabej.com>
* Pass args as a tuple
Add tests for the git command created from different repo and version combinations
* changelog
Co-authored-by: Tomas Babej <tomas@tbabej.com>
* add tests for fail filter
also tests that fail does not block inspectability
* add fail filter
fallback message is a bit clunky,
since you can't invoke a filter without specifying an input.
That is, "{{ fail }}" doesn't work,
so you have to do "{{ None | fail }}"
* document 'fail' filter
* add changelog fragment
* fail filter uses default message on Undefined or emptystring
makes it slightly easier to use the default message:
```diff
- "{{ None | fail }}"
+ "{{ '' | fail }}"
```
and the user sees a slightly more relevant message
if the message itself is undefined:
```diff
- The error was: {{ failmsg | fail }}: 'failmsg' is undefined
+ The error was: {{ failmsg | fail }}: Mandatory variable has not been overridden
```
* rebuild as the builtin `Undefined`
* harmonise `hint` parameter for make_undefined with jinja
* use code block for documentation item
[ref](https://github.com/ansible/ansible/pull/75435#discussion_r707661035)
* rename to `undef` to expose less Python into the Jinja
[ref](https://github.com/ansible/ansible/pull/75435#pullrequestreview-757799031)
* explicitly instantiate undefined value now that it's possible
see I knew we would break something with reflection
* preserve test coverage of undefined variable
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
Change:
- Remove only user-facing use of ANSIBLE_ASYNC_DIR.
- Remove two comments saying to change things that, apparently, we
aren't going to change...
Test Plan:
- ci_complete
Tickets:
- Fixes#74139
- Fixes#74138
- Refs #74226
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable ansible-galaxy to specify client id override with Keycloak Token
* Specify ability to provide override of client_id
* Test client_id can be configured for individual servers
* Add issue link to changelog
* Document client_id as a config option and add an example
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* apt module: add option to allow package downgrades
* Add new option to module so users don't have to force downgrades which
is insecure and dangerous
* Add integration tests similar to upgrade integration tests
* Changelog
* Update changelog fragment
* Update changelogs/fragments/74852-apt-allow-downgrade.yaml
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
* Update lib/ansible/modules/apt.py
Co-authored-by: Amin Vakil <info@aminvakil.com>
Co-authored-by: Amin Vakil <info@aminvakil.com>
* ensure dump produces json
* clog and tests
* remove library additions since they are not usable
* avoid any/all dirs
* ensure we dont use local dir as playbookdir
* If an exception occurs when getting a collection's metadata, continue to the next in the server list.
* Warn for unknown exceptions when finding versions of a collection
* Test that an invalid server is no longer fatal if a subsequent server has the collection
* Fix server for verify tests - compare checksums against the server from which it was installed
* Add tests for verify and fix that code path to mirror install/download behavior for server errors
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* pip - Use pip from the current Python interpreter.
If `executable` and `virtualenv` were not specified, and
the `pip` Python module is available for the current interpreter,
use that `pip` module instead of searching for a `pip` command.
* Add comment about needing `__main__` to run `pip`.
* Fix unit test.
* Add porting guide entry.
* Update changelog to match porting guide description.
ci_complete
* Make individual dirs that should exist instead of using shutil.copytree
* Test build_ignore when installing collections in git repos
* changelog
* Fix assertion
Fix git directory
* Fix nested content while building the collection dir
Test installing a collection from a git repo and artifact have the same result
Refactor to use variables
removing out of date example ansible.cfg, show user how to generate from ansible-config
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Support templating _from_files with --extra-vars for imported roles
* Add tests for templating the filenames for import_role, import_playbook, and import_tasks
* Add documentation
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Test a passlib wrapped algo with a password lookup
* Fix error when passlib is used with a wrapped algo
The exception was:
An unhandled exception occurred while running the lookup plugin 'password'.
Error was a <class 'TypeError'>, original message: issubclass() arg 1 must be a class
and can be reproduced using the following command:
ansible localhost -i localhost, -mdebug -amsg="{{ lookup('password', '/dev/null encrypt=ldap_sha512_crypt') }}"
The concerned algo are: bsd_nthash, django_argon2, django_bcrypt, ldap_bcrypt,
ldap_bsdi_crypt, ldap_des_crypt, ldap_hex_md5, ldap_hex_sha1, ldap_md5_crypt,
ldap_pbkdf2_sha1, ldap_pbkdf2_sha256, ldap_pbkdf2_sha512, ldap_sha1_crypt,
ldap_sha256_crypt, ldap_sha512_crypt, roundup_plaintext
Rather than adding further complexity to the regex, preprocess the output to
remove any empty lines. Now the only purpose of the regex is to fix
wrapped lines.
Fixes#70949
* avoid getting delegated vars w/o inventory host
fixes#75512
In the case of imports, we don't have a host, so getting host vars for
the delegated host makes no sense and should be avoided.
* also avoid error on vars_files with per host vars
* test
* testing given case
* oops
* add 'file options' for become and connection pass
* implemented getting passwords from file or script
* added config entry
* fixed env var name and noted executable behaviuor
Co-authored-by: Rick Elrod <rick@elrod.me>
* Add new comment attribute to template plugin
Add comment_start_string and comment_end_string attribute to template
plugin
Co-authored-by: Hossein Zolfi <h.zolfi@inside.sahab.ir>
* user - Remove unused code.
* Replace deprecated abstractproperty decorator.
* Fix __all__ to be a tuple.
* Use a generator in subelements lookup.
* Use from import in basic.py
* Add changelog fragment.
* Fix selinux unit test.
* Improve error for InconsistentCandidate exceptions
* Add test case for installing a collection with an inconsistent version
* Add test case for installing a collection that has a dependency with an inconsistent version
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Add basic schema for attributes.
* Support attributes for version_added_collection adding mechanism.
* Add changelog fragment.
* Support new attribute properties from #74331.
* Require additional properties strictly.
* Fix a bug with the dnf module not using all components of a package name when filtering to determine if it's installed
* changelog
* Simplify splitting on the last '.'
* Update lib/ansible/modules/dnf.py
Push /usr/bin/python to almost the bottom of the barrel. This makes the strategy to prefer
specific versions of the "mystery meat" version.
* Change INTERPRETER_PYTHON default to 'auto'
Change description to match change in behavior.
Change deprecation message to a warning.
* Update docs
* Add porting guide entry
* Update unit tests
* Update integration test
* Allow INTERPRETER_PYTHON_FALLBACK to be configure using a variable
* Prefer platform-python above other Python 2 interpreters
* Add Python 3.10 to the list of interpreters
* Make undefined variables in handler names non-fatal if the handler is not used
* If the handler has no way to be notified (i.e. the name can't be templated and the handler has no listen topics), display a warning
* Add tests for variables in handler names
* changelog
* Fix when evaluation on Native Jinja and Python 3.10
* Add unit test
* Add explaining comment
* Enable jinja2_native before tests
Co-Authored-By: Matt Martz <matt@sivel.net>
* Sanity
* Return native template module instead of modifying globals
Co-authored-by: Matt Martz <matt@sivel.net>
* Make sure AnsibleModule positional arguments are validated.
* Extract ANSIBLE_MODULE_CONSTURCTOR_ARGS with inspection.
* Remove no longer necessary return value.
* Fix PR #.
* argument_spec might not have been specified, as in community.general.xenserver_facts.
* Fix typo.
* Add packaging to requirement of ansible-test
Fix#75353
After requires_ansible field was added as mandatory to runtime.yml
file, ansible-test fails to check this field if it doesn't have
packaging module.
[1] https://github.com/ansible/galaxy-importer/pull/124
* Update network user guide to point to ansible.utils.cli_parse module as ansible.netcommon.cli_parse is deprecated from ansible.netcommon 2.0.0 release
* modules moved to use best_parsable_locale
* fixed invocations
* better better
* also module_utils
* converted to function as per fb
* patch testt
* whitespace
* set host_key_checking check to False, rather than if not (which captures False and None)
* add host_key_checking default to ssh.py / update documentation
* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play
* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list
* Add validation for action_group metadata and a toggle to disable the warnings
* Handle groups recursively referring to each other
* Remove special-casing for non-fqcn actions in module_defaults groups
* Error for actions and groups in module_defaults that can't be resolved
* Error for fully templated module_defaults
* Add integration tests for action_groups
* Changelog
Since the task and connection both have the same 'retries' keyword, the task default
would override the connection value.
Do not pass 'retries' from the task to the connection options.
* Set ssh_connection retries default value back to 0
It was 0 before the move to config and was changed to 3 by accident.
* module_common - handle cache directory creation collision
Occasionally multiple workers can try to create the cache directory if it exists. Catch the exception
and handle it.
* Just re-raise if it doesn't exist rather than trying again
* Use exists_ok rather than handling an exception
* Remove unused import and unused variable
* Go back to try/except but with OSError
Since we do not have split controller remote, this needs to be able to run in Python 2 in order
to be merged currently.
* The resolved_action is the formatted version of the final plugin in the PluginLoadContext's redirect_list
* Collection plugins are represented as FQCN
* Legacy plugins are represented with only the plugin name
* Add tests
* Changelog
* Fix struture of cli/doc.py snippet code.
A couple releases ago, cli/doc.py was modified to mostly conform to the
data processing pipeline steps. format_plugin_doc() was the biggest
exception in that refactor. When the snippet code was made generic
instead of being only for modules, the new code should have conformed to
the data processing pipeline too.
* Move the decision to output a snippet to the run() method alongside
the decision to output a listing versus plugin_docs.
* Move the test for invalid plugin_types to the run() method as it
affects all snippets in this run, not just a single snippet. (-t can
only be specified once)
* Rename get_snippet_text() to format_snippet() as:
* This is the data formatting step
* The format_snippet() name matches with its conceptual sibling,
format_plugin_doc().
* Use ValueError inside of format_snippet() to flag unrecoverable errors
formatting a single snippet.
* Emit a warning when format_snippet() raises ValueError and continue to
the next snippet.
* If the yaml(?) or toml inventory plugin is specified for snippet output,
raise ValueError() so that the user sees a warning instead of simply
seeing blank output.
* Do not modify arguments passed into format_snippet(). This is the
formatting step so data should not be modified.
* Change _do_yaml_snippet() and _do_lookup_snippet() to operate side
effect free.
* Fix raising of exceptions when formatting requred options for snippets.
* Unrelated: Use to_text() instead of to_native when calling
display.warning(). to_native() is used for raising exceptions. Not
for display methods.
* Add a changelog
specially for when you have parameters in unicode but need
to scrape responses, C is still the fallback
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Can now specify plugin/plugin type for list and dump actions
New 'init' action to create usable config sources
Co-authored-by: Felix Fontein <felix@fontein.de>
* Change tests to pytest-style tests
* Add tests for invalid hosts
* Validate host inputs
- check for empty values
- check for None
- check for values that are not a sequence and are not strings
- add unit tests
* Move play name setting to get_name() and out of load()
* Add _validate_hosts() method
By defining this method, it gets called automatically by FieldAttributeBase.validate().
* Improve the error handling code
Rather than multiple return paths, have a single return and set the message based
on the type of failure.
* Add another test for non-specific failures
* Reorganize tests so failure tests are in one tasks file
* Remove os.stat() call and add changelog
* subversion - set LC_ALL for accurate command output parsing
When LC_ALL is not set, the output language of commands will differ based on locale. There
is a lot of history of trying to fix this. See the following pull requests:
https://github.com/ansible/ansible-modules-core/pull/4358https://github.com/ansible/ansible-modules-core/pull/4358
This patch attempts to fix this my setting LC_ALL to a UTF-8 locale. Setting LC_ALL to C reintroduces this bug https://github.com/ansible/ansible-modules-core/issues/4178.
I'm sure there are some problems I am not seeing with setting this to en_US.UTF-8, but that is
the only way I could find to fix this bug without reintriducing the bug mentioned above.
* Rather than setting locale, just check for matches before trying to get groups
This is a pragmatic solution to avoid the stack trace since setting the locale correctly
to ensure message parsing is accurate is problematic.
* Improve regexps for finding revision and URL
Since moving to distro, it is possible to return this information for all platforms, not just Linux.
Also return version information for all platfrom not just Linux.
Update unit tests.
Remove some duplicate unit tests though I think there are more to remove.
* Fix docstring formatting
* Minor docstring changes
* Mock distro.id for Solaris service test
* Update comment
Brian Coca