* dont warn about using a yescrypt hash as password
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
* add changelog
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
* add yescrypt test
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
---------
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
* Fix locale related parsing error in git.py
This fixes the issue found at https://github.com/ansible/ansible/issues/77213 which got introduced by changing over to "best parseable locale" approach, but missing out the requirement of locales other than `L` having to have `LANGUAGE` specified as well. For further details, check this post within the issue discussion: https://github.com/ansible/ansible/issues/77213#issuecomment-1446919617
* Add changelog
Create 81931-locale-related-parsing-error-git.yml
Also update tests to support the format on modules/plugins
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Adds an option that can have an action plugin tell the module to ignore
options that do not fit its arg spec. This is to enable support for core
running modules that exist outside of the collection that may not be new
enough to support some of the options supplied to it.
* Reword the error message when the module fails to parse parameters
in JSON format
* misc typo fixes
Fixes: #81188
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Improve host_group_vars efficiency:
* normalize the basedir with `os.path.realpath()` once and cache it
* cache missing paths/files
* reduce the calls to `isinstance`
Add a couple more general improvements in vars/plugins.py get_vars_from_path():
* call `PluginLoader.all()` once for vars plugins and reload specific
plugins subsequently
* don't reload legacy/builtin vars plugins that are not enabled
Add a test for host_group_vars and legacy plugin loading
Co-authored-by: Matt Davis <mrd@redhat.com>
* changelog
* Add a new is_stateless attribute to the vars plugin baseclass
update integration tests to be quieter and use the same test pattern
Fix deprecation and adjust test that didn't catch the issue (deprecation only occured when the value was False)
move realpath cache to host_group_vars (do not smuggle call state as instance data)
refactor under a single 'if cache:' statement
Call os.path.isdir instead of always calling os.path.exists first. Just call os.path.exists to differentiate between missing and non-directory.
remove call to super(VarsModule, self).get_vars()
use the entity name as the cache key instead of variable location
Remove isinstance checks and use a class attribute just in case any plugins are subclassing Host/Group
Replace startswith by checking index 0 of the name instead, since host/group names are required
* rename is_stateless to cache_instance to make it more clear what it does
* add plugin instance cache using the path to plugin loader
reduce loading stage option if a new instance isn't created
don't require a known subdir on PluginLoader instantiation for backwards
compatibility
rename attribute again
contain reading from/initializing cached instances to a plugin loader method
* Deprecate v2 vars plugins
* Refactor to use the cache in existing plugin loader methods
Rename the attribute again
Refactor host_group_vars with requested changes
Make changelog a bugfixes fragment
Add a deprecation fragment for v2 vars plugins.
Add type hints
* unbreak group_vars
* Apply suggestions from code review
* misc tweaks
* always cache instance by both requested and resolved FQ name
* add lru_cache to stage calculation to avoid repeated config consultation
* handle KeyError from missing stage option
---------
Co-authored-by: Matt Davis <mrd@redhat.com>
When templating tags (which happens outside of standard `post_validate`) we
need to template each object in the inheritance chain and set the templated
values on those objects individually. That way when `task.tags` is called the
`extend` functionality properly picks up the templated values of all
parents into one flatten list.
Fixes#81053
revert to previous behavior to push vars to play at compile time
add `public` parameter to allow per import control of exporting (vs just the global config)
Co-authored-by: tchernomax <maxime.deroucy@gmail.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Provide more information about plugin usage which needs to be
used instead of the deprecated plugin in the deprecation message.
Fixes: #80561
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* The module systemd is renamed to systemd_service to maintain
the scope of the module. Mention this in the module description.
* Misc typo fixes.
Fixes: #80917
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
While creating a symlink, we apply fs attributes changes
to the destination. But in case of a non-existent destination, we warn
the user. This warning message can be suppressed by setting follow=false.
Add a note of this behavior in the `follow` parameter.
Fixes: #50543
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* ansible-galaxy fix scm dependency error
also changed usage of 'virtual colleciton' to actual type
avoid error by filtering out virtual collections that dont have
expected properties
simplified as per webknjaz
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Make sure paths are correct when building collection files manifest
This commit makes sure the path of the files part of
the collection build manifest are correct.
This commit uses os.path.commonprefix instead of
dealing with strings.
Signed-off-by: Carlos Camacho <ccamacho@redhat.com>
Bugfix Pull Request
Fixes: #81618
* Revert the change note type to `minor_changes`
* Clarify the change note with user-oriented details
---------
Signed-off-by: Carlos Camacho <ccamacho@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
PR #81606.
Prior to this patch, when `--pre` CLI flag was not passed, the
dependency resolver would treat concrete collection dependency
candidates (Git repositories, subdirs, tarball URLs, or local dirs or
files etc) as not meeting the requirements.
This patch makes it so pre-releases in any concrete artifact
references, and the ones being specifically pinned dependencies or
user requests, met anywhere in the dependency tree, are allowed
unconditionally.
This is achieved by moving the pre-release check from
`is_satisfied_by()` to the `find_matches()` hook, following the
Pip's example.
As a bonus, this change also fixes the situation when a collection
pre-releases weren't considered if it didn't have any stable releases.
This now works even if `--pre` wasn't requested explicitly.
Finally, this patch partially reverts commit
6f4b4c345b, except for the tests. And it
also improves the `--pre` hint warning to explain that it mostly
affects Galaxy/Automation Hub-hosted collection releases.
Ref #73416
Ref #79112Fixes#79168Fixes#80048Resolves#81605
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Symlinks in the collection might be pointing to non-existent
targets. Check and report the failure to the user.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* when --diff is used with content parameter, print destination
path instead of temporary file path.
Fixes: #79749
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* If all tasks in the role are skipped or unreachable, the role is not marked as complete for the host.
* Only mark the role as complete if a task in the role succeeds or fails for the host.
The `handler_templar` in `StrategyBase._process_pending_results` was
used for templating handler names which is no longer the case. Now it is
only used for templating `changed_when`/`failed_when`. To prevent
re-creating templars each time results are processed, just create it
on-demand when whens are processed.
* Before performing shredding the original file, check if the
destination file location is writable or not. This will
prevent corruption of original file.
Fixes: #81455
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
The phrase `Does nothing when used with other platforms.` sometimes appears before the supported platforms. Combine this phrase with the supported platform line.
* Add type hints to ansible.utils.display::Display
Fixes#80841
* Avoid circular import
* Fix sanity
* type hint some of the functions of the module?
* Fix units
* Not sure about this
* Fix some of the issues from reviews
* Add changelog
* ...
* Update lib/ansible/utils/display.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* remove py2 boilerplate
---------
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* changes made in become link
* change become link
* Update lib/ansible/plugins/action/__init__.py
---------
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Python 3.11.4 introduces a new parameter 'filter' in extract and
extractall in tarfile. Handle deprecation warning message emitted
in Python 3.12.
* added probing mechanism in ansible-galaxy code to detect broken
data filter implementation in tarfile.
Fixes: #80832
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Matt Clay <matt@mystile.com>
Improves the error checking when running PowerShell modules using
Ansible.ModuleUtils.Legacy. It will only return an rc of 1 if both the
PowerShell module runner signalled an error occurred and those error
records were present in the output. This should reduce some false
positive errors when using the older module style.
* apt: ignore fail_on_autoremove and allow_downgrade when using aptitude
* fail_on_autoremove (--no-remove) and allow_downgrade (--allow-downgrades)
parameters are only valid for apt-get and not for aptitude. Ignore them when
aptitude is detected and used.
Fixes: #77868
Document path_join behavior -
If a path component is an absolute path, then all previous components
are ignored and joining continues from the absolute path.
Fixes: #81446
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Gather value of password using debconf-get-selections command,
use this information for idempotency.
Fixes: #47676
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Document return values of template module
Add documentation for some return values of template module.
I have not tested this extensively, so it's possible that under special
circumstances this might not be entirely accurate, but briefly looking
at the code as far as I can tell they are correct.
Co-authored-by: mprasil <mprasil@mpT15p>
* Back out use of communicate, add better comments, add bufsize, and align with subprocess._communicate
* tests
* re-order logic slightly
* more comments
* loopty loop
* yet another comment
* Revert "yet another comment"
This reverts commit 96cd8ada5fa0441b92f2298bdaa6cb40594847d2.
* Revert "loopty loop"
This reverts commit 96ea066f6a7d18902c04a14f18dd79b38e56f5e7.
* ci_complete
* Copy in comment too
* Wording updates
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Back out bufsize
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* a recent optimization lost the unsafe lookup disable behavior when templating conditionals with inline templates that referred to untrusted values
* added regression test to catch this case
* iptables chain creation does not populate with a rule
fixes#80256
* Add changelog fragment
* Add rules and flush chain during integration tests
* Check chain rule on comment
* Update test/integration/targets/iptables/tasks/chain_management.yml
* User-provided collection type might differ from collection
source. Cross-check the type before proceeding
Fixes: #79463
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* keepcache parameter can not be set in repository file
instead it goes in yum.conf or dnf.conf
Fixes: #78693
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* service action: improve documenation about service and service
moudule used as backup and cli that confuses everyone
* Update service.py
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* ansible-galaxy - improve ignoring multiple signature status codes when using --ignore-signature-status-code
* fix backwards compatibility by adding a new plural option instead, and hide the singular from --help
* add type hints for DataLoader methods
* fix list of tuples
* remove mention of pre-2.4 vault method
* Add None return type
Use | instead of typing.Optional or typing.Union
* Remove datetime.datetime.utcfromtimestamp and datetime.datetime.uctnow
from controller code since they are deprecated in Python 3.12.
* Update target side code to use new utcfromtimestamp and utcnow utils in ansible.module_utils.compat.datetime that return aware datetime objects on Python 2.7 and 3.
Co-authored-by: Matt Clay <matt@mystile.com>
* update docker containers versions to use newer ansible-test ref in the pre-built venvs
* Allow invoking ansible-test with Python 3.12
* Add python3.12 to the INTERPRETER_PYTHON_FALLBACK
* changelog
* add Python 3.12 as a non-default Python version for the test containers
* Update mypy ignores for Python 3.12
* Add Python 3.12 to CI matrix for unit tests, generic tests, and galaxy
* Update unit test for using the Python 2 collection loader path with Python 3.
Skip the existing test on Python 3.12, since find_module is removed.
Suppress the pre-existing deprecation warnings using the Python 2
codepath with Python 3.
Add a test for Python >= 3.12, which doesn't call find_module.
* Ignore sanity test errors on systems without libselinux present.
* Only install collections which can't be satisfied by a collection in any of the configured paths.
* Improve warning for unexpected collection install path
Fix warning when path is configured, but is a pip-managed path
Normalize the path before validating to fix warning consistency
* Add test for 256-color configuration values
See #78607.
* color is not restricted to 16 choices
currently supports up to 256, not listing them all
TOOD: create examples and point to/list the basic 16
---------
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* replace: handle exception while parsing escape char
* Fail early when bad escape character is provided in replace module
Fixes: #79364
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Run tests in Python 3.6 or greater env
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
---------
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* first_found lookup, let lookup handle templating errors
Avoids case in which TE was not sending valid and templatable entries to the lookup
The lookup already handles the case TE was attempting to itself, so no need for this code anymore.
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Add type annotation for connection plugins
* Use new | syntax instead of Union/Optional
* Fix pep issue
* Use ParamSpec and other minor fixes
* Fix up ParmaSpec args and kwargs type
It adds exception treatment when execute a inventory based on script with the --host argument
---------
Co-authored-by: Everson Leal <everson.leal@sonda.com>
* Improve readability of unit test output
This drops the trailing `-expectedXXX` suffixes from test names generated by parametrize.
* Add more splitter unit tests
This fills in code coverage gaps in the exising unit tests.
* Bug fixes and code cleanup
- Fix IndexError exceptions caused by parsing a leading newline, space or escaped space.
- Fix an AttributeError exception in `parse_args` when parsing `None`.
- Fix incorrect parsing of multi-line Jinja2 blocks, which resulted in doubling newlines.
- Remove unreachable exception handlers in the `parse_kv` function.
The unreachable code was verified through analysis of the code as well as use of the `atheris` fuzzer.
- Remove unnecessary code in the `split_args` function.
- Add an optimization to `split_args` for the empty args case.
* Add unit tests for bug fixes
The splitter code is now fully covered by unit tests.
* Add another issue ref in changelog
this will handle escaping correctly in tempaltes that override the default jinja variable tokens
also fix optimization for single var path when overrides happen
* Avoid usage of deprecated importlib.abc.TraversableResources
This fixes ansible-compat test failures with Python 3.12.
* Add deprecated: marker for compat code
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* add declarative deprecation comment to < 3.9 case
Co-authored-by: Matt Clay <matt@mystile.com>
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
This fixes the issue where handlers notifying other handlers are
not properly run because the notification is not registered unless
another flush_handlers occurs. Instead, if the current host state
is iterating handlers we immediately register the handler to be
run so the notification is not lost.
Fixes#80880
* Check if skeleton is /dev/null while creating home folder
* Add test for linux
Co-authored-by: Holger Dörner <h.doerner@bitexpert.de>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* replace deprecated ast.value.s with ast.value.value
the s attribute is deprecated since Python 3.8 and emits a warning in
3.12 causing some test failures
* setup module, retry facter to handle --puppet errors
facter versions have changed how they deal with the --puppet flag
when puppet is not present, most versions will just ignore it and not error,
but initial versions of facter 4 changed the behaviour (later reverted).
fixes#80496
* 📝🐛 Fix RST list @ `ansible-console` manpage
Said CLI entrypoint docstring is used to construct the corresponding
manpage. `docutils` were producing the following warning with the
current RST markup used there, as follows:
:36: (WARNING/2) Bullet list ends without a blank line; unexpected unindent.
This change puts a portion of the first list entry on a separate line
which allows the RST parser stomach it without any problems and
produce correct output without gibberish in it.
* 🎨📝 Use RST inline code @ ansible-console doc
The single backtick syntax was causing things to be interpreted as
italics text. The inline code needs double backticks.
Instead of using Templar.environment in Templar.do_template for
accessing/mutating the environment, myenv local variable should be used
because it is the environment used for actual templating. It can either
point to Templar.environment or newly created environment overlay.
Fixes#80605
Atthe moment if a users wants to protect virtualenv_command by using
quotes around 'venv', module will fail out as literal parsing is used
at the moment. In order to imrpove things, argparse is leveraged to
parse out passed value to the virtualenv_command
Closes-Bug: #76372
Signed-off-by: Dmitriy Rabotyagov <noonedeadpunk@gmail.com>
* Don't include a default for CLI as it overrides config with lesser precedence
connection plugins should already use the same config entries (and more)
than DEFAULT_TIMEOUT.
fixes#66434
* removed flag setting from enable/disable on openbsd
service was changing permanent config on openbsd, which is not expected from the this module
* if in check mode, module should not stop at enable/disable
* simplify and clean up opensd service enable/disable
* does break for those that were using service for configuring flags
* Fix installing signed collections by using the fqcn, version, source, and type as a unique identifier.
Define __hash__ and __eq__ methods to handle Candidate/Requirement comparison excluding signatures which aren't fully populated until install time.
* Remove PinnedCandidateRequests since it is redundant now.
* Fix verifying against a signed remote when the keyring isn't configured
* Add TTY check and argument to disable it (#50603)
* Fix formatting
* add changelog
* rename flag and updated help description
* add tests for tty check
* replace deprecated uses of assertRaisesRegexp to assertRaisesRegex
* fix yaml syntax
* shorten line 79
* Revert "replace deprecated uses of assertRaisesRegexp to assertRaisesRegex"
This reverts commit cea5fe1655.
* change back to assertRaisesRegexp
* fix meaning of parallel in gather_facts
* Update docs with note about parallel not always being faster
* add 'smarter' usage of gahter_timeout for parallel tasks
* restore async when needed, not always
* added typing
* parallelism tests
* Add GALAXY_COLLECTIONS_PATH_WARNING option.
This allows users to disable warnings from `ansible-galaxy collection
install` about `--collections-path` missing from Ansible's configured
collections_paths.
* Only bypass type validation for null parameters if the default is None. A default is mutually exclusive with required.
* Prevent coercing None to str type. Fail the type check instead.
* Validate task attributes `run_once` and `action` with finalized attrs after individual loop results
* Validate task attribute `ignore_unreachable` using individual loop results
Once there's a way to post validate only certain fields, we can use self._task.post_validate() instead
This replaces the fix introduced in https://github.com/ansible/ansible/pull/80051.
Fixes#73643
* clear_notification method and simplify ifs
* Deduplicate code
* Limit number of Templar creations
* Fix sanity
* Preserve handler callbacks order as they were notified
* Symbolic modes with X or =[ugo] always use original mode (Fixes#80128)
Here's what's happening, by way of this mode example: u=,u=rX
At the first step in the loop, the "u" bits of are set to 0. On the next
step in the loop, the current stat of the filesystem object is used to
determine X, not the "new_mode" in the previous iteration of the loop. So
while most operations kind of operate left to right, "X" is always going
back to the original file to determine whether to set x bit.
The Linux "chmod" (the only one I've tested) doesn't operate this way. In
it, "X" operates on the current state the loop understands it is in,
based on previous operations (and starting with the file permissions).
This is an issue with "X" and any of the "=[ugo]" settings, because
they are lookups. For example, if a file is 755 and you do "ug=rx,o=u",
file module produces 0557 and chmod produces 0555.
This really becomes a problem when you want to recursively change a
directory of files, and the files are currently 755, but you want to
change the directory to 750 and the files to 640. In chmod you can do
"a=,ug=rX,u+w" (or "a=,u=rwX,g=rX"), and have it apply equally to the
directory and the files. I can't come up with a single way in the ansible
file module to deterministically, recursively, set a directory to 750
and the contents to 640 no matter what the current permissions are,
as the code currently is.
The fix is to pass in "new_mode" to _get_octal_mode_from_symbolic_perms
in lib/ansible/module_utils/basic.py inside _symbolic_mode_to_octal. And
then take "new_mode" as an argument and use it instead of the filesystem
object stat.st_mode value.
* Fixing my new unit test, fixing bug in test comments
Reduce the number of Galaxy API calls made during dependency resolution by fetching remote signatures afterwards, since these are not used in backtracking.
Reduce the verbosity to `-vvvv` (to match other Galaxy API calls) to see this activity.
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Ensure we default to show all tags when listing tags
'never' was being excluded by default,as it is not part of the 'run tags'
runtime default ('all'). For listing we now add it to the default 'run tags'.
* fix using templated values for include/import role options 'public', 'allow_duplicates', and 'rolespec_validate'
* pass templated values without changing the instance
* Fix templating by setting always_post_validate to True and calling IncludeRole.post_validate() instead
ci_complete
* add changelog
* Improve Ansible.Basic.cs tempdir uniqueness
The current tempdir naming scheme can result in the same name if the
remote worker starts at the same time as another. By using the process
id it should add enough uniqueness to avoid this situation.
* Fix sanity issues
* Fix up compile issue on older hosts
* password lookup, handle ident properly when saved
Currently we format and save ident when present but we didn't account for this when reading the saved file
Also added some more robust error handling.
* ansible-galaxy - support `resolvelib >= 0.5.3, < 1.1.0`
<https://pypi.org/project/resolvelib/1.0.1> released on 2023-03-09:
- <https://github.com/sarugaku/resolvelib/blob/main/CHANGELOG.rst#101-2023-03-09>
- <https://github.com/sarugaku/resolvelib/releases/tag/1.0.1>
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* Trigger CI by pinning resolvelib with latest version
Since resolvelib is pre-installed in our test containers, we should temporarily pin the latest version allowed to force the tests to run with that version. Once the tests have passed that commit can be reverted.
Please make those changes without force pushing, so that we keep the reference to the passing CI run. We can squash the commits when merging the PR so the temporary commits won't be in the final commit merged to the devel branch.
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* https://github.com/ansible/ansible/pull/80196#discussion_r1136003637
Also test resolvelib with multiple supported versions.
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* Revert "Trigger CI by pinning resolvelib with latest version"
This reverts commit 5518e5dbca.
---------
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* clog frag
* Fix retries so that each explicit call to _call_galaxy is retried for the correct number of attempts. Fixes#80174
* Extend retry logic to common URL related connection errors. Fixes#80170
* Extend retries to downloading artifacts
* Extend param docs for change
* Rework the exception handling
* Don't be overly broad, reduce to TimeoutError, and BadStatusLine for now
* _download_file needs to raise AnsibleError.orig_exc
* Remove unused import
* Add IncompleteRead
* Add socket.timeout for py39
* Add 502 to retry codes
* Move http error code checking first
* Use itertools.tee to replay the backoff_iterator instead of using a callable
* Actually set a CLI default of 60s for timeout, to prevent implicit galaxy from using 10s as default from Request.open
* Import typing
* fix type hints
* Use http.HTTPStatus instead of int HTTP error codes where feasible
* Split exception handling
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add missing import
---------
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Add more details about what "true" and "false" mean for the
force_basic_auth setting. Give example scenarios when clients may want
to use this setting.
* unarchive - properly handle relative path for dest
* Add integration test
* Return output of underlying commands with increased verbosity
* Revert "Return output of underlying commands with increased verbosity"
This reverts commit a2790c8275cdc5697b65670a0beffdc74b741bf6.
* Warn when a relative destination path was provided
* Create a queue per WorkerProcess to receive intra-task updates
* Update `pause` action to use the worker queue
* Deprecate ConnectionBase()._new_stdin
* Add new `Display` convenience method `prompt_until` to manage both controller- and worker-sourced prompting without cross-fork stdin sharing, in-worker mechanism to handle request-response over new worker queue.
* Add condition that causes a when to skip a task
* Fix up tests
* Use false_condition instead of failed_condition
* Remove formatting accidentially added
* Fix sanity
* ansible-inventory, add --limit option
* also graph
* optimize the when not providing limit
* added tests and clog
* avoid empty groups, fix tests
* i swear there as an ignore_errors there already!!?!?!?
* Fix stdout test fallback
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* simpler approach using subset against inventory directly
* tyupose
* renamed funciton to what it actuall does
got yaml in line with others about removing empty groups
* have graph ignore limts, also note same for --host
* fixed typo long line
* better test
* cause hosvars are not a thing
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Remove unneeded fallback code for nested defined/undefined tests
The code used to handle defined/undefined tests on objects containing
nested undefined variables. Due to changes in lazy evalution of Jinja2
expressions it is no longer needed, see #56116.
* Change ansible_job_id format
...to be something that does not resemble float or other type which
value could be changed by literal_eval that jinja2_native uses.
Specifically the format of '%d.%d' is converted from str to float and
then back to float which may result in truncating the number resulting
in the job not being found because the job id does not exist.
* Fix detection of available hashlib algorithms
Detection of hashlib algorithms now works on Python 3.x.
The new implementation works on Python 2.7 and later.
Test coverage is provided by both integration and unit tests.
* Add additional details about hashlib in docs
* Improving the documentation on how we generate the default value of the filename parameter
* fix pep8
* removing unnecessary documentation and improving the module's return
* making the RETURN docs
* pep8
* version_added and changelog
* module._diff
* module._diff fix
* add rudimentary tests for new outputs
---------
Co-authored-by: Matt Davis <mrd@redhat.com>
* Update `collections.abc` imports
- Use `six.moves` for modules and module_utils
- Use `collections.abc` for controller code
This avoids using `ansible.module_utils.common._collections_compat`,
which was added before the vendored `six` was updated to provide these
imports.
* Update _collections_compat to use six.moves
Also update the custom pylint rule to reflect this change.
* add null 'manifest' key to metadata for git repo collections containing MANIFEST.json
changelog
* set to Sentinel instead of None
* Test installing a collection in a git repo that contains a MANIFEST.json
* fix test
* Update changelogs/fragments/ansible-galaxy-install-git-src-manifest.yml
* Normalize deprecation records.
* Fix alias deprecations in suboptions.
* Report in which option an alias warning happened for suboptions.
* Add deprecation tests for suboptions.
* Also test deprecation in list of dicts.
* Adjust unit tests for toplevel alias deprecation field name change.
* Fix notes in regex_replace doc
The documentation says that this maps to 'regex.replace' when it should say 're.replace'. The distinction is important because of the third-party 'regex' module that has more features than the built-in 're' module. https://pypi.org/project/regex/
* Fix notes in re.search docs
The documentation says that this maps to 'regex.search' when it should say 're.search'. The distinction is important because of the third-party 'regex' module that has more features than the built-in 're' module. https://pypi.org/project/regex/
* Remove obsolete 'if'
The result dict returned by super.run() will never have a 'skipped' entry.
Without the if, it is more clear that wrap_async is actually defined at the end of the function.
* Remove more dead code.
The result dict will also not contain invocation.
* Remove notes.
* Add support for importlib.resources
* Remove the importlib.resources imports
* return the correct data
* Some code comments, and re-order for consistency
* Disallow traversing packages below an individual collection
* Add a traversable class for namespaces
* Re-use variable
* Utilize itertools.chain.from_iterable
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Simplify logic to check for packages from ansible loaders
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Just a generator expression, instead of a generator
* docstrings
* Add comment about find_spec for our namespaces
* Add some initial unit tests for importlib.resources
* normalize
* Utilize importlib.resources for listing collections
* collections_path is already in config, just use config
* install uses a different default for collections_path
* Remove unused import
* Remove duplicate __truediv__
* Bring back TraversableResources
* Apply some small suggestions from code review
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Remove cross contamination between plugin loader code and CLI code
* Remove unused import
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Fix ignoring certs when downloading tarballs
Fix ignoring certs when downloading a collection from a specific source that isn't in the configured servers list
* background threads writing to stdout/stderr can cause children to deadlock if a thread in the parent holds the internal lock on the BufferedWriter wrapper
* prevent writes to std handles during fork by monkeypatching stdout/stderr during display startup to require a mutex lock with fork(); this ensures no background threads can hold the lock during a fork operation
* add integration test that fails reliably on Linux without this fix
* Remove unused mock from test_password_already_created_encrypt
The _get_paths mock is never used in the
test_password_already_created_encrypt test case.
* Add test to assert the password file is not rewritten
If the password file already contains the salt and the hasing algorithm
does not use the ident parameter, the password lookup should not write
to the password file.
* Fix "changed" if using "encrypt" in password lookup
When using the "encrypt" parameter to the password lookup without the
ident parameter, the password file was always marked as "changed". This
caused the file to be rewritten with the same content. This is fixed by
only marking the file as changed, if an "ident" value needs to be added
to the file.
Fixes#79430.
Add changelog entry
* galaxy: Add license_file to manifest directives
* ag collection build: Test license handling
This adds tests to ensure that
- REUSE licensing files: .reuse/dep5, LICENSES/*, anyfile.license
- galaxy.yml license_file
are always included in the manifest.
* local connection: avoid tb when running in container with invalid user
* clog
* cannot use uid, leave empty and ~/ will resolve itself
* get back to what it did
* lib/ansible/executor/playbook_executor.py: fix minor typo in comment
* lib/ansible/executor/playbook_executor.py: move 'running playbook inside collection' from warning to -v debugging output
* give a warning when no roles match the search instead of returning rc 1
* porting guide note
* Update docs/docsite/rst/porting_guides/porting_guide_core_2.15.rst
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Use GalaxyAPI for isinstance check instead of RoleDistributionServer, since the latter is defined in __main__ sometimes (when running integration tests or ansible-galaxy from source) and importing from ansible.cli.galaxy won't reference the same object.
* issue #79022 file parameter import_tasks and include_tasks
* adding details back
* Update lib/ansible/modules/import_tasks.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/modules/include_tasks.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* use pipe as in original version
Co-authored-by: Felix Fontein <felix@fontein.de>
* fix empty str default for config/base.yml config options by rendering '' instead of ``''``
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* copy module - fix copying directories containing modified subdirs with remote_src=True. Previously, the first changed subdir would prevent recursively checking for changes for in subdirs at the same level.
* Fix reporting changed for copying empty directories with remote_src=True. If a directory is created on the remote but nothing else, changed is True.
* ensure we only use shorthand for assigned values
* fix ansible-config std display
* split baby
* killed offending whitespace
* moved to func instead
* the revenge of whitespace
* Custom salt for ansible-vault encrypt
add VAULT_ENCRYPT_SALT config
add salt testing
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Piotr Stawarski <p@stawarski.pl>
* Fix installation from source transforms symlinks of dirs to empty dirs
* Add test to check symlinks to dirs are respected when installing from source
* Add changelog for collection install from source symlink to dirs issue
* Ensure that we do not squash keywords in validate. Fixes#79021
* become_user: nobody should only apply to the test tasks, not the setup_test_user role
* Update how become_user is specified
* Add test to ensure keyword inheritance is working for become
* Add clog frag
* Cache fattributes to prevent re-calculation
* ci_complete
* Remove unnecessary getattr
After changes:
```
"ansible_locally_reachable_ips": {
"ipv4": [
"127.0.0.0/8",
"127.0.0.1",
"192.168.0.1",
"192.168.1.0/24"
],
"ipv6": [
"::1",
"fe80::2eea:7fff:feca:fe68",
...
]
},
```
192.168.1.0/24 is a local prefix, where any IP address inside this range
is reachable locally (or outside this host if this prefix is announced via
EGP/IGP).
Signed-off-by: Donatas Abraitis <donatas.abraitis@hostinger.com>
* Use python re to parse service output instead of grep. Fixes#78541
* Add clog frag
* Add an extra guard to abort if rc is 4, and /etc/init.d is missing
* ansible-vault: add newline to the output of the 'encrypt_string' command
* Add comment with issue description and improve string joining
* Add changelog fragment
* Update changelogs/fragments/79017-ansible-vault-string-encryption-ending-with-newline.yml
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Fixed some typos in changelog fragment
* Improve comment regarding newline character at the end of file
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Do not crash templating when filter/test name is not a valid Ansible plugin name.
* Store and re-raise KeyError if there was one.
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* omit keyword should reset to context
ensure we use context/inheritance when calculating value,
using default only when context is unavailable.
fixes#75692
- correct 'vars:' precedence to allow phasing out of include_params
- actually merge vars and always include role_vars
- avoided dupe deps from giving wrong vars
- use 'first' instance of dep as others are from previous instances/invocations
and can have diff values for vars
- ensured deps only provide exportable vars themselves
- added COMMENTS
- added tests
- apply export restrictions setting to defaults
- use 'public' as cutoff
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
ci_complete
* winrm - fix reducing 'Read timed out' error
This fix reduces the exceptiopn 'Read timed out. (read timeout=nnn)' occurring seemingly randomly and
unpredictable on some networks (which have long latency and/or many hops or other problems). The fix
ensures that the involved timeout is set to a value so this issue is reduced.
* Add changelog fragment
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Add --offline option to 'ansible-galaxy collection install' to prevent querying distribution servers
This allows installing/upgrading individual tarfiles to have dependency resolution.
Previously needed to be done manually with --no-deps or else all collections and dependencies needed to be included in the requirements.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add OSMC to Debian OS_FAMILY_MAP
- os_family fact of the Debian-based OSMC distribution was not detected correctly
* tweak changelog
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* new _fqcn attribute to plugin objects
* unbreak plugins in subdirs
* Fix inadventent changes to _load_name and use existing vars
* add plugin aliases and name property, and replace plugin._load_name where incorrect
* Fix listing plugin names
Fix listing legacy and builtin together
test deprecated plugin documentation
fix doc extensions
remove sometimes inaccurate _load_name handling from plugin.name
* Add tests for REJECT_EXTS and doc extensions
Fix unpredictable collection redirects so non-fqcns in the redirect list are guaranteed to be legacy (instead of determined by the collections keyword)
Move aliases and name properties to _update_object so all plugin types, including doc fragments, can use them
* make legacy plugin names internally consistent
* rename attributes to ansible_name and ansible_aliases
This fixes bug which prevents identification of broken playbook when
passing multiple playbooks as arguments, something that is common
especially with `--syntax-check`.
Affects: ansible-lint
* Recover unix load average into an ansible_loadavg fact
The average load is available out of the box with os.getloadavg().
It adds useful context to the existing ansible_memory_mb and ansible_mounts
facts that provide size and utilization or memory and disk space.
* Add an integration test
* Fix distro fact handling for Flatcar
The existence of the file /etc/flatcar/update.conf depends on
bootstrap configuration typically provided by the user. For that
reason this file is unsuitable for determining distro facts for
Flatcar Container Linux.
The distribution_release fact is meaningless in the case of Flatcar
since Flatcar doesn't have named releases. The distribution_version
fact, however, IS meaningful and should contain a number such as
"3139.2.0".
- Use /etc/os-release instead of /etc/flatcar/update.conf.
- Drop the distribution_release fact.
- Set the distribution_version fact.
- Update distro test fixture for Flatcar
- Generate the fixture using gen_distribution_version_testcase.py.
- Override result.distribution and result.os_family manually as the
generator script gives wrong values.
- Use a recent Flatcar version.
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
* Limit Galaxy API calls during ansible-galaxy collection dependency resolution when possible
Installing a tarfile with a dependency from a Galaxy server (e.g. dependencies: {'ns.coll': '>=1.0.0'}) does not get the available versions of the dependency from the galaxy server if a sufficient version is already installed.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* refactor and remove redundant code in documentation
allow location and building api to be more accessible
fix issues with displaying ansible.legacy and ansible.builtin
ensure we don't x2 process tokens (some modules reference them also) fixes#77764
move to constants vs hardcoded
more informative errors and comments
now have actual filter/test plugins, which expose the filter/test functions
moved filter/test loading/finding logic into jinja2pluginloader, removed dupe implementations
added tests for case in which we unique by basename when listing
Update lib/ansible/utils/plugin_docs.py
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Add a warning for collections that are attempting to be autoloaded to no effect
* Deprecate REQUIRES_WHITELIST and add support for REQUIRES_ENABLED so the docs are accurate
* Fix documentation
* add more vars plugin tests
* Simplify code and add a FIXME for another bug
* fix precedence
* Make setting the class attr at all a warning, even if it's True
* Add fun parsing for _load_name
* include _load_name in messages
* add required file to collection skeleton
This file is required to be able to upload a collection.
It is present in
https://github.com/ansible-collections/collection_template/blob/main/meta/runtime.yml
but that does not get used by default.
Without this, if you use the "ansible-galaxy collection init" command
and you try and publish that collection without adding this file, you
get the error:
"ERROR! Galaxy import process failed: 'requires_ansible' in
meta/runtime.yml is mandatory, but no meta/runtime.yml found (Code:
UNKNOWN)"
Also updates relevant test and adds a changelog fragment
* yum_repository: Do not set default value for async
The `async` repository option is deprecated in RHEL 8, but Ansible sets
```ini
async = 1
```
even when it's omitted from the module options, which causes `dnf` to complain
about an unknown configuration option.
This commit removes the default value from the `async` parameter, which means
it won't be added to the repository file if omitted from the module parameters.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
* Document the specific versions with the default True and the option deprecation for RHEL 8
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
Replace get_persistent_connection_options with get_options
Remove special case for network sub_plugin in _set_plugin_options
Try to avoid mock connection pretending to be persistent
Rename variables->options to reflect what they actually are
Gather options for ssh_type_conn on network_cli
Drop reliance on sub_plugin["type"]
* ansible-config init fixes
now handles --disabled correctly for 'vars' format
also does not display internal config entries anymore
fixes#78438
note plugins have _prefixed settings that are public, this should change
* Don't actually update the cache in check mode
Add tests for updating the cache in check mode
* Don't mark packages as manually installed in check mode and add a test for it
* changelog
* Update test/integration/targets/apt/tasks/apt.yml
* fix test
* update attributes of files that are symlinks
* update attributes of files that are hard links
* fix default state in documentation
* remove unnecessary suppression
* add to changelog
* Rethread pr/70185 through the dependency resolver
Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath
Don't error while listing collections if a collection's metadata is missing keys required for building a collection.
Give an informative warning if metadata has been badly formatted.
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Report OpenSuSE >= 15 as opensuse
Make distro.id() report newer versions of OpenSuSE (at least >=15) also report
as opensuse. They report themselves as opensuse-leap.
* Add a test
* Fix KeyError for ansible-galaxy when caching paginated responses from v3
* changelog
* generate responses in loop for test
Co-authored-by: Matt Martz <matt@sivel.net>
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes#78214
* Write non-ascii via script instead of static file
* Use contexlib.closing to support py27 context manager
* Use executable from task, instead of shebang
* Update encoded bytes to utf16
- `processor_count` was erroneously set to the number of cores
- `processor_cores` was erroneously set to the number of threads per core
- `processor_vcpus` and `processor_threads_per_core` were not set
- `processor` was a string, while it's supposed to be a list
Before:
```
"ansible_processor": "PowerPC_POWER7",
"ansible_processor_cores": 4,
"ansible_processor_count": 12,
```
After:
```
"ansible_processor": [
"PowerPC_POWER7"
],
"ansible_processor_cores": 12,
"ansible_processor_count": 1,
"ansible_processor_threads_per_core": 4,
"ansible_processor_vcpus": 48,
```
Also add a unit test.
Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
* listify_lookup_plugin_terms deprecate dataloader
deprecated useless dataloader pass to function
also removed from callers in core
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
I'm not aware of a way to easily get vault secrets decoded on the
ansible-connection side without sending the vault secrets over the
connection in the same way, so just decode them for transport.
* Fix workding for :ref|term: subsitution
now matches 'seealso' and makes more sense:
```
delay:
applies_to:
- Task
description: Number of seconds to delay between retries. This setting is only used
in combination with `until`.
```
vs
```
delay:
applies_to:
- Task
description: Number of seconds to delay between retries. This setting is only used
in combination with website for `until`.
```
* updated unit tests
* match see also
* more sanity
* Update dpkg_selections.py to include an syntax example showing how to remove package holds
* Fix pep8 issue: dpkg_selections.py
Missed some trailing spaces when adding example text.
* Move undefined check from concat to finalize
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.
Fixes#78156
ci_complete
* Fix sanity
* ...
* sigh
* [dnf] Fix skip_broken, add test coverage
Change:
- skip_broken was set in config but not actually used in calls to
base.install()
- added a lot of test cases with specialized repo
- got rid of external (docker repo) nobest test cases since the
specialized repo works well for those too
- Slight cleanup and adding comments in dnf module
Test Plan:
- ci_complete
Tickets:
- Fixes#73072
Original-author: Rick Elrod <rick@elrod.me>
* Use a better test for checking results list
ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
##### SUMMARY
The sample value given for the 'mode' parameter is shown without quotes, but the data type is string. If you actually try to use an unquoted numeric string for this value you're in for a nasty surprise! I added quotation marks to the sample value.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* ansible-galaxy configurable timeouts
- also fixed issues with precedence,
so --ignore-certs now overrides config
- made galaxy_timeout generic setting,
if set, it becomes default for server configs,
but now specific servers can override
- updated tests or added notes (some tests ignore/override precedence)
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Use ctypes to access shadow, instead of spwd which is deprecated
* Remove sanity ignore
* cast to str to satisfy mypy
* Add typing.cast polyfill
* Appease mypy redefine
* Add clog frag
* Remove .travis.yml files from galaxy init skeletons
These files have existed for many years but in light of the recent
circumstances around Travis, we should not include them by default in
the galaxy init skeletons.
* Add breaking_changes changelog fragment with recommendation to use a custom skeleton if the .travis.yml file is still needed.
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* Fix 'Permission denied' in user module while generating SSH keys
Fix#78017
Use try/except for spwd usage to prevent "Permission denied".
Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* facts: Add uptime fact for NetBSD
This is copied from the current OpenBSD facts code.
Tested on the GCC compile farm: https://cfarm.tetaneutral.net/
* Added changelog fragment
Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Add configurable option for paramiko banner_ssh timeout
This PR adds a configurable option for paramiko connection
to control the value of banner timeout in paramiko library.
The banner timeout value can be set in ansible.cfg file as
The default value of banner timeout is set to 30 seconds in
paramiko connection plugin.
```
[paramiko_connection]
banner_timeout = 30
```
or using enviornment variable
```
export ANSIBLE_PARAMIKO_BANNER_TIMEOUT=30
```
* Fix CI failure
* Fix review comments
* Fix review comment
Add a new handler class ZipZArchive to use unzip -Z as an alternative to zipinfo
Run 'unzip -Z' in can_handle_archive so we fall back to the next handler if it's not available (failing in is_unarchived is too late)
* Add a test for unzip -Z when zipinfo is not available
* Update test for missing binary altogether by removing /usr/bin from the PATH
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1
Test incompatibilities are removed for resolvelib >= 0.6.0
Test against the latest 0.8.x version and fix requirements
* Fix tests - use a venv for testing the range of resolvelib versions
* Update temporary hardcoded fallback for ansible-test
* Update hardcoded upperbound for sanity tests
* Make error check more flexible
checksum can also accept a checksum only file (no filename beside the checksum).
fixes#54390
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
* apt: fix virtual package install version detection
Change 4a62c4e3e4 introduced version
matching in installation.
The problem stems from
if version_installable or version:
pkg_list.append("'%s=%s'" % (name, version_installable or version))
When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.
For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with
$ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...
In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.
The tests are updated to install and remove this package.
Fixes: #76779
* Fix traceback when a supported version of resolvelib is not installed
Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).
* Add tests for unsupported resolvelib versions
* Resolve remaining import sanity test issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults
Add a toggle for action plugins to prefer the resolved module when loading module_defaults
Allow moving away from modules intercepted as actions pattern
Fixes#77059
* winrm, psrps added missing var entry
this handles issue with the default being set to inventory_hostname
but defaults not being templated implicitly
fixes#77841
* rename systemd module to services only
disambiguates what it handles since systemd is now much more
that a service manager, but the module is specific to services
* Expand ansible-doc to tests/filters and fix existing issues
enable filter/test docs if in single file or companion yaml
add docs for several filters/tests plugins
allow .yml companion for docs for other plugins, must be colocated
verify plugins are valid (not modules, cannot)
fix 'per collection' filtering
limit old style deprecation (_ prefix) to builtin/legacy
start move to pathlib for saner path handling
moved some funcitons, kept backwards compat shims with deprecation notice
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Prevent losing unsafe from lookups
This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.
Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar
Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.
The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.
It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.
For more examples see unit and integration tests included in this patch.
Fixes#77535
* git fix docs and wrapper script
fixes#77582
now env var is set to wrapper or full command depending on version
as was the intent of previous PR
added ref to git commit from git for why/how we used the env vars
* handle key_file
When looking up the `no_log` setting for a parameter that is an alias in
`AnsibleModule._log_invocation()`, the alias value will always be an
empty dictionary since `self.aliases` on the `AnsibleModule` instance is
never updated after initialization. Since the `no_log` setting is on the
canonical parameter not the alias, an incorrect warning is issued if the
parameter matches `PASSWORD_MATCH`.
This PR returns the aliases dictionary as an attribute of the
`ValidationResult` and updates the `aliases` attribute on the
`AnsibleModule` instance.
* Revert "Revert "Config, ensure templating happens at functions (#77483)""
This reverts commit 94c9106153.
* removed update configdata, which is unused
* removed test for action we don't perform anymore
* removed unused configdata
move away from special templating in constants.py
simplified main get, unified paths to get info
* Let native jinja deal with eval for type
* make constants available for templating defaults
* ensure we use config file
* ansible-connection verboistery
for cli, just use normal parser creation
this also adds --help, but that seems fine
also some error cleanup
Co-authored-by: Nathaniel Case <this.is@nathanielca.se>
* replace hardcoded '~/.ansible' to C.ANSIBLE_HOME
* rename previously existing env ANSIBLE_HOME in env-setup script
* modify cache dir monkeypatching in galaxy api unit tests
* update "version_added" to 2.14 for ANSIBLE_HOME
* fix description of collections with proper use of ANSIBLE_HOME
Co-authored-by: htol <github@h-tol.net>
Co-authored-by: stefanwascoding <stefan@syntaxhelden.de>
* Proper error on missing jinja2 override separator
Properly catch the case when no (or a wrong) separator is used in a
jinja2 override, and return a useful error message to the user.
* Support colons in jinja2 override value
By limiting the split to 1, any colons in the value are preserved and
passed on.
* Fall back to implicit namespace.name from the path if the metadata is invalid
* Test listing a collection with null namespace/name/version fields in its galaxy.yml
* Add option --no-fail-on-errors to return errors for ansible-doc --metadata-dump in JSON result instead of failing.
* Adjust changelog fragment.
* Add basic tests.
* Support ignoring of certificates for ansible-galaxy during SCM cloning
* Add integration tests installing a role from an untrusted repository
Test installing the role without --ignore-certs fails
Test installing the role with --ignore-certs is successful
* Add a toggle to control the number of signatures required to verify the authenticity of a collection
* Make the default number of required valid signatures 1
* Add option to make signature verification strict and fail if there are no valid signatures (e.g. "+1")
* Use a regex to validate --required-valid-signature-count
* Add a toggle to limit the gpg status codes that are considered a failure
* Update documentation and changelog
* Add unit and integration tests for the new options
* Fixes#77146
Fix using user-provided signatures when running 'ansible-galaxy collection verify ns.coll --offline'
Add a test for a user-provided signature when running ansible-galaxy collection verify with --offline
Fix displaying overall gpg failure without extra verbosity
Add a test for displaying gpg failure without verbosity
Improve documentation to be more clear that signature verification only currently applies to collections directly sourced from Galaxy servers
For IPv6 addresses, Ansible already provides the prefix length for IP
addresses in the `prefix` fact. This patch adjusts the facts for IPv4
addresses to also contain the prefix length in the prefix fact. This
makes it easier to use the facts consistently when the CIDR notation is
needed.
Signed-off-by: Till Maas <opensource@till.name>
* Add more sanity tests on schema level (so they also work for plugins).
* Fix various issues the sanity test reported.
* Add changelog fragment.
* Fix function name.
* Fix ansible console to respect ANSIBLE_STDOUT_CALLBACK
This makes the ansible-console able to customize the stdout callback.
Ansible console currently uses statically minimal callback.
* Add changelog
* Move the fragment changelog in the right directory
`with_first_found` only takes a list of things, it's not like `tags`
where it can take a thing or a list of things.
Inspired by locally discovered breakage and then finding
https://github.com/ansible/ansible/issues/77136 where someone was also
doing this and getting into trouble.
Also, bump the default. A number of people have reported problems with
this 20 second timeout for a couple of years on
https://github.com/ansible/galaxy/issues/2302
It is mentioned in the issue that resolving dns can sometimes take up to
30 seconds, and that this timeout includes dns resolution.
Includes a changelog message
* apply owner/group permissions to top folder
* remove unused var unarchive30
* fix permissions for top folders if the tarball include multiple top folders
* added test for top folder ownership
* fix facts delegation loop overwrite
partial revert of change to allow facts to be present in each loop iteration
was not needed in final results as result processing alreayd had the disctiontion
and ended up breaking the assumptions in the calling code.
fixes#76676
* play_context, compensate for existing plugins
some connection plugins are not fully using the correct configuration,
but this was previously hidden from them as play_context was providing
the info instead, now play_context provides the 'correct' info, but hitting
these bad configurations.
* ansible-galaxy collection install|verify:
- Support verifying the origin of the MANIFEST.json when the Galaxy server has provided signatures.
- Allow supplemental signatures to use during verification on the CLI/requirements file.
* ansible-galaxy collection install:
- Support disabling signature verification. This silences the warning provided by ansible-galaxy if the Galaxy server provided signatures it cannot use because no keyring is configured.
- Store Galaxy server metadata alongside installed collections for provenance. This is used by 'ansible-galaxy collection verify --offline'.
* Add unit tests for method that gets signatures from a Galaxy server
* Add integration tests for user-provided signature sources
- Test CLI option combinations
- Test installing collections with valid/invalid signature sources
- Test disabling GPG verification when installing collections
- Test verifying collections with valid/invalid signature sources
* Make signature verification advisory-by-default if signatures are provided by the Galaxy server
- Make the default keyring None
- Warn if the keyring is None but the Galaxy server provided signatures
- Error if the keyring is None but the user supplied signatures
- Error if the keyring is not None but is invalid
* changelog
* add ansible-galaxy user documentation for new options
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Andy Mott <amott@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
* Return rc=0 on success.
Error handling in playbooks generally expects `rc` to be set to 0 when a module has not failed. Playbook authors should not have to check for the existence of `rc` first.
* Use single definition and added changelog
* Fix up tests with new return value
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* add DebianStrategy tests
* ensure hostname can be changed by using become
* use Systemd strat for debian and Base for generic.
* add test to ensure all strategies are available
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
avoid polluting vars with incorrect settings
simplify variables and templars
- variables, original, only updated with final results, reset are copies of this
- tempvars used for preliminary templating
- cvars used for connection/shell/become plugins, delegation aware
- vars_copy ignore tempvars updates and use connection plugin to get
finalized version per loop item/delegation, also used to store temp results
till we are ready to update 'variables'
- fine tune nolog just cause we are here
- also fix inventory_hostname_short for IP addresses
git module now uses env vars exclusively
- updated docs to clarify usage
- now env vars append instead of overwrite to allow existing custom setups to keep working
fixes#38104, #64673, #64674
- added note for hostkeychecking more securely
fixes#69846
- keep script cause old versions still choke on env
- env var cannot hold more than 'command' for older versions
- all ssh_opts in one place
ssh plugin, use 'correct' information source in all cases
* still fallback to pc
* added inventory to new test
* undef var can still show as parser error on pc
now task_exectuer has a more accurate error handling
* ansible-test - Use relative paths in junit output.
Also fix a traceback in the junit callback during automatic fact gathering.
* ansible-test - Handle out-of-tree JUnit paths.
* updated tests to conform to new block inheritance
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* minor refactor in other options by pushing common code into functions
* consolidate coll_filter
* more normalizing loader
* dont pass plugin_loader, its global import
* Also dump roles and collections
* adjusted tests to new err msg
* disable namespace filter (unused)
Aditya Putta