Prior to this change, when the dependency resolver started looping over multiple versions of the same collection due to backtracking, it might take a lot of time to consider and disregard tens or hundreds of versions. But to the end-user, it looks like “nothing is happening, the program is *stuck*”. Even worse if such a time-consuming backtracking hits multiple collections and it “hangs” for longer cumulative period of time.
This patch improves the perceived responsiveness by printing out informational messages with the current status whenever the backtracking for a collection happens for the first, the eighth and the thirteenth times. The last message also reminds them that they can interrupt the process and attempt to adjust the constraints.
In debug mode, it also shows what caused conflicts leading up to candidate rejections. But this is only available with `resolvelib >= 0.9.0`.
The improvement is heavily inspired by https://github.com/pypa/pip/commit/9731131.
PR #81709
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Removes the use of pexpect in the winrm connection plugin and rely on
just subprocess. In the past pexpect was used for macOS compatibility so
that it could handle the TTY prompt but after testing it seems like
subprocess with `start_new_session=True` is enough to get it reading
from stdin on all platforms. This simplifies the code as there's no
longer an optional library changing how things are called and will work
out of the box.
Fix up bug when attempting to run any module on a Windows host that has
been configured with WDAC and Dynamic Code Security in audit mode. This
does not enable WDAC support with signed scripts so Ansible will still
not pass the audit events but it no longer fails to run.
* Ensure implicit flush_handlers have a parent block
To avoid getting tracebacks when calling methods like ``get_play()`` on
them.
* task needs to be copied
* copy only when necessary
No the file name that caused the error will be apparent
---------
Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <Akasurde@redhat.com>
* Fixed various become-related issues in `local` connection plugin.
* Fixed various issues in `sudo` and `su` become plugins.
* Added unit and integration test coverage.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
* Link to amazon.aws.aws_ec2 inventory plugin.
* Mention that refresh_inventory does not update the selected hosts for a play.
* Address comments by bcoca.
* Apply suggestions from code review
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Previously, requirement version specs starting with `!=` were
incorrectly considered as pinned release requests because the
comparison was being made against a one-char string while the
operator is two-char. This patch changes the check to test against `!`
which is enough to detect this case.
* rc-status commands returns unwanted lines with service names
and their status. Skip such lines while parsing service names
Fixes: #84512
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* apt-key module updates due to debian removal
Still kept for now for backwards compat, but removing from testing when not present
And adding more explicit mesasges to errors to point to new module
* added docs and error msg
* clog
* aslkdfj
* no docs to document doc changes
* also add warning to apt_repository
* clog on apt_repo too
* fix string concat
* Apply suggestions from code review
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
---------
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Improves the logic for parsing CLIXML values in the stderr returned by
SSH. This fixes encoding problems by having a fallback in case the
output is not valid UTF-8. It also can now extract embedded CLIXML
sequences in all of stderr rather than just at the start.
When a user doesn't exist and user module is used to create the user and the
homedir, adduser is called which parses HOME_MODE from /etc/login.defs, and when
not set calculates the mode from UMASK from the same file.
When a user already exists without homedir, and the user module is used to add a
home dir, it incorrectly ignores HOME_MODE, resulting in a world-readable home
dir when UMASK is not set. This is for example the case in Debian trixie and
later, and likely Ubuntu 25.04 and later.
Signed-off-by: Lee Garrett <lgarrett@rocketjump.eu>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
prompt now only errors if stdin is specifically triggered and not due to lack of other args
fixes#84489
---------
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* fix[doc.py]: path will be undefined and a direct reference will throw an UnboundLocalError.
If none of the files in files exists, path will be undefined and a direct reference will throw an UnboundLocalError.
Repair function file parameter type annotation is inaccurate
* Update changelogs/fragments/fix-cli-doc-path_undefined.yaml
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
---------
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Fixes:
Traceback (most recent call last):
File "/home/shertel/ansible/lib/ansible/plugins/vars/host_group_vars.py", line 139, in get_vars
FOUND[key] = found_files = loader.find_vars_files(opath, entity_name)
^^^
UnboundLocalError: cannot access local variable 'key' where it is not associated with a value
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/shertel/ansible/lib/ansible/plugins/vars/host_group_vars.py", line 151, in get_vars
raise AnsibleParserError(to_native(e))
ansible.errors.AnsibleParserError: cannot access local variable 'key' where it is not associated with a value
Also remove redundant check from tqm
Now covers module output (set_fact/include_vars)
Includes play objects at any stage (tasks that error were not covered)
Added tests, moved them to role structure
* Added docstrings to V2 methods in the CallbackBase Class (4 & 5 of 27)
* Made corrections as requested by webknjaz.
* Cleaned up whitespace issues.
* Corrections to customization note for review by webknjaz.
* Added rtype to return in docstrings.
* Simplified docstrings.
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
gather_facts, fix network_os and smart logic and defaults
setup will be default for smart only if network_os is not set, now you get warnings and errors when missing a valid facts module for a network os
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify redundancy with AnsibleModule set_*_if_different methods
* simplify copying a source directory to a dest directory without modifying behavior
* Allow user to control the trimming of blocks while templating
* Added tests
* Updated documentation and example
Fixes: #75962
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Clamp zip timestamps to representible values when unpacking zip files on
platforms that use 32-bit time_t (e.g. Debian i386). This is a
non-issue in practice (in 2024), but should allow the test suite to pass
on Debian i386.
We use a round value of 2038-01-01 00:00:00 for simplicity, and to avoid
running into timezone offsets closer to the actual limit.
MR #81520 introduced sanity-checking tests that used dates not
representable with a 32-bit time_t.
* ssh: Test reset_connection with templated ansible_ssh_executable
Add failing test to confirm subsequent fixes are necessary & sufficient.
* ssh: Fix reset_connection with templated ansible_ssh_executable
Signed-off-by: Alex Willmer <alex@moreati.org.uk>
Open the double quotes at the beginning of the entire `failed_when` value.
Individual conditions work well, yet fail each time they are combined with a logic `or` (as per the example).
Double quoting the entire string solved the problem.
This prevents "[WARNING]: Reset is not implemented for this connection" when
using wait_for_connection with localhost or other local hosts.
It's arguable (from a consistency/correctness perspective) that
`ansible.plugins.connection.local.Connection.reset()` should call
`Connection.close()`. I went for a no-op on the basis of "if it aint broke
don't fix it", and erred on the side of keeping existing semantics. However
either option would be fine with me.
Fedora 41 no longer has python3-rpm installed by default either,
so package_facts blows up on Fedora 41 hosts unless you make sure
python3-rpm is installed first. Not sure we can do a lot about
this besides extending this note.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
Fixes the logic when attempting to become the SYSTEM user using the
runas plugin. It was incorrectly assumed that calling LogonUser with the
SYSTEM username would produce a new token with all the privileges but
instead it creates a copy of the existing token. This reverts the logic
back to the original process and adds in new logic to avoid any tokens
that are restricted from creating new processes.
though the previous docs were the 'intended' outcome,
the current behaviour has been active for a long time and should
not be removed due to backward compatibility issues.
One thing we can do going forward is deprecate substitution enabled by default.
* Update resolvelib upperbound to 2.0.0
Minor SemVer releases should not contain breaking changes
* Test the latest minor release and reduce number of resolvelib versions tested for efficiency
* Update the documentation for check_required_by
* Fix return value for check_required_by (now returns empty list on success)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Removed deprecated pycompat24 and importlib
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Make CI green
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ignore basic.py
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Make CI green III
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Make CI green IV
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
---------
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Several tasks within the modification_time.yml and state_link.yml task
lists have explicitly enabled diff_mode because these tests previously
assumed a diff attribute would always be returned from the file module.
While showing the deprecation message, mention the collection name
from which the module is removed.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* fixes for CVE-2024-8775
* propagate truthy `_ansible_no_log` in action result (previously superseded by task-calculated value)
* always mask entire `include_vars` action result if any file loaded had a false `show_content` flag (previously used only the flag value from the last file loaded)
* update no_log tests for CVE-2024-8775
* include validation of _ansible_no_log preservation when set by actions
* replace static values with dynamic for increased robustness to logging/display/callback changes (but still using grep counts :( )
* changelog
* use ternary, coerce to bool explicitly
In ``Handler.notify_host`` we ensure that ``Handler.notified_hosts`` can contain
particular host at most once. Therefore for removing a host it should be
faster to use ``list.remove`` which removes the first item in the list,
as opposed to using list comprehension removing all such items.
* Update triple single quotes to triple double quotes
This change was fully automated.
The updated Python files have been verified to tokenize the same as the originals, except for the expected change in quoting of strings, which were verified through literal_eval.
* Manual conversion of docstring quotes
* Fix installing collections|roles from git repos with GALAXY_IGNORE_CERTS
* Fix installing collections from git repos with --ignore-certs
* Update unit test
* Add test case
This greatly reduces run time on large inventories since meta tasks are
executed in the main process sequentially and just executing them is expensive.
This change avoids running the following implicit meta tasks:
* ``flush_handlers`` on hosts where no handlers are notified
* ``noop`` for the linear strategy's lockstep, instead hosts that are
not executing the current task are just not part of the current host loop
A playbook consiting of two simple plays both running on ~6000 hosts
runs in:
devel: 37s
this PR: 1.3s
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Since we don't really care about the type we don't have to
query for it and just retrieve the filename value.
Unfortunately we cannot use module_utils.urls.get_response_filename
as we don't have the response object, so just utilize
email.message.Message to parse the filename
instead of manually doing the work ourselves.
Fixes: #83690
* INVENTORY_IGNORE_EXTS stop ignoring ini
Originally added to avoid configuration files for inventory scripts
now clashes with the much more common ini inventory files.
* add a note to script plugin
* Update lib/ansible/plugins/inventory/script.py
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* facts: Skip path if the distribution path is directory
Skip path if the distribution path is directory instead of file.
Handle exception raised while handling distribution path.
Fixes: #84006
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Review requests
---------
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
The following Python output proves that the second member of the resulting list should be " a", not "a":
```txt
$ python3
Python 3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> string = "jojo is, a"
>>> string.split(',')
['jojo is', ' a']
```
* Add a mount_facts module capable of gathering mounts skipped by default
fact gathering
* By default, collect mount facts from standard locations including
/etc/mtab, /proc/mounts, /etc/fstab, /etc/mnttab, /etc/vfstab, and on AIX,
/etc/filesystems.
When no file-based source for the current mounts can be found
(like /proc/mounts), the module falls back to using mount as a source.
This allows BSD and AIX to collect the existing mounts by default, without
causing Linux hosts to use both /proc/mounts and mount output.
* Non-standard locations and "mount" can be configured as a sources.
* Support returning an aggregate list of mount points in addition to first
found.
When there are multiple mounts for the same mount point in an
individual source, a warning is given if the include_aggregate_mounts
option is not configured.
* Add options to filter on fstypes and devices (supporting UNIX shell
wildcards).
* Support configuring a timeout and timeout behavior to make it easier
to use the module as a default facts module without risking a hang.
* Include the source and line(s) corresponding to a mount for easier
debugging.
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* add descriptions for `ansible-galaxy install` and `ansible-galaxy role|collection install`
* fix the usage for installing roles and collections together and include collections in the description for -r
Closes#81159
Co-authored-by: Alan Rominger <arominge@redhat.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Instead of re-using the token used in impersonation, this change will
create a new token for the SYSTEM account as returned by LogonUser. The
benefits of this is that the token will contain the full privileges for
the SYSTEM account rather than potentially one that has restricted
privileges we used during impersonation. It should also help avoid
problems on Windows that fails on status 0x0000016F when the
impersonated token during become was from a process that is restricted
from creating sub processes.
* Fact gathering fix 'no shm' branhc
* Use concurrent.futures instead of multiprocessing
This entirely avoids the need for fallback logic since the concurrent.futures thread pool does not depend on `/dev/shm`.
Co-authored-by: Matt Clay <matt@mystile.com>
* add a loop_control break_when directive to break out of a loop after any item
* remove loop var as normal exit would
* example usage:
- name: generate a random password up to 10 times, until it matches the policy
set_fact:
password: "{{ lookup('password', '/dev/null', chars=character_set, length=length) }}"
loop: "{{ range(0, 10) }}"
loop_control:
break_when:
- password is match(password_policy)
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
* Add DaemonThreadPoolExecutor impl
* Provide a simple parallel execution method with the ability to abandon timed-out operations that won't block threadpool/process shutdown, and without a dependency on /dev/shm (as multiprocessing Thread/Process pools have).
* Create module_utils/_internal to ensure that this is clearly not supported for public consumption.
* psrp - Remove extras lookups
Removed the extras variable lookups for the psrp connection plugin. All
valid options are already documented and the extras functionality is
slated to be deprecated at a future point in time. This should have
affect on existing user's playbooks.
* Fix up sanity tests and add explicit boolean conversion test
Adds the datastore details to the parser error when attempting to
include tasks that contain include_tasks without a filename set. This
change will now display the exact location of the include_tasks that
failed like any normal syntax error.
Expands the test matrix used for testing on Windows to cover the three
connection plugins we support for all the tasks. This change also
changes how raw commands are run over SSH to avoid starting a
`powershell.exe` process that was uneeded in the majority of cases used
in Ansible. This simplifies our code a bit more by removing extra
Windows specific actions in the ssh plugin and improves the efficiency
when running tasks.
Added configuration options, including environment variables to control the polling
No-Issue
---------
Signed-off-by: James Tanner <tanner.jc@gmail.com>
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Rely on dnf.base.remove, no special handling isn't needed,
let the dnf internals figure out what is needed to be done.
This is more in line with what dnf cli does.
* "already installed" in Exception (if it is even a thing) should be
caught by special exceptions like MarkingError or CompsError. This
appears to be a historic check that is no longer needed.
Supersedes: #83295
* Fix up raw_params for ansible.windows modules
Fixes up the logic for detecting if using ansible.windows.win_command or
ansible.windows.win_shell with _raw_params. These two modules are
special in that they can be referenced in 4 different ways but the
ansible.windows collection specific prefix needs to be manually added to
the list.
* Fix up sanity issue
Currently we match the load name, which can be an fqcn, but most users expect the 'naked' name
Now plugins can declare that name by setting _extras_prefix property or fallback to 'non fqcn' if no extras prefix
* ssh and psrp - Support more complex chars in fetch_file
Fixes the psrp and ssh (with piped) fetch function to work with paths
that contains glob like characters in the path. For Windows this was
needed when using paths that contain `[]` in the path. For ssh this was
a problem with FreeBSD when using the piped transfer method with similar
characters.
Also tidies up the psrp logic to not inject the paths and buffer size
in the script but pass it as an object through an argument/parameter.
* Fix sanity check
Ensure we force mtime/atime update when using copystat
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* fix creating file in directory with setgid bit
* add a test using the copy module's content option to create a file in a directory with setgid bit
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Enable validation of subkeys in rpm key module
A gpg subkey may change while the primary key remains the same. Due to
this behavior, there are situations where validation of the primary gpg
key fingerprint is not sufficient because the desired target is actually
the gpg subkey. This change allows the user to validate against either
the fingerprint of the primary gpg key or its subkey.
Signed-off-by: Kellin <kellin@retromud.org>
* Improve tests, add multi-fingerprint
- Improve tests to cover all cases
- add multi fingerprint validation
Signed-off-by: Kellin <kellin@retromud.org>
add caplevel to display to pass through
also reverse dict order as 'last update wins'
added tests ... and also log severity to log
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* winrm - quota retry handling
Add a retry attempt when receiving ERROR_WSMAN_QUOTA_MAX_OPERATIONS when
starting a command. This can occur when running a loop with multiple
iterations or an action plugin that runs multiple commands.
* Update pywinrm constraint for test
* Add verbose hint and mark test as destructive
Removes the docs for the environment keyword in the shell base plugins
as they are a no-op in the plugins themselves. The environment value is
provided by the action base which gets it from the action base on the
task.environment value. This should avoid confusion around how its being
set and removes some code that is not used at all.
While upgrade process removes a package, module should
report changed=True instead of changed=False
Fixes: #46314
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* lift code that normalizes value type for boolean vtype to cover both
branches of conditional.
* remove obsolete and incomplete conversion of type in set_selection.
Fixes: #83594
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Previous change overlooked 'uncommenting' the variable entry as a way to update this to keep the functionality.
Co-authored-by: Glandos <bugs-github@antipoul.fr>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* dnf: follow-up on is_newer_installed arches fix
* fix for the non package object case
* prevent early bailout in dnf _is_newer_version_installed
* non-installed available arches would fail out of the check early
---------
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
Using this dictionary to store the return values results in
the return values showing up in the returned
`invocation['module_args']`, which is confusing. It also causes all
module arguments to be returned, which is preserved by this change but
should ideally be removed in the future.
previouslly we recorded but did not show to avoid spam
since we could not dedup from forks, that was already
fixed in another PR so now we can show/display them.
Also:
* funcitonalize deprecation msg construct from docs
* reuse formatting func in cli
* normalize alternatives: most of the code used intended plural
but some and most data/tests used the singular
* update schemas and tests
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Replace references to archived ansible/ansible-examples repository
* Update examples to reference ansible repository
* Include HTTPS among SSH references in examples
Previously, `support_discard` simply returned the value of
`/sys/block/{device}/queue/discard_granularity`. When its value is `0`, then
the block device doesn't support discards; _however_, it being greater than
zero doesn't necessarily mean that the block device _does_ support discards.
But another indication that a block device doesn't support discards is
`/sys/block/{device}/queue/discard_max_hw_bytes` being equal to `0` (with the
same caveat as above). So if either of those are `0`, set `support_discard` to
zero, otherwise set it to the value of `discard_granularity` for backwards
compatibility.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
Identical branches in conditional node: 'elif name != permanent_hostname' and 'else'. First bruch is not nessesary
Co-authored-by: mar.s <marchenkov.s.d@gmail.com>
Previously, if the checksum of the downloaded file did not match the
specified checksum, the *destination* file was removed. This possibly
leaves the system that is being provisioned in an invalid state.
Instead, the checksum should be calculated on the temporary file only.
If there's a mismatch, delete the *temporary* file, not the destination
file.
This requires checking the checksum before moving the file.
When vault password file env variable is set to blank,
this value is converted to CWD and passed for further
processing.
Check if ANSIBLE_VAULT_PASSWORD_FILE is not a directory.
Fixes: #42960
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
also make the YAML booleanization the same as spec/JSON/module function
previous 'aproximation' was missing several options.
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Improves the Add-Type temporary directory handler to include a retry
mechanism and not fail on an error. Deleting a temporary file used in
compilation is not a critical error and should improve the reliability
of Ansible on Windows hosts.
* Fix task.resolved_action for callbacks when playbooks use action or local_action
* Fix using module_defaults with 'action' and 'local_action' task FA and add a test case
Fixes#81905
If user provides checksum URL which requires basic auth,
then module parameter username and password is used to
download the checksum file.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Correct a code comment in the PlaybookExecutor class that explains
the host failure condition under which the PBE breaks out of the
run. This comment previously stated that the maximum failure
percentage was checked, but that logic has been refactored into
the linear strategy plugin.
* Documentation on the timeout keyword
* wording changes
* keyword description wording edit
* removed whitespace at the end of lines.
* Changed to more accurate wording
---------
Co-authored-by: Niko Mlynarčik <536617@mail.muni.cz>
Abhijeet Kasurde