Commit Graph

3443 Commits (9f16bdbdda930c9a7bf1f030d7c8855833ee4b58)

Author SHA1 Message Date
Jordan Borean ad79c1e0d0
Add support for json in adhoc -a (#78114)
Co-authored-by: Sandra McCann <samccann@redhat.com>
2 years ago
Matt Clay d8fefba20e
ansible-test - Update locale logic to match core. (#78389)
Now that core requires UTF-8 filesystem encoding, ansible-test does as well.

Additionally, the `en_US.UTF-8` or `C.UTF-8` encoding must be available.
Previously the `en_US.UTF-8` encoding was requested, but its availability was never verified.
The fallback to `C.UTF-8` maintains UTF-8 encoding while allowing more flexibility in the running environment.
2 years ago
Sloane Hertel 05608b20e8
Fix listing collections that are missing the metadata required by build (#76596)
* Rethread pr/70185 through the dependency resolver

Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath

Don't error while listing collections if a collection's metadata is missing keys required for building a collection.

Give an informative warning if metadata has been badly formatted.

Co-authored-by: Sam Doran <sdoran@redhat.com>
2 years ago
Brian Coca 9b79d6ba35
config error origin (#78407) 2 years ago
Sloane Hertel a2174fc6d9
Fix removing existing dir/files with `ansible-galaxy collection init --force` (#78403)
* Remove collection contents when re-initializing with --force


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2 years ago
Martin Krizek ce6c9befb8
Do not allow handlers from dynamic includes to be notified (#78399)
* Do not allow handlers from dynamic includes to be notified
2 years ago
Matt Martz f6d2b18322
Correctly get the filename from a url in fetch_file. Fixes #29680 (#78383) 2 years ago
Brian Coca e4890afc4e
Pause module, return actual echo value (#78378) 2 years ago
Matt Clay b0b58fa7c2
ansible-test - Update base and default containers. (#78377) 2 years ago
Matt Davis 1368bfa348
PluginLoader now installs module-to-be-imported in sys.modules before exec (as Python import does). (#78364) 2 years ago
Ruediger Pluem 0ef1376966
Make distro.id() report newer versions of OpenSuSE (at least >=15) also report as opensuse (#76121)
* Report OpenSuSE >= 15 as opensuse

Make distro.id() report newer versions of OpenSuSE (at least >=15) also report
as opensuse. They report themselves as opensuse-leap.

* Add a test
2 years ago
Matt Clay a3c90dd0bc
ansible-test - Fix TTY and output handling. (#78350) 2 years ago
Brian Coca c135a47f44
chmod aix errors with 255 is not unreachable (#78249)
* chmod aix errors with 255 is not unreachable

  fixes #78210

* added another 'aix chmod msg'
check stderr also
2 years ago
Sloane Hertel 5728d72cda
Fix KeyError for ansible-galaxy when caching paginated responses from v3 (#78325)
* Fix KeyError for ansible-galaxy when caching paginated responses from v3

* changelog

* generate responses in loop for test

Co-authored-by: Matt Martz <matt@sivel.net>
2 years ago
Matt Martz 1429672213
Add pep440 version_type for version test (#78308) 2 years ago
Matt Martz 9d4ced1237
wait_for - Read file and perform comparisons using bytes to avoid decode errors (#78317)
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes #78214

* Write non-ascii via script instead of static file

* Use contexlib.closing to support py27 context manager

* Use executable from task, instead of shebang

* Update encoded bytes to utf16
2 years ago
Matt Clay 0012263c7a
ansible-test - Fix handling of args after `--` (#78328) 2 years ago
Martin Krizek b0a84cc9ca
dnf: clarify comparison operators in docs (#78316)
Fixes #78295
2 years ago
Martin Krizek b56b38b3ba
Use classproperty to access field attributes of a class (#78180)
* Use classproperty to access field attributes of a class, consolidate into one class
2 years ago
Matt Martz 884244f1b2
Python 3.9 min for controller (#77566) 2 years ago
Matt Clay 90ef914f77
ansible-test - Drop Python 3.8 controller support. (#78237) 2 years ago
Matt Clay f3f4ad9332
ansible-test - Use sudo for all remotes. (#78310) 2 years ago
Matt Clay 8565deaae3
ansible-test - Install `acl` on Alpine remotes. (#78303) 2 years ago
Matt Clay 8bb20fe06b
ansible-test - Enable ACL support on more remotes. (#78299) 2 years ago
Matt Clay b63812bc08 ansible-test - Enable ACLs on FreeBSD remotes.
This allows integration tests to switch from one unprivileged user
to another, sharing files between the users using ACLs.
2 years ago
Martin Krizek 9afdb7fec1
template module/lookup: fix convert_data for macros (#78259)
Fixes #78141
2 years ago
Matt Martz de810d5799
Deprecate use of Python crypt (#78257) 2 years ago
James Livulpi d635b871d1
Cleanup write_file method in uri (#76412) 2 years ago
Mahesh b2cde3a896
Add default value for lvm_facts when lvm or lvm2 is not installed or … (#75989)
* Add default value for lvm_facts when lvm or lvm2 is not installed or there are no lvm facts
2 years ago
zorun a6e671db25
Fix AIX processor facts and add unit test (#78223)
- `processor_count` was erroneously set to the number of cores
- `processor_cores` was erroneously set to the number of threads per core
- `processor_vcpus` and `processor_threads_per_core` were not set
- `processor` was a string, while it's supposed to be a list

Before:

```
"ansible_processor": "PowerPC_POWER7",
"ansible_processor_cores": 4,
"ansible_processor_count": 12,
```

After:

```
"ansible_processor": [
    "PowerPC_POWER7"
],
"ansible_processor_cores": 12,
"ansible_processor_count": 1,
"ansible_processor_threads_per_core": 4,
"ansible_processor_vcpus": 48,
```

Also add a unit test.

Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
2 years ago
Brian Coca 1f59bbf4f3
listify_lookup_plugin_terms deprecate dataloader (#78244)
* listify_lookup_plugin_terms deprecate dataloader

  deprecated useless dataloader pass to function
  also removed from callers in core


Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Kate Case fff14d7c1d
Decode vaulted args before sending over ansible-connection. (#78236)
I'm not aware of a way to easily get vault secrets decoded on the
ansible-connection side without sending the vault secrets over the
connection in the same way, so just decode them for transport.
2 years ago
Kay Yan 2e4b0fefbf
Added Kylin Linux Advanced Server Support (#78232) 2 years ago
Jordan Borean 75de4a37bc
validate-modules - allow raw module return type (#78231) 2 years ago
Martin Krizek 2bc2153c01
yum: fix releasever for latest (#78066)
Fixes #78058
2 years ago
Martin Krizek 630616103e
dnf: fix output parsing on non-English LANGUAGE systems (#78233)
Fixes #78193
2 years ago
Matt Martz b1dd2af4ca
Preflight utf8 encodings check (#78175) 2 years ago
Brian Coca d26801e994
Fix wording for :ref|term: substitution (#78216)
* Fix workding for :ref|term: subsitution

 now matches 'seealso' and makes more sense:

  ```
  delay:
  applies_to:
  - Task
  description: Number of seconds to delay between retries. This setting is only used
    in combination with `until`.
  ```
 vs
  ```
  delay:
  applies_to:
  - Task
  description: Number of seconds to delay between retries. This setting is only used
    in combination with website for `until`.
  ```

* updated unit tests

* match see also

* more sanity
2 years ago
Brian Coca f6419a53f6
Ensure atomic writes for cache file (#78208)
* Ensure atomic writes for cache file

 helps avoid errors in highly concurrent environments
2 years ago
Martin Krizek 17d52c8d64
Move undefined check from concat to finalize (#78165)
* Move undefined check from concat to finalize

In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.

Fixes #78156

ci_complete

* Fix sanity

* ...

* sigh
2 years ago
LRitzdorf 6bcb494f83
[dnf] Fix skip_broken, add test coverage (v2) (#78158)
* [dnf] Fix skip_broken, add test coverage

Change:
- skip_broken was set in config but not actually used in calls to
  base.install()
- added a lot of test cases with specialized repo
- got rid of external (docker repo) nobest test cases since the
  specialized repo works well for those too
- Slight cleanup and adding comments in dnf module

Test Plan:
- ci_complete

Tickets:
- Fixes #73072

Original-author: Rick Elrod <rick@elrod.me>

* Use a better test for checking results list

ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>

Co-authored-by: Rick Elrod <rick@elrod.me>
2 years ago
Matt Clay b9d13d222c ansible-test - Improve pip bootstrap download. 2 years ago
Brian Coca fa35aa4865
Galaxy server config updates (#77106)
* ansible-galaxy configurable timeouts

  - also fixed issues with precedence,
    so --ignore-certs now overrides config
  - made galaxy_timeout generic setting,
    if set, it becomes default for server configs,
    but now specific servers can override
  - updated tests or added notes (some tests ignore/override precedence)

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Matt Clay ca04dc28a2 ansible-test - Remove deprecated test platforms. 2 years ago
Brian Coca 34f8168afc
ansible-console fixes (#78064)
* list collection task actions too
* dynamically add execute/help functions when module is found
* handle redirection and short names
2 years ago
Brian Coca cea18bf60a
password lookup argument parsing fix (#78080)
fixes #78079
2 years ago
Matt Clay 24d91f552c ansible-test - Add support for more remotes. 2 years ago
Matt Clay 5666c6d6a3 ansible-test - More flexible become support. 2 years ago
Chris James ea351f0ae2
user - Use -n instead of -N for luseradd on all distros (#75042)
* Use -n instead of -N for luseradd on all distros

Co-authored-by: Chris James <git@etcet.net>
2 years ago
Brian Coca 3a5a61b783
hide distro map since it is not configurable (#78068)
make it a private setting for internal use and avoid
 user confusion by not displaying it with normal configuration options
2 years ago
Matt Martz 18992b7947
Add new `loop_control.extended_allitems` option (#75760)
* Add new `loop_control.extended_allitems` option. Fixes #75216

* Add test for extended_allitems

* docs code block fix
2 years ago
Matt Martz a90f666ab3
Use ctypes to access shadow, instead of spwd which is deprecated (#78050)
* Use ctypes to access shadow, instead of spwd which is deprecated

* Remove sanity ignore

* cast to str to satisfy mypy

* Add typing.cast polyfill

* Appease mypy redefine

* Add clog frag
2 years ago
David Moreau Simard d81a519f25
Remove .travis.yml files from galaxy init skeletons (#75740)
* Remove .travis.yml files from galaxy init skeletons

These files have existed for many years but in light of the recent
circumstances around Travis, we should not include them by default in
the galaxy init skeletons.

* Add breaking_changes changelog fragment with recommendation to use a custom skeleton if the .travis.yml file is still needed.

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
Sergey 30a923fb5c
Fix 'Permission denied' in user module while generating SSH keys (#78040)
* Fix 'Permission denied' in user module while generating SSH keys

Fix #78017
Use try/except for spwd usage to prevent "Permission denied".

Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Brian Coca 3fa377387e
shell plugins give out user friendly type error (#77983)
shell plugins give out user friendly type error
# TODO: this should really have been taken care of via config or fieldattribute validation
2 years ago
zorun 05fc96c927
facts: Add uptime fact for NetBSD (#75431)
* facts: Add uptime fact for NetBSD

This is copied from the current OpenBSD facts code.

Tested on the GCC compile farm: https://cfarm.tetaneutral.net/

* Added changelog fragment

Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
2 years ago
Felix Fontein 570379ef98
Do not crash callbacks if exception is not a string. (#77781) 2 years ago
Matt Clay a4e4413cde ansible-test - Update distro containers to 4.3.0.
The primary change in this update is removal of `/etc/ansible/hosts` from the containers.
2 years ago
Matt Clay 5b3557f8ba ansible-test - Allow docstring in docs-only module 2 years ago
Matt Davis 6e78425f8d
fix pip module resolution (#78000)
* `importlib.util` appears to be lazily imported and is sometimes unavailable as an attribute of `importlib` without an explicit import
2 years ago
Matt Clay c2b36ba04d ansible-test - Update base and default containers. 2 years ago
Matt Clay 23914d3f0b
ansible-test - Adjust unit test mock usage. (#77961) 2 years ago
Matt Clay 5a0b230e24 Make fuzzy plugin matching deterministic. 2 years ago
Sloane Hertel 9d6cc7b576
unarchive - fall back to unzip -Z if zipinfo is not available (#76971)
Add a new handler class ZipZArchive to use unzip -Z as an alternative to zipinfo

Run 'unzip -Z' in can_handle_archive so we fall back to the next handler if it's not available (failing in is_unarchived is too late)

* Add a test for unzip -Z when zipinfo is not available

* Update test for missing binary altogether by removing /usr/bin from the PATH
2 years ago
Sloane Hertel e89176caac
Pin PyYAML version compatible with Python 3.8+ (#77936)
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2 years ago
Matt Martz 5e369604e1
Forked display via queue (#77056)
* Forked Display via queue

* Docs and simple code cleanup

* Only proxy Display.display

* Remove unused import

* comment

* Update deadlock comment, remove py3 check

* Don't flush display, and don't lock from forks

* clog frag

* ci_complete ci_coverage

* Add units for queue proxying

* Cleanup flush

* ci_complete

* Only lock the write, switch to RLock

* Remove unused import
2 years ago
Sloane Hertel 143e7fb45e
ansible-galaxy - support resolvelib versions >= 0.5.3, < 0.9.0 (#77649)
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1

Test incompatibilities are removed for resolvelib >= 0.6.0

Test against the latest 0.8.x version and fix requirements

* Fix tests - use a venv for testing the range of resolvelib versions

* Update temporary hardcoded fallback for ansible-test

* Update hardcoded upperbound for sanity tests

* Make error check more flexible
2 years ago
Matt Clay df765c0ecf ansible-test - Update Alpine, openSUSE containers 2 years ago
Brian Coca 89c6547892
preserve add_host/group_by on refresh (#77944)
* preserve add_host/group_by on meta: refresh_inventory

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2 years ago
Martin Krizek 39b3581316
Deprecate non lists lookups (#77875) 3 years ago
Sloane Hertel c908d782fb
get_url - permit to have a checksum only file (#77948)
checksum can also accept a checksum only file (no filename beside the checksum).
fixes #54390

Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
3 years ago
Sloane Hertel f270b4e224
Fix items2dict errors for incorrect list item type and missing keys (#77946)
* Fix items2dict errors for incorrect list item type and missing keys

* keep dict comprehension
3 years ago
Martin Krizek 058a69e6b0
Properly send a skipped message for empty templated loop (#77945)
Fixes #77934
3 years ago
Matt Clay 81351022d8 ansible-test - Prefer unittest.mock for core. 3 years ago
Matt Clay f933314101 ansible-test - Fix shell target options handling. 3 years ago
Kristian Heljas 9767cda507
Make `ignore_unreachable` to count `ignored` counter
Fixes #77690
3 years ago
Julien Lecomte 33beeace10
Fix lusermod using a group name and not id (#61965) (#77914)
* Fix lusermod using a group name and not id (#61965)
3 years ago
Sloane Hertel 1214b63f4f
Fix 'ansible-config dump --only-changed -t all' verbosity (#77898)
* Fix 'ansible-config dump --only-changed -t all' to only display headers if plugin options are changed

* changelog

* add a test
3 years ago
Matt Clay 289cba333b ansible-test - More multi-arch containers. 3 years ago
OscarBell 74eb8b779d
Blockinfile multiline search (#75090) 3 years ago
Matt Davis 813afcbbb4
swallow all exceptions in type annotation support shim imports (#77860)
* swallow all exceptions in type annotation support shim imports

* add changelog
3 years ago
Brian Coca f7d7604454
Add uri, urn and url test plugins (#77423)
and docs!

Co-authored-by: Tabah Baridule <dulemartins07@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
3 years ago
Ian Wienand e4c0bbf885
apt: fix virtual package install version detection (#76781)
* apt: fix virtual package install version detection

Change 4a62c4e3e4 introduced version
matching in installation.

The problem stems from

 if version_installable or version:
     pkg_list.append("'%s=%s'" % (name, version_installable or version))

When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.

For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with

 $ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
 ... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...

In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.

The tests are updated to install and remove this package.

Fixes: #76779
3 years ago
Sloane Hertel 82f3a57bee
Fix ansible-galaxy traceback when unexpected version of resolvelib is installed (#77630)
* Fix traceback when a supported version of resolvelib is not installed

Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).

* Add tests for unsupported resolvelib versions

* Resolve remaining import sanity test issues.

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
3 years ago
Sloane Hertel 621e782ed0
Add toggle to fix module_defaults with module-as-redirected-action on a per-module basis (#77265)
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults

Add a toggle for action plugins to prefer the resolved module when loading module_defaults

Allow moving away from modules intercepted as actions pattern

Fixes #77059
3 years ago
Brian Coca eecbaee7f4
Fix windows connection options (#77844)
* winrm, psrps added missing var entry

 this handles issue with the default being set to inventory_hostname
 but defaults not being templated implicitly

 fixes #77841
3 years ago
Robert Muir 0f882d010f
Supply 'dev' keyword for linux network facts discovery (#77793)
This avoids incorrect results when the device name conflicts with
another 'ip' keyword. For example, if the device name is 'primary'.

Closes #77792
3 years ago
Sloane Hertel 400475acc0
validate meta/main.yml dependencies and meta/requirements.yml are both lists before concatenating them together (#77821) 3 years ago
Brian Coca c83419627a
apt_repository add falalback to deprecated apt-key usage (#77340)
New path uses gpg directly and the stated directories and keystores to use by debian docs

Co-authored-by: Matt Clay <matt@mystile.com>
3 years ago
Matt Clay 13d03c3c22
ansible-test - Add RHEL 9 / FreeBSD 13.1 remotes. (#77851) 3 years ago
Matt Clay 3d2c2e60aa
ansible-test - Update distro containers. (#77847)
* ansible-test - Add a Ubuntu 22.04 container.
* ansible-test - Add a Fedora 36 container.
* ansible-test - Update distro containers.
* Fix dnf test on Fedora 36.
* Work around scp test issues.
3 years ago
Matt Clay dfde4be444 Add Python 3.11 support.
ci_complete
ci_coverage
3 years ago
Brian Coca 4706091a38
rename systemd module to services only (#77644)
* rename systemd module to services only

 disambiguates what it handles since systemd is now much more
 that a service manager, but the module is specific to services
3 years ago
Joseph Torcasso c9ce7d08a2
template/__init__.py - fix KeyError when wantlist=False with non-list 3 years ago
Brian Coca 5e50284693
Restrict role loading .. to role ... (#77683)
* Ansible will now error out if you try to use the `*_from` to load files from outside the role being loaded by `{import,include}_role`
3 years ago
Martin Krizek e9af6efee6
Raise a proper error when include/import_role is used as a handler (#77807) 3 years ago
Matt Clay 1e3cf9f732
ansible-test - Make distro containers multi-arch. (#77814) 3 years ago
Brian Coca 0d6aa2e87e
More information on doc parsing errors (#77685)
* More information on doc parsing errors

  fixes #77675
3 years ago
Abhijeet Kasurde 524d30b8b0
distribution: add distribution_minor_version for Debian Distro (#74721)
* Added a fact about minor version details for Debian Distro

Fixes: #74481

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
3 years ago
Matt Clay 449d82a521 ansible-test - Add RHEL 8.6 support. 3 years ago