Commit Graph

3288 Commits (ff4bb5500a79609de7d23253b2f25713eeaf6bfe)

Author SHA1 Message Date
Matt Davis ff4bb5500a
PluginLoader now installs module-to-be-imported in sys.modules before exec (as Python import does). (#78364) (#78367)
(cherry picked from commit 1368bfa348)
3 years ago
Matt Clay 426e4899a3
[stable-2.13] ansible-test - Parse content config only once. (#78418) (#78426)
(cherry picked from commit f2abfc4b3d)
3 years ago
Brian Coca cbec27fefc
chmod aix errors with 255 is not unreachable (#78249) (#78356)
* chmod aix errors with 255 is not unreachable

  fixes #78210

* added another 'aix chmod msg'
check stderr also

(cherry picked from commit c135a47f44)
3 years ago
Martin Krizek 2aa3cc9e40
2.13: template module/lookup: fix convert_data for macros (#78259) (#78269)
* template module/lookup: fix convert_data for macros (#78259)

Fixes #78141

(cherry picked from commit 9afdb7fec1)

* Fix templating nested vars with convert_data=False (#78273)

Regression introduced in #78259.

(cherry picked from commit d070b03ad8)
3 years ago
Nicolas Karolak bf1e031eb2
honor use_proxy parameter (#77312) (#78261)
* honor use_proxy parameter
* fix uri test with "use_proxy: no"
* fix urls.py module

Co-authored-by: Carlos <Juan.Carlos.Cardenas.Viera@ibm.com>
(cherry picked from commit 1d9c68d27e)

Co-authored-by: h4rr21 <juankchess13@hotmail.com>
3 years ago
Martin Krizek fc82d215a6
yum: fix releasever for latest (#78066) (#78242)
Fixes #78058

(cherry picked from commit 2bc2153c01)
3 years ago
Martin Krizek 63f6fa0714
dnf: fix output parsing on non-English LANGUAGE systems (#78233) (#78241)
Fixes #78193

(cherry picked from commit 630616103e)
3 years ago
Kate Case 31db1a3a11
Decode vaulted args before sending over ansible-connection. (#78236) (#78246)
I'm not aware of a way to easily get vault secrets decoded on the
ansible-connection side without sending the vault secrets over the
connection in the same way, so just decode them for transport.

(cherry picked from commit fff14d7c1d)
3 years ago
Matt Martz 70e5673319
[stable-2.13] Fix KeyError for ansible-galaxy when caching paginated responses from v3 (#78325) (#78405)
* Fix KeyError for ansible-galaxy when caching paginated responses from v3

* changelog

* generate responses in loop for test

Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 5728d72)

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
3 years ago
Matt Clay 24359537bb [stable-2.13] ansible-test - Fix TTY and output handling. (#78350).
(cherry picked from commit a3c90dd0bc)

Co-authored-by: Matt Clay <matt@mystile.com>
3 years ago
Matt Clay 4bb1907084 [stable-2.13] ansible-test - Fix handling of args after `--` (#78328)
(cherry picked from commit 0012263c7a)

Co-authored-by: Matt Clay <matt@mystile.com>
3 years ago
Sandra McCann 8f96331f4c
Backportapalooza 07 21 (#78326)
* Update cheetsheet to fix meaning of -k (#78271)

(cherry picked from commit 691dde645c)

* Add note about package managers (#78260)

(cherry picked from commit fedd386998)

* dnf: clarify comparison operators in docs (#78316)

Fixes #78295

(cherry picked from commit b0a84cc9ca)

* Fix minor typos (#78294)

Signed-off-by: Hu Shuai <hus.fnst@fujitsu.com>
(cherry picked from commit fd810e88d6)

* Add cli docs for interacting with gpg for collection signing (#78321)

Co-authored-by: Jan-Piet Mens <jp@mens.de>
Co-authored-by: Desmond Obisi <51109125+DesmondSanctity@users.noreply.github.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Hu Shuai <hus.fnst@fujitsu.com>
Co-authored-by: Matt Martz <matt@sivel.net>
3 years ago
jamesmarshall24 0863427230
New release v2.13.2 (#78276) 3 years ago
jamesmarshall24 a689972b56
New release v2.13.2rc1 (#78229) 4 years ago
Brian Coca 69562a8db6
Fix wording for :ref|term: substitution (#78216) (#78221)
* Fix workding for :ref|term: subsitution

 now matches 'seealso' and makes more sense:

  ```
  delay:
  applies_to:
  - Task
  description: Number of seconds to delay between retries. This setting is only used
    in combination with `until`.
  ```
 vs
  ```
  delay:
  applies_to:
  - Task
  description: Number of seconds to delay between retries. This setting is only used
    in combination with website for `until`.
  ```

* updated unit tests

* match see also

* more sanity

(cherry picked from commit d26801e994)
4 years ago
Brian Coca 357ca6394e
Ensure atomic writes for cache file (#78208) (#78217)
* Ensure atomic writes for cache file

 helps avoid errors in highly concurrent environments

(cherry picked from commit f6419a53f6)
4 years ago
Brian Coca 3ef4609bca
password lookup argument parsing fix (#78080) (#78102)
fixes #78079

(cherry picked from commit cea18bf60a)
4 years ago
Sergey bdeeaa528d
Fix 'Permission denied' in user module while generating SSH keys (#78040) (#78054)
* Fix 'Permission denied' in user module while generating SSH keys

Fix #78017
Use try/except for spwd usage to prevent "Permission denied".

Signed-off-by: Sagi Shnaidman <sshnaidm@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 30a923fb5c)
4 years ago
Martin Krizek 6468bb0cb6
Move undefined check from concat to finalize (#78165) (#78179)
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.

Fixes #78156

ci_complete

(cherry picked from commit 17d52c8d64)
4 years ago
Felix Fontein 036fc02c27
[stable-2.13] ansible-doc: list modules in collections recursively (#78137)
* List modules in collections recursively.

* Skip symlinks when listing plugins and modules.
4 years ago
Matt Clay 99217ca2b6 [stable-2.13] ansible-test - Improve pip bootstrap download.
(cherry picked from commit b9d13d222c)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Dimitri Savineau e9f633ec72
New release v2.13.1 (#78086) 4 years ago
Dimitri Savineau d34ba977fb
New release v2.13.1rc1 (#78033) 4 years ago
Sloane Hertel cf5c0c9b06
ansible-galaxy - support resolvelib versions >= 0.5.3, < 0.9.0 (#77649) (#78008)
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1

Test incompatibilities are removed for resolvelib >= 0.6.0

Test against the latest 0.8.x version and fix requirements

* Fix tests - use a venv for testing the range of resolvelib versions

* Update temporary hardcoded fallback for ansible-test

* Update hardcoded upperbound for sanity tests

* Make error check more flexible

(cherry picked from commit 143e7fb45e)
4 years ago
Sloane Hertel 977c2480db
[2.13] Fix ansible-galaxy traceback when unexpected version of resolvelib is installed (#77901)
* Fix ansible-galaxy traceback when unexpected version of resolvelib is installed (#77630)

* Fix traceback when a supported version of resolvelib is not installed

Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).

* Add tests for unsupported resolvelib versions

* Resolve remaining import sanity test issues.

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 82f3a57bee)

* Fix boolean condition so that ansible-galaxy collection install works when a valid resolvelib is installed. (#77906)

(cherry picked from commit 6fbc8bd2bc)

* ansible-galaxy - ensure variable is defined for any error when getting the ansible-core distribution (#77993)

(cherry picked from commit db335498d0)

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Matt Davis 3957913485
fix pip module resolution (#78000) (#78006)
* `importlib.util` appears to be lazily imported and is sometimes unavailable as an attribute of `importlib` without an explicit import

(cherry picked from commit 6e78425f8d)
4 years ago
Brian Coca be9c208cb4
apt: fix virtual package install version detection (#76781) (#77909)
* apt: fix virtual package install version detection

Change 4a62c4e3e4 introduced version
matching in installation.

The problem stems from

 if version_installable or version:
     pkg_list.append("'%s=%s'" % (name, version_installable or version))

When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.

For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with

 $ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
 ... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...

In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.

The tests are updated to install and remove this package.

Fixes: #76779
(cherry picked from commit e4c0bbf885)

Co-authored-by: Ian Wienand <iwienand@redhat.com>
4 years ago
Sloane Hertel 63a0860231
Add toggle to fix module_defaults with module-as-redirected-action on a per-module basis (#77265) (#77899)
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults

Add a toggle for action plugins to prefer the resolved module when loading module_defaults

Allow moving away from modules intercepted as actions pattern

Fixes #77059

(cherry picked from commit 621e782ed0)
4 years ago
Brian Coca f80f58903b
Fix windows connection options (#77844) (#77894)
* winrm, psrps added missing var entry

 this handles issue with the default being set to inventory_hostname
 but defaults not being templated implicitly

 fixes #77841

(cherry picked from commit eecbaee7f4)
4 years ago
Joseph Torcasso e5e3bc849d
template/__init__.py - fix KeyError when wantlist=False with non-list (#77824)
(cherry picked from commit c9ce7d08a2)
4 years ago
Brian Coca f02e870b1b
dont rely on path to set config defs for plugins (#77659) (#77694)
(cherry picked from commit a3cc6a581e)
4 years ago
Sloane Hertel f026e3dcf9
Fix 'ansible-config dump --only-changed -t all' verbosity (#77898) (#77922)
* Fix 'ansible-config dump --only-changed -t all' to only display headers if plugin options are changed

* changelog

* add a test

(cherry picked from commit 1214b63f4f)
4 years ago
Sloane Hertel 0c76edb63a
Pin PyYAML version compatible with Python 3.8+ (#77936) (#77986)
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit e89176caac)
4 years ago
Matt Clay 6291280ef8
[stable-2.13] Make fuzzy plugin matching deterministic. (#77997)
(cherry picked from commit 5a0b230e24)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay 567a295c00
[stable-2.13] ansible-test - Adjust unit test mock usage. (#77961) (#77999)
(cherry picked from commit 23914d3f0b)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay ae380e3bef
[stable-2.13] ansible-test - Multiple backports (#77951)
* ansible-test - Backport `InternalError`

NOTE: This is a partial backport, including only one new class.

(cherry picked from commit b960641759)

* ansible-test - Fix subprocess management. (#77641)

* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.

(cherry picked from commit 5c2d830dea)

* ansible-test - Add support for remote Ubuntu VMs.

(cherry picked from commit 6513453310)

* ansible-test - Fix remote completion validation.

(cherry picked from commit e2200e8dfc)

* ansible-test - Add multi-arch remote support.

(cherry picked from commit 2cc74b04c4)

* ansible-test - Enhance the shell command. (#77734)

* ansible-test - Add shell --export option.

* ansible-test - Support cmd args for shell command.

Also allow shell to be used without a valid layout if no delegation is required.

* ansible-test - Improve stderr/stdout consistency.

By default all output goes to stdout only, with the exception of a fatal error.

When using any of the following, all output defaults to stderr instead:

* sanity with the `--lint` option -- sanity messages to stdout
* coverage analyze -- output to stdout if the output file is `/dev/stdout`
* shell -- shell output to stdout

This fixes issues two main issues:

* Unpredictable output order when using both info and error/warning messages.
* Mixing of lint/command/shell output with bootstrapping messages on stdout.

* ansible-test - Add changelog fragment.

(cherry picked from commit fe349a1ccd)

* ansible-test - Fix remote args restriction.

The platform-specific and global fallbacks were not working with the `--remote` option.

This regression was introduced by https://github.com/ansible/ansible/pull/77711

(cherry picked from commit 76ead1e768)
4 years ago
Matt Clay 2ac74197af [stable-2.13] ansible-test - Prefer unittest.mock for core.
(cherry picked from commit 81351022d8)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay 44ce9fe998 [stable-2.13] ansible-test - Fix shell target options handling.
(cherry picked from commit f933314101)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay 2b6df9ea36
Add RHEL 9.0 to ansible-test and CI. (#77923)
* ansible-test - Add RHEL 9.0 remote support.

* Add RHEL 9.0 to CI. (#77853)

* Add RHEL 9.0 to CI.

* Restrict network manager inspection to RHEL8

* Skip module tests when astream_name is undefined, undefine it for RHEL9 until 9.1

* Remove redundant test.

Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 43d650f924)

* Include test fix from https://github.com/ansible/ansible/pull/77847
4 years ago
Dimitri Savineau ad609fa57e
New release v2.13.0 (#77799) 4 years ago
David Schmidt f282306b1b
New release v2.13.0rc1 (#77717) 4 years ago
Martin Krizek 89e6dcda26
Prevent losing unsafe from lookups (#77609) (#77650)
* Prevent losing unsafe from lookups

This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.

Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar

Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.

The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.

It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.

For more examples see unit and integration tests included in this patch.

Fixes #77535

(cherry picked from commit 3980eb8c09)
4 years ago
Christian Adams 5d9d0fe39b
New release v2.13.0b1 (#77629) 4 years ago
Matt Clay 9a7b645c50 ansible-test - Update default containers to 5.9.0. 4 years ago
Sam Doran 91fb18bf31
[stable-2.13] arg_spec - Return aliases in validation result and update aliases (#77576) (#77601)
When looking up the `no_log` setting for a parameter that is an alias in
`AnsibleModule._log_invocation()`, the alias value will always be an
empty dictionary since `self.aliases` on the `AnsibleModule` instance is
never updated after initialization. Since the `no_log` setting is on the
canonical parameter not the alias, an incorrect warning is issued if the
parameter matches `PASSWORD_MATCH`.

This PR returns the aliases dictionary as an attribute of the
`ValidationResult` and updates the `aliases` attribute on the
`AnsibleModule` instance.
(cherry picked from commit 1b947eaf92)

Co-authored-by: Sam Doran <github@samdoran.com>
4 years ago
Brian Coca 3e1db5d753
git fix docs and wrapper script (#77588) (#77603)
* git fix docs and wrapper script

 fixes #77582

 now env var is set to wrapper or full command depending on version
 as was the intent of previous PR
 added ref to git commit from git for why/how we used the env vars

* handle key_file

(cherry picked from commit d06d99cbb6)
4 years ago
Brian Coca e51e36ea6a
varaiblemanager, more efficienet vars file reads (#77570) (#77596)
(cherry picked from commit 2c2a204dc6)
4 years ago
Matt Clay 9deb5ad45a
[stable-2.13] Exclude Python 2.6 from interpreter discovery. (#77528)
(cherry picked from commit 32d67ce998)

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Dimitri Savineau 220917e2af
New release v2.13.0b0 (#77502) 4 years ago
Sloane Hertel 2cba3e8c42
Get git executable for collections in git repos (#77493) (#77500)
* Fix traceback installing collections from git repos if git is not installed

(cherry picked from commit 477c55b0d2)
4 years ago