Commit Graph

3089 Commits (614dc10ef09c9832499be2490ea36460d2327cae)

Author SHA1 Message Date
Matt Martz 5e369604e1
Forked display via queue (#77056)
* Forked Display via queue

* Docs and simple code cleanup

* Only proxy Display.display

* Remove unused import

* comment

* Update deadlock comment, remove py3 check

* Don't flush display, and don't lock from forks

* clog frag

* ci_complete ci_coverage

* Add units for queue proxying

* Cleanup flush

* ci_complete

* Only lock the write, switch to RLock

* Remove unused import
2 years ago
Wouter Schoot 711b51fad6
Remove double spaces in errormessage (#77839) 2 years ago
Sloane Hertel 621e782ed0
Add toggle to fix module_defaults with module-as-redirected-action on a per-module basis (#77265)
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults

Add a toggle for action plugins to prefer the resolved module when loading module_defaults

Allow moving away from modules intercepted as actions pattern

Fixes #77059
2 years ago
Matt Clay 2595b42aca
Prefer unittest.mock over mock. (#77886) 2 years ago
Matt Clay 3a9a23fb1a
Use unittest.mock instead of mock. (#77883)
This can only be done for controller tests.
2 years ago
Matt Clay 8ca28acd0d Fix unit tests for Python 3.11. 2 years ago
Matt Clay 5f74350fd5 Avoid deprecated method in unit tests. 2 years ago
Martin Krizek e9af6efee6
Raise a proper error when include/import_role is used as a handler (#77807) 2 years ago
Abhijeet Kasurde 524d30b8b0
distribution: add distribution_minor_version for Debian Distro (#74721)
* Added a fact about minor version details for Debian Distro

Fixes: #74481

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Martin Krizek ed30fc9a06
Remove deprecated loading cache plugins directly without cache_loader (#77655) 2 years ago
Matt Clay 5c2d830dea
ansible-test - Fix subprocess management. (#77641)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
2 years ago
Brian Coca b439e41a91
expand ansible-doc coverage (#74963)
* Expand ansible-doc to tests/filters and fix existing issues

  enable filter/test docs if in single file or companion yaml
  add docs for several filters/tests plugins
  allow .yml companion for docs for other plugins, must be colocated
  verify plugins are valid (not modules, cannot)
  fix 'per collection' filtering
  limit old style deprecation (_ prefix) to builtin/legacy
  start move to pathlib for saner path handling
  moved some funcitons, kept backwards compat shims with deprecation notice

  Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
  Co-authored-by: Felix Fontein <felix@fontein.de>
  Co-authored-by: Sandra McCann <samccann@redhat.com>
2 years ago
Martin Krizek 3980eb8c09
Prevent losing unsafe from lookups (#77609)
* Prevent losing unsafe from lookups

This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.

Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar

Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.

The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.

It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.

For more examples see unit and integration tests included in this patch.

Fixes #77535
2 years ago
Matt Clay 26fd5a8c3a Revert "ansible-test - Fix subprocess management. (#77638)"
This reverts commit 62d03c8e75.
2 years ago
Matt Clay 62d03c8e75
ansible-test - Fix subprocess management. (#77638)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
2 years ago
Sam Doran 1b947eaf92
arg_spec - Return aliases in validation result and update aliases (#77576)
When looking up the `no_log` setting for a parameter that is an alias in
`AnsibleModule._log_invocation()`, the alias value will always be an
empty dictionary since `self.aliases` on the `AnsibleModule` instance is
never updated after initialization. Since the `no_log` setting is on the
canonical parameter not the alias, an incorrect warning is issued if the
parameter matches `PASSWORD_MATCH`.

This PR returns the aliases dictionary as an attribute of the
`ValidationResult` and updates the `aliases` attribute on the
`AnsibleModule` instance.
2 years ago
Jack 4d69c09695
ansible-galaxy collection - ensure dependencies are a dict (#77561)
* fix traceback when installing collection with dependencies set to None
2 years ago
Brian Coca abdd237de7
Yolo (#77554)
* Revert "Revert "Config, ensure templating happens at functions (#77483)""

This reverts commit 94c9106153.

* removed update configdata, which is unused
* removed test for action we don't perform anymore
* removed unused configdata
2 years ago
Martin Krizek cbe42bff7f
Allow for lazy evaluation of Jinja2 expressions (#56116) 2 years ago
Brian Coca f2ab920822
Better info sourcing (#77511)
Task is  authoritative
   also includes latest per loop info
   and fix tests
2 years ago
Brian Coca 3b9592fcaf
Move to display as single source of verbosity (#77498)
* Move to display as single source of verbosity

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2 years ago
htol d7fbde4ea9
Make "~/.ansible" path configurable (#76114)
* replace hardcoded '~/.ansible' to C.ANSIBLE_HOME
* rename previously existing env ANSIBLE_HOME in env-setup script
* modify cache dir monkeypatching in galaxy api unit tests
* update "version_added" to 2.14 for ANSIBLE_HOME
* fix description of collections with proper use of ANSIBLE_HOME

Co-authored-by: htol <github@h-tol.net>
Co-authored-by: stefanwascoding <stefan@syntaxhelden.de>
2 years ago
Martin Krizek 94eff70030
PlayIterator - remove deprecated states (#77444) 2 years ago
Sloane Hertel 477c55b0d2
Get git executable for collections in git repos (#77493)
* Fix traceback installing collections from git repos if git is not installed
2 years ago
Sloane Hertel 87a8fedd94
Fix type for ansible-galaxy server config definitions (#77424)
* Fix type for ansible-galaxy server config definitions

* changelog

* check attr without api call

* pep8
2 years ago
Martin Krizek e080bae766
Remove deprecated ALLOW_WORLD_READABLE_TMPFILES config option (#77410)
Fixes #77393
2 years ago
Jordan Borean 60b4200bc6
winrm - ensure callers PATH for kinit is set (#77401)
* winrm - ensure callers PATH for kinit is set

* Fix unit test expectations

* Fix type annotation
2 years ago
Sloane Hertel f96a661ada
ansible-galaxy - add configuration options for more flexible collection signature verification (#77026)
* Add a toggle to control the number of signatures required to verify the authenticity of a collection

* Make the default number of required valid signatures 1

* Add option to make signature verification strict and fail if there are no valid signatures (e.g. "+1")

* Use a regex to validate --required-valid-signature-count

* Add a toggle to limit the gpg status codes that are considered a failure

* Update documentation and changelog

* Add unit and integration tests for the new options

* Fixes #77146

Fix using user-provided signatures when running 'ansible-galaxy collection verify ns.coll --offline'

Add a test for a user-provided signature when running ansible-galaxy collection verify with --offline

Fix displaying overall gpg failure without extra verbosity

Add a test for displaying gpg failure without verbosity

Improve documentation to be more clear that signature verification only currently applies to collections directly sourced from Galaxy servers
2 years ago
ChanthMiao 34e60c0a7a
ansible_distribution: Add support for Uos/Deepin (#77275)
* ansible_distribution: Add support for Uos/Deepin

* Add changelog fragment.
2 years ago
Felix Fontein e3c72230cd
unarchive: fix io_buffer_size option, remove ignore.txt entry (#77271)
* Fix io_buffer_size option.

* Remove ignore.txt entry by adding action plugin only options to module's argument spec.

* Add changelog fragment.

* Adjust unit tests.
2 years ago
Matt Clay f68c66a3ef Remove collections compat from controller code. 2 years ago
Abhijeet Kasurde 4f48f375a0
parameters: handle blank values when argument is a list (#77119)
Fixes: #77108

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Matt Clay 1a5853d794
Remove obsolete units.compat.mock compat layer. (#77118)
* Remove obsolete units.compat.mock compat layer.
* Update remaining units.compat.mock references.
2 years ago
Matt Clay b613808277 Remove obsolete unit test builtins compat. 2 years ago
Matt Clay 2cd6cdc6a7
Fix duplicate unit test function names. (#77115)
* Fix duplicate unit test function names.
* Disable faulty tests.
2 years ago
Matt Clay 0bd8106d15 Remove unused mock compat code.
The code is unreachable since Python 3 versions before 3.5 are not supported.
2 years ago
Matt Clay de9a3bda2c Fix warning in unit tests for _yaml import. 2 years ago
Sloane Hertel 43e55db208
ansible-galaxy - add signature verification of the MANIFEST.json (#76681)
* ansible-galaxy collection install|verify:

  - Support verifying the origin of the MANIFEST.json when the Galaxy server has provided signatures.
  - Allow supplemental signatures to use during verification on the CLI/requirements file.

* ansible-galaxy collection install:

  - Support disabling signature verification. This silences the warning provided by ansible-galaxy if the Galaxy server provided signatures it cannot use because no keyring is configured.
  - Store Galaxy server metadata alongside installed collections for provenance. This is used by 'ansible-galaxy collection verify --offline'.

* Add unit tests for method that gets signatures from a Galaxy server

* Add integration tests for user-provided signature sources

- Test CLI option combinations
- Test installing collections with valid/invalid signature sources
- Test disabling GPG verification when installing collections
- Test verifying collections with valid/invalid signature sources

* Make signature verification advisory-by-default if signatures are provided by the Galaxy server

- Make the default keyring None
- Warn if the keyring is None but the Galaxy server provided signatures
- Error if the keyring is None but the user supplied signatures
- Error if the keyring is not None but is invalid

* changelog

* add ansible-galaxy user documentation for new options

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Andy Mott <amott@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
2 years ago
Thomas Sjögren b145732973
Resubmit: Use SystemdStrategy for Debian in the hostname module (#76929)
* add DebianStrategy tests
* ensure hostname can be changed by using become
* use Systemd strat for debian and Base for generic.
* add test to ensure all strategies are available

Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
2 years ago
Éloi Rivard 3889ddeb4b
iptables - added a ``chain_management`` parameter to control chain (#76378)
creation and deletion

fixes #25099
closes #32158
2 years ago
Brian Coca 0ff80a15ba
ssh connection avoid parsiing own debug (#76732)
- Avoids false positives on become strings being echoed back 
   by ssh cli itself
  - added test for debug lines
  - also simplified some of existing test code
3 years ago
Brian Coca 9142be2f6c
Allow specifying specific python via shebang (#76677)
modules with python were always normalized to /usr/bin/python,
  while other interpreters could have specific versions.

* now shebang is always constructed by get_shebang and args are preserved
* only update shebang if interpreter changed
* updated test expectation
* added python shebang test
3 years ago
Adrian Likins 20cc87f059
Expect upper case message 'levels' for galaxy publish results (#63530) 3 years ago
Sloane Hertel 76220c4a7b
ansible-galaxy - fix the --ignore-certs flag for the implicit galaxy server (#76735)
* ansible-galaxy - fix the --ignore-certs flag for the implicit galaxy server

* changelog

* Add a test without the server config

* Fix respecting --ignore-certs for individual --server URLs also

* Update changelogs/fragments/76735-ansible-galaxy-fix-ignore-certs.yaml
3 years ago
Matt Martz 0ef5274a3c Don't assert stdlib behavior, just assert that urllib was called 3 years ago
Jordan Borean aaa7944b02
Fix tests failing on pytest 7 (#76670)
* Fix pytest 7 failures with leaky fixture
3 years ago
Matt Clay 97104f1221
Avoid deprecated TestCase functions in unit tests. (#76678)
* Avoid deprecated TestCase functions in unit tests.
* Add assertRaisesRegex for Python 2.7.
* Fix indentation.
3 years ago
Jaromaz 16def8050a
facts - Set correct OS family for EuroLinux (#76624)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
3 years ago
Sloane Hertel ed6581e4db
check finder type before passing path (#76448)
* check finder type before passing path

ci_complete

* Reduce nesting

* Test find_module does not cause a traceback with Python 3 FileFinder

* Update lib/ansible/utils/collection_loader/_collection_finder.py
3 years ago
Jordan Borean 8482ee4e9a
galaxy build - ignore existing MANIFEST and FILES (#76479) 3 years ago