Commit Graph

6301 Commits (2cd6cdc6a74beb2413383837ffc25cdd902264e8)

Author SHA1 Message Date
Sviatoslav Sydorenko 392b16c21c
Test integration of loop and until options (#44927)
Ref #44741
Ref ansible/proposals#140

* Replace select filter with a more portable thing
* Add context
    This is needed for split controller/remote

Co-authored-by: Sam Doran <sdoran@redhat.com>
3 years ago
Thomas Stähle cd2d8b77fe
apt - add allow_change_held_packages option (#73629)
* Add changelog fragment and integration tests
* Break up function signature into multiple lines
    It passed the 160 character width. Change it to multi-line for easier reading
    and better diffs in the future.
* Remove unused import
* Update version added for new feature
* Move changelog fragment to correct location

Co-authored-by: Thomas Stähle <thomas.staehle@check24.de>
Co-authored-by: Sam Doran <sdoran@redhat.com>
3 years ago
Sam Doran d8aeffc8b7
Add macOS 12 to CI (#76328)
* Disable lookup_url test of macOS 12
3 years ago
Matt Clay 21ac52435b
ansible-test - Add support for RHEL 8.5 instances. (#76305)
* ansible-test - Add support for RHEL 8.5 instances.
* Clear libdnf checksum cache from unsigned package in dnf integration test.

Co-authored-by: Matt Martz <matt@sivel.net>
3 years ago
Matt Martz 0668538a3e
Default callback result format (#76166)
* First pass of adding yaml result format output to default callback

* Add clog frag

* flow style always false

* Further normalization of results across distros

* no lossy, now pretty

* Fix env var in runme.sh

* Rename variable to better self document

* include NativeJinjaUnsafeText

* Linting cleanup

* Add tests specific to the prior yaml callback

* Make data munging more exact, following the checks in libyaml/pyyaml

* Remove unused import

* Extend comment

* more correct and some comments

* Fix consistency of tab vs space in this section of the file

* Add str representer

* be cooler

* faster character filtering

* Clarify None as a sentinel for default behavior

* Flip filtering logic to avoid CRLF issues with 'space break' filtering

* Py3 change and comment about the use of __call__

Co-authored-by: Matt Davis <mrd@redhat.com>
3 years ago
Jordan Borean 9985b8a975
ansible-test pssa update and new rules (#76256) 3 years ago
Matt Clay b6725ec6c9 Fix pip integration test.
- Use `fallible==0.0.1a2` instead of `black==19.10b`
- Test on both Python 2 and 3.
3 years ago
Matt Clay f42ffe16d2
ansible-test - Add Fedora 35 test container. (#76228)
* ansible-test - Add Fedora 35 test container.
* Added Fedora 35 to CI and removed Fedora 33 from CI.
* Fix dnf integration test.
* Fix connection_paramiko_ssh test.
3 years ago
Matt Clay 57989c2bcb Remove install of setuptools in venv for pip test.
This task was previously added in https://github.com/ansible/ansible/pull/25243 as a work-around for a setuptools bug.

The pinned version does not work with Python 3.10, and the task should no longer be needed.
3 years ago
Matt Clay 39509ff216 Fix pip test to use setup_remote_tmp_dir. 3 years ago
Matt Clay 730194e2b6
ansible-test - Add RHEL 9 support. (#76217)
* Move RHEL pip package install into function.
* Add RHEL 9 support to bootstrap script.
* ansible-test - Add RHEL 9 support.
* Add install and uninstall instructions for RHEL9 to paramiko tests
* Skip incidental_inventory_docker_swarm on RHEL 9.
* Update dnf tests for rhel9
* Fix prepare_http_tests on RHEL 9.
* fix RHEL9 Python discovery tests to match distro map
* Skip subversion test on RHEL 9.0b.

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Davis <mrd@redhat.com>
3 years ago
Matt Martz 27a5116e19
Explicitly require iptables for incidental_setup_docker on RHEL8 (#76212) 3 years ago
Martin Krizek 150faf25d3
dnf: add more specific error message for GPG check (#76194)
Fixes #76192
3 years ago
Sloane Hertel b5ed41edb3
Skip recursive suboption validation if sub_parameters is not a dict (#75635)
* Skip recursive suboption validation if sub_parameters is not a dictionary

* Ensure sub parameter elements is a sequence to prevent iterating over string characters and causing duplicate error messages for the same param
3 years ago
Brian Coca 7bec196061
loop/fact delegation fix (#75768)
now set_fact and include_vars intermediate results are congruent with delegation
3 years ago
Sloane Hertel 8bbecc7cac
exhaust list of vault ids before failing (#75540)
* Try to find any valid vault id before raising an exception
* Apply the same logic during vault password file loading
* Also. Improve the warning msg and tests
3 years ago
Martin Krizek 47ee282227
Parser errors from within includes should not be rescueable (#73722)
* Parser errors from within includes should not be rescueable
* Also fixes unit tests
Fixes #73657
3 years ago
Tadej Borovšak f92830d16e
Fix zip content filtering in unarchive module (#76069)
When we introduced an include parameter to the unarchive module, we
inadvertenly flipped the exclusion logic. This flip meant that the
unarchive module started rejecting files that should be extracted.

This commit flips the bad logic and adds some tests that will make
sure things do not go bad again.
3 years ago
Jonas I. Liechti 0b0acef423
bug fix: using non-FQDN in plugin choices fails plugin name verification (#75991)
* enable both non-FQDN and FQDN plugin name
* add info about matching requirement of plugin name in config and plugin source file
3 years ago
Martin Krizek 26707a3c6b
Heisen jinja2_native (#75587)
* Use NativeEnvironment for all templating

ci_complete

* Keep Templar.copy_with_new_env for backwards compat

* Mention that AnsibleUndefined.__repr__ changed in the porting guide

* Templar.copy_with_new_env backwards compat

* ci_complete
3 years ago
Matt Clay cae7d2a671
Overhaul ansible-test integration tests. (#76111)
* Overhaul ansible-test integration tests.
* ansible-test - Fix import test pyyaml usage.
* ansible-test - Remove unused import.
* ansible-test - Fix traceback when pip is unavailable.
* ansible-test - Fix typo in port forwarding message.
* ansible-test - Fix controller logic in requirements install.
* Fix unit tests in ansible-test integration test.

Unit tests are now run for available Python versions which
provide `virtualenv` (Python 2.x) or `venv` (Python 3.x).
3 years ago
Matt Martz e40a0e5c90
Don't use output_dir in target tests (#76107)
* Don't use output_dir in user tests

* Move blockinfile tests from using output_dir to depending on setup_remote_tmp_dir

* Don't use output_dir in git tests

* Don't use output_dir in uri tests
3 years ago
Martin Krizek 7621784b94
Require Jinja2 3.0.0 (#75881)
* Require Jinja2 3.0.0

ci_complete

* Fix sanity

* Remove Jinja min/max tests

* ansible-test changes

* ci_complete

* More cleanup

ci_complete

* Revert _count_newlines_from_end :( and other stuff

* Fix sanity

* It's using host_vars ...

* Unused import

* Remove overridden groupby filter

* environmentfilter -> pass_environment

* Explain preserve_trailing_newlines

* Add changelog

* ci_complete

* Deprecated ANSIBLE_JINJA2_NATIVE_WARNING

* native_helpers.py cleanup

* More cleanup in the find intgration test
3 years ago
Matt Martz 66a83314b9
Modernize install (#76021)
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
Martin Krizek afb8a008c2 Remove incidental_deploy_helper 3 years ago
Martin Krizek 0c1b839c10 Remove incidental_win_copy
ci_complete
ci_coverage
3 years ago
Martin Krizek 90f670090c Remove incidental_setup_tls
ci_complete
ci_coverage
3 years ago
Martin Krizek 7e79411701 Remove incidental_cs_common 3 years ago
Martin Krizek 23f95300bd
Intentional tests (#76051)
ci_complete
ci_coverage
3 years ago
Matt Clay 2af4d30b9b Remove incidental mongodb tests.
They no longer provide unique coverage.

ci_complete
ci_coverage
3 years ago
Matt Clay cca9013df8 Remove incidental_win_data_deduplication test.
It no longer provides unique coverage.

ci_complete
ci_coverage
3 years ago
Martin Krizek 7cf29ded9c Intentional tests
ci_complete
ci_coverage
3 years ago
Matt Clay 609cba5017 Remove incidental_win_dsc test.
It no longer provides any unique test coverage.

ci_complete
ci_coverage
3 years ago
Matt Clay 319b50f04c Fix subversion test.
Switch to a different repo that isn't giving 503 errors since GitHub hasn't fixed the issue yet.
3 years ago
Matt Clay 3d46670622
Remove incidental_win_ping test. (#76031)
It no longer provides any unique test coverage.

ci_complete
ci_coverage
3 years ago
Matt Clay 2437fb3718 Add more ansible-test cloud tests. 3 years ago
Martin Krizek b1211429d5
Fix executing includes in the free strategy (#75649)
Fixes #75642
3 years ago
Matt Martz 8510db4935
Allow ca_path to point to a bundle (#75894)
* Allow ca_path to point to a bundle. Fixes #75015
3 years ago
Matt Martz 8a295b8ada
Remove incidental_cloud_init_data_facts integration tests (#75885)
* Remove incidental_cloud_init_data_facts integration tests

* Remove cloud_init_data_facts.py

* ci_coverage ci_complete
3 years ago
Brian Coca a22964609e
Fix lineinfile test (#75785)
* fix remote test after controller split

* other files had it too!
3 years ago
Sloane Hertel 845935f74f
Revert "Compile role with parents in the task's dependency chain (#75165)" (#75779)
This reverts commit 440cf15aeb.
3 years ago
Jordan Borean a2d4a8a9f3
PowerShell - Support optional module util imports (#75187) 3 years ago
Matt Clay 78ff1ab546 Add constraints to prepare_http_tests target. 3 years ago
Sloane Hertel 440cf15aeb
Compile role with parents in the task's dependency chain (#75165)
* Use the task dependency chain to compile the role instead of the parent role's _parents list

* keep existing role inheritance, but limit it to roles in the current task dependency chain

* Test getting role parents recursively by using the current task dependency chain

* changelog
3 years ago
Thomas Sjögren 57359d0174
use passlib default if `password_hash` option isn't set (#75043)
* add changelog fragment
* password_hash|length == 120
* mention the new default bcrypt format in the porting guide
3 years ago
Sloane Hertel 4ab90f3afc
Re-enable old_style_cache_plugin test target (#75748)
Fix tests for cache plugins written before ConfigManager
3 years ago
Matt Clay 4ea8d9a782
ansible-test - split controller/target testing (#75605) 3 years ago
Daniel Goldman 989eeb243f
Add an `undef` global Jinja function (#75435)
* add tests for fail filter

also tests that fail does not block inspectability

* add fail filter

fallback message is a bit clunky,
since you can't invoke a filter without specifying an input.
That is, "{{ fail }}" doesn't work,
so you have to do "{{ None | fail }}"

* document 'fail' filter

* add changelog fragment

* fail filter uses default message on Undefined or emptystring

makes it slightly easier to use the default message:
```diff
- "{{ None | fail }}"
+ "{{ '' | fail }}"
```

and the user sees a slightly more relevant message
if the message itself is undefined:

```diff
- The error was: {{ failmsg | fail }}: 'failmsg' is undefined
+ The error was: {{ failmsg | fail }}: Mandatory variable has not been overridden
```

* rebuild as the builtin `Undefined`

* harmonise `hint` parameter for make_undefined with jinja

* use code block for documentation item

[ref](https://github.com/ansible/ansible/pull/75435#discussion_r707661035)

* rename to `undef` to expose less Python into the Jinja

[ref](https://github.com/ansible/ansible/pull/75435#pullrequestreview-757799031)

* explicitly instantiate undefined value now that it's possible

see I knew we would break something with reflection

* preserve test coverage of undefined variable

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
3 years ago
Rick Elrod 61f5c22551
Remove (only) user-facing use of ANSIBLE_ASYNC_DIR (#74249)
Change:
- Remove only user-facing use of ANSIBLE_ASYNC_DIR.
- Remove two comments saying to change things that, apparently, we
  aren't going to change...

Test Plan:
- ci_complete

Tickets:
- Fixes #74139
- Fixes #74138
- Refs #74226

Signed-off-by: Rick Elrod <rick@elrod.me>
3 years ago
Matt Clay 54a795896a
Integration test and sanity index generation fixes. (#75731)
* Fix test usage of `which python`.

Also use `command -v` instead of `which` where needed.

* Fix testing_formatter.sh file checking.

The index is longer expected to exist when generating it.
The generation script is run with `set -eux`.
3 years ago
Jordan Borean 097bc07b66
PowerShell - Ignore LIB env var when building C# code (#75698)
* PowerShell - Ignore environment variables when building C# code

* Just unset LIB for now

* Fix sanity issue
3 years ago
Sam Doran a11bb8b4d3
Revert "user - consistently create user home directory on Linux (#71952)" (#75704)
This reverts commit 2f7e0b8489.
3 years ago
maulikd 9558f53a58
Fix git submodules path issue (#75655)
* Fix git path used for submodules

* Fix git submodules integration test
3 years ago
hyperreality c3fc8fb99a
apt module: add option to allow package downgrades (#74852)
* apt module: add option to allow package downgrades

* Add new option to module so users don't have to force downgrades which
  is insecure and dangerous

* Add integration tests similar to upgrade integration tests

* Changelog

* Update changelog fragment

* Update changelogs/fragments/74852-apt-allow-downgrade.yaml

Co-authored-by: Amin Vakil <info@aminvakil.com>

* Update lib/ansible/modules/apt.py

Co-authored-by: Amin Vakil <info@aminvakil.com>

* Update lib/ansible/modules/apt.py

Co-authored-by: Amin Vakil <info@aminvakil.com>

Co-authored-by: Amin Vakil <info@aminvakil.com>
3 years ago
Brian Coca 3d83c325dd
Fix metadump (#75668)
* ensure dump produces json

* clog and tests

* remove library additions since they are not usable

* avoid any/all dirs

* ensure we dont use local dir as playbookdir
3 years ago
Sloane Hertel 469b559ebe
Try all galaxy servers when locating available versions for a collection (#75468)
* If an exception occurs when getting a collection's metadata, continue to the next in the server list.

* Warn for unknown exceptions when finding versions of a collection

* Test that an invalid server is no longer fatal if a subsequent server has the collection

* Fix server for verify tests - compare checksums against the server from which it was installed

* Add tests for verify and fix that code path to mirror install/download behavior for server errors

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
devon-mar 3e7a622204
Fix unexpected exception when a role has an empty argument_specs.yml (#75604)
* Fix role with empty argument_specs.yml

* Use try/except and add changelog fragment

* Always return a dict

* Add test for empty argument_specs key
3 years ago
Matt Martz 55d448baa7
Run apache via async (#75682) 3 years ago
Sloane Hertel 649a5bbda2
Fix cleaning up user for unarchive tests (#75681)
Become root to clean up user

Clean up users with force and remove enabled

Co-authored-by: Matt Martz <matt@sivel.net>
3 years ago
Matt Martz 05464a0ff7
Ensure we set perms on htpasswd file to avoid permissions errors (#75679) 3 years ago
Sloane Hertel 7cbb53fe9a
Fix test interactions by cleaning up users with the same uid by using 'force: yes' and 'remove: yes' (#75670)
Co-authored-by: Matt Martz <matt@sivel.net>
3 years ago
Martin Krizek 63413dd483
Remove incidental_win_security_policy (#75654)
* Add an intentional test for diff.prepared

ci_complete
ci_coverage

* Remove incidental_win_security_policy

* Forgot to remove this
3 years ago
Matt Clay 8d84f3c593
Fix fetch integration test. (#75661) 3 years ago
Matt Clay fc8197e326
AnsiballZ - Make sure cwd is accessible. (#75636) 3 years ago
Sloane Hertel f38a97cece
Fix build_ignore when installing a collection from source (#75547)
* Make individual dirs that should exist instead of using shutil.copytree

* Test build_ignore when installing collections in git repos

* changelog

* Fix assertion
Fix git directory

* Fix nested content while building the collection dir

Test installing a collection from a git repo and artifact have the same result

Refactor to use variables
3 years ago
Martin Krizek 9c2f44b884
Fix templating task action with host-specific vars (#75600)
Fixes #75568
3 years ago
Matt Clay bee5e02232 Rename integration test targets.
Improve consistency with naming of tests for Ansible CLI commands.
Fix naming of module_utils tests to match the files under test.
3 years ago
Sloane Hertel e5f6c2d141
Fix rekey_on_member when key is undefined (#75131)
* Fix rekey_on_member when key is undefined and add a test case

* Add documentation about error handling in filter and test plugin sections
3 years ago
Sloane Hertel db3e8f2c1c
import_role - support templating _from_files with --extra-vars (#75269)
* Support templating _from_files with --extra-vars for imported roles

* Add tests for templating the filenames for import_role, import_playbook, and import_tasks

* Add documentation

Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
3 years ago
Martin Krizek d36116ef1c
dnf: ensure we use the correct locale for dnf API (#75264)
Fixes #75021
3 years ago
Martin Krizek 2ba9e35d09
yum: fix yumstate when wildcards are used in list arg (#75545)
Fixes #74557

* map is not available on centos 6's jinja...
3 years ago
Brian Coca 91319c5cfc
Avoid accessing delegated vars when you dont have a host (#75524)
* avoid getting delegated vars w/o inventory host

 fixes #75512

 In the case of imports, we don't have a host, so getting host vars for
 the delegated host makes no sense and should be avoided.

* also avoid error on vars_files with per host vars

* test

* testing given case

* oops
3 years ago
Martin Krizek 1c34492413
dnf: align the return value of the list argument with yum (#75501)
* dnf: align the return value of the list argument with yum

Fixes #75483

* Add integration test
3 years ago
Hossein Zolfi 3d872fb5e8
Add new comment attribute to template plugin (#69253)
* Add new comment attribute to template plugin

Add comment_start_string and comment_end_string attribute to template
plugin


Co-authored-by: Hossein Zolfi <h.zolfi@inside.sahab.ir>
3 years ago
Sloane Hertel e24eb59de5
Improve ansible-galaxy error for InconsistentCandidate exception (#75235)
* Improve error for InconsistentCandidate exceptions

* Add test case for installing a collection with an inconsistent version

* Add test case for installing a collection that has a dependency with an inconsistent version

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
3 years ago
Sloane Hertel e23fdee45e
Fix dnf test assertion (#75426) 3 years ago
Sloane Hertel b541a148d5
Fix dnf package matching (#75411)
* Fix a bug with the dnf module not using all components of a package name when filtering to determine if it's installed

* changelog

* Simplify splitting on the last '.'

* Update lib/ansible/modules/dnf.py
3 years ago
Martin Krizek 5a38076568
Globals should be accessible when importing a template without the context (#75384)
Fixes #75371
3 years ago
Sam Doran fa7482c63a
Change interpreter discovery defaults to silently prefer Python 3 (#75049)
Push /usr/bin/python to almost the bottom of the barrel. This makes the strategy to prefer
specific versions of the "mystery meat" version.

* Change INTERPRETER_PYTHON default to 'auto'
    Change description to match change in behavior.
    Change deprecation message to a warning.

* Update docs
* Add porting guide entry
* Update unit tests
* Update integration test
* Allow INTERPRETER_PYTHON_FALLBACK to be configure using a variable
* Prefer platform-python above other Python 2 interpreters
* Add Python 3.10 to the list of interpreters
3 years ago
Sloane Hertel c8d413164d
ignore/warn for undefined vars in unused handler names (#75244)
* Make undefined variables in handler names non-fatal if the handler is not used

* If the handler has no way to be notified (i.e. the name can't be templated and the handler has no listen topics), display a warning

* Add tests for variables in handler names

* changelog
3 years ago
Brian Coca 22aa6dedc9
fix yaml version issue with config yaml dump (#75344)
now tests can run in other pythons (really yaml pkg versions)
  also cleaned up diff_failure for unimplemented flow
3 years ago
Martin Krizek 6607051b75
docs: clarify regex_search return value (#75343) 3 years ago
Martin Krizek b22f976480
Remove duplicate any_errors_fatal test file (#75345)
test/integration/targets/any_errors_fatal/18602.yml is not run via
runme.sh and is testing the same as
test/integration/targets/any_errors_fatal/always_block.yml which is
actually run. The former file is safe to remove.
3 years ago
Sergey 40ca87a963
Add packaging to requirement of ansible-test (#75356)
* Add packaging to requirement of ansible-test

Fix #75353

After requires_ansible field was added as mandatory to runtime.yml
file, ansible-test fails to check this field if it doesn't have
packaging module.

[1] https://github.com/ansible/galaxy-importer/pull/124
3 years ago
Matt Martz 4dca539a29
Expose `unredirected_headers` to `uri` and `get_url` modules (#75308) 3 years ago
Martin Krizek 767b2f07b0
Ensure Jinja2 template header overrides are used (#75306)
Fixes #75275
3 years ago
Brian Coca e5a2fe4c41
fix ssh cli defaults (#75245)
* fix ssh cli defaults

  fixes #75221
3 years ago
Sloane Hertel 4d78b58540
Fix resolution of action/module names in module_defaults (#75284)
* Fix resolution of action/module names in module_defaults

* changelog

* Include legacy module_defaults entries for builtin plugins
3 years ago
Sloane Hertel 1fc1ab89ae
Add a test for deduplicating dependencies with include_role (#75259) 3 years ago
Sloane Hertel 3b861abce1
add action_groups support to collections (#74039)
* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play

* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list

* Add validation for action_group metadata and a toggle to disable the warnings

* Handle groups recursively referring to each other

* Remove special-casing for non-fqcn actions in module_defaults groups

* Error for actions and groups in module_defaults that can't be resolved

* Error for fully templated module_defaults

* Add integration tests for action_groups

* Changelog
3 years ago
Sloane Hertel bd03fa811b
Include parent role vars as well as _role_vars (#75034) 3 years ago
Brian Coca 363c1a3fba
Vaultfilters (#74998)
add vault and unvault filters

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
3 years ago
Brian Coca 60c9f045cc
Unfrack extra vars (#75127)
* Ensure we keep teh original path as per cli

  fixes #75126

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
3 years ago
Martin Krizek f176cc1eda
Remove create-repo.py script as it was changed to module (#75124) 3 years ago
Martin Krizek 03cb40e712
Fix baseurl handling in yum_repository test (#75096)
Since we moved yum_repository_test_repo.baseurl to file:/// format,
using urlsplit filter on it returned an empty string. Using an empty
string as a left operand of `in` was then always evaluated to True.

This was discovered with native jinja being on where urlsplit returns
None which results in a hard fail as None cannot be a left
operand of `in <string>`.
3 years ago
Sloane Hertel 865bda3a11
Add a resolved_action task attribute (#74709)
* The resolved_action is the formatted version of the final plugin in the PluginLoadContext's redirect_list

* Collection plugins are represented as FQCN

* Legacy plugins are represented with only the plugin name

* Add tests

* Changelog
3 years ago
Brian Coca 50b6d28ee1
Config init+ (#74914)
Can now specify plugin/plugin type for list and dump actions
 New 'init' action to create usable config sources 

Co-authored-by: Felix Fontein <felix@fontein.de>
3 years ago
David Shrewsbury 703cb79442
Implement async callbacks (#74953)
* add changelog and output from default callback
* add test
* add comments about TE task
3 years ago
Sloane Hertel ca6123e0ee
Template suboptions in a role's arg spec (#75073) 3 years ago
Junegunn Choi 8e755707b9
Add 'hash_behaviour' option to include_vars module (#72944) 3 years ago