Commit Graph

5405 Commits (0f2b3fde76ac0d2a423010f8ff83acb9c04ab605)

Author SHA1 Message Date
Sam Doran 71e33d2578
version test - improve message when value is empty (#74754)
When an empty value is provided, no `version` attribute will exist on the `LooseVersion` or
`StrictVersion` object. We catch and handle this, but it's not immediatebly clear that an
AttributeError means an empty value was provided.

Specifically handle the case where value or version are empty and add more
helpful error messages.

Add integration tests.
4 years ago
Brian Coca d2d1f01f9d
avoid literal quoting in config lists (#74740)
* added tests
4 years ago
Sam Doran c0cb353ce1
Remove PyCrypto (#74699)
* Remove PyCrypto from setup.py and packaging script
* Remove mention of pycrpto from installation docs
* Remove PyCrypto from vault
* Remove pycryto constraint and unit test requirement
* Remove PyCrypto tests from unit tests
* Add docs and fix warning message
* Remove section about cryptography library in Ansible Vault docs
4 years ago
Sam Doran d8e6f21d9f
ini lookup - add case sensitive option (#74630)
* Add tests for case-sensitive option
* Run all test playbooks from a single file
4 years ago
Sam Doran 8d3dce49bf Remove incidental_setup_rabbitmq integration test
All tests using this have been removed.
4 years ago
Sloane Hertel bab7acb1ad
update cron test added in #74497 for /etc/crontab (#74689) 4 years ago
elara-leitstellentechnik c6ac9de67b
Do not remove non-empty cron tabs (#74497)
* Only remove crontabs if they are empty

* Add integration test to ensure system cron tab doesn't get removed. Increase cron integration tests separation.
* Also detect crontab which only contains whitespace as empty.
* cron integration test: Adjust system crontab path to be distribution specific.

Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
4 years ago
Sam Doran 0affe4d027
ini lookup - catch and handle duplicate key and missing section errors (#74629)
* Remove unused import
* Add integration tests for errors
* Cleanup formatting on existing tests
* Add changelog
4 years ago
Sam Doran f99d024851
CI and compat fixes for Jinja2 >= 3.0 (#74666)
* Add constraint for MarkupSafe

MarkupSafe >= 2.0.0 requires Python >= 3.6.0. Add a constraint for older Python versions
and fix the `groupby_filter` test.

* Fix template_jinja2_latest test.

* patch filter decorators on newer Jinja2

* Jinja2 >= 3.0 renames several filter decorators used by Ansible itself, as well as by filters in collections. This patch ensures that the old names are usable within Ansible and by collections without warnings or errors.

* Ignore docs-build issues.

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Matt Martz 8d1cf7f266
Vendor `distutils.version` (#74644)
* Vendor distutils.version

* Fix import order. ci_complete

* remove distutils warning filter

* Don't remove warnings filter from importer

* ci_complete

* Add pylint config for preventing distutils.version

* Add changelog fragment
4 years ago
Sloane Hertel cf11c38caf
Exclude unreachable hosts in ansible_play_batch between plays (#74625)
ci_complete
4 years ago
David Shrewsbury 8fb54885bf
Role argspec: allow new argument spec file (#74582)
* support separate role argspec file in ansible-doc

* support separate role argspec file in ansible-core

* support both .yml and .yaml extensions on argspec file in ansible-doc

* fix filename building bug and rename some argspec files to test variations

* use yaml extensions from constants

* add superfluous meta/main.yml files to tests

* Update lib/ansible/cli/doc.py

Co-authored-by: Sam Doran <sdoran@redhat.com>

* update docs

* ci_complete

* add changelog and allow for main.yml variations

* add collection role testing

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Matt Clay d244e47822
Update Python versions for controller-only tests. (#74652)
* Update Python versions for controller-only tests.

Both `ansible-galaxy` and `ansible-test` integration tests only run on the controller.

* Use a venv to make requirements available.
4 years ago
Ruediger Pluem 2f7e0b8489
user - consistently create user home directory on Linux (#71952)
Always use create_homedir when we are asked to create a home directory
in the User class. Don't use the -m and -k parameters from
useradd / luseradd as they behave differently with respect to
preexisting home directories. Instead always specify -M to ensure
that useradd / luseradd do not try to create the home directory.

This does not change potential different behaviours in child classes
of the User class.

Consider the new umask option from #73821 in create_homedir as well as
we do not let luseradd / useradd create the home directory any longer.
4 years ago
René Moser 403a5d147d
modules: get_url: Fix checksum binary validation (#74502)
From the sha512sum man page:

... The default mode is to print a line with checksum, a character indicating type ('*' for binary, ' ' for text), and name for each FILE.
4 years ago
Brian Coca 35ff4ea95b
warn on walking through pathing issues (#74514)
* warn show pathing issues when wallking through provided paths

  moved issues from msg to actual warnings AND a specific return field

  fixes #25314

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Abhijeet Kasurde 38dd49eb00
ansible-inventory: Handle exception in toml parsing (#74486)
Handle stacktrace raise while parsing inventory in toml format

Fixes: #74404

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
David Shrewsbury c6fda9fc1a
[coverage] improve test coverage for together lookup (#74549) 4 years ago
David Shrewsbury 4f0a9488bb
[coverage] improve test coverage for indexed_items lookup (#74548) 4 years ago
David Shrewsbury 244bf9c640
[coverage] improve test coverage for vars lookup (#74546) 4 years ago
Brian Coca 30912b6a47
dont specify default for port, allow ssh/config (#74526)
* dont specify default for port, allow ssh/config

 also added general note on how defaults work.

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Sam Doran 728e43bc59 callback_default test - Reduce sleep time to correct CI failures
For unknown reasons, these tests started failing recently because the results
from testhost11 are being returned before testhost10, but only when coverage is
enabled.

Setting the initial host sleep time to 0 fixes this failure.

ci_coverage
4 years ago
Matt Clay ca507ff477 Integration test fixes. 4 years ago
Rick Elrod d8c20a73a4
[ansible-test] Add Fedora 34, update Fedora 33 (#74506)
Change:
- Added Fedora 34 container image to docker.txt
- Dropped Fedora 32 from CI
- Added Fedora 34 to CI
- Updated Fedora 32 and 33 containers for ssh-related fixes
- Move cron fix from 698eae3f3d into
  cron_setup and make it more generic; it can affect modern distros too.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Maxim Masiutin 6ea81b84ed
apt_key - correct error message about the failure to import a 'gpg' (#74476) 4 years ago
Maxim Masiutin d147c1ef1f
apt_key test - Change the URL of the GPG key to S3 and remove it from the repo (#74483) 4 years ago
Abhijeet Kasurde 1a7923e318
pip: Idempotency tests (#74473)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: David Moreau Simard <dmsimard@redhat.com>
4 years ago
David Shrewsbury f1a5c411d9
Fix sequence lookup message and add tests (#74472)
* add test for bad kv arg value
* add simple form parsing tests and make error messages the same
* add changelog
4 years ago
Maxim Masiutin 3bc2e7783c
Binary GnuPG keys downloaded via URLs by the 'ansible.builtin.apt_key' module were corrupted so 'gpg' could not import them (https://github.com/ansible/ansible/issues/74424) (#74474) 4 years ago
Sam Doran 62cba4a6ad incidental_setup_rabbitmq test - Use official repo for rabbitmq-erlang
Previously it was hosted on bintray, but that service is shutting down on May 1. Using the new
repository also required using a newer version of RabbitMQ.
4 years ago
Alexandre Garnier 8698855ffd
Case-insensitive set theory filters (#74256)
Fixes #74255

* Fix call to 'unique(case_sensitive=False)' triggering error when falling back to Ansible's version which **is** case-sensitive
* Test multiple situations of 'unique' filter errors with fallback not handling specific parameters

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 6cfe36fa1f
Set ansible-test min controller Python to 3.8. (#74395)
- Version neutral sanity tests now require Python 3.8 or later.
- Unit tests for controller-only code now require Python 3.8 or later.
4 years ago
Alexander Sowitzki ff88a2a06d Fix combinated run of unarchive and uri tests under centos 6
The unarchive integration test installs pinned packages via pip,
the uri test installs them from system packages. This explodes.
PR lets unarchive uninstall packages after test ran.
4 years ago
Matt Martz bf7d4ce260
Abstract away libyaml details into yaml utils (#72607)
* Add yaml utils file and use it

* Linting fix and missing import

* Abstract a few more details

* Parser imports

* Don't use CSafeDumper in AnsibleDumper

* Move and use convert_yaml_objects_to_native when libyaml is present

* yaml_load and yaml_dump, instead of safe_*

* re-use HAS_LIBYAML from utils.yaml

* add changelog fragment

* Address recent changes

* Use representer instead of recursive type converter

* Restore needed import

* move yaml utils to module_utils

* Properly guard imports

* Update from_yaml(_all)? to handle text wrappers with CSafeLoader

* Use yaml utils for legacy_collection_loader

* Add HAS_YAML, and ignore pylint issue

* oops

* GPL->BSD
4 years ago
Brian Coca 6418f368e3
Fix pb subs collapse (#74301)
* fix subdir parsing for plays

  fixes #74283

  Co-authored-by: Nikolaos Kakouros <nkak@kth.se>
4 years ago
Matt Martz a3e7cf0110
Update alpine repo to community for libuser (#74374) 4 years ago
Rick Elrod 738132da95
[ansible-test] add freebsd/13.0 remote (#74328)
Change:
- FreeBSD 13.0 was released on the 13th, add it to ansible-test.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
elara-leitstellentechnik 437a08eb6d
Map Debian 8 to Python 2 (#74152)
* Map Debian 8 to Python 2

If Python 3 is installed on Debian 8 Ansible cannot run, as the version
is too old (3.4)

* Add integration test for python interpreter discovery on Debian 8

* fix test issue on Debian 9, add changelog

* un"fix" not broken test :D

Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Amin Vakil 49d4442378
Add umask option to user module (#73821)
* Add umask option to user module

* Fail on setting both umask and local: True

* Add integration test

* Add changelog

* Run integration tests only if HOME_MODE is not set

* Run integration tests only on Linux

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
David Shrewsbury d44eb03f49
Remove include from tests (#74330) 4 years ago
Rick Elrod 698eae3f3d [tests] Work around vixie-cron and PAM issue, el6
Change:
- This works around an issue that causes the cron test to fail sometimes
  on el6.

Test Plan:
- ansible-test integration cron --docker centos6

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 17237c1d88
Deprecate Py2.6 support for target nodes (#74165) 4 years ago
Amin Vakil a7c8093ce4
[2.12] Add accept_newhostkey option to git module (#73819)
* Add integration test for accept_newhostkey option

* Add changelog
4 years ago
Jordan Borean 019452dda7
uri - fix traceback on multipart-form int value (#74302) 4 years ago
Amin Vakil fdee5ca16d
Add cacheonly option to yum and dnf modules (#73820)
* Add integration test

* Add changelog

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Martin Krizek 7fecb7ccc3
Actually run blocks/finalized_task test (#74291) 4 years ago
Rick Elrod 7ef3dc2b8b
[controller ansiballz] escape directory regex (#74270)
Change:
- We were passing a directory name directly to re.compile().
  If the directory isn't valid regex (or is) this can have odd side
  effects, such as crashing.
- Fix a few other similar cases, but less likely to be a real issue.

Test Plan:
- New test

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 1082e2ab79
Catch errors getting filters (#74127)
* Catch errors getting filters, and fail

* Add changelog

* Switch to warnings instead of errors, to match other plugin loader behavior

* Add tests

* Handle collections
4 years ago
Martin Krizek 664531d7d6
Prevent ansible_failed_task from further templating (#74290)
* Prevent ansible_failed_task from further templating

Fixes #74036

* Add changelog
4 years ago
Brian Coca c6945de899
Ini fixes (#74285)
* avoid 'mixed' param formats

* added tests

* clog

* fixed alignment
4 years ago
Brian Coca b91749d671
fix config manager list loption with choices (#74267)
fixes #74225

  Co-authored-by: Kim Nørring <github@norring.dk>
4 years ago
Brian Coca 84e473a26e
All lookups ported to config system (#74108)
* all lookups to support config system

 - added get_options to get full dict with all opts
 - fixed tests to match new error messages
 - kept inline string k=v parsing methods for backwards compat
 - placeholder depredation for inline string k=v parsing
 - updated tests and examples to also show new way
 - refactored and added comments to most custom k=v parsing
 - added missing docs for template_vars to template
 - normalized error messages and exception types
 - fixed constants default
 - better details value errors

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Brian Coca b3b1dde648
various fixes to command (#74212)
* various fixes to command

  - Updated splitter to allow for all expected args in ad-hoc
  - Ensure we always return the returns we promissed to always return (i.e stderr/stdout)
  - Updated docs to clarify creates/removes precdence in checking
  - Removed abspath from chdir to allow reporting to handle symlinks correctly
  - Corrected tests to new output messages

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
abikouo 29aea9ff34
constructed - Add keyed_groups suboptions 'default_value' and 'trailing_separator' (#74005)
* Support omitting the trailing separator when a dictionary key's value is an empty string

* Support a default value when the value used in the group name is an empty string

* Add tests

* change log
4 years ago
David Shrewsbury 5e55af2f02
[test coverage] Add tests for set_stats (#74243)
* satisfy the all-knowing, ever present God of Sanity and use 'grep -c' vs. 'wc -l'
4 years ago
David Shrewsbury 93fdba7013
[bugfix][coverage] find module should consider file size with file_type=any (#74241)
* add changelog
* fix cl text
* Update changelogs/fragments/74241-find-checks-size-with-any.yml

Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod fa0bccf6a1
[module_utils.distro] Fall back to bundled (#74229)
Change:
- When a "distro" package exists in PYTHONPATH but isn't what we expect,
  fall back to our own vendored one and use it. This prevents a
  traceback if someone has some random thing that provides "distro" but
  isn't actually the "distro" library we need.

Test Plan:
- new tests

Tickets:
- Fixes #74228

Signed-off-by: Rick Elrod <rick@elrod.me>

* nuke playbook test file

Signed-off-by: Rick Elrod <rick@elrod.me>

* test fixes

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod aae5bc5b9e
[advanced_host_list] 100% coverage + fix a bug (#74227)
Change:
- Tests for advanced_host_list; it is now at 100% coverage.
- There was a typo (host vs hostnames) where when the host pattern
  failed to parse, instead of treating the name as a literal as
  intended, it would trigger UnboundLocalError. This didn't fatal
  as there's a global "Exception" handler below, but it did lead to
  an ugly error and incorrect behavior.

Test Plan:
- New tests
- Local experimenting

Signed-off-by: Rick Elrod <rick@elrod.me>

* changelog

Signed-off-by: Rick Elrod <rick@elrod.me>

* fix tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod bd1acb74c8
[playbook/play.py] Increase error test coverage (#74217)
Change:
- Mostly increase error coverage for various conditions in play.py
- Also fix a string in an error, where get_name() was called before
  self.name was read in, so get_name() was always ''.

Test Plan:
- new tests

Signed-off-by: Rick Elrod <rick@elrod.me>

* Fix regex for py2 and py3

Signed-off-by: Rick Elrod <rick@elrod.me>

* py2 hates me

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 19aeb4706d
[conditional] Remove support for bare variables (#74208)
Change:
- Variables used in ``when`` conditionals are no longer parsed and
  attempted to be converted to booleans. All non-empty strings are
  considered true (empty strings, false).

Test Plan:
- Updated existing tests
- Added a bunch of new tests with various kinds of truthy/falsy
  values.

Tickets:
- Fixes #74134

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay b752d07163 Overhaul ansible-test container management.
This brings ansible-test closer to being able to support split controller/remote testing.
4 years ago
Brian Coca 9f856a4964
config lookup can query plugin settings (#74186)
* config lookup can query plugin settings
4 years ago
Rick Elrod 3504f4c45f
[yum] report upgraded multiarch packages (#73548)
Change:
- Previously when the same package name was installed twice under
  different architectures, we only reported it once in changes.updated.
- This was the result of using a dict internally and keying on package
  name alone.
- This change still keys on package name but turns the values into lists
  which can contain multiple packages per name.

Test Plan:
- Added a lot of tests around multi-arch support
- Added some tests around virtual provides as well

Tickets:
- Fixes #73284

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod ef554d0378
[playbook] error on empty, error on 'include' (remove two deprecations) (#74172)
Change:
- Remove two deprecated features
- We now error if a playbook is an empty list instead of just skipping
- We now error if using 'include' instead of 'import_playbook'

Test Plan:
- Added new tests for new errors

Tickets:
- Fixes #74133

Signed-off-by: Rick Elrod <rick@elrod.me>

* sanity & changelog

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod d7df8a4484
[cron] Remove deprecated features (#74197)
Change:
- Require name always
- Drop 'reboot' parameter in favor of 'special_time: reboot'

Test Plan:
- CI

Tickets:
- fixes #74132

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
David Shrewsbury d5ce6e6bed
Increase test coverage for command module (#74183) 4 years ago
David Shrewsbury d1842afd59
ansible-inventory - add coverage tests (#74130) 4 years ago
David Shrewsbury 1cacf933b7
lineinfile - improve test coverage (#74122) 4 years ago
Brian Coca f9f839fa08
Fix debug factsetter (#74067)
* prevent debug from setting namespaced facts as tlv
* also added tests
4 years ago
Matt Davis 454c7e37ec
nonzero exit code on `ansible galaxy collection verify` failures (#74051) 4 years ago
Alexander Sowitzki 1527078a8f
pause - do not continue with '\r' when timeout is set (#74030)
Original function of pause was, to only allow user input
(finished with enter) when no timeout was set. This restores
the behaviour.
4 years ago
Matt Davis a84c1a5669
add --offline option to galaxy collection verify (#74040)
* --offline allows in-place verify for installed collections with manifests
* manifest hash, collection name, version, and path are now always displayed
* test updates
4 years ago
Brian Coca 4a82e2c486
Fix setup subset (#74022)
* fix error msg on bad subset

* added test
* handle more raised but not handled fact exceptions
4 years ago
Felix Fontein 940b7af7d8
argspec validation - fix apply_defaults (#74029)
* Add test
4 years ago
Sam Doran fa1b52ce55
dnf test - update libmodulemd when updating python3-dnf (#74025)
The dependency version is set too low in the latest version of the package.

https://bugzilla.redhat.com/show_bug.cgi?id=1942236
4 years ago
Sam Doran c4e211a429 apt_key - update key ID and URL used in test
The GPG key in getfedore.org/static/fedora.gpg changed and caused the test to fail. Update
to using the new key ID and save the GPG file in our S3 to prevent spontaneous changes/breakage.
4 years ago
Felix Fontein a1ece49006
Add integration tests for add_collection_to_versions_and_dates(), and extend ansible-doc tests (#73601)
* Add add_collection_to_versions_and_dates integration tests.

* Add doc fragment tests with fragments from other collections.

* Remove trailing whitespace (Python 2's json.dump inserts some).

* Use ' ' instead of '\s'.
4 years ago
Brian Coca 089d0a0508
find - set proper default based on use_regex (#73961)
When using "use_regex: yes" and setting an excludes: without
specifying a pattern: the existing code passes the file-glob '*' to
the regex matcher.  This results in an internal invalid-regex
exception being thrown.

This maintains the old semantics of a default match-all for pattern:
but switches the default to '.*' when use_regex is specified.

The code made sense as-is before excludes: was added (2.5).  In that
case, it made no sense to set use_regex but *not* set a pattern.
However, with excludes: it now makes sense to only want to exclude a
given regex but not specify a specific matching pattern.

Closes: #50067

* moved change to new location
added changelog

* Update lib/ansible/modules/find.py


Co-authored-by: Ian Wienand <iwienand@redhat.com>
4 years ago
Sloane Hertel 48c0fbd1cb
Fix a bug adding unrelated candidates to the plugin loader redirect_list (#73863)
* Fix a bug adding unrelated candidates to the plugin loader redirect_list

* Add tests for the redirect list

  * test redirect list for builtin module
  * test redirect list for redirected builtin module
  * test redirect list for collection module
  * test redirect list for redirected collection module
  * test redirect list for legacy module

* changelog
4 years ago
sommersoft 4add723107
ansible-pull: Run All Playbooks When Multiple Are Supplied (#73172)
* ansible-pull: run all playbooks when multiple are supplied

* add test for ansible-pull with multiple playbooks supplied from cli

* add changelog fragment
4 years ago
Matt Martz 832631b888
Ensure task from the worker is finalized/squashed (#73881)
* Ensure task from the worker is finalized/squashed. Fixes #57399. Fixes #49942
4 years ago
David Shrewsbury 1e5ccb326f
Allow for searching handler subdir for included tasks (#73809)
* Allow for searching handler subdir for included tasks
4 years ago
Matt Clay 3a8c9242e1 Use virtual host style S3 bucket names in tests. 4 years ago
Matt Davis 3e1f6484d7
add optional module_utils import support (#73832)
* add optional module_utils import support

Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).

* sanity test fixes

ci_complete
4 years ago
Brian Coca 9db557e431
Nonfatal facts (#73804)
continue with local facts vs at script error
 actually capture execution errors
 better error messages in general
 add more local facts tests

 fixes #52427
4 years ago
David Shrewsbury 78d3810fdf
Auto cleanup of async cache file (#73760)
* Auto cleanup of async cache file

* Add changelog
4 years ago
Brian Coca ce1de28061
Fix notify on import_tasks (#73572)
* Fix notify on import_tasks

  also able to apply to blocks now.
4 years ago
Brian Coca 935528e22e
finish migrating ssh plugin to config system (#73708)
* finish migrating ssh plugin to config system

  fixes #72739
  fixes #57220

* fix connection detection in reset
* correct options for connection meta reset

Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
4 years ago
Brian Coca 43300e2279
module output is only json objects (#73765)
* module output is only json objects

 remove json lists as they are not valid from modules
 fixes #73744
4 years ago
Brian Coca 527bff6b79
Don't fail for mixed typed keys (#73726)
* Don't fail for mixed typed keys

  but warn that content cound not be sorted because of this

* added tests
4 years ago
Felix Fontein 3cc693b92c
Import test: improve docs, add more tests (#73600) 4 years ago
Abhijeet Kasurde 6cb324bb0e
selinux: return selinux_getpolicytype facts (#73609)
* selinux: return selinux_getpolicytype facts

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

* add basic selinux facts tests

* fix selinux facts test when selinux missing

Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Alexander Sowitzki e804fccf1c Let file module not change link to absolute when src not given (#65448)
The file module changes existing sym links from relative to absolute
if the src is not stated in the tasks since it uses `os.path.realpath`
to fetch the link source and not `os.readlink`. Changed that.
4 years ago
Sloane Hertel 8628c12f30
find module - stop traversing directories with os.walk when depth is already exceeded (#73718) 4 years ago
Matt Martz 950ab74758
Normalize ConfigParser between Python2 and Python3 (#73715)
* Normalize config parser between py2 and py3

* Add tests and changelog

* Use different config entry, since we supply certain env vars
4 years ago
manas-init eb72c36a71
galaxy: Handle ignored directory names in role skeleton (#72035)
* galaxy: restore left hand slicing in assignment

Fix 'ansible-galaxy role init --role-skeleton=role-skeleton' when the role skeleton
contains an ignored directory.

The issue was because the 'dirs' variable was changed to reference a different list,
but needs to be mutated instead to stop os.walk from traversing ignored directories.

Fixes: #71977

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Martz 11f1177e6c
Operate pexpect with bytes to limit encoding issues (#73255)
* Operate pexpect with bytes to limit encoding issues

* Update tests to ensure no pepxect encoding issues

* Add changelog fragment

* Add multiline note

* Use rst formatting directly
4 years ago
Sam Doran 3ab52f2965
Disable junit callback on roles_arg_spec_tests (#73663)
This prevents the role arg spec validation tasks from showing up as failure
in the CI output.
4 years ago
Rick Elrod fa046d302c
[InventoryManager] Fix two unhandled exceptions (#73667)
Change:
- Fix regression: unhandled exception when given inventory directory
  is empty or contains empty subdirectories.
- Fix unhandled exception when limit file is actually a directory
  instead of a file.
- Fix inventory tests which previously could never fail due to missing
  `set -e`. Fixed up tests that failed after `set -e` was added. Added
  several tests.

Test Plan:
- New tests
- Fixed existing tests which previously could never fail

Tickets:
- Fixes #73658

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 133a29acb4 Add test to verify pkg_resources imports work. 4 years ago
Sam Doran 18f7282ccf
ansible-doc - account for empty meta/main.yml (#73590)
* ansible-doc - account for empty meta/main.yml

 from_yaml() will return None when encrounting an empty file.
4 years ago
David Shrewsbury d1d9406066
Add rolespec_validate to import/include_role (#73589)
* Add rolespec_validate to import/include_role

* Add changelog

* fix sanity, not private
4 years ago
David Shrewsbury f0ec10dbc3
Role arg spec validation implementation (#73152)
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
4 years ago
Brian Coca ea2f37d253
allow constructed to use vars plugin (#73418)
Allow constructed to optionally use vars plugin data

* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
4 years ago
Alexander Sowitzki d0fda3e901
Let vault lookup output unicode string. (#73571)
Until now, the lookup plugin returned a byte string.
Changed this to output a unicode string instead.
4 years ago
Brian Coca ec8a556538
only add data when there is data to add (#54559)
Only add data when there is data to add

  also avoid clobbering existing data with empty file
  fixes #45843

* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
4 years ago
Matt Davis 4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
4 years ago
Matt Clay 8a175f59c9
Upgrade pylint used by ansible-test. (#70155)
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
4 years ago
Jeff Squyres 6fd0a74601
unarchive: add support for .tar.zst (zstd compression) (#73265) 4 years ago
pushkarkumar15 4344607d7d
user - add password expiration parameters (#69531)
* Add integration test

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Sam Doran 7d18ea5e93
default callback - add "show_task_path_on_failure" option (#73260)
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.

* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()

* Keep color output consistent for now
  Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
  Instead of the color of the path changing based on verbosity level, just keep it at the
  currently behavior of COLOR_DEBUG. Having the color of the same information change
  based on verbosity level seems incorrect and makes visual parsing of the information
  more difficult.

Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
4 years ago
Matt Martz d3441bc313
Add new split filter (#73532)
* Add version added
4 years ago
Matt Clay 1a2da990a4
Temporary fix for cryptography issues. (#73530) 4 years ago
Rick Elrod aca5b0e43b
[setup_rpm_repo test] Ensure rpm-build is present (#73516)
Change:
- Other targets might remove rpm-build as they clean up after
  themselves. Ensure that it's present in setup_rpm_repo because
  rpmfluff needs it.

Test Plan:
- Local experimentation with yum_repository and mysql_db (the latter of
  which depends on a handler which was removing rpm-build) on
  stable-2.9.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 125c220343
Make non-lockstep callback tests more deterministic (#73511) 4 years ago
Shane McDonald 67f5bb39c7
galaxy - add format options for collection list (#73474)
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
4 years ago
Matt Martz d3f3784b86
Don't treat host_pinned as lockstep (#73484)
* Don't treat host_pinned as lockstep. Fixes #73364

* Add intg tests
4 years ago
Florian Heiderich 8d6136eab9
uri - add ca_path parameter (#71979)
* add changelog fragment for #71979 (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
  and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
4 years ago
Jordan Borean 0cdc410dce
no_log mask suboption fallback values and defaults CVE-2021-20228 (#73487)
* no_log mask suboption fallback values and defaults

* Added changelog

* Remove lambda expression
4 years ago
Florian Heiderich 8f887b7adb
add self-signed https endpoint for ansible-test (#73463)
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
4 years ago
Sam Doran 997b2d2a19
Make yum_repository test more reliable (#73467)
* Improve setup_rpm_repo

- add handlers to remove repos
- add variable to control whethere or not repos are created

* Use local repo for all distros
* Change repo creation script to module
4 years ago
Sloane Hertel 9759e0ca49
add option to upgrade collections (#73336)
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Jose Angel Munoz 69631da889
lineinfile - add search_string parameter for non-regexp searching (#70647)
* Add tests for search_string
* Improve examples
* Add changelog
4 years ago
Alexander Sowitzki 1c83672532
Allow `$` & `~` inside paths in galaxy roles (#72966)
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.

Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
4 years ago
Sloane Hertel 840bdc1e10
Fix warning for nonexistent inventory cache (#72840)
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
4 years ago
Sloane Hertel 728dafb6eb
Make `ansible-galaxy collection verify` error out on no `MANIFEST.json`
PR #73403

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
4 years ago
Sviatoslav Sydorenko 6f4b4c345b
Match user-requested transitive pre-releases in collection dependency resolver (#73416)
* Add direct+transitive pre-release regression test

* Match user-requested transitive pre-releases

This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.

Co-authored-by: Jordan Borean <jborean93@gmail.com>
4 years ago
Brian Coca 5aa4295d74
change detection and check mode fixes for apt_key (#73334)
Change detection and check mode fixes for apt_key

* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
4 years ago
Sviatoslav Sydorenko 595413d113
Replace the inhouse collection dependency resolver with `resolvelib`
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
4 years ago
Brian Coca c7d4acc12f
allow env to override unspecified unsafe_writes (#73282)
* allow env var for fallback value for unspecified unsafe_writes
4 years ago
Martin Krizek a2af8432f3
Local vars should have highest precedence in AnsibleJ2Vars (#72830)
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix #6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.

Also removes AnsibleJ2Vars.extras instance variable which is not used.

Also adds missing test for #6653.

Fixes #72262
Fixes #72615

ci_complete
4 years ago
Alexander Sowitzki e6da544310 Let `command` always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
4 years ago
Rick Elrod 1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test (#69161)
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes #69203

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 4357a78130
[ansible-test] First attempt at freebsd/11.4 (#72655)
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes #48782

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 30d93995dd
Use a pty for local connections (#73023)
* Use a pty for local connections

Fixes #38696

Co-authored-by: James Cammarata <jimi@sngx.net>
4 years ago
Brian Coca 1e27d4052a
fix inventory source parse error handling (#73160)
fixes #51025

added test cases
4 years ago
Brian Coca 1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
4 years ago
Sloane Hertel 0c4de6839b
Add integration tests to ansible-galaxy-collection for 'ansible-galaxy collection verify' (#73229) 4 years ago
Sam Doran 0e6c334115
pause - adjust warning when run in background (#73182)
When the pause module is run in the background and seconds parameter is provided,
do not warn.

* Add tests
* Fix existing tests
  The test wasn't failing when it should have.
4 years ago
Sloane Hertel 9de2da8a7e
Add tests for merging and replacing vars from inventory sources (#73181) 4 years ago
Sam Doran a7e834071c
Add macOS 11 to CI (#72622)
* Fix connection_paramiko_ssh test for macOS 11
* Update Azure Pipelines config
* Add changelog
4 years ago
Sam Doran 264e08f21a
user - properly handle password and password lock when used together (#73016)
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.

On BSD, run separate commands as appropriate since locking and setting the password cannot 
be done in a single action.

FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.

* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied

* Simplify rc collection in FreeBSD class
  Since the _handle_lock() method was added, the rc would be set to None, which could make
  task change reporting incorrect. My first attempt to solve this used a set and was a bit too
  complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.

* Improve the Linux password hash and locking behavior
  If password lock and hash are provided, set the hash and lock the account by using a password
  hash since -L cannot be used with -p.

* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.

* Clarify password_lock behavior.
4 years ago
Matt Clay 456e9b7a33 Update pip integration test to use venv on py3. 4 years ago
Rick Elrod 44ee04bd1f
[dnf] Make "remove" filtering closer to dnf CLI (#73033)
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
  pkg specs (nv, nvr, nvra, wildcard, etc.)

Test Plan:
- New tests
- Local experiments with DNF API via PDB.

Tickets:
- Fixes #72809

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
David Shrewsbury c8666f07d7
Add support for argument_specs attribute to RoleMetadata class (#73120) 4 years ago
Rick Elrod b96732fa6b [ansible-test] Add Fedora 33 support and drop F31
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Sloane Hertel a469a4455a
Add ansible-galaxy-collection-scm tests for requirements.yml (#73029)
* Add ansible-galaxy-collection-scm tests for requirements.yml format

* Add a test for --no-deps

Fix assertion
4 years ago
Sam Doooran 003a9e890d
Constrain pexpect and ptyprocess (#73109)
The recent update of ptyprocess to 0.7.0 is incompatible with Python 2.6 and
is causing test failures.

* Add setup_pexpect role to expect test
4 years ago
pva 42bc03f0f5
[get_url] skip checksum during --check (#66700)
Fix get_url failure in check mode (--check) when using the checksum
format <algorithm>:<url>.

Regression introduced in (#20532)

Fixes: #61369
4 years ago
Brian Coca 932ba36160
ensure unsafe writes fallback (#70722)
* Ensure we actually fallback to unsafe_writes when set to true

 add integration test
 add fix for get_url not passing the parameter from args
4 years ago
Sam Doooran 48803604cd
systemd - do not overwrite unit name when searching (#72985)
* systemd - do not overwrite unit name when searching

PR #72702 introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.

* Use facts rather than a manual probe to determine if systmed is present

* Remov unnecessary block

* Use vars files instead of set_fact

* Add tests for using a templated unit file

* Update changelog fragment

* Use template to get correct path to sleep binary
4 years ago
Sam Doooran 8e022ef00a
import_playbook - change additional params to deprecation (#72987)
* import_playbook - change additional params to deprecation

I incorrectly recommended this be set as a warning when it should have been a deprecation.

* Fix deprecation sanity test to not required a collection name when not inside a collection
4 years ago
Felix Fontein dbc2c996ab Allow key None to prevent errors with import test. 4 years ago
Matt Clay 37d09f2488
Update ansible-test pylint Python support. (#72972)
* Update ansible-test pylint Python support.

* Python 3.8 is now officially supported.
* Python 3.9 is now skipped with a warning.
4 years ago
Sloane Hertel b0c78208fd
Add test for ansible-galaxy collection install
Test installing a collection to a directory containing other collections without any metadata (#72971)
4 years ago
Brian Coca d22804c4fb
saner path dir management (#72648)
* saner path dir management

   fixes #72628

   ensure we always store paths w/o a_c

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
4 years ago
Sam Doran 5157a92139 Remove exit code from traps in tests
Having the trap exit with a specific code will override
the exit code that caused the trap to run, which could
mask errors
4 years ago
David Shrewsbury 3a18ef7159
Change role argspec file used by ansible-doc (#72927)
* Change role argspec file used by ansible-doc

This changes the file used for role argument specs from meta/argument_specs.yml
to meta/main.yml. The argument specs are expected to be in that file under the
top-level entry of `argument_spec`.

* Switch to argument_specs
4 years ago
Toshio Kuratomi e7dee73774
Collection list site packages (#70173)
* ansible-galaxy collection list and verify now utilize collections in site-packages.

This is a short term fix for #70147.  The long term fix needs to handle
install (but that discussion is also bound up in how upgrade is going to
work and where things can get installed so it's deferred for 2.11.)

* Add test for ansible-galaxy collection list with site-packages

* Fix sanity issue

Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
4 years ago
Sloane Hertel 0a60e5e341
Add a couple more tests to ansible-galaxy-collection (#72931)
* Test installing a dependency compatible with multiple collection requirements

Test downloading a collection without dependencies

* Move cleanup to the end of the test suite
4 years ago
Matt Martz a3b6485073
Fix reset_connection paramiko, winrm, psrp (#72688)
* Ensure we only reset the connection when one has been previously established. Fixes #65812

* Ensure psrp doesn't trace

* winrm too

* Indentation fix
4 years ago
Matt Martz 83764ad506
Fix async interpreter parsing (#72636)
* Fix async interpreter parsing. Fixes #70690

* Target localhost instead of remote host

* Don't forget inventory

* Address shellcheck issue
4 years ago
Martin Krizek 1b70260d5a
Add implicit role_complete block instead of role._eor (#72208)
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes #69848
4 years ago
David Shrewsbury fe17cb6eba
Fix fileglob parameter order bug (#72879) 4 years ago
Sijis Aviles 034e9b0252
unarchive - add include option (#40522)
This should allow users to extract specific files from an archive as
desired.

Fixes #16130, #27081.

* Rebase and make a few minor changes
* Add changelog
* Improve tests

- move to separate tasks file
- change assertions to check for exactly one file
- use remote_tmp_dir for output dir

* Make exclude and include mutually exclusive
* Don't remove files needed by other tasks
* Fix sanity tests
* Improve feature documentation
* Skip tests that use map() on CentOS 6
* Use fnmatch on include for zip archives
  This matches the behavior of exclude

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Pilou b464d18fd1
AnsibleModule.set_mode_if_different: handle symlink is in a sticky directory (#45198)
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory

Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
4 years ago
Rick Elrod 9e7616b208 Fix yum_repository for more centos6 EOL fallout
Change:
- Use a version specific vars file to specify archive url for EPEL.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
David Shrewsbury 2eb9795543 Test pip fix 4 years ago
Yadnesh Kulkarni 3add96909d
Fix parsing of values when using an empty string as key (#57132) (#72545)
Signed-off-by: Yadnesh Kulkarni <ykulkarn@redhat.com>
4 years ago
Jordan Borean fb092a82a1
ansible-galaxy - source deps from all servers and not just parent (#72576)
* ansible-galaxy - source deps from all servers and not just parent

* Added integration tests for this scenario
4 years ago
Alan Rominger 1fbac24739
Update ansible-runner tests version and deps (#72197)
We could not find reason for the docutils install so trying out removing it.
Also bumping to latest version of ansible-runner

Skip all python2 versions because next ansible-runner drops it

Co-authored-by: Elijah DeLee <kdelee@redhat.com>
4 years ago
David Shrewsbury 570aed0913
ansible-doc role arg spec support (#72120)
* Support listing roles in text and JSON

* Change tests for unfrack'd playbook_dir var

These tests were using '/tmp' for testing the setting of the playbook_dir
var. Now that we unfrack that var, MacOS will change this to '/private/tmp'
causing the tests to fail. We can choose a path that does not exist (since
unfrack does not validate existence) so that we can guarantee unfracking
will not change the value.
4 years ago
Brian Coca 13bc35e13c
actual min changes needed for rhel8 (#72477)
* actual min changes needed for rhel8

 - specifiy releaserver as dnf/vars is ignored
4 years ago
Rick Elrod 6894ae7d1d
Rename to ansible-core (#72594)
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base

Test Plan:
- ci_complete
4 years ago
Sam Doran 4b8cb6582b
pause - do not hang if run in the background (#72065)
* Consolidate logic for determining whether or not session is interactive
  into a single function, is_interactive()
* Increase test coverage

I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
4 years ago
Jordan Borean aee7a3ed68
Fix FreeBSD HTTP Kerberos setup (#72595) 4 years ago
Jordan Borean 03a395cba4
Rollback pulp container to stabilise CI (#72580) 4 years ago
Jordan Borean a521ea0a13
Tweak galaxy CI tests to bring stability (#72578) 4 years ago
Brian Coca 4f0e2fff95
avoid fatal exception on invalid collection name (#72296)
* avoid fatal exception on invalid collection name used in ansible-doc filter
4 years ago
Brian Coca 48c08f410c
allow any type of unsafe data (#72547)
* allow any type of unsafe data

  dont limit to strings
4 years ago
Jordan Borean d451433e5d
Add RHEL 7.9 - ci_complete (#72558) 4 years ago
Brian Coca aa4d53ccdf
ensure local exposes correct user (#72543)
* ensure local exposes correct user

  avoid corner case in which delegation relied on
  playcontext fallback which was removed

  fixes #72541
4 years ago
Jordan Borean de5858f48d
Added caching mechanism for Galaxy API requests (#71904)
* Added caching mechanism for Galaxy API requests

* Add cache options and split up code

* Added unit tests

* Fix sanity test

* Use modified date and fix up caching for explicit servers

* Make sure credentials are not in cached server name

* Added test for getting updated cache version

* Changes from review
4 years ago
Brian Coca a1730af91f
Ensure blockinfile correctly returns backupfile (#72544)
* Ensure blockinfile correctly returns backupfile

  Fixes #27626
  based on #27859

Co-authored-by: Giovanni Sciortino (@giovannisciortino)
4 years ago
Rick Elrod 11b7091c84 fix paramiko deps for freebsd/12.2
Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 8eab113cb1
show keyword documentation in ansible-doc (#72476)
* show keyword documentation in ansible-doc

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Sloane Hertel 09ad4baf5b
Remove incidental_setup_ec2 test target (#72523) 4 years ago
Sloane Hertel 0ed7bfc694
Fix task get_name to always prepend the role name (#72511)
* Fix 'role_name : tast_name' notation if task contains role name

* Add tests for notifying handler names which contain the role name

Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
4 years ago
Matt Martz c8590c7482
Various intentional tests (#72485)
* Add tests for argspec choices type=list

* Add explicit interpreter discovery tests to validate modules returning ansible_facts still set interp

* Add explicit tests for missing_required_lib

* Add explicit tests for recursive_diff

* ci_complete ci_coverage

* Update data to cover more code/tests

* ci_complete ci_coverage

* Add argspec tests for aliases, and no_log

* Forgotten file

* ci_complete ci_coverage

* Add argspec tests for type int

* ci_complete ci_coverage

* Remove incidental_k8s

* ci_complete ci_coverage

* fix missing newline

* Remove incidental_sts_assume_role

* ci_complete ci_coverage
4 years ago
Rick Elrod d8c637da37
[dnf] Some fixes around filtering (#72483)
Change:
- Docs: Add note that security/bugfix apply to dependencies too, like
  the dnf command.

- dnf: security/bugfix only makes sense for updates, so limit the
  package query sack to available updates.

- tests: Limit tests to our known-good test packages, so that RHEL
  packages marked security/bugfix without similarly marked dependencies
  don't fail our tests.

Test Plan:
- Tested with `dnf upgrade-minimal --bugfix` and reproduced the same
  error currently seen in CI, showing that we are consistent with what
  dnf does.

Tickets:
- Likely fixes #72316

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 880087748c
Remove incidental tower integration tests (#72461)
* Add explicit test for dict repr in argspec

* Add explicit test for undefined repr

* ci_complete ci_coverage

* Skip old jinja2

* ci_complete ci_coverage

* Remove incidental_tower_receive

* ci_complete ci_coverage

* Remove incidental_tower_credential_type

* ci_complete ci_coverage

* Remove ignore entries

* ci_complete ci_coverage
4 years ago
Felix Fontein da60525610
Fix missing ansible.builtin FQCNs in hardcoded action names (#71824)
* Make sure hard-coded action names also check for FQCN.
* Use _add_internal_fqcn() to avoid hardcoded lists and typoes.
4 years ago
Felix Fontein 6b30efa454 Fix fallout from #70028. 4 years ago
Matt Martz ccc63abc8e
Remove incidental vmware tests (#72420)
* Add explicit tests for ansible.module_utils.common.network.is_mac

* Test missing ANSIBLE_CONFIG

* Add explicit inventory script tests

* Add explicit tests for _consume_options

* Adjust perms, remove extra file

* ci_complete ci_coverage

* remove incidental_inventory_vmware_vm_inventory

* ci_complete ci_coverage

* Remove incidental_script_inventory_vmware_inventory

* ci_complete ci_coverage

* Remove incidental_vmware_guest

* ci_complete ci_coverage

* Remove incidental_vmware_host_hyperthreading and incidental_vmware_prepare_tests

* ci_complete ci_coverage

* newline and comment about what the test is doing

* ci_complete ci_coverage
4 years ago
Matt Martz 6543c7bc5d
Remove incidental_cs_role_permission (#72380)
* Add explicit argspec tests for choices

* ci_complete ci_coverage

* Remove incidental_cs_role_permission

* ci_complete ci_coverage

* ci_complete ci_coverage
4 years ago
Matt Martz 2ee5af514b
Update pip tests to omit install dev extras to avoid dep issues (#72436) 4 years ago
Brian Coca 4b673484f0
rethink wording (#70028)
* rethink wording
* removed unrequired requirement
* fix tests
* fixed versions
Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
Brian Coca cb94c0cc55
added timeout options to adhoc and console (#71230)
* added timeout options to adhoc and console

* added test

* fix typosesz

* fix conflict

* task_timeout

* fix timeout option, added extra vars to console

* actually use right cli switch .. DUH!

* added timeout to include 'valid' but ignored keys

* fix default

* fixes per review
4 years ago
Brian Coca 96c1972439
find will give more specific reasons on skip (#68823)
* find will give more specific reasons on skip

 - added more error handling
 - use random path for test
4 years ago
Sam Doran fb09fd2a23
wait_for - ignore psutil related errors (#72401)
When enumerating connections with psutil, catch and ignore errors to avoid returning a stack trace.

Co-authored-by:  Matt Martz <matt@sivel.net>
4 years ago
Matt Martz e73a0b2460
Explicitly get the include task, and not assume it is the parent (#72378)
* Explicitly get the include task, and not assume it is the parent. Fixes #6571
4 years ago
Sam Doran a51a6f4a25
reboot - add reboot_command parameter (#69847)
Fixes #51359
* Update default search paths
* Fix returns for args and command, don't allow conversion
* Reorganize tests
* Fix test for Azure Pipelines
4 years ago
Brian Coca e05c62547b
run playbook from collections (#67435)
* fixes for collection playbooks

 - add fqcn invocation, also handles extensions
 - bring import_playbook into new normal
 - avoid adding collection playbooks to adjacency
 - deal with extensions or lack of em
 - added tests
 - fix bugs with multiple playbooks supplied
 - unicode/bytes/native fixes
 - fix modulenotfound for < py3.7
 - tests
4 years ago
Jordan Borean 4856ab0e68
Galaxy server update (#72286)
* Update galaxy server image used in CI

* Fix port and migration issue

* Add delete step, still need to deal with pagination

* Make cleanup more efficient

* Remove testing code
4 years ago
Sam Doran 7352457e7b
hostname - add macOS (#54439)
* Add DarwinStrategy class and integration tests

macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.

* Update documentation

* Account for virtualization type returned on Azure Pipelines

* Do not be dependent on order of self.name_types

Use the scrubbed name when the name type is LocalHostName
4 years ago
Sam Doran c51438312a
blockinfile - properly insert block when no trailing new line exists (#72350) 4 years ago
Matt Clay b848fa0fc7 Update hostname test to skip containerd. 4 years ago
Sloane Hertel 8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
4 years ago
Felix Fontein 1489bf9190
Adds argspec tests for required, required_one_of and required_by (#72245)
* Improve variable names.

* Add test for required.

* Add test for required_one_of.

* Add test for required_by.
4 years ago
Matt Martz be5fc4e642
Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
4 years ago
Rick Elrod 9ddb1d76af
[tests/dnf] work around dnf packaging issue (#72249)
Change:
- In this test we end up upgrading dnf (and python3-dnf) so that we can
  test its new logging behavior. However, the latest Fedora 32 dnf had a
  packaging issue which caused it to not pull in the latest
  python3-libdnf. This is fixed, but not synced out to mirrors yet.
  Fixing it in this test will get CI passing again in the meanwhile.

Test Plan:
- CI

Tickets:
- https://bugzilla.redhat.com/show_bug.cgi?id=1887502

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 5ec53f9db8
make collection callbacks follow normal flow (#59932)
make collections whitelist follow normal flow

* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
4 years ago
Jordan Borean caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
4 years ago
Matt Martz c4acd41d6e
Ensure delegate vars calculation has correct loop context. Fixes #37132 (#71477) 4 years ago
Matt Martz 5d811fc551
Remove incidental_win_psexec (#71953)
* Explicit coverage to remove incidental_win_psexec

* rc=1 failed

* ci_complete ci_coverage

* Test rc!=0 without failed and until

* ci_complete ci_coverage

* Remove incidental_win_psexec

* Reduce delay

* ci_complete ci_coverage
4 years ago
Amin Vakil bb2c96aacc
Fix subversion_pacakges name (#72138) 4 years ago
Matt Clay 5f76bd2af7 Support collection constraints in ansible-test.
This allows collections to specify requirements and constraints for packages that ansible-test has requirements or constraints for.
4 years ago
Matt Martz 2f8dbf673e
Remove incidental_zabbix_host (#72142)
* Add explicit apt tests for fnmatch and update_cache

* Add explicit apt_key tests for fetching key directly from url

* ci_complete ci_coverage

* Remove repo only by repo

* ci_complete ci_coverage

* Add apt cache update after apt_repository to show that the cache doesn't update

* ci_complete ci_coverage

* Add systemd tests for enabling and disabling a service

* ci_complete ci_coverage

* Remove incidental_zabbix_host

* ci_complete ci_coverage
4 years ago
Matt Martz cf0cd4b50b
Remove incidental_aws_codebuild (#72140)
* Remove incidental_aws_codebuild

* ci_complete ci_coverage
4 years ago
Sloane Hertel 102e429285
Add test to replace exclusive incidental_cloudformation coverage (#72137)
* Add intentional coverage for incidental_cloudformation

* Remove the incidental_cloudformation test target and the now unused supporting modules
4 years ago
Matt Martz cfa41898c4
Add explicit coverage of argspec choices with strings that shadow YAML bools (#72122)
* Add explicit coverage of argspec choices with strings that shadow YAML bools

* ci_complete ci_coverage

* Remove incidental_ufw

* ci_complete ci_coverage
4 years ago
Matt Martz 50c8c87fe2
Add explicit coverage of suboptions=list without elements (#72108)
* Add explicit coverage of suboptions=list without elements

* ci_complete ci_coverage

* Remove incidental_vmware_guest_custom_attributes

* ci_complete ci_coverage
4 years ago
Sloane Hertel 960e4c0809
Add intentional coverage for incidental_ec2_instance (#72028)
* Add a test suite for module_utils.common.dict_transformations

* ci_complete

ci_coverage

Add a wait_for test using delegate_to

* Remove incidental_ec2_instance

* Remove unused test support modules

* Requested changes

ci_complete

ci_coverage

* Oops, put everything back to test coverage again

ci_complete

ci_coverage

* Remove incidental_ec2_instance tests and supporting modules
4 years ago
Rick Elrod ebc91a9b93
[unarchive] work on older RHEL with group: <gid> (#72098)
Change:
- No longer fail due to old Fedora/RHEL and our failure to try to cast
  gids to integers before trying to pass them to getgrgid() before
  trying to use them.
- Add tests for user/mode for various unarchive formats.

Test Plan:
- New integration tests, ran against centos6 container

Tickets:
- Fixes #71903
4 years ago
Matt Martz 460ba041c8
Add explicit coverage of required_together (#72107)
* Add explicit coverage of required_together

* ci_complete ci_coverage

* Remove incidental_hcloud_server

* Remove hcloud from shippable matrix

* ci_complete ci_coverage
4 years ago
Matt Martz 7f1c32f5ce
Rewrite _is_role and remove incidental_flatpak_remote (#72105)
* Build explicit list instead of complicated nested loop

* ci_complete ci_coverage

* Remove incidental_flatpak_remote

* ci_complete ci_coverage
4 years ago
Matt Martz 35809806d3
Remove incidental_timezone (#72063)
* Add explicit coverage for templated undefined with_items element

* ci_complete ci_coverage

* Remove incidental_timezone

* ci_complete ci_coverage
4 years ago
Matt Martz ab2b339dd6
More explicit argspec tests (#72064)
* Add more explicit coverage of argspec functionality

* fail_on_missing_params

* ci_complete ci_coverage

* Remove incidental_aws_step_functions_state_machine

* ci_complete ci_coverage

* Remove incidental_cs_service_offering

* ci_complete ci_coverage
4 years ago
Ruediger Pluem a7170da851
user - allow local users with an expiry date to be created (#72022)
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.

Fixes: #71942

In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.

Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.
4 years ago
Sloane Hertel cceba07114
VariableManager - add the 'vars' key before getting delegated variables (#71214)
* VariableManager - add the 'vars' key before getting delegated variables
4 years ago
Erik Zettel 2b79cd2c0f
Fix typos (#72072) 4 years ago
Jordan Borean 09c2475368
Update the tests to work with new httptester container (#72013) 4 years ago
Matt Martz 9081b22868
Remove incidental_vyos_static_route (#72024)
* Add explicit tests for required_together suboptions

* ci_complete ci_coverage

* Remove incidental_vyos_static_route

* ci_complete ci_coverage

* Add explicit coverage of suboptions required_if

* ci_complete ci_coverage

* Remove incidental_vyos_logging

* ci_complete ci_coverage
4 years ago
Matt Martz 6f4aed5377
Remove incidental_nios_txt_record (#72009)
* Add explicit coverage of argspec type=dict

* Non string mapping failure

* ci_complete ci_coverage

* Remove incidental_nios_txt_record and associated files

* Don't forget the ignore.txt changes

* ci_complete ci_coverage
4 years ago
Amin Vakil 8f02819db0
Replace deprectated include with {import,include}_tasks in apt{,_repository,_key} integration tests (#71964)
* Replace deprectated include with import_tasks

* Fix
4 years ago
Matt Martz f2512f7931
Remove incidental_xml integration tests (#71917)
* Add explicit integration tests to cover incidental_xml coverage

* normalize diff

* More normalization

* diff without changed

* ci_complete ci_coverage

* nuke incidental_xml

* ci_complete ci_coverage
4 years ago
Rick Elrod 6a7e637c5f
[tests] Make setup_docker clean up better (#71949)
Change:
- pip packages should get removed after, not try to add them again
- Try removing containerd.io package too

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 35b0fef536
Changes required for macOS 10.15 (#71841)
* Use default group of staff on macos to prevent sudo issues
* Install gnu-tar for macos in git and unarchive tests
* Enable timezone module to support py3 on macos
* If the virtualenv command is missing, try python -m virtualenv
* Install passlib for filter_core on macos
* Install paramiko via pip on macos for paramiko tests
* Normalize discovered python interpreter on macos
* Get pip tests passing, by ensuring we have wheel installed
* Create /etc/ansible for ca certs on mac, list lookup_url as destructive
* Fixups for CA certs
* Include macos
* Dynamically get cafile instead of hardcoding the path
4 years ago
Martin Krizek 90134b0ac3
Ensure native jinja is off for non-native tests (#71914)
Even if native jinja is off by default now it is better to be
explicit and it will be prepared in case we changed that default.
4 years ago
Rick Elrod 31ddca4c0d
[incidental_setup_docker] changes to upstream repo (#71897)
Change:
- The docker-ce.repo file for centos does not work on RHEL since it uses
  $releasever and on RHEL that is, e.g., "7Server".
- Instead, set up the repo manually.
- Additionally, the docker centos8 repo no longer has old versions, so
  we use the (only) version in the repo instead.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Amin Vakil 98d10cd6ee
Fix conditional for the dnf nobest test (#71805) 4 years ago
Rick Elrod 5cd489af06
Let get_file_attributes() work without `lsattr -v` (#71845)
* Let get_file_attributes() work without `lsattr -v`

Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
  in some cases, it may not.
- The function now takes an optional include_version bool parameter,
  which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
  it returns, we now call it with include_version=False.

Test Plan:
- New unit tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod abfb7919dc
[default callback] Show include_tasks task banner (#71821)
Change:
- In some cases (always with free strategy, sometimes with linear), the
  default callback would not show the task banner for include_tasks.
- This only affects the include_tasks task itself, not the tasks in the
  included file.

Test Plan:
- Updated default callback tests

Tickets:
- Fixes #71277

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz a99212464c
Remove incidental_consul tests (#71811)
* Add explicit intg tests for argspec functionality

* ci_complete ci_coverage

* Remove incidental_consul and incidental_setup_openssl

* ci_complete ci_coverage
4 years ago
Martin Krizek 00b22ab55e
Provide more information in AnsibleUndefinedVariable (#71666)
* Provide more information in AnsibleUndefinedVariable

Fixes #55152
4 years ago
Sam Doran 6984081111
Test changes to allow using "macos" test image (#71849)
* Add link to remote.sh from macos.sh
* Add skip/macos to tests that have skip/osx
4 years ago
Matt Clay 782effbb89 Fix hostname test on docker.
The test is not supported when running in a container. It now recognizes both 'docker' and 'container' as virtualization types that should cause the test to be skipped.
4 years ago
Matt Clay 17765cd4e8
Fix attribute testing in file integration test. (#71843)
The file test will no longer attempt to test attributes if `lsattr -vd` does not work on the system under test.
4 years ago
Matt Martz e6e9840717
Remove incidentals without coverage (#71788)
* Remove incidental_lookup_hashi_vault
* Remove incidental_connection_chroot
* Remove incidental_selinux
* Remove incidental_win_hosts
4 years ago
Matt Martz d63d880383
Remove incidental_synchronize tests (#71776)
* Remove incidental_synchronize tests

* Update sanity ignore
4 years ago