Commit Graph

10899 Commits (78f34786dd468c42d7a222468685590207e74679)

Author SHA1 Message Date
Alexei Znamensky 5654de6fce
Return error if cwd directory does not exist (#72390)
* Return warning or error if cwd directory does not exist, in AnsibleModule.run_command()
4 years ago
Matt Martz 880087748c
Remove incidental tower integration tests (#72461)
* Add explicit test for dict repr in argspec

* Add explicit test for undefined repr

* ci_complete ci_coverage

* Skip old jinja2

* ci_complete ci_coverage

* Remove incidental_tower_receive

* ci_complete ci_coverage

* Remove incidental_tower_credential_type

* ci_complete ci_coverage

* Remove ignore entries

* ci_complete ci_coverage
4 years ago
Jordan Borean 1eae2bb890
Update Galaxy NG version used in CI (#72469) 4 years ago
Felix Fontein da60525610
Fix missing ansible.builtin FQCNs in hardcoded action names (#71824)
* Make sure hard-coded action names also check for FQCN.
* Use _add_internal_fqcn() to avoid hardcoded lists and typoes.
4 years ago
Matt Clay 79021df2e4 Restore ansible-test coverage config.
Reverts a portion of the changes made in https://github.com/ansible/ansible/pull/70028
4 years ago
Felix Fontein 6b30efa454 Fix fallout from #70028. 4 years ago
Matt Martz ccc63abc8e
Remove incidental vmware tests (#72420)
* Add explicit tests for ansible.module_utils.common.network.is_mac

* Test missing ANSIBLE_CONFIG

* Add explicit inventory script tests

* Add explicit tests for _consume_options

* Adjust perms, remove extra file

* ci_complete ci_coverage

* remove incidental_inventory_vmware_vm_inventory

* ci_complete ci_coverage

* Remove incidental_script_inventory_vmware_inventory

* ci_complete ci_coverage

* Remove incidental_vmware_guest

* ci_complete ci_coverage

* Remove incidental_vmware_host_hyperthreading and incidental_vmware_prepare_tests

* ci_complete ci_coverage

* newline and comment about what the test is doing

* ci_complete ci_coverage
4 years ago
Felix Fontein 64a809d2b6
Improve ansible-test classifications for collections (#72353) 4 years ago
Matt Martz 6543c7bc5d
Remove incidental_cs_role_permission (#72380)
* Add explicit argspec tests for choices

* ci_complete ci_coverage

* Remove incidental_cs_role_permission

* ci_complete ci_coverage

* ci_complete ci_coverage
4 years ago
Matt Martz 2ee5af514b
Update pip tests to omit install dev extras to avoid dep issues (#72436) 4 years ago
Jordan Borean 809d5fc398
ansible-test - skip installing PowerShell sanity test reqs if they are already installed (#72423) 4 years ago
Brian Coca 4b673484f0
rethink wording (#70028)
* rethink wording
* removed unrequired requirement
* fix tests
* fixed versions
Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
Brian Coca cb94c0cc55
added timeout options to adhoc and console (#71230)
* added timeout options to adhoc and console

* added test

* fix typosesz

* fix conflict

* task_timeout

* fix timeout option, added extra vars to console

* actually use right cli switch .. DUH!

* added timeout to include 'valid' but ignored keys

* fix default

* fixes per review
4 years ago
Brian Coca 96c1972439
find will give more specific reasons on skip (#68823)
* find will give more specific reasons on skip

 - added more error handling
 - use random path for test
4 years ago
Sam Doran fb09fd2a23
wait_for - ignore psutil related errors (#72401)
When enumerating connections with psutil, catch and ignore errors to avoid returning a stack trace.

Co-authored-by:  Matt Martz <matt@sivel.net>
4 years ago
Matt Martz e73a0b2460
Explicitly get the include task, and not assume it is the parent (#72378)
* Explicitly get the include task, and not assume it is the parent. Fixes #6571
4 years ago
Sam Doran a51a6f4a25
reboot - add reboot_command parameter (#69847)
Fixes #51359
* Update default search paths
* Fix returns for args and command, don't allow conversion
* Reorganize tests
* Fix test for Azure Pipelines
4 years ago
Brian Coca e05c62547b
run playbook from collections (#67435)
* fixes for collection playbooks

 - add fqcn invocation, also handles extensions
 - bring import_playbook into new normal
 - avoid adding collection playbooks to adjacency
 - deal with extensions or lack of em
 - added tests
 - fix bugs with multiple playbooks supplied
 - unicode/bytes/native fixes
 - fix modulenotfound for < py3.7
 - tests
4 years ago
Sam Doran 8f9cf456b0
url lookup - set default user agent (#72324)
* Add unit tests
* Add note about when default changed
4 years ago
Jordan Borean 4856ab0e68
Galaxy server update (#72286)
* Update galaxy server image used in CI

* Fix port and migration issue

* Add delete step, still need to deal with pagination

* Make cleanup more efficient

* Remove testing code
4 years ago
Sam Doran 7352457e7b
hostname - add macOS (#54439)
* Add DarwinStrategy class and integration tests

macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.

* Update documentation

* Account for virtualization type returned on Azure Pipelines

* Do not be dependent on order of self.name_types

Use the scrubbed name when the name type is LocalHostName
4 years ago
Sam Doran c51438312a
blockinfile - properly insert block when no trailing new line exists (#72350) 4 years ago
Felix Fontein 233e7beb5b
Fix processing of add_file_common_args=True when argument_spec is not specified as kwarg. (#72334) 4 years ago
Matt Clay b848fa0fc7 Update hostname test to skip containerd. 4 years ago
Rick Elrod 2293b327c0
[ansible-test] Limit cryptography to <3.2 (#72342)
Change:
- Cryptography 3.2 drops support for OpenSSL 1.0.2. Some of our CI
  infrastructure still uses this version (FreeBSD, namely). For now,
  just add a constraint to use old cryptography.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Sloane Hertel 8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
4 years ago
Matt Davis 83909bfa22
Remove ansible-galaxy login (#72288)
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries

* remove dead code/config, update messages and porting guides
4 years ago
Matt Clay 3c2e8b99be
Fix ansible-test handling of user-defined docker networks. (#72256)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry.
4 years ago
Felix Fontein 1489bf9190
Adds argspec tests for required, required_one_of and required_by (#72245)
* Improve variable names.

* Add test for required.

* Add test for required_one_of.

* Add test for required_by.
4 years ago
Matt Martz be5fc4e642
Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
4 years ago
Rick Elrod 9ddb1d76af
[tests/dnf] work around dnf packaging issue (#72249)
Change:
- In this test we end up upgrading dnf (and python3-dnf) so that we can
  test its new logging behavior. However, the latest Fedora 32 dnf had a
  packaging issue which caused it to not pull in the latest
  python3-libdnf. This is fixed, but not synced out to mirrors yet.
  Fixing it in this test will get CI passing again in the meanwhile.

Test Plan:
- CI

Tickets:
- https://bugzilla.redhat.com/show_bug.cgi?id=1887502

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 5ec53f9db8
make collection callbacks follow normal flow (#59932)
make collections whitelist follow normal flow

* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
4 years ago
Matt Clay 2ef4b7e07e Fix ansible-test Azure Pipelines container auth. 4 years ago
Rick Elrod 3b40c6f3b7
[config] coerce more to string when 'type: str' (#72172)
Change:
- When a plugin defines `type: str` on a parameter, treat more kinds of
  input as a string instead of whatever it is parsed as.

Test Plan:
- New unit tests
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
dberg1 cdf62edc65
service_facts: return service state information on AIX (#72073)
* service_facts: return service state information on AIX

AIX uses the System Resource Controller (SRC) to manage services.
See https://www.ibm.com/support/knowledgecenter/ssw_aix_72/osmanagement/sysrescon.htm
Use lssrc command on AIX to return service state information.


Co-authored-by: dberg1 <dberg1@github.com>
4 years ago
Jordan Borean caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
4 years ago
Matt Martz c4acd41d6e
Ensure delegate vars calculation has correct loop context. Fixes #37132 (#71477) 4 years ago
Matt Martz 5d811fc551
Remove incidental_win_psexec (#71953)
* Explicit coverage to remove incidental_win_psexec

* rc=1 failed

* ci_complete ci_coverage

* Test rc!=0 without failed and until

* ci_complete ci_coverage

* Remove incidental_win_psexec

* Reduce delay

* ci_complete ci_coverage
4 years ago
Florian Heiderich e673b8e4ec
fix typo (foward -> forward) (#72182) 4 years ago
Amin Vakil bb2c96aacc
Fix subversion_pacakges name (#72138) 4 years ago
Matt Clay 5f76bd2af7 Support collection constraints in ansible-test.
This allows collections to specify requirements and constraints for packages that ansible-test has requirements or constraints for.
4 years ago
Matt Martz 2f8dbf673e
Remove incidental_zabbix_host (#72142)
* Add explicit apt tests for fnmatch and update_cache

* Add explicit apt_key tests for fetching key directly from url

* ci_complete ci_coverage

* Remove repo only by repo

* ci_complete ci_coverage

* Add apt cache update after apt_repository to show that the cache doesn't update

* ci_complete ci_coverage

* Add systemd tests for enabling and disabling a service

* ci_complete ci_coverage

* Remove incidental_zabbix_host

* ci_complete ci_coverage
4 years ago
Matt Martz cf0cd4b50b
Remove incidental_aws_codebuild (#72140)
* Remove incidental_aws_codebuild

* ci_complete ci_coverage
4 years ago
Sloane Hertel 102e429285
Add test to replace exclusive incidental_cloudformation coverage (#72137)
* Add intentional coverage for incidental_cloudformation

* Remove the incidental_cloudformation test target and the now unused supporting modules
4 years ago
Jordan Borean 1ae3683d0c
Fix example to use correct shebang (#72129)
* Fix example to use correct shebang

* Fix other example modules as well

* Ignore shebang test
4 years ago
Matt Martz cfa41898c4
Add explicit coverage of argspec choices with strings that shadow YAML bools (#72122)
* Add explicit coverage of argspec choices with strings that shadow YAML bools

* ci_complete ci_coverage

* Remove incidental_ufw

* ci_complete ci_coverage
4 years ago
Matt Martz 50c8c87fe2
Add explicit coverage of suboptions=list without elements (#72108)
* Add explicit coverage of suboptions=list without elements

* ci_complete ci_coverage

* Remove incidental_vmware_guest_custom_attributes

* ci_complete ci_coverage
4 years ago
Sloane Hertel 960e4c0809
Add intentional coverage for incidental_ec2_instance (#72028)
* Add a test suite for module_utils.common.dict_transformations

* ci_complete

ci_coverage

Add a wait_for test using delegate_to

* Remove incidental_ec2_instance

* Remove unused test support modules

* Requested changes

ci_complete

ci_coverage

* Oops, put everything back to test coverage again

ci_complete

ci_coverage

* Remove incidental_ec2_instance tests and supporting modules
4 years ago
Rick Elrod ebc91a9b93
[unarchive] work on older RHEL with group: <gid> (#72098)
Change:
- No longer fail due to old Fedora/RHEL and our failure to try to cast
  gids to integers before trying to pass them to getgrgid() before
  trying to use them.
- Add tests for user/mode for various unarchive formats.

Test Plan:
- New integration tests, ran against centos6 container

Tickets:
- Fixes #71903
4 years ago
Rick Elrod 0332046699
[ansible-test] attempt to work around podman (#72096)
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.

Test Plan:
- Tested with podman 2.0.6 in Fedora 31.

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Matt Martz 460ba041c8
Add explicit coverage of required_together (#72107)
* Add explicit coverage of required_together

* ci_complete ci_coverage

* Remove incidental_hcloud_server

* Remove hcloud from shippable matrix

* ci_complete ci_coverage
4 years ago
Matt Martz 7f1c32f5ce
Rewrite _is_role and remove incidental_flatpak_remote (#72105)
* Build explicit list instead of complicated nested loop

* ci_complete ci_coverage

* Remove incidental_flatpak_remote

* ci_complete ci_coverage
4 years ago
Evan Van Dam 3db08adbb1
Add optional attribute arg for min and max filters (#50909)
* Pass **kwargs to min and max filters

* Use the jinja2 filters if available

* Add unit tests

* Add examples to docs passing attribute
4 years ago
Rick Elrod 709484969c
sysctl/openbsd fact fixes (#72070)
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`

Test Plan:
- New unit tests

Tickets:
- Fixes #71968
- Refs #72025
- Refs #72067

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
4 years ago
Matt Martz 35809806d3
Remove incidental_timezone (#72063)
* Add explicit coverage for templated undefined with_items element

* ci_complete ci_coverage

* Remove incidental_timezone

* ci_complete ci_coverage
4 years ago
Matt Martz ab2b339dd6
More explicit argspec tests (#72064)
* Add more explicit coverage of argspec functionality

* fail_on_missing_params

* ci_complete ci_coverage

* Remove incidental_aws_step_functions_state_machine

* ci_complete ci_coverage

* Remove incidental_cs_service_offering

* ci_complete ci_coverage
4 years ago
Ruediger Pluem a7170da851
user - allow local users with an expiry date to be created (#72022)
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.

Fixes: #71942

In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.

Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.
4 years ago
Sloane Hertel cceba07114
VariableManager - add the 'vars' key before getting delegated variables (#71214)
* VariableManager - add the 'vars' key before getting delegated variables
4 years ago
Matt Martz fb03ac7019
Add libyaml info to version output, and restore git info (#71948)
* Add libyaml info to version, restore git functionality

* Add clog fragment

* Update tests to reflect new version output
4 years ago
Erik Zettel 2b79cd2c0f
Fix typos (#72072) 4 years ago
Felix Fontein 1107aace1b
Add deprecation collection name to plugin options (#71735)
* Add changelog fragment.

* Use correct field that's expected by lib/ansible/cli/__init__.py..

* Add basic unit tests.
4 years ago
Jordan Borean 672df07a33
Bump httptester version (#72036) 4 years ago
Jordan Borean 09c2475368
Update the tests to work with new httptester container (#72013) 4 years ago
Matt Martz 9081b22868
Remove incidental_vyos_static_route (#72024)
* Add explicit tests for required_together suboptions

* ci_complete ci_coverage

* Remove incidental_vyos_static_route

* ci_complete ci_coverage

* Add explicit coverage of suboptions required_if

* ci_complete ci_coverage

* Remove incidental_vyos_logging

* ci_complete ci_coverage
4 years ago
Christian Loos ea119d3089
fix distribution fact for SLES4SAP (#71559)
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.

This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.

The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
4 years ago
Matt Martz 6f4aed5377
Remove incidental_nios_txt_record (#72009)
* Add explicit coverage of argspec type=dict

* Non string mapping failure

* ci_complete ci_coverage

* Remove incidental_nios_txt_record and associated files

* Don't forget the ignore.txt changes

* ci_complete ci_coverage
4 years ago
Felix Fontein a077bca5d5
ansible-test: improve version number validation, validate some semantic versioning properties (#71679)
* Validate removal versions.
* Validate that removal collection versions and version_added collection versions conform to semver spec.
* Validate removal version numbers in meta/runtime.yml.
* Stricter validation for isodates (f.ex. YYYY-M-D is not allowed).
* Improve error reporting.
* Validate removal collection versions.

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
4 years ago
Amin Vakil 8f02819db0
Replace deprectated include with {import,include}_tasks in apt{,_repository,_key} integration tests (#71964)
* Replace deprectated include with import_tasks

* Fix
4 years ago
flowerysong 91a153434a
Fix sorting of ignore.txt (#71930)
Mostly out of order from directory flattening
4 years ago
Matt Martz f2512f7931
Remove incidental_xml integration tests (#71917)
* Add explicit integration tests to cover incidental_xml coverage

* normalize diff

* More normalization

* diff without changed

* ci_complete ci_coverage

* nuke incidental_xml

* ci_complete ci_coverage
4 years ago
Jorge Vallecillo 94522b7c10
Fix typos (#71970) 4 years ago
Rick Elrod 6a7e637c5f
[tests] Make setup_docker clean up better (#71949)
Change:
- pip packages should get removed after, not try to add them again
- Try removing containerd.io package too

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 35b0fef536
Changes required for macOS 10.15 (#71841)
* Use default group of staff on macos to prevent sudo issues
* Install gnu-tar for macos in git and unarchive tests
* Enable timezone module to support py3 on macos
* If the virtualenv command is missing, try python -m virtualenv
* Install passlib for filter_core on macos
* Install paramiko via pip on macos for paramiko tests
* Normalize discovered python interpreter on macos
* Get pip tests passing, by ensuring we have wheel installed
* Create /etc/ansible for ca certs on mac, list lookup_url as destructive
* Fixups for CA certs
* Include macos
* Dynamically get cafile instead of hardcoding the path
4 years ago
Alicia Cozine 27826827e9
updates intersphinx references for docs links (#71921)
* DOCS: updates intersphinx references for docs links
* TESTS: Raise the number of bytes scanned to determine if a file is binary. The newest ansible-2.10.inv file has its first null byte at position 2261. 4096 is still a cheap chunksize to read so it still makes sense to raise this.

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
4 years ago
Sviatoslav Sydorenko ece0be30ab
Move ansible.galaxy.collection module into package
This change preserves how all the external imports refer to this code
while allowing us to start cutting the spaghetti into more easily
maintainable pieces.

This is a start of the upcoming refactoring effort destined to
eliminate tight coupling, implicit data manipulation, god objects,
abstraction leaks and other code smells.

Essentially, `ansible.galaxy.collection` is going to be a package that
holds parts of the collection management code spread across loosely
coupled modules.

PR: #71931
4 years ago
Matt Clay 92b66e3e31
CI provider fixes for ansible-test. (#71929)
* Make Azure Pipelines resource_prefix lowercase.

* Make classification of CI files consistent.

* Update package-data sanity test for AZP.
4 years ago
Martin Krizek 90134b0ac3
Ensure native jinja is off for non-native tests (#71914)
Even if native jinja is off by default now it is better to be
explicit and it will be prepared in case we changed that default.
4 years ago
René Moser 5348e7bf61
test: cloudstack: Use test-container 1.4.0 (#71905) 4 years ago
Rick Elrod 31ddca4c0d
[incidental_setup_docker] changes to upstream repo (#71897)
Change:
- The docker-ce.repo file for centos does not work on RHEL since it uses
  $releasever and on RHEL that is, e.g., "7Server".
- Instead, set up the repo manually.
- Additionally, the docker centos8 repo no longer has old versions, so
  we use the (only) version in the repo instead.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Amin Vakil 98d10cd6ee
Fix conditional for the dnf nobest test (#71805) 4 years ago
Rick Elrod 5cd489af06
Let get_file_attributes() work without `lsattr -v` (#71845)
* Let get_file_attributes() work without `lsattr -v`

Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
  in some cases, it may not.
- The function now takes an optional include_version bool parameter,
  which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
  it returns, we now call it with include_version=False.

Test Plan:
- New unit tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Amin Vakil 3e9943bc5e
Reduce ignored module sanity tests (#70319)
* Reorder test/sanity/ignore.txt to better see changes
* Remove extra empty line causing pass fail
* Apply suggestions made by Akasurde
* Minor fix in package_facts and remove two unnecessary ignores
* Fix subversion based on a suggestion made by felixfontein
* Apply suggestions made by felixfontein and Andersson007
* Fix subversion.py as suggested by felixfontein
* Minor reformatting in yum_repository description
* Reformat changelog
* Add key to apt_key, add deprecated changelog
* Add PR url to changelog
* Ignore paramater-type-not-in-doc in favour of adding key back to apt_key
* Fix apt_key
* Remove undocumented-paramater from apt_key ignore
* Ignore doc-choices-do-not-match-spec in package_facts
* Fix package_facts
* Fix filter option in setup module
4 years ago
Rick Elrod abfb7919dc
[default callback] Show include_tasks task banner (#71821)
Change:
- In some cases (always with free strategy, sometimes with linear), the
  default callback would not show the task banner for include_tasks.
- This only affects the include_tasks task itself, not the tasks in the
  included file.

Test Plan:
- Updated default callback tests

Tickets:
- Fixes #71277

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz a99212464c
Remove incidental_consul tests (#71811)
* Add explicit intg tests for argspec functionality

* ci_complete ci_coverage

* Remove incidental_consul and incidental_setup_openssl

* ci_complete ci_coverage
4 years ago
Martin Krizek 00b22ab55e
Provide more information in AnsibleUndefinedVariable (#71666)
* Provide more information in AnsibleUndefinedVariable

Fixes #55152
4 years ago
Sam Doran 6984081111
Test changes to allow using "macos" test image (#71849)
* Add link to remote.sh from macos.sh
* Add skip/macos to tests that have skip/osx
4 years ago
Matt Clay 782effbb89 Fix hostname test on docker.
The test is not supported when running in a container. It now recognizes both 'docker' and 'container' as virtualization types that should cause the test to be skipped.
4 years ago
Matt Clay 17765cd4e8
Fix attribute testing in file integration test. (#71843)
The file test will no longer attempt to test attributes if `lsattr -vd` does not work on the system under test.
4 years ago
Matt Martz e6e9840717
Remove incidentals without coverage (#71788)
* Remove incidental_lookup_hashi_vault
* Remove incidental_connection_chroot
* Remove incidental_selinux
* Remove incidental_win_hosts
4 years ago
Matt Martz d63d880383
Remove incidental_synchronize tests (#71776)
* Remove incidental_synchronize tests

* Update sanity ignore
4 years ago
Matt Martz 195c8b496e
Add integration coverage to replace hash_vault and consul_session (#71775)
* Add integration coverage to replace hash_vault and consul_session

ci_complete
ci_coverage

* Remove errant list element
4 years ago
Sam Doran 65cdb86c8a incidental_setup_openssl - Pin version of Python packages
A recent update to cffi that was yanked is still being installed on our
Mac OS X 10.11 test image since the version of pip there is very old and
does not ignore yanked packages.

Pin the version of pyOpenSSL and its dependencies to fix this and avoid
future spontaneous failures.
4 years ago
Rick Elrod fdf80690e4
[dnf] accumulate update filters (#71726)
Change:
- Previously when `security: true` and `bugfix: true` were both given,
  only security updates would get applied. Filters now accumulate so
  that both get applied in this case.

Test Plan:
- New integration tests for both check_mode and not. These tests make
  use of a contrived yum repository which is stored in S3.

Tickets:
- Fixes #70854

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Martz <matt@sivel.net>

Co-authored-by: Matt Martz <matt@sivel.net>
4 years ago
Abhijeet Kasurde 173091e2e3
distro: Add support for Pardus distribution (#71663)
Fixes: #71636

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Davis e813b0151c
fix coverage output from synthetic packages (#71727)
* fix coverage output from synthetic packages

* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.

* add unit test
4 years ago
Tadej Borovšak 74a103d655 Remove em dash from the pytest configuration file
On some systems (tested with official Debian Buster-based Python 3.6
docker image), setting the LC_ALL environment variable to en_US.UTF-8
will trigger Python into switching its preferred encoding to ASCII.
If any python process tries to read a non-ASCII file in this scenario,
it will terminate with an error.

And this is exactly what happens to pytest when it tries to load its
configuration that ansible-test supplies because the configuration
contains an em dash.

In order to bypass this issue, we replaced the em dash with a regular
dash and things started working again.

An alternative solution would be to replace the en_US.UTF-8 locale with
something safer, but unfortunately, the en_US.UTF-8 is probably as safe
as it gets.
4 years ago
Martin Krizek 59a240cd31
Fix with_dict/with_list being passed incorrect type tests (#71699)
These tests were failing due to a variable passed being undefined
instead of being of an incorrect type which is the actual
intent of the tests.
4 years ago
Jérôme Loyet daca3ade99
Allow list of filters for the setup module (#68551)
The setup module can now filter out multiple pattern by providing a list
to the filter parameter instead of just a string. Single string sill
works. Previous behaviour remains.

(cherry picked from commit b5c36dac483fdd74d6c570d77cc8f3e396720366)
4 years ago
Matt Martz 61f6aa55b6
Add semver support to the jinja2 version test (#71600) 4 years ago
Jordan Borean b615789fcc
psrp - fix hang when copying an empty file (#71649) 4 years ago
Matt Martz 48f12c14e9
AnsibleVaultEncryptedUnicode should be considered a string (#71609)
* AnsibleVaultEncryptedUnicode should be considered a string

* linting fix

* clog frag
4 years ago
Andrew Klychkov d398a4b4f0
file: module should warn in check_mode when path and owner/group don't exist (#69640)
* file: module must fail in check_mode when path and owner/group don't exist
4 years ago
Martin Krizek a3b954e5c9
Force template module to use non-native Jinja2 (#68560)
Fixes #46169
4 years ago
Matt Clay 3d769f3a76 Add pause to avoid same mtime in test. 4 years ago
Matt Martz db98433e7a
Use InventoryManager for inventory_hostnames lookup (#71573)
* Use InventoryManager for inventory_hostnames lookup. Fixes #17268

* Add test for using patterns

* s/it's/its/

* Allow bypassing parse_sources in InventoryManager

* Remove unneeded method call
4 years ago
Matt Martz 05788c05d3 Remove incidental_postgresql_user tests 4 years ago
Rick Elrod 64e2b9da3b
[yum tests] add missing "always" block (#71564)
Change:
- So that if yum tests get re-run, we don't error out the second time.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod db519bc17c
[tests] Bump container versions (#71518)
Change:
- This pulls in python 3.9.0rc1

Test Plan:
- CI, hopefully

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 98febab975 Use new endpoint for Parallels based instances. 4 years ago
Tadej Borovšak b9e2c4e37d
Add tests for ignore functionality of collection build (#69345)
In order to test this piece of functionality, we create a dedicated
collection that contains files that should be ignored when building
the collection tarball.

After the collection is built, we make sure the produced tarball does
not contain any files that should be ignored.
4 years ago
Abhijeet Kasurde 159544610e
get_url: Handle same SHA sum for downloaded files (#71435)
Fixes: #71420

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Rick Elrod 0f3b5bb4a6
[tests] fix fallout from dnf gpg fix (#71548)
Change:
- Missed needed gpg ignores

Test Plan:
- CI
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Sam Doran 785012321f
Fix setup_paramiko test (#71547) 4 years ago
Rick Elrod 9bea33ffa3
[dnf] ensure packages are gpg-verified (#71537)
Change:
- By default the dnf API does not gpg-verify packages. This is a feature
  that is executed in its CLI code. It never made it into Ansible's
  usage of the API, so packages were previously not verified.
- This fixes CVE-2020-14365.

Test Plan:
- New integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Martin Krizek d083307e36
Set global skipped result flag for looped tasks (#67847)
This allows for the skipped filter to be used on a registered looped
task results.

Fixes #16949
4 years ago
Matt Clay 2bbcbe99fd Support macOS 10.15 for ansible-test --remote.
Use of this new version is experimental, so it is not enabled in CI yet.
4 years ago
Martin Krizek fbdfb85c20
Remove duplicate unit test (#71488)
test_dict_undefined_values is a duplicate of test_dict_undefined_values_is_defined.
It seems originally test_dict_undefined_values was supposed
to do what test_dict_undefined_values_bare does so it is safe to remove.
4 years ago
Martin Krizek 50f221c63e
Fix incorrect msg in the results dict in loops (#71482) 4 years ago
Matt Martz a1a624b150
Normalize how None is handled in quote filter. Fixes #32174 (#71473) 4 years ago
Matt Martz fdf5dd02b3
Updates for password hashing (#71120)
* Validate salt when using crypt. Respect salt_size in password lookup. Repair salt for bcrypt. Fixes #71107. Fixes #53750. Fixes #36129.

* Handle algorithms we don't know about, and make sure to return the salt

* Account for old passlib

* Add tests for salt constraints

* Add changelog fragment

* Add test for #36129
4 years ago
Martin Krizek 9792d631b1
_check_failed_state: always use the current/nested state (#71347)
Fixes #71306
4 years ago
Matt Clay f5b6df14ab
Fix ansible-test coverage traceback. (#71446)
* Add integration test for ansible-test coverage.

* Fix ansible-test coverage traceback.

* Fix coverage reporting on Python 2.6.
4 years ago
Jordan Borean 72a7cb4a2c
powershell - fix quoting values (#71411)
* powershell - fix quoting values

* Add ignore for smart quote skip
4 years ago
Jordan Borean 8897d7e2ff
powershell - fix nested CLIXML parser (#71412) 4 years ago
Matt Clay d099591964
Update ansible-test remote endpoint handling. (#71413)
* Request ansible-core-ci resources by provider.
* Remove obsolete us-east-2 CI endpoint.
* Add new --remote-endpoint option.
* Add warning for --remote-aws-region option.
* Update service endpoints.
* Allow non-standard remote stages.
* Add changelog fragment.
4 years ago
Sloane Hertel 1425e3597b
Allow meta tasks to use tags (#67508)
* Support tags for explicit meta tasks

Add --list-tasks tests for meta tags

Add breaking_changes changelog fragment and porting guide
4 years ago
Rick Elrod e5bb7b1a16
Add skip_reason to meta task skips (#71355)
Change:
- Make them more consistent with other tasks.

Test Plan:
- CI, new test

Tickets:
- Refs #71009

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Martin Krizek 2675768e3d
Make noop tasks implicit (#71344)
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca f8aa307659
add extra vars to inventory plugins (#70970)
* add extra vars to inventory plugins
* disable by default for backwards compat
* also allow extra vars for options
4 years ago
Rick Elrod ea58d7c233
Make it so callback plugins can act on implicit/explicit meta tasks (#71009)
Change:
- Now sends meta tasks to the task start callback
- Lets callback plugins opt-in to receiving implicit tasks

Test Plan:
- New integration tests

Tickets:
- Indirectly fixes #71007 by allowing custom callbacks with this data

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Abhijeet Kasurde 4f993922c8
Add documentation about info/facts module development (#71250)
Fixes: #40151

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
David Shrewsbury f2f6c34632
Fix play stats when rescue block is a child block (#70922)
* check run state of current block only

* Add changelog and test

* Add test for issue 29047

* Fix for both tests

* blerg

* Change test messages

* fix tests

* Add multi-level block in rescue test case

* Add recursive rescue check and multi-level test

* Should probably run the new test

* ci_complete

* Merge new tests

* ci_complete
4 years ago
Matt Martz 6289570234
epoch can be a float with strftime filter. Fixes #71257 (#71314) 4 years ago
Edwin Hermans eb8b3a8479
get_url - Allow checksum file to be local file:// url (#71205)
This would be a partial solution for #69364 in that the SHASUMS file can be downloaded and gpg verified but then used from the downloaded location to verify the get_url's file.
* Make checksum url parsing more explicit

Use urlsplit to test if the checksum string has a (currently tested and) supported url scheme.

* Fix whitespace
* Changelog fragment
* Added tests
* Fix typo in test setup
4 years ago
Lorenzo Castelli a1a50bb3cd
systemd - supports new systemctl output message for chroot (#71197)
The message generated by systemctl has been updated in 9321e23c40, which requires a corresponding change in the systemd module.

In addition, this fixes the module when the SYSTEMD_OFFLINE environment variable is set.
4 years ago
Matt Martz 5821128995
Allow callbacks from forks (#70501)
* POC for supporting callback events that come from the worker

* linting fixes. ci_complete

* fix up units. ci_complete

* Try moving the sentinel put higher. ci_complete

* safeguards. ci_complete

* Move queue killing to terminate

* LINTING. ci_complete

* Subclass Queue, to add helper send_callback method

* Just use _final_q instead of adding another queue and thread

* Revert a few changes

* Add helper for inserting a TaskResult into the _final_q

* Add changelog fragment

* Address rebase issue

* ci_complete

* Add test to assert async poll callback from fork

* Don't use full path

* ci_complete

* Use _results_lock as a context manager

* Add new generic lock decorator, and use it with send_callback
4 years ago
Toshio Kuratomi d816a5966e Force an upgrade to a newer version of antsibull 4 years ago
Florent PIGOUT 336c176ebc
Fix cron file regression (#71207) 4 years ago
Sam Doran 888be697cb
Revert "Change default file permissions so they are not world readable (#70221)" (#71231)
* Revert "Change default file permissions so they are not world readable (#70221)"

This reverts commit 5260527c4a.

* Revert "Fix warning for new default permissions when mode is not specified (#70976)"

This reverts commit dc79528cc6.
4 years ago
Matt Martz c59472715a
Add support for non-yaml EXAMPLES (#71184)
* Add support for non-yaml EXAMPLES

* Make pattern raw string

* fmt tag should be the first non-whitespace line

* Add changelog fragment
4 years ago
Martin Krizek b66d66027e
Skip literal_eval for string filters results in native jinja. (#70988)
Fixes #70831
4 years ago
Matt Martz ae1291004e
Make sure that apache2-webdav related packages get upgraded (#71187) 4 years ago
Rick Elrod 80ad1a17df
Remove incidental_lvg target (#71174)
Change:
- This no longer has any incidental coverage.

Test Plan:
- Coverage report

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 4f54fb8d43
Remove incidental azure tests (#71164)
Change:
- All incidental coverage from these tests has moved to intentional
  coverage.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jordan Borean 3f22f79e73
Ensure -k is set to delegated hosts without a pass (#71136)
* Ensure -k is set to delegated hosts without a pass

* Fix up some broken tests

* Update task_executor.py

one possible fix, the other is updating winrm to normalize on 'password' like the other connection plugins

* Add alias for winrm and fix incorrect assumption

* Make sure aliases are used for keyword options

* Conditionally run test if sshpass is present, fix sanity

Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
4 years ago
Matt Martz 7c60dadb9a
Updates to Integration tests to pass against Alpine (#70946)
* Start of alpine testing

* More updates

* Add forgotten file

* remove debug

* Add alpine3

* equal

* group 4

* group 4

* group 5

* Try to decrease test length

* libuser only available in testing

* Remove debug

* Make loops target work on hosts without gnu date

* Enable alpine testing

* ci_complete

* Don't specify uid for creating test user

* ci_complete

* Re-sort docker completion

* use newer container image

* ci_complete

* fix indentation

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Martz 410b6a7109
Update integration tests to support rpmfluff-0.6 (#71155) 4 years ago
Felix Fontein 050841324c
ansible-test: bump acme test container version to 2.0.0 (#71097) 4 years ago
Sam Doran 6ac4439a6a
Drop "rhui-" prefix from RHEL repositories in AMI (#71130)
The repository names seem to have changed and no longer have the "rhui-" prefix
4 years ago
Felix Fontein fbfc0f99eb
Make changelog tool be more strict about suffixes (#70909)
* Bump antsibull-changelog version.

* Flag all dotfiles, except .keep and .gitkeep.

* Enable ignoring other fragment extensions.
4 years ago
Toshio Kuratomi fb144c4414
Update ansible doc formats (#71070)
* Fix tty_ify bugs and refactor

* Move tty_ify() and supporting attributes to the DocCLI class as that's
  the only thing using it.
* Add unittest for the code.
* Fix a bug where the substitution macros can be detected when they are
  a part of another word.
* Add support for L(), R(), and HORIZONTALLINE which were added to the
  website docs many years ago.

* Update test/units/cli/test_doc.py

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Brian Coca 662d34b9a7
add jinja2 global to reserved names (#71088)
* add jinja2 global to reserved names

  also allow expansion by additional context provided from caller

  fixes #41955

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Abhijeet Kasurde 504ef607f3
Misc typo fixes (#71089) 4 years ago
Brian Coca f9af27c631
dont clobber facts in loop (#71032)
* dont clobber facts in loop

	fixes #71031
4 years ago
Rick Elrod 0d7c144ce4
Allow macOS ACLs to work for unpriv -> unpriv (#70785)
Change:
- Use `chmod +a` in the fallback chain to allow MacOS to use ACLs to
  allow an unprivileged user to become an unprivileged user.

Test Plan:
- CI, new tests

Tickets:
- Fixes #70648

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 4e27569347
Add more include and yaml parsing tests (#70506)
These additional tests should provide coverage for features currently tested by the postgres incidental tests.
4 years ago
Rick Elrod cdcf0aa42a
Intentional coverage, incidental_azure_rm_resource (#71052)
Change:
- Adds some intentional coverage around PluginLoader for cases that
  incidental_azure_rm_resource covered.
- Specifically, modules starting with an underscore, and starting with
  an underscore but a symlink.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 8af3079640
Add intentional coverage for incidental_lvg (#71043)
Change:
- Add hardware_facts test target which manually sets up some LVM devices
  and tests facts against them.

Test Plan:
- New integration tests

Tickets:
- Refs #71041 and #71042 both of which I discovered during this

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 810a9a5593
find: Allow reading whole file for contains regex (#71083)
Change:
- Add a parameter `read_whole_file` which allows for reading the whole
  file when doing a `contains` regex search.
- This allows for (for example) matching a pattern at the very end of
  a file.

Test Plan:
- New integration tests

Tickets:
- Fixes #63378

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Martin Krizek 5ca3aec3c4
native types: properly handle Undefined in nested data (#68432) 4 years ago
Rick Elrod 3bec27dc34
Allow cache_timeout=0 to reach inv. cache plugins (#70977)
Change:
- Previously, `cache_timeout=0` was seen as falsey and not passed along
  to cache plugins as an option. Now only "nulley" values are not sent.

Test Plan:
- New integration tests

Tickets:
- Fixes #70702

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jordan Borean f6b3b4b430
ansible-galaxy - fix download for subdirs in SCM (#71005) 4 years ago
Matt Davis 86b6c4bbb6
fix collection package root location under pytest >=6.0.0 (#70963) 4 years ago
Matt Clay b53e7f8720 Fix ansible-test self-test change classification.
Changes to sanity and unit tests now trigger the ansible-test self-test integration tests.

No changelog entry since this only affects tests for ansible itself and not collections.
4 years ago
Rick Elrod 7f62a9d7b5
Make cache adjudicator's flush call plugin flush (#70987)
Change:
- Previously CachePluginAdjudicator#flush only removed entries from the
  cache backend that it knew about by using them earlier. Now it calls
  the underlying plugin's flush() method.

Test Plan:
- New unit tests

Tickets:
- Fixes #68770

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Toshio Kuratomi e10902d744
Add the ansible_2_10 intersphinx reference. (#71028)
There are links from the scenario guides to collections but collections
docs aren't built in testing or locally when we're on the devel branch.
Due to that we need to make sure those references resolve to the
production docsite.  We can use intersphinx to make sure that happens.

* The test for binary files wasn't reading enough of the file.

Checking for null bytes in the first 1024 bytes failed to diagnose the
ansible_2_10.inv file as binary
4 years ago
Sloane Hertel 991714b9d1
copy - redact 'content' from invocation in check mode (#71033)
* sanitize copy module invocation secrets in check mode
4 years ago
Rick Elrod 74c14c6743
Strip spaces in module names in explicit actions (#71040)
* Strip spaces in module names in explicit actions

Change:
- When an action is called like "action: copy foo=bar", strip spaces
  around the action name.
- This allows "action: copy        foo=bar" to work as expected.

Test Plan:
- New integration tests

Tickets:
- Fixes #62136

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Ganesh Nalawade 6160e82bf2
Update network integration auth timeout (#71057)
*  Incase of network integration test for connection local
   test the paramiko_ssh auth_timeout is the value of timeout
   under defaults section which is 10 seconds.
*  For slower connection 10sec timout value result in authentication
   timeout error hence increase the timeout value to 90 seconds
4 years ago
Rick Elrod 96aa81e8ff
Remove incidental_azure_rm_mariadbserver tests (#71037)
Change:
- Remove incidental test which has no remaining incidental coverage.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Sam Doran f99f96ceb6
Fix unstable unarchive test (#71004)
* Add mode to copy tasks
* Fix unreliable test by ignoring errors
4 years ago
Martin Krizek bf7276a4e8
Emit proper error for `x in y` when y is undefined (#70990)
Fixes #70984
4 years ago
Philip Douglass ac5f3f8bef
unarchive - Check 'fut_gid' against 'run_gid' in addition to supplemental groups (#65666)
Add integration tests for unarchiving as unprivileged user
Break tasks into separate files for easier reading and maintenance

Create a user by specifying a default group of 'staff' for macOS.

The user module does not actually remove the user directory on macOS,
so explicitly remove it.

Put the removal tasks in an always block to ensure they always run

Co-authored-by: Philip Douglass <philip.douglass@amadeus.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Sloane Hertel 54e2ae79e7
fix downloading collections in git repos and tar.gz artifacts (#70524)
* Fix downloading tar files

* Fix downloading SCM collections

* changelog
4 years ago
Sam Doran e6bf202738
linux facts - return proper broadcast address (#64528)
* linux facts - return proper broadcast address

Check that the value being returned is actually a broadcast address

* Add tests

* Cleanup tests
4 years ago
Sam Doran c4f442ed5a
facts - fix incorrect time for some date_time_facts (#70665)
The iso8601_micro and iso8601 facts incorrectly called now.utcnow(), resulting
in a new timestamp at the time it was called, not a conversion of the previously
stored timestamp.

Correct this by capturing the UTC timestamp once then calculating the local
time using the UTC offset of the current system.

* Use time.time() for getting the current time
* Convert from that stored epoch timestamp to local and UTC times
* Used existing timestamp for epoch time
* Add unit tests that validate the formate of the return value rather than an exact value since mocking time and timezone is non-trivial
4 years ago
Matt Clay 2e0097ada3 Fix ansible-test relative import analysis. 4 years ago
Sam Doran dc79528cc6
Fix warning for new default permissions when mode is not specified (#70976)
Follow up to #70221
Related to #67794
CVE-2020-1736

When set_mode_if_different() is called with mode of 'None', ensure we issue
a warning about the change in default permissions.

Add integration tests to ensure the warning works properly.

* Fix tests
- actually use custom module 🤦‍♂️
- verify file permission on created files
- use remote_tmp_dir so we're ready for split controller
- improve test module so we can skip the call to set_fs_attributes_if_different()
- fix tests for CentOS 6
4 years ago
Rick Elrod 14dc4de424
Update docs for --tags default, and add some tests (#70939)
Change:
- Clarify that not passing `--tags` will cause `ansible_run_tags` to
  default to `["all"]`.
- Add some extra coverage around `ansible_run_tags`

Test Plan:
- New integration and unit tests

Tickets:
- Fixes #69619

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jordan Borean b1cb2553af
ansible-galaxy - fix fallback for AH searches (#70957) 4 years ago
Rick Elrod 20a7476064
git: verify, only use --raw when we need it (#70900)
Change:
- Allow older git to verify tags again
- Enable verification tests everywhere, even if most of them only work
  on newer git. Some of them work on older git and they test the --raw
  parameter.

Test Plan:
- Re-enabled subset of git tests

Tickets:
- Fixes #64469

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca f9c3c6cba6
Allow changed/failed mgmt on strategy actions (#70919)
* Allow changed/failed mgmt on strategy actions
4 years ago
Matt Clay 3a8ac62596 Cap pytest version to avoid relative import issue. 4 years ago
Sloane Hertel 5c8520fbd9
Fix verbosity for var_templating test (#70943)
* Fix verbosity for var_templating test

Display the subtests

* fix specificity and set ANSIBLE_BECOME_ALLOW_SAME_USER

Co-authored-by: Matt Martz <matt@sivel.net>

Co-authored-by: Matt Martz <matt@sivel.net>
4 years ago
Rick Elrod 7d32129efb
[dnf] show installations/removals in check_mode (#70892)
Change:
- Previously, we only showed that something would have changed, not what
  would have changed. This allows us to show what will chang as well.

Test Plan:
- Local RHEL8 VM
- New integration tests

Tickets:
- Fixes #66132

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Felix Fontein 7e2cc7db12
validate-modules: fix version_added validation for top-level, fix error codes (#70869)
* Also validate top-level version_added.

* Fix error code.

* Produce same version_added validation error in schema than in code (and stop returning it twice).

* Return correct error codes for invalid version_added for options and return values.

* Add changelog.

* Fix forgotten closing braket.

* Accept 'historical' for some top-level version_added.
4 years ago
Rick Elrod be4be926c4
subversion Give subversion module a validate_certs option (#70890)
* Give subversion module a validate_certs option

Change:
- Add `validate_certs` option to subversion module. Defaults to off for
  backwards compatibility.

Tickets:
- Fixes #22599

Signed-off-by: Rick Elrod <rick@elrod.me>

* Update changelogs/fragments/22599_svn_validate_certs.yml

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>

* test verify_certs codepaths

Signed-off-by: Rick Elrod <rick@elrod.me>

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Amin Vakil 9d2982549d
dnf: Add nobest option (#70318)
* dnf: Add nobest option

* dnf: Fix indent, add nobest specifically to dnf not yum

* Add changelog for dnf: add nobest option

* dnf: Add nobest to yumdnf module argument_spec

* dnf: remove nobest from module paramaters in yumdnf.py

* dnf: Add test for nobest option

* dnf: Cleanup packages in nobest test at last

* dnf: Cleanup manually added repos in nobest test at last

* dnf: Remove dnf-plugins-core as well in nobest test

* dnf: Change nobest release version to 2.11

* Change changelog number according to change in PR number

* Change changelog number according to change in PR number
4 years ago
Brian Coca 1223ce656a
reset logging to INFO (#70878)
- due to CVE-2019-14846
 - also added comments and test to avoid 'oportunistic' reversion
4 years ago
Matt Clay 8b96caf712
Update ansible-base test container to 1.5.0. (#70856) 4 years ago
Jordan Borean 5e1a968983
Do not add connection vars to the output results (#70853)
* Do not add connection vars to the output results

* Also revert the delgated scenario JIC

* Added regression test
4 years ago
Matt Davis 4c0af6c808
fix internal cases of actions calling unqualified module names (#70818)
* fix internal cases of actions calling unqualified module names

* add porting_guide entry
* misc other fixes around action/module resolution broken by redirection

ci_complete

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* address review feedback

* pep8

* unit test fixes

* win fixes

* gather_facts fix module args ignores

* docs sanity

* pep8

* fix timeout test

* fix win name rewrites

Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Abhijeet Kasurde e439194c8c
basic: Add name of aliases in error message (#69427)
With this patch, user will be notified with available aliases
of arg parameter.

Fixes: #58752

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doran 5260527c4a
Change default file permissions so they are not world readable (#70221)
* Change default file permissions so they are not world readable

CVE-2020-1736

Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.

A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.

- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
4 years ago
David Shrewsbury bf98f031f3
Sanitize URI module keys with no_log values (#70762)
* Add sanitize_keys() to module_utils.

* More robust tests

* Revert 69653 change

* Allow list or dict

* fix pep8

* Sanitize lists within dict values

* words

* First pass at uri module

* Fix insane sanity tests

* fix integration tests

* Add changelog

* Remove unit test introduced in 69653

* Add ignore_keys param

* Sanitize all-the-things

* Ignore '_ansible*' keys

* cleanup

* Use module.no_log_values

* Avoid deep recursion issues by using deferred removal structure.

* Nit cleanups

* Add doc blurb

* spelling

* ci_complete
4 years ago
Brian Coca 84adaba6f5
Allow hostvars delegation (#70331)
* ensure hostvars are available on delegation
* also inventory_hostname must point to current host and not delegated one
* fix get_connection since it was still mixing original host vars and delegated ones
* also return connection vars for delegation and non delegation alike
* add test to ensure we have expected usage when directly assigning for non delegated host
4 years ago
Matt Martz 96b74d3e0b
Ensure single vaulted values aren't counted as sequences. Fixes #70784 (#70786) 4 years ago
Baptiste Mille-Mathias 0ead4306a8
user - Create home and parent directories only when requested (#70600)
The home user and the parents directories should only be created when
create_home == True
4 years ago
psi / Ryo Hirafuji 5ce47646ad
cron - Allow non-ascii (UTF-8) chars in cron file paths and jobs (#70426)
* Encode/Decode files in UTF-8
* Use helper function in ansible
* Add an integration test
* Use emoji in test data.
* add changelog
* Also support non-ascii chars in filepath and add tests about this.
* Also use non-ascii chars in replaced text and ensure not to break cron syntax.
* rename self.existing to self.n_existing
* rename crontab.existing to crontab.n_existing
4 years ago
Abhijeet Kasurde 4f96f9826c
distribution: Add support for DragonFly (#70748)
partially fixes #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Rick Elrod 69472a5f8d
Refactor _fixup_perms2 to remove way-nested logic (#70701)
Change:
- Refactoring to make it harder to get wrong and easier to read.
- Generalize become_unprivileged tests and fix some that never worked
  but also never failed.

Test Plan:
- CI, new units/integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Kohles 77d0effcc5
Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577 (#69578)
* Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577

* adding changelog

* fixing typo in changelog entry

* adding test case

Adding test case written by bmillemayhias.

* using $HOME instead of ~

* fixing commit measage

* Update 69578-shell-remote_tmp-quoting.yaml

Co-authored-by: Brian Kohles <me@briankohles.com>
4 years ago
Abhijeet Kasurde c1402ddee0
validate_modules: added missing single quote (#70744)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Martz f90aa5599f
Don't treat empty excludes as a match. Fixes #70640 (#70710) 4 years ago
Eduard Rozenberg 566c5e6ce1
Handle Slackware OS version strings containing a plus (“+”) (#68142)
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.

Rearrange distribution_files unit tests to easily support more tests
  - add conftest with common fixtures 
  - use parametrize for testing multiple scenarios

* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version

Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
4 years ago
Toshio Kuratomi 9dda393d70
Collections docs generation (#59761)
* Build documentation for Ansible-2.10 (formerly known as ACD).

Builds plugin docs from collections whose source is on galaxy

The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.

* Update the python syntax checks
  * docs builds can now require python 3.6+.

* Move plugin formatter code out to an external tool, antsibull-docs.
  Collection owners want to be able to extract docs for their own
  websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs

* Cleanup and bugfix to other build script code:
  * The Commands class needed to have its metaclass set for abstractmethod
    to work correctly
  * Fix lint issues in some command plugins

* Add the docs/docsite/rst/collections to .gitignore as
  everything in that directory will be generated so we don't want any of
  it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages

* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs

* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined

* Remove references to plugin documentation locations that no longer exist.
  * Perhaps the pages in plugins/*.rst should be deprecated
    altogether and their content moved?
  * If not, perhaps we want to rephrase and link into the collection
    documentation?
  * Or perhaps we want to link to the plugins which are present in
    collections/ansible/builtin?

* Remove PYTHONPATH from the build-ansible calls
  One of the design goals of the build-ansible.py script was for it to
  automatically set its library path to include the checkout of ansible
  and the library of code to implement itself.  Because it automatically
  includes the checkout of ansible, we don't need to set PYTHONPATH in
  the Makefile any longer.

* Create a command to only build ansible-base plugin docs
  * When building docs for devel, only build the ansible-base docs for
    now.  This is because antsibull needs support for building a "devel
    tree" of docs.  This can be changed once that is implemented
  * When building docs for the sanity tests, only build the ansible-base
    plugin docs for now.  Those are the docs which are in this repo so
    that seems appropriate for now.
4 years ago
Matt Davis c616e54a6e
refactor Python module_utils locator (#70610)
* refactor Python module_utils locator

* no longer recursive
* embed special-case module code internally
* share common code between collections/not cases
* fixes #70134
* properly support subpackage redirection
* adds support for FQCN redirect targets used by migration (expands to FQ Python name)
* add tests

* add changelog
4 years ago
Sloane Hertel 8c213c9334
template connection variables accessed directly before using (#70657)
* template variables accessed directly when using them instead of FieldAttributes
4 years ago
Abhijeet Kasurde 055871cbb8
api: time.clock compatible code (#70650)
time.clock is removed in Python 3.8. Add time.clock
compatible code.

Fixes: #70649

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doran 9d27d7c8b1
ansible_runner test - Add constraints (#70667)
A recent updated to psutil, which is a dependency of ansible-runner, fails
to install on older versions of pip.

Commit with the breaking change:

  135628639b
4 years ago
Rick Elrod 8455bc6838
Some more coverage for csvfile (#70661)
* Some more coverage for csvfile

Change:
- Add a few tests

Test Plan:
- CI, new tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod e1a33a6a84
inventory_hostnames lookup plugin -> 100% coverage (#70628)
Change:
- Add tests to get inventory_hostnames up to 100% coverage.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 01e30993fd
Get subelements lookup plugin coverage to 100% (#70634)
Change:
- Add a bunch of tests to bump coverage to 100%

Test Plan:
- CI, new tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Mykola Grygoriev fe86a93482
Add a new `date_time` fact to provide DST timezone
PR #70449
4 years ago
Jordan Borean e22e103cdf
winrm - Added kinit_args to control the args for kinit calls (#70624) 4 years ago
Matt Martz a77dbf0866
Allow single vault encrypted values to be used directly as module parameters. Fixes #68275 (#70607) 4 years ago
jabdr 0690b68bd3
Support datetime.date object in module result (#70595)
* Support datetime.date object in module result

Fixes #70583

* change blank lines for pep8 sanity test
4 years ago
Mark Chappell 8d160b1881
Remove remaining examples of 1.1.1.1 (#70552)
* Remove remaining examples of 1.1.1.1

* Update ec2_group.py
4 years ago
Matt Clay 1e02a201a6 Fix ansible-test virtualenv management. 4 years ago
Matt Clay 09f02980a4 Update ansible-test change detection for plugins. 4 years ago
Rick Elrod d372ce2c5d
Get m_u.facts.utils coverage up to 100% (#70614)
* Get m_u.facts.utils coverage up to 100%

Change:
- Add tests to 'gathering_facts' integration target to get
  module_utils.facts.utils coverage up to 100%.
- This also clears incidental coverage from incidental_selinux.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod d16018fe72
Add intentional coverage for an async_wrapper case (#70593)
Change:
- Test async_wrapper when the module it runs has stderr output

Test Plan:
- CI
- Looked at coverage report and saw green for a few lines that weren't
  previously green.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod d5480572c8
varnames: add tests, fix exception grammar (#70573)
Change:
- Add integration tests for various cases
- Fix wrong use of "its" in an exception thrown in varnames when it
  throws an AnsibleError, given a term of the wrong type.

Test Plan:
- new tests, CI

Tickets:
- Fixes #70546

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay df45dcdae0
Add integration tests for test plugins. (#70576) 4 years ago
Brian Coca cf89ca8a03
Make filter type errors 'loop friendly' (#70417)
- ensure we preserve the typeerror part of the exception so loop defereed error handling
 can postpone those caused by undefined variables until the when check is done.
 - fix tests to comply with the 'new normal'

 - human_to_bytes and others can issue TypeError not only on 'non string'
 but also bad string that is not convertable.

Co-authored-by: Sloane Hertel <shertel@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
Felix Fontein 24dcaf8974
plugin loader: return collection name; ansible-doc: handle ansible.builtin correctly (#70026)
* Determine collection in plugin loader.

* Fix test.

* Use PluginPathContext objects in PluginLoader._plugin_path_cache instead of tuples.
4 years ago
Rick Elrod 1b4fd23ba6
csvfile: use parse_kv() for args, add tests (#70550)
Change:
- Use parse_kv() for parsing in the csvfile lookup plugin. This allows
  us to handle multi-word search keys and filenames. Previously, the
  plugin split on space and so none of these things worked as expected.
- Add integration tests for csvfile, testing a plethora of weird cases.

Test Plan:
- New integration tests, CI

Tickets:
- Fixes #70545

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Felix Fontein f4c89eab23
ansible-doc: include collection name in text output (#70401)
* ansible-doc: include collection name in text output

* Be more careful to not accidentally pass ansible.builtin for user-supplied modules.
4 years ago
Amin Vakil 9c40b1b2ff
Replace filename with file in apt_key (#70492)
* Replace filename with file from apt_key check

  one is internal variable, the other is actual parameter used and required for parameter check.
4 years ago
Rick Elrod 9fbd65958d
hostname: hostnamectl check -> SystemdStrategy (#70532)
Change:
- Move hostnamectl check out of GenericStrategy because it was incorrect
  for everything except the SystemdStrategy which is where it belongs.
- Add some initial tests for the hostname module, though we are limited
  by the fact that we can't do much testing with it in containers.

Test Plan:
- new hostname integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Matt Clay bbd8f15a58 Correct name of fileglob lookup integration test. 4 years ago
Mykola Grygoriev 71c378e139
Fix decrypt argument in assemble module (#70465)
* Do not pass decrypt parameter to assemble module

* Add integration tests where decrypt=True

* Add changelog #70465
4 years ago
Alan Rominger b0d9deeae3
Fix ansible-test error in community.aws (#70507)
* Fix ansible-test error in community.aws

* Add changelog entry for fix

* Change check from None to string_types

* Update changelogs/fragments/70507-validate-null-author.yaml

clarify wording "or a list of strings"

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py

clarify wording - single string or not specified valid

Co-authored-by: Felix Fontein <felix@fontein.de>

* Do not fail but return None when given outside list

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
David Shrewsbury 8aca464b8b
Make sure ansible_become treated as a boolean (#70484)
* Make sure ansible_become treated as a boolean
4 years ago
Rick Elrod f3ef4ed076
Remove some constants.py deprecated items (#70466)
Change:
- Remove mk_boolean
- Remove BECOME_METHODS
- Remove get_config

Test Plan:
- CI, removed mk_boolean unit tests

Tickets:
- Fixes #69678

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 41414ed475
Deprecate command warnings feature (#70504)
Change:
- The command warnings feature which suggests that users use modules
  instead of certain commands is now deprecated. Its `warn` paramater
  and `COMMAND_WARNINGS` configuration options are also deprecated.
  Their use will become an error in version 2.13.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Rick Elrod 688cd8657b
Fix string/bytestring comparsion in m_u.basic (#70439)
Change:
- module_utils.basic.is_special_selinux_path() used a string ==
  bytestring comparison which returned False and made Ansible think that
  certain filesystems aren't, in fact, special-cased, when they should
  be. Ensure both sides of the == are bytestrings.

Test Plan:
- Added `copy` integration tests for this case.

Tickets:
- Fixes #70244

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Abhijeet Kasurde f5037314e3
sanity: remove ansible-bad-function (#70431)
* Updated docs and ignore.txt
* Replaced with correct function calls

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sloane Hertel 5b03267a1f
Add ansible-doc tests for documentation containing YAML anchors (#70436)
Co-authored-by: Tadej Borovšak <tadej.borovsak@xlab.si>
4 years ago
Felix Fontein 9164b96774
ansible-doc man formatter: fail with better error message when description isn't there (#70046)
* ansible-doc man formatter: do not crash when description isn't there.
* Change to report a better error message when description is not there.
* Add test.
4 years ago
Rick Elrod 1be78dbfc5
Updated bundled libraries (#70418)
Change:
- Update bundled six to 1.13 (last with py2.6 support)
- Make it pass lint
- Fix check to allow skipping over compat __init__.py files we authored
- Fix check to allow files that can't be updated for some reason

Test Plan:
- ansible-test sanity --docker
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 7a15a3a109
fix flatten handling of nulls/nones (#70141)
* fix flatten loop control issue (break -> continue)

fix issue #69012

(cherry picked from commit 2127be5ec5)

* fixed null break bug and added option to include

 fixes #69012
 fixes #69013

Co-authored-by: pseudocoder <borisovano@users.noreply.github.com>
Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
4 years ago
Matt Martz ce570691e7
Test against galaxy_ng (#70303)
* Test against galaxy_ng

* Switch container image

* Remove redundant |default

* Re-enable

* Update image

* Update wording

* Don't use pulp as the container name
4 years ago
Rick Elrod 91aea92c62
Add ability to fallback to chgrp remote_tmp and its files. (#68627)
* Add ability to fallback to chgrp remote_tmp and its files.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod f7078c1f8f
Throw a prettier error in m_u.basic syslog (#70312)
Change:
- In certain situations, such as when the input string contains null
  bytes (\0), syslog.syslog will throw a TypeError. Handle that and
  fail_json instead.

Test Plan:
- New test
- ansible-test --docker centos[68] (for py2 and py3 respectively)

Tickets:
- Refs #70269

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jordan Borean 40ce448657
ansible-test - do not validate blacklisted ps modules (#70376)
* ansible-test - do not validate blacklisted ps modules

* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Abhijeet Kasurde a8ae8f4d72
setup: Handle CIFS share with backward slash (#70006)
CIFS can be mounted using backward slash as well in /etc/fstab like

\\Windows\share /data/ cifs credentials=/root/.creds 0 0

Handle this condition while gather mount information in Linux.

Fixes: #48813

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Dmitriy Rabotyagov 8324db7cb5
Add new option for YUM4/DNF repositories (#70042)
Documentation reference:
  https://dnf.readthedocs.io/en/stable/conf_ref.html

Co-authored-by: Estelle Poulin <dev@inspiredby.es>
4 years ago
Abhijeet Kasurde 74bedab8a9
known_hosts: update documentation (#70342)
* Update documentation as per sanity tests
* Added example about custom SSH port in example section

Fixes: #29236

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
psi / Ryo Hirafuji 4997063b4a
apt - add fail_on_autoremove option to avoid unintended package removals (#70056)
* Ensure not to remove existing packages while installing apt packages.
* Make all lines shorter than 160 characters
* Allow removing packages only when upgrading.
* Add integration tests
4 years ago