Commit Graph

2775 Commits (aa12af1d346cefc296fb862705cbf62b42bfc1d3)

Author SHA1 Message Date
Brian Coca ea2f37d253
allow constructed to use vars plugin (#73418)
Allow constructed to optionally use vars plugin data

* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
3 years ago
Matt Clay 30a4ef4414 Update ansible-test default test containers. 3 years ago
Gonéri Le Bouder 0a8d5c0983 ansible-test: yamllint, check the assigment
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.

See: https://github.com/ansible/ansible/pull/73322
3 years ago
Matt Clay f533d46572
Fix ansible-test constraints issues. (#73578)
* Silence Python 3.5 EOL notice in pip.
* Skip import test on compat/selinux.py utils.
* Improve Python version skip warning.
* Use Python 3.6 as minimum Python for sanity tests.
* Improve min Python handling for code-smell tests.
* Overhaul test-constraints sanity test.
* Merge sanity test constraints with requirements.
* Remove legacy content specific constraints.
* Add changelog fragment.
3 years ago
Sam Doran b6811dfb61
Add argument spec validator (#73335)
Add argument spec validator class
3 years ago
Alexander Sowitzki d0fda3e901
Let vault lookup output unicode string. (#73571)
Until now, the lookup plugin returned a byte string.
Changed this to output a unicode string instead.
3 years ago
Brian Coca ec8a556538
only add data when there is data to add (#54559)
Only add data when there is data to add

  also avoid clobbering existing data with empty file
  fixes #45843

* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
3 years ago
Matt Davis 4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
3 years ago
Matt Clay 8a175f59c9
Upgrade pylint used by ansible-test. (#70155)
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
3 years ago
Jeff Squyres 6fd0a74601
unarchive: add support for .tar.zst (zstd compression) (#73265) 3 years ago
pushkarkumar15 4344607d7d
user - add password expiration parameters (#69531)
* Add integration test

Co-authored-by: Sam Doran <sdoran@redhat.com>
3 years ago
Sam Doran 2f5c83dfb1
Add AlmaLinux to the family of Red Hat-like operating systems (#73541)
* Add changelog and fixtures for AlmaLinux support

Co-authored-by: Christoph Schug <com+github@schug.net>
3 years ago
Sam Doran 7d18ea5e93
default callback - add "show_task_path_on_failure" option (#73260)
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.

* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()

* Keep color output consistent for now
  Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
  Instead of the color of the path changing based on verbosity level, just keep it at the
  currently behavior of COLOR_DEBUG. Having the color of the same information change
  based on verbosity level seems incorrect and makes visual parsing of the information
  more difficult.

Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
3 years ago
Felix Fontein ca448f7c35
Update collection routing (#73046)
* Docker content from community.general migrated to community.docker.

* Hetzner Robot content from community.general migrated to community.hrobot.

* RouterOS content from community.network migrated to community.routeros.

* PostgreSQL content from community.general migrated to community.postgresql.

* OC content from community.general migrated to community.okd.

* Cisco NSO content from community.network migrated to cisco.nso.

* Add changelog fragment.

* Google content from community.general migrated to community.google.

* Hashi Vault content from community.general migrated to community.hashi_vault.

* KubeVirt content from community.general migrated to community.kubevirt.

* FortiOS content from community.network migrated to community.fortios.

* Update changelog fragment.

* DellEMC content from community.general migrated to dellemc.openmanage.
3 years ago
Eugene Vilensky 65fc8c9e35
INTERPRETER_PYTHON_DISTRO_MAP: Treat oracle same as rhel/centos (#73498)
* treat oracle same as rhel/centos

* fix misunderstood use of yaml anchors

* add changelog fragment

* Update changelogs/fragments/73498-INTERPRETER_PYTHON_DISTRO_MAP-Treat-oracle-same-as-rhel-centos.yml
3 years ago
Matt Martz d3441bc313
Add new split filter (#73532)
* Add version added
3 years ago
Matt Clay 1a2da990a4
Temporary fix for cryptography issues. (#73530) 3 years ago
Shane McDonald 67f5bb39c7
galaxy - add format options for collection list (#73474)
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
3 years ago
Matt Martz d3f3784b86
Don't treat host_pinned as lockstep (#73484)
* Don't treat host_pinned as lockstep. Fixes #73364

* Add intg tests
3 years ago
Florian Heiderich 8d6136eab9
uri - add ca_path parameter (#71979)
* add changelog fragment for #71979 (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
  and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
3 years ago
Felix Fontein 82cdd7e735 Bump antsibull-changelog version for changelog sanity test. 3 years ago
jakegatsby f271d02a9f
bugfix: use to_text(stdout) in psrp.Connection.put_file method (#73491)
Co-authored-by: jakegatsby <jakegatsby@example.com>
3 years ago
Jordan Borean 0cdc410dce
no_log mask suboption fallback values and defaults CVE-2021-20228 (#73487)
* no_log mask suboption fallback values and defaults

* Added changelog

* Remove lambda expression
3 years ago
Michael Shnit 4315e18807
Fix regression introduced in b77abd0491 causing bug in inventory modu… (#73429)
* Fix regression introduced in b77abd0491 causing bug in inventory modules which break functionality in user setting  use_contrib_script_compatible_sanitization  parameter.

* Add changelog

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
3 years ago
Florian Heiderich 8f887b7adb
add self-signed https endpoint for ansible-test (#73463)
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
3 years ago
Sloane Hertel 9759e0ca49
add option to upgrade collections (#73336)
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
Gonéri Le Bouder 7cf80f50d1
validate_modules: fails with .id attribute not found (#73322)
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
3 years ago
Jose Angel Munoz 69631da889
lineinfile - add search_string parameter for non-regexp searching (#70647)
* Add tests for search_string
* Improve examples
* Add changelog
3 years ago
Rick Elrod 9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros (#73445)
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes #73084

Signed-off-by: Rick Elrod <rick@elrod.me>
3 years ago
Alexander Sowitzki 1c83672532
Allow `$` & `~` inside paths in galaxy roles (#72966)
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.

Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
3 years ago
Matt Clay a9b5bebab3
Overhaul ansible-test SSH key management. (#73451)
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
3 years ago
Alexander Sowitzki 218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set (#17029) (#73439)
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
3 years ago
Rick Elrod bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) (#73446)
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
3 years ago
Sloane Hertel 840bdc1e10
Fix warning for nonexistent inventory cache (#72840)
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
3 years ago
Matt Clay 81cd8e46f2 Update ansible-test distro containers to 1.29.0. 3 years ago
Matt Clay c7cb944315 Always use python exec wrapper in ansible-test. 3 years ago
Matt Martz c986cbb996
Add Python 3.8 and Python 3.9 to the fallback list (#73405) 3 years ago
Matt Clay 902c402b6f Update ansible-test default containers. 3 years ago
Matt Clay ded2d0b90e Use `sh` for ansible-test raw remote shell. 3 years ago
Matt Clay b1344a1847 Avoid sphinx usage in changelog sanity test. 3 years ago
Matt Clay 418b9b781d No longer warn about missing pip/python pair.
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.

In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
3 years ago
Matt Clay 672941fb0d
Add test support for Python 3.8 on RHEL and FreeBSD. (#73393)
* Support Python 3.8 on RHEL 8.2+ in ansible-test.

* Support Python 3.8 on FreeBSD in ansible-test.

* Use libyaml with PyYAML on FreeBSD.
3 years ago
Matt Clay 1b157ef42f
Limit rstcheck sanity test to ansible-core. (#73391) 3 years ago
Brian Coca 5aa4295d74
change detection and check mode fixes for apt_key (#73334)
Change detection and check mode fixes for apt_key

* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
3 years ago
Sviatoslav Sydorenko 595413d113
Replace the inhouse collection dependency resolver with `resolvelib`
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
3 years ago
Abhijeet Kasurde 82b74f7fd7
iptables: Fixed get_chain_policy API (#69152)
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.

Fixes: #68612

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
3 years ago
Brian Coca c7d4acc12f
allow env to override unspecified unsafe_writes (#73282)
* allow env var for fallback value for unspecified unsafe_writes
3 years ago
Brian Coca 2b0cd2c13f
use correct executable and options in all cases (#73323)
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
Matt Clay 13d08d232c Rename ansible-test functions to match constants. 3 years ago
Matt Clay 73fadc5e97
Clean up code in ansible-test. (#73379)
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
3 years ago
Martin Krizek a2af8432f3
Local vars should have highest precedence in AnsibleJ2Vars (#72830)
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix #6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.

Also removes AnsibleJ2Vars.extras instance variable which is not used.

Also adds missing test for #6653.

Fixes #72262
Fixes #72615

ci_complete
3 years ago
Matt Clay fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 3 years ago
Matt Clay 484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 3 years ago
Brian Coca e0c9f285ff
undeprecated hash_merge setting (#73328)
added updates from sloan's PR
 feedback from others

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
3 years ago
Alexander Sowitzki e6da544310 Let `command` always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
3 years ago
Rick Elrod 1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test (#69161)
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes #69203

Signed-off-by: Rick Elrod <rick@elrod.me>
3 years ago
Rick Elrod 4357a78130
[ansible-test] First attempt at freebsd/11.4 (#72655)
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes #48782

Signed-off-by: Rick Elrod <rick@elrod.me>
3 years ago
Alexander Sowitzki fe792fdcd2
Vanquish rare container bootstrap failure (#73288)
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.

Co-authored-by: Matt Clay <matt@mystile.com>
3 years ago
Joshua Bayfield 823c72bcb5
Shadow input for encrypt_string by default unless asked (fixes #71618) (#73263)
* Shadow input for encrypt_string by default unless asked (fixes #71618)
4 years ago
Jordan Borean bc60d8ccda
Galaxy - make versions list consistent across versions (#72932)
* Galaxy - make versions list consistent across versions

* Fix up unit tests
4 years ago
Oldřich Jedlička ec55a221f6
Do not pretend expression is filename in compile() Python built-in call (#73113)
When calling compile(), the filename argument should be either a real file
name or a string. According to Python docs, suggested one is '<string>'.
 Keep the current behaviour (encapsulate the actual expression), 
but enclose it into angle brackets.

Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
4 years ago
Sam Doran e8d4b62b41
Fix YAML error message when error is at the end of the file (#73241)
* Fix YAML error message when error is at the end of the file

If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.

When reading the file lines, we use that index to the get the relevant line.  If the index value is out
of range, the relevant line is lost for error reporting.

Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.

* Update existing tests and add new tests
4 years ago
Brian Coca 30d93995dd
Use a pty for local connections (#73023)
* Use a pty for local connections

Fixes #38696

Co-authored-by: James Cammarata <jimi@sngx.net>
4 years ago
Brian Coca 1e27d4052a
fix inventory source parse error handling (#73160)
fixes #51025

added test cases
4 years ago
Sandra McCann 9d71bbd13e
add changelog fragment for 73176 (#73259) 4 years ago
Brian Coca 1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
4 years ago
Sam Doran e2308ed6e3
ansible-galaxy - correct collections-path command line arg (#73193)
It should be plural to match other options and environment variables.
4 years ago
Felix Fontein df9cf368c0
Always mention the name of the deprecated plugin in routing deprecation messages (#73059) 4 years ago
Sam Doran df451636e7
facts - properly report virtualization facts for Linux guests on bhyve (#73204) 4 years ago
Sam Doran 0e6c334115
pause - adjust warning when run in background (#73182)
When the pause module is run in the background and seconds parameter is provided,
do not warn.

* Add tests
* Fix existing tests
  The test wasn't failing when it should have.
4 years ago
Rick Elrod 7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 4164cb26f0 Remove obsolete `tower` plugin from ansible-test. 4 years ago
Matt Clay 21f1811ddf
Cleanup provisioning code in ansible-test. (#73207)
* Remove unused code in ansible-test.

* Remove obsolete endpoint logic from ansible-test.

* Remove obsolete region selection in ansible-test.

* Remove obsolete port logic in ansible-test.

* Clean up ansible-test remote providers.
4 years ago
Brian Coca d6670da1d7
Added clog missing for issue 70722 (#73175) 4 years ago
Martin Zimmermann be59caa59b
iptables: Added set module/match_set support (#72984) 4 years ago
Sam Doran a7e834071c
Add macOS 11 to CI (#72622)
* Fix connection_paramiko_ssh test for macOS 11
* Update Azure Pipelines config
* Add changelog
4 years ago
Sam Doran 264e08f21a
user - properly handle password and password lock when used together (#73016)
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.

On BSD, run separate commands as appropriate since locking and setting the password cannot 
be done in a single action.

FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.

* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied

* Simplify rc collection in FreeBSD class
  Since the _handle_lock() method was added, the rc would be set to None, which could make
  task change reporting incorrect. My first attempt to solve this used a set and was a bit too
  complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.

* Improve the Linux password hash and locking behavior
  If password lock and hash are provided, set the hash and lock the account by using a password
  hash since -L cannot be used with -p.

* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.

* Clarify password_lock behavior.
4 years ago
Matt Clay 05ba49cbba ansible-test - No virtualenv install on Python 3
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.

On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
4 years ago
Brian Coca b1ee1a285a
Added NO_COLOR as config option (#73105) 4 years ago
Rick Elrod 44ee04bd1f
[dnf] Make "remove" filtering closer to dnf CLI (#73033)
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
  pkg specs (nv, nvr, nvra, wildcard, etc.)

Test Plan:
- New tests
- Local experiments with DNF API via PDB.

Tickets:
- Fixes #72809

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay 77942acefc Refactor ansible-test cryptography install code. 4 years ago
David Shrewsbury c8666f07d7
Add support for argument_specs attribute to RoleMetadata class (#73120) 4 years ago
Abhijeet Kasurde 25792d2388
galaxy: handle plus sign in user token appearing in role url (#73057)
Fixes: #45475

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sviatoslav Sydorenko 5b4f0b3370 Declare support for Python 3.9 in the metadata
Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod b96732fa6b [ansible-test] Add Fedora 33 support and drop F31
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca de3844cba9
discourage hexdigits in password lookup (#73106)
* actually add new docs

* Update lib/ansible/plugins/lookup/password.py

Co-authored-by: Sandra McCann <samccann@redhat.com>
4 years ago
Lloyd Parkes e0c75a6756
Replace the use of the function string.replace with the method str.replace (#68793)
* Replace the use of the function string.replace() with the method str.replace()
because that's what works for both Python 2 and 3.

* Cleanup the unused string import.

Added a changelog fragment.

* The documentation for os.write() seems a bit iffy, but in Python 3 we
definitely cannot pass it a string and we need to encode it into
bytes.

The Python documentation at
https://docs.python.org/3/howto/pyporting.html#text-versus-binary-data
says that this code will work in Python 2 as well.

Co-authored-by: Lloyd Parkes <lloyd@must-have-coffee.gen.nz>
4 years ago
Rick Elrod 20509b6507
[facts] fix version facts on FreeBSD RC/PRERELEASE (#73020)
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
  -RELEASE, -STABLE, or -CURRENT.

Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
  version of FreeBSD.

Tickets:
- Fixes #72331

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jean-Pierre Matsumoto 3074ede8a3
ExecutionPolicy default value is RemoteSigned (#72993)
* ExecutionPolicy default value is RemoteSigned

##### SUMMARY
In Windows setup steps, settings `ExecutionPolicy` is restored to value `Restricted`. But actually default value for Windows server is not `Restricted` but `RemoteSigned` ([source](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1#default)).

I have got issue with `win_dsc` because `ExecutionPolicy` was `Restricted`.

##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr

* Add changelog fragment

* Apply suggestions from code review

Co-authored-by: Sandra McCann <samccann@redhat.com>
4 years ago
pva 42bc03f0f5
[get_url] skip checksum during --check (#66700)
Fix get_url failure in check mode (--check) when using the checksum
format <algorithm>:<url>.

Regression introduced in (#20532)

Fixes: #61369
4 years ago
Abhijeet Kasurde aa56a2ff6a
galaxy: Handle token as dict while loading using yaml (#70911)
Malformed galaxy_token file creates stacktrace.

Fixes: #70887

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doooran 5ecfb19cad
arg_spec - move validator lookup method to a function (#72667)
* arg_spec - move type checking lookup method to a function

* Change get_wanted_type name and behavior

Change the name to get_validator to bette describe what it is doing.

Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.

* Add param to check_type_str()

Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.

* Remove private attribute mapping types to validator

Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.

Update tests that were referencing the private attribute.

* Use private method for 'str' type

To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.

Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.

Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.

* Add changelog

* Change function name to better reflect its... function

* Change dict name to better reflect its contents

CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS

* Fix changelog
4 years ago
Sam Doooran 48803604cd
systemd - do not overwrite unit name when searching (#72985)
* systemd - do not overwrite unit name when searching

PR #72702 introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.

* Use facts rather than a manual probe to determine if systmed is present

* Remov unnecessary block

* Use vars files instead of set_fact

* Add tests for using a templated unit file

* Update changelog fragment

* Use template to get correct path to sleep binary
4 years ago
Sam Doooran 8e022ef00a
import_playbook - change additional params to deprecation (#72987)
* import_playbook - change additional params to deprecation

I incorrectly recommended this be set as a warning when it should have been a deprecation.

* Fix deprecation sanity test to not required a collection name when not inside a collection
4 years ago
Sam Doooran 850a77f639
ansible-test - prefer venv over virtualenv on Python 3 (#73000)
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).

On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
4 years ago
Sloane Hertel 5e03e322de
Pass the top level dictionaries to combine_vars (#72979)
* Pass the top level dictionaries to combine_vars

combine_vars uses dict.update() to replace keys
4 years ago
Matt Clay 37d09f2488
Update ansible-test pylint Python support. (#72972)
* Update ansible-test pylint Python support.

* Python 3.8 is now officially supported.
* Python 3.9 is now skipped with a warning.
4 years ago
Felix Fontein 0ba96d2be8
Compare removal and deprecation dates and versions in runtime metadata against current version and today (#72625)
* Check deprecation and removal versions w.r.t. current version.

* Check removal dates from tombstones w.r.t. today.

* Add changelog.
4 years ago
Brian Coca d22804c4fb
saner path dir management (#72648)
* saner path dir management

   fixes #72628

   ensure we always store paths w/o a_c

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
4 years ago
Martin Zimmermann 83fb24b923
iptables: Adding multiport module support (#72928) 4 years ago
Adam Miller 0044091a05
Update yum/dnf module docs to include version comp (#72763)
* Update yum/dnf module docs to include version comp

Fixes #61234

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix up docs syntax to sanity tests are happy

Signed-off-by: Adam Miller <admiller@redhat.com>
4 years ago
Toshio Kuratomi e7dee73774
Collection list site packages (#70173)
* ansible-galaxy collection list and verify now utilize collections in site-packages.

This is a short term fix for #70147.  The long term fix needs to handle
install (but that discussion is also bound up in how upgrade is going to
work and where things can get installed so it's deferred for 2.11.)

* Add test for ansible-galaxy collection list with site-packages

* Fix sanity issue

Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
4 years ago
Matt Martz a3b6485073
Fix reset_connection paramiko, winrm, psrp (#72688)
* Ensure we only reset the connection when one has been previously established. Fixes #65812

* Ensure psrp doesn't trace

* winrm too

* Indentation fix
4 years ago
Matt Clay 36ab3d1189
Fix cryptography constraints in ansible-test. (#72914) 4 years ago
Matt Martz 83764ad506
Fix async interpreter parsing (#72636)
* Fix async interpreter parsing. Fixes #70690

* Target localhost instead of remote host

* Don't forget inventory

* Address shellcheck issue
4 years ago
Martin Krizek 1b70260d5a
Add implicit role_complete block instead of role._eor (#72208)
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes #69848
4 years ago
Martin Krizek 252685092c
Use _wrap_native_text only for builtin STRING_TYPE_FILTERS (#71801) 4 years ago
Sven Wegener 08cc6edc64
vault: Read stdin data as binary on python3 (#52229)
On python3 sys.stdin is an encoded file object that does not support
reading raw binary data. Use the supplied buffer object to do so.

Signed-off-by: Sven Wegener <sven.wegener@inovex.de>

Co-authored-by: Sven Wegener <sven.wegener@inovex.de>
4 years ago
log-e 8f77e95765
Add alinux hostname module support (#72894) 4 years ago
David Shrewsbury fe17cb6eba
Fix fileglob parameter order bug (#72879) 4 years ago
David Shrewsbury be2c376ab8
Extract embedded function to RoleMixin method and add tests (#72754)
* Add changelog
* Simplify return
4 years ago
Sijis Aviles 034e9b0252
unarchive - add include option (#40522)
This should allow users to extract specific files from an archive as
desired.

Fixes #16130, #27081.

* Rebase and make a few minor changes
* Add changelog
* Improve tests

- move to separate tasks file
- change assertions to check for exactly one file
- use remote_tmp_dir for output dir

* Make exclude and include mutually exclusive
* Don't remove files needed by other tasks
* Fix sanity tests
* Improve feature documentation
* Skip tests that use map() on CentOS 6
* Use fnmatch on include for zip archives
  This matches the behavior of exclude

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Felix Fontein f569d80fde
Improve deprecations (#72697)
* Remove space before comma in '... bla , use ...'

* 'why' is inserted in the middle of a sentence, between two commas.

* Make deprecations from base.yml show source ansible-core.

* Add changelog fragment.

* Improve some more 'why's.

* Add PR URL to fragment.
4 years ago
Pilou b464d18fd1
AnsibleModule.set_mode_if_different: handle symlink is in a sticky directory (#45198)
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory

Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
4 years ago
Felix Fontein 5226ac5778
The implicit default for type=bool options is not 'false', but 'none' (#72699)
* The implicit default for type=bool options is not 'false', but 'none'.

* Fix modules, resp. add ignore.txt entries.
4 years ago
Felix Fontein f94ba68d8f
ansible-test sanity: fix UnicodeDecodeError for `--python 2.7 --docker` (#72623)
* Store target paths as unicode.

* Add changelog fragment.
4 years ago
Matt Clay 08842cd6bb
Fix Azure Pipelines change detection. (#72824) 4 years ago
Matt Clay 2b24fae429 Replace `exit` with `sys.exit` in Galaxy CLI. 4 years ago
Sam Doran a788ea0132
systemd - account for templated unit files when searching for service (#72702)
Related to issue #71528 and PR #72337

Co-authored-by: Martin Polden <mpolden@mpolden.no>
4 years ago
Rick Elrod a59825128d [centos6] update container for EOL
Change:
- Reference:
  https://lists.centos.org/pipermail/centos-devel/2020-December/056208.html
- Bump centos6 container to 1.25.0

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Paul Belanger a95213d2f5
Switch to hashlib.sha256() for ansible-test (#72411)
When FIPs mode is enable on centos-8, we are not able to load md5
functions.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
4 years ago
Abhijeet Kasurde a223ea5185
distribution - handle NetBSD OS Family (#70799)
Fixes: #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doran 1db9588279
ansible-test - add cryptography constraint for cffi (#72761)
The recently released version of cffi fails to install on systems with an older version of gcc. In
our case, this in the CentOS 6 test image. There is a fix but it has not yet been released.

https://foss.heptapod.net/pypy/cffi/-/issues/480
4 years ago
Martin Krizek ae08c6a639
Ensure Ansible's unique filter preserves order (#67856)
Fixes #63417
4 years ago
Yadnesh Kulkarni 3add96909d
Fix parsing of values when using an empty string as key (#57132) (#72545)
Signed-off-by: Yadnesh Kulkarni <ykulkarn@redhat.com>
4 years ago
Matt Martz 46198cf80a
Add orig_exc context to error messages (#72677)
* Add orig_exc context to error messages. Fixes #68605

* Fix string formatting
4 years ago
Jordan Borean fb092a82a1
ansible-galaxy - source deps from all servers and not just parent (#72576)
* ansible-galaxy - source deps from all servers and not just parent

* Added integration tests for this scenario
4 years ago
Matt Martz f8ef34672b
Provide better decryption errors for single vault values (#72362)
Fixes #72276
Fixes #72281
4 years ago
Sam Doran e889b1063f
arg_spec - rework _check_arguments() (#72447)
* Move _syslog_facitily to __init__
  No good reason it should not be set for each object

* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
  Change the behavior to return a set, either empty or populated, with unsupported keys.
  Accept legal_inputs as optional which will not required calling handle_aliases before calling
  get_unsupported_parameters().

* Add changelog
* Rework function behavior and documentation
  I realized I missed the original intent of this method when moving it to a function. It
  is meant to compared the parameter keys to legal inputs always, not compare
  parameter keys to argument spec keys, even though the argument spec keys should
  be a subset of legal inputs.

* Add tests
* Fix typo.
* Set internal properties when handling suboptions
4 years ago
Alan Rominger 1fbac24739
Update ansible-runner tests version and deps (#72197)
We could not find reason for the docutils install so trying out removing it.
Also bumping to latest version of ansible-runner

Skip all python2 versions because next ansible-runner drops it

Co-authored-by: Elijah DeLee <kdelee@redhat.com>
4 years ago
Brian Coca 07248e5ec1
avoid key errors on environment access (#72620) 4 years ago
Brian Coca d8baf4d38d
Document and warn when max_fail and free are used (#72616)
Fixes #16666
4 years ago
Brian Coca 13bc35e13c
actual min changes needed for rhel8 (#72477)
* actual min changes needed for rhel8

 - specifiy releaserver as dnf/vars is ignored
4 years ago
Rick Elrod 6894ae7d1d
Rename to ansible-core (#72594)
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base

Test Plan:
- ci_complete
4 years ago
Sam Doran bbef250c2b
Fix super annoying Python 2.6 multiprocessing.Queue stack trace in CI (#72604)
* Fix super annoying Python 2.6 multiprocessing.Queue stack trace in CI

A bug exists in Python 2.6 that sometimes raises an exception during interpreter shutdown. We
encounter this frequently in our CI since we run tests on CentOS 6 as the control node, which
has Python 2.6.6 with this bug.

This PR adds a very minor sleep only on Python 2.6 which gets around this issue. I did lot of testing
using a standalon script I found that easily duplicated the issue to find the minimum sleep value
needed to avoid this issue.

CPython issue: https://bugs.python.org/issue4106
Fix in CPython: https://hg.python.org/cpython/rev/d316315a8781

* Use correct attribute
4 years ago
Sam Doran 4b8cb6582b
pause - do not hang if run in the background (#72065)
* Consolidate logic for determining whether or not session is interactive
  into a single function, is_interactive()
* Increase test coverage

I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
4 years ago
Paul 44a38c9f33
add return documentation for unarchive. (#72546)
* add return documentation for unarchive, add fragment for changelog, apply suggestions from code review

Co-authored-by: Amin Vakil <info@aminvakil.com>
4 years ago
Jordan Borean bdd0c48837
Added changelog for RHEL 7.9 (#72572) 4 years ago
Brian Coca 4f0e2fff95
avoid fatal exception on invalid collection name (#72296)
* avoid fatal exception on invalid collection name used in ansible-doc filter
4 years ago
Brian Coca 48c08f410c
allow any type of unsafe data (#72547)
* allow any type of unsafe data

  dont limit to strings
4 years ago
Egor Margineanu e879f12fb9
Fix AIX networks facts when nestat is either missing or has incorrect permissions (#72516)
* Added check for none on netstat_path variable

* Added changelog
4 years ago
Matt Clay fa2be89cd4 Add coverage exporting to ansible-test
A new `--export` option for `ansible-test coverage combine` allows multi-step aggregation of code coverage for CI pipelines.
4 years ago
Matt Clay f022dedd0a Fix container discovery for the acme test plugin. 4 years ago
Brian Coca aa4d53ccdf
ensure local exposes correct user (#72543)
* ensure local exposes correct user

  avoid corner case in which delegation relied on
  playcontext fallback which was removed

  fixes #72541
4 years ago
Jordan Borean de5858f48d
Added caching mechanism for Galaxy API requests (#71904)
* Added caching mechanism for Galaxy API requests

* Add cache options and split up code

* Added unit tests

* Fix sanity test

* Use modified date and fix up caching for explicit servers

* Make sure credentials are not in cached server name

* Added test for getting updated cache version

* Changes from review
4 years ago
Brian Coca a1730af91f
Ensure blockinfile correctly returns backupfile (#72544)
* Ensure blockinfile correctly returns backupfile

  Fixes #27626
  based on #27859

Co-authored-by: Giovanni Sciortino (@giovannisciortino)
4 years ago
Rick Elrod e7bf0696ef
[ansible-test] use newer container images (#72126)
Change:
- Bump default, ansible-base, distro containers
- We do NOT add fedora33 yet, because it doesn't work right on Shippable
  due to an old kernel. This will be added post-AZP.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Amin Vakil c1da427a5e
iptables: Reorder comment postition (#71496)
* Reorder comment postition

* Add comment unit test

* Fix unit test

* Fix unit test

* Add changelog

* Add paramaters which would be problematic without this fix

* Fix typo

* Fix unit test

* Fix unit test
4 years ago
Rick Elrod 3b379f372e [ansible-test] first attempt at freebsd 12.2
Change:
- Add freebsd/12.2 remote target

Test Plan:
- ci_complete

Tickets:
- Fixes #72366

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 8eab113cb1
show keyword documentation in ansible-doc (#72476)
* show keyword documentation in ansible-doc

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Matt Martz 96ad5b799e
Emit warning when running on the controller with a Python older than 3.8 (#72467)
* Emit warning when running on the controller with a Python older than 3.8

* Add spaces

Co-authored-by: Matt Clay <matt@mystile.com>

* and more spaces

Co-authored-by: Matt Clay <matt@mystile.com>

* s/Py/Python/

Co-authored-by: Matt Clay <matt@mystile.com>

* Add note to Control node requirements about Py3.8 requirement

* Add collection_name to deprecated call

* more spaces

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Clarify that we are only packaging for py3.8+

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Sloane Hertel 0ed7bfc694
Fix task get_name to always prepend the role name (#72511)
* Fix 'role_name : tast_name' notation if task contains role name

* Add tests for notifying handler names which contain the role name

Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
4 years ago
Martin Krizek a2593b5e27 CLI - Specify jinja version in --version output 4 years ago
Brian Coca 488b9d6c35 remove redundant remote_user for local setting
local action plugin already does and this also should fix
  fork/thread issue by removing use of pwd library

  fixes #59642
4 years ago
Sam Doran b33d7e2e29
Add changelog (#72502) 4 years ago
Felix Fontein c7a4b39633
Adjust action groups to moved modules (#72428)
* Support docker and k8s action groups for moved modules in community.docker and community.kubevirt.

* Also support k8s action group for community.okd.

* Also add kubernetes.core.

* Adjust PR #.

* Fix changelog fragment.

* Remove community.okd.

* Revert "Remove community.okd."

This reverts commit 812b5aa6e2.
4 years ago
Alexei Znamensky 5654de6fce
Return error if cwd directory does not exist (#72390)
* Return warning or error if cwd directory does not exist, in AnsibleModule.run_command()
4 years ago
Felix Fontein da60525610
Fix missing ansible.builtin FQCNs in hardcoded action names (#71824)
* Make sure hard-coded action names also check for FQCN.
* Use _add_internal_fqcn() to avoid hardcoded lists and typoes.
4 years ago
Felix Fontein 64a809d2b6
Improve ansible-test classifications for collections (#72353) 4 years ago
Jordan Borean 809d5fc398
ansible-test - skip installing PowerShell sanity test reqs if they are already installed (#72423) 4 years ago
Felix Fontein 4fb336cef1
ansible-doc: export has_action when --json is used (#72359)
* ansible-doc: export has_action when --json is used.
* Remove docuri and now_data, which were not used resp. ignored in format_plugin_doc and the functions it calls anyway.
* Add function _combine_plugin_doc.
4 years ago
Brian Coca cb94c0cc55
added timeout options to adhoc and console (#71230)
* added timeout options to adhoc and console

* added test

* fix typosesz

* fix conflict

* task_timeout

* fix timeout option, added extra vars to console

* actually use right cli switch .. DUH!

* added timeout to include 'valid' but ignored keys

* fix default

* fixes per review
4 years ago
Brian Coca 96c1972439
find will give more specific reasons on skip (#68823)
* find will give more specific reasons on skip

 - added more error handling
 - use random path for test
4 years ago
Sam Doran fb09fd2a23
wait_for - ignore psutil related errors (#72401)
When enumerating connections with psutil, catch and ignore errors to avoid returning a stack trace.

Co-authored-by:  Matt Martz <matt@sivel.net>
4 years ago
Matt Martz e73a0b2460
Explicitly get the include task, and not assume it is the parent (#72378)
* Explicitly get the include task, and not assume it is the parent. Fixes #6571
4 years ago
Sam Doran a51a6f4a25
reboot - add reboot_command parameter (#69847)
Fixes #51359
* Update default search paths
* Fix returns for args and command, don't allow conversion
* Reorganize tests
* Fix test for Azure Pipelines
4 years ago
Sam Doran 8f9cf456b0
url lookup - set default user agent (#72324)
* Add unit tests
* Add note about when default changed
4 years ago
Sam Doran d6115887fa
systemd - use list-unit-files rather than list-units (#72363)
list-unit-files will return all files on the system. list-units omits those
that are disabled.

Co-authored-by: Ken Dreyer <ktdreyer@ktdreyer.com>
4 years ago
Sam Doran 7352457e7b
hostname - add macOS (#54439)
* Add DarwinStrategy class and integration tests

macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.

* Update documentation

* Account for virtualization type returned on Azure Pipelines

* Do not be dependent on order of self.name_types

Use the scrubbed name when the name type is LocalHostName
4 years ago
Sam Doran c51438312a
blockinfile - properly insert block when no trailing new line exists (#72350) 4 years ago
Felix Fontein 233e7beb5b
Fix processing of add_file_common_args=True when argument_spec is not specified as kwarg. (#72334) 4 years ago
Sloane Hertel dee8a4a8f1
Use type 'path' for fact_caching_connection in jsonfile cache plugin (#72317) 4 years ago
Sam Doran db84e2c989
systemd - fix issue with capbpf and newer kernel (#72337)
A bug existed in systemd 245 that did not properly handle unknown kernel
capabilities gracefully. This resulted in incomplete output when querying
for the service status. It is possible to get service status by other means.
This PR works around this issue by getting service status using other commands
in the event of a failure due to this bug.
4 years ago
Rick Elrod 2293b327c0
[ansible-test] Limit cryptography to <3.2 (#72342)
Change:
- Cryptography 3.2 drops support for OpenSSL 1.0.2. Some of our CI
  infrastructure still uses this version (FreeBSD, namely). For now,
  just add a constraint to use old cryptography.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Sloane Hertel 8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
4 years ago
Felix Fontein 0b2b82c0b6
Add changelog fragment for #71736. (#72323) 4 years ago
Matt Davis 83909bfa22
Remove ansible-galaxy login (#72288)
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries

* remove dead code/config, update messages and porting guides
4 years ago
Matt Clay 3c2e8b99be
Fix ansible-test handling of user-defined docker networks. (#72256)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry.
4 years ago
Ruediger Pluem e64ba97e8a
Fix the reversed order of return values (#72088)
* Fix the reversed order of return values

AnsibleModule.run_command returns a tuple of return code, stdout and stderr.
The module main function of the user module expects user.create_user to
return a tuple of return code, stdout and stderr.
Fix the locations where stdout and stderr got reversed.

* Fix another missed occurance

* Add changelog

* Fix further reversed order of return values occurrences

Followup to 72c3ba62c8e2606a6a35d51075fba29ce5faf34e:
Fix further reversed order of return values occurrences.
4 years ago
Rick Elrod 69e510e767
[virt facts] recognize containerd as a container (#72210)
Change:
- containerd is now recognized as container tech

Test Plan:
- Tested with Docker inside Github Actions in a private repo:

        "ansible_virtualization_role": "guest",
        "ansible_virtualization_tech_guest": [
            "containerd",
            "container",
            "VirtualPC"
        ],
        "ansible_virtualization_tech_host": [],
        "ansible_virtualization_type": "containerd",

Tickets:
- Fixes #66304 because this is what docker containers show up as
  in Github Actions.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jordan Borean c9fa1d0e7e
Fix race condition when creating async dir (#72069)
* Fix race condition when creating async dir

* Simplify exception wrapper

* Remove var used for testing
4 years ago
Matt Martz be5fc4e642
Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
4 years ago
Brian Coca 5ec53f9db8
make collection callbacks follow normal flow (#59932)
make collections whitelist follow normal flow

* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
4 years ago
Matt Clay 2ef4b7e07e Fix ansible-test Azure Pipelines container auth. 4 years ago
Rick Elrod 3b40c6f3b7
[config] coerce more to string when 'type: str' (#72172)
Change:
- When a plugin defines `type: str` on a parameter, treat more kinds of
  input as a string instead of whatever it is parsed as.

Test Plan:
- New unit tests
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
dberg1 cdf62edc65
service_facts: return service state information on AIX (#72073)
* service_facts: return service state information on AIX

AIX uses the System Resource Controller (SRC) to manage services.
See https://www.ibm.com/support/knowledgecenter/ssw_aix_72/osmanagement/sysrescon.htm
Use lssrc command on AIX to return service state information.


Co-authored-by: dberg1 <dberg1@github.com>
4 years ago
Jordan Borean caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
4 years ago
Matt Martz c4acd41d6e
Ensure delegate vars calculation has correct loop context. Fixes #37132 (#71477) 4 years ago
Matt Clay 5f76bd2af7 Support collection constraints in ansible-test.
This allows collections to specify requirements and constraints for packages that ansible-test has requirements or constraints for.
4 years ago
Abhijeet Kasurde 9ffa84cc1c
basic: Handle exception in default selector (#72101)
In Python 2.7, default selector raises IOError
instead of OSError. Fix handles this exception.

Fixes: #71704

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Rick Elrod ebc91a9b93
[unarchive] work on older RHEL with group: <gid> (#72098)
Change:
- No longer fail due to old Fedora/RHEL and our failure to try to cast
  gids to integers before trying to pass them to getgrgid() before
  trying to use them.
- Add tests for user/mode for various unarchive formats.

Test Plan:
- New integration tests, ran against centos6 container

Tickets:
- Fixes #71903
4 years ago
Rick Elrod a90e37d017
[virt facts] identify KubeVirt properly (#72092)
Change:
- Make KubeVirt be identified as such, and with a guest role.
- This is a breaking change, since playbooks could be relying on the
  incorrect behavior previously.

Test Plan:
- Issue author

Tickets:
- Fixes #72001

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 0332046699
[ansible-test] attempt to work around podman (#72096)
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.

Test Plan:
- Tested with podman 2.0.6 in Fedora 31.

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Evan Van Dam 3db08adbb1
Add optional attribute arg for min and max filters (#50909)
* Pass **kwargs to min and max filters

* Use the jinja2 filters if available

* Add unit tests

* Add examples to docs passing attribute
4 years ago
Rick Elrod 709484969c
sysctl/openbsd fact fixes (#72070)
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`

Test Plan:
- New unit tests

Tickets:
- Fixes #71968
- Refs #72025
- Refs #72067

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
4 years ago
Ruediger Pluem a7170da851
user - allow local users with an expiry date to be created (#72022)
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.

Fixes: #71942

In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.

Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.
4 years ago
Sloane Hertel cceba07114
VariableManager - add the 'vars' key before getting delegated variables (#71214)
* VariableManager - add the 'vars' key before getting delegated variables
4 years ago
Matt Martz fb03ac7019
Add libyaml info to version output, and restore git info (#71948)
* Add libyaml info to version, restore git functionality

* Add clog fragment

* Update tests to reflect new version output
4 years ago
Matt Martz 0e480d38dd
uri status_code elements are type int (#72062) 4 years ago
Felix Fontein 1107aace1b
Add deprecation collection name to plugin options (#71735)
* Add changelog fragment.

* Use correct field that's expected by lib/ansible/cli/__init__.py..

* Add basic unit tests.
4 years ago