Commit Graph

2424 Commits (9081b228689ca97261fae42ee84fde7e0bb48d6e)

Author SHA1 Message Date
Brian Coca 8789d7968d
try to capture better winrm/put_file error (#70508)
* try to capture better winrm/put_file error

fixes #70361

* Update lib/ansible/plugins/connection/winrm.py

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
4 years ago
Amin Vakil 9c40b1b2ff
Replace filename with file in apt_key (#70492)
* Replace filename with file from apt_key check

  one is internal variable, the other is actual parameter used and required for parameter check.
4 years ago
Mykola Grygoriev 71c378e139
Fix decrypt argument in assemble module (#70465)
* Do not pass decrypt parameter to assemble module

* Add integration tests where decrypt=True

* Add changelog #70465
4 years ago
Rick Elrod 28fda23284
command warnings: don't send the param from action (#70531)
Change:
- Followup to #70504. We need to not pass the 'warn' parameter from the
  action plugin either, unless it's True. Otherwise, even though it
  defaults to false, we always show the deprecation.

Test Plan:
- Local

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz c4fd5bee00
Speedup modify module (#70475)
* speed up modify_module

* Remove debugging

* ci_complete

* Simplify generic_visit, alias to visit, eliminate some attr lookups

* ci_complete

* Add changelog fragment
4 years ago
Alan Rominger b0d9deeae3
Fix ansible-test error in community.aws (#70507)
* Fix ansible-test error in community.aws

* Add changelog entry for fix

* Change check from None to string_types

* Update changelogs/fragments/70507-validate-null-author.yaml

clarify wording "or a list of strings"

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py

clarify wording - single string or not specified valid

Co-authored-by: Felix Fontein <felix@fontein.de>

* Do not fail but return None when given outside list

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
David Shrewsbury 8aca464b8b
Make sure ansible_become treated as a boolean (#70484)
* Make sure ansible_become treated as a boolean
4 years ago
Rick Elrod f3ef4ed076
Remove some constants.py deprecated items (#70466)
Change:
- Remove mk_boolean
- Remove BECOME_METHODS
- Remove get_config

Test Plan:
- CI, removed mk_boolean unit tests

Tickets:
- Fixes #69678

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 41414ed475
Deprecate command warnings feature (#70504)
Change:
- The command warnings feature which suggests that users use modules
  instead of certain commands is now deprecated. Its `warn` paramater
  and `COMMAND_WARNINGS` configuration options are also deprecated.
  Their use will become an error in version 2.13.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Rick Elrod 707458cc8c
Make netbsd virtualization facts more specific (#70467)
Change:
Our handling of NetBSD virtualization facts led to facts that were just
plain incorrect. One example is reporting Xen even when the system is
running on something completely different (like KVM).

As stated by the reporter of #69352, NetBSD has a better sysctl setting
to use for this information, machdep.hypervisor.

This PR does the following:

- Try to use machdep.hypervisor sysctl value if the other sysctl values
  we check don't end up with enough information to be useful
- Only look for /dev/xencons and assume Xen if nothing else works
  (Really this should probably return 'unknown' since the file exists on
  non-Xen systems and is not very useful).
- Add a few more patterns (Xen matches and also Hyper-V) to
  VirtualSysctlDetectionMixin#detect_virt_product.

This change is slightly breaking:
- If the first two attempts at using sysctl worked before,
  (machdep.dmi.system-product and machdep.dmi.system-vendor), they will
  continue to work.
- For cases when those values didn't work, previously the existence of
  /dev/xencons was checked, and if found, we reported 'xen' (even on
  non-Xen systems when the file existed). After this PR, we try the
  machdep.hypervisor sysctl key before still falling back to
  /dev/xencons. This means that in some cases, we might go from
  (wrongly) saying "xen" to giving a more accurate value such as "kvm"
  or "Hyper-V".

Test Plan:
- Tested with local NetBSD VM and got 'kvm' instead of 'xen' back.

Tickets:
- Fixes #69352

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 688cd8657b
Fix string/bytestring comparsion in m_u.basic (#70439)
Change:
- module_utils.basic.is_special_selinux_path() used a string ==
  bytestring comparison which returned False and made Ansible think that
  certain filesystems aren't, in fact, special-cased, when they should
  be. Ensure both sides of the == are bytestrings.

Test Plan:
- Added `copy` integration tests for this case.

Tickets:
- Fixes #70244

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Felix Fontein 9164b96774
ansible-doc man formatter: fail with better error message when description isn't there (#70046)
* ansible-doc man formatter: do not crash when description isn't there.
* Change to report a better error message when description is not there.
* Add test.
4 years ago
Andreas Schleifer 7d7f15fc9b
apt - make errors more transparent (#70099)
Include error from apt Python library in module error output

Co-authored-by: Andreas Schleifer <aschleifer@bigpoint.net>
4 years ago
Satyajit Bulage 4f0ec5a9a0
Suppress warning for user directory in ansible-inventory (#65344)
When user uses home directory in --playbook-dir option
of ansible-inventory command, it warns user about this.
This PR suppress the warning message for user's home directory usage
in ansible-inventory command.

Fixes: #65262

Signed-off-by: Satyajit Bulage <sbulage@redhat.com>
4 years ago
Sam Doran b4184aa50e
ssh connection - use get_option() rather than _play_context (#70438)
The ssh_args were sometimes not correctly applied to the connection
when using _play_context. Use get_option() instead to ensure the
correct ssh_args are always applied.
4 years ago
peanutduck 36d7ba1408
Fixed TypeError instancemethod expecting at least 2 arguments (#69463)
* Fixed TypeError instancemethod expecting at least 2 arguments

* added changlelog for 69463
4 years ago
Brian Coca 7a15a3a109
fix flatten handling of nulls/nones (#70141)
* fix flatten loop control issue (break -> continue)

fix issue #69012

(cherry picked from commit 2127be5ec5)

* fixed null break bug and added option to include

 fixes #69012
 fixes #69013

Co-authored-by: pseudocoder <borisovano@users.noreply.github.com>
Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
4 years ago
Abhijeet Kasurde 5709173c32
with_sequence: example using vars (#69369)
Added an example for using vars in with_sequence.

Fixes: #68836

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Martz 1f1d6e5aec
Use the first galaxy server supporting v1 for roles (#70375)
* Use the first galaxy server supporting v1 for roles. Fixes #65440

* Add changelog fragment

* This is best effort, fall back to original behavior if something bad happens
4 years ago
Matt Davis b9e38e8b55
misc collection metadata fixes (#70403)
* misc collection metadata fixes

* parse collection meta with libyaml if available
* require only Mapping for validation
* add explanatory text for _meta_yml_to_dict

* ignore custom pylint rule

* this code shouldn't import a bunch of stuff from ansible, since it's run under the import sanity test
4 years ago
Rick Elrod f7078c1f8f
Throw a prettier error in m_u.basic syslog (#70312)
Change:
- In certain situations, such as when the input string contains null
  bytes (\0), syslog.syslog will throw a TypeError. Handle that and
  fail_json instead.

Test Plan:
- New test
- ansible-test --docker centos[68] (for py2 and py3 respectively)

Tickets:
- Refs #70269

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Jordan Borean 40ce448657
ansible-test - do not validate blacklisted ps modules (#70376)
* ansible-test - do not validate blacklisted ps modules

* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Dmitriy Rabotyagov 8324db7cb5
Add new option for YUM4/DNF repositories (#70042)
Documentation reference:
  https://dnf.readthedocs.io/en/stable/conf_ref.html

Co-authored-by: Estelle Poulin <dev@inspiredby.es>
4 years ago
Felix Fontein 5e4f708241
ansible-doc: avoid problems with YAML anchors when formatting man page (#70045)
* Avoid problems with YAML anchors when formatting man page.

* Add changelog.
4 years ago
psi / Ryo Hirafuji 4997063b4a
apt - add fail_on_autoremove option to avoid unintended package removals (#70056)
* Ensure not to remove existing packages while installing apt packages.
* Make all lines shorter than 160 characters
* Allow removing packages only when upgrading.
* Add integration tests
4 years ago
Martin Krizek b05e00e99a
Fix delegate_facts with interpreter not being set (#70293)
Fixes #70168

ci_complete

Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Mykola Grygoriev 70bb2b33ca
Improve error message when ssh client is not found on the host (#70122)
* Add changlelog
4 years ago
Laurent Coustet e396715d7b
git - add single_branch option (#28465)
In some usecases, we want to be able to clone a single branch
of a repository, without using --depth (which implies --single-branch).

* Use branch name when available
  -  update description of parameter
  - consolidate branch or tag checking for easy reuse

* Add changelog
* Use static task imports rather than dynamic includes
* Add integration tests for single_branch
* Account for older versions of git
* Minor tweak to warnings

Co-authored-by: Laurent Coustet <laurent.coustet@clarisys.fr>
Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Hideki Saito 2d59e548f6
Improve ansible-galaxy STDOUT messages for collections (#70040)
- Fix issue #70010
- Add installation successful message
- This feature targets "collection" sub-command and does not affect "role" sub-command

Signed-off-by: Hideki Saito <saito@fgrep.org>
4 years ago
Felix Fontein 689cfd1983
Top-level deprecation of plugin did not get collection_name added when deprecating by version (#70344)
* Top-level deprecation of plugin did not get collection_name added when deprecating by version.

* Add changelog fragment.
4 years ago
s-hamann e39a9bf583
Add support for Parrot Linux, a Debian derivate (#69158)
Co-authored-by: black <invalid>
4 years ago
老广 c3fc2d27d9
pause - fix curses.setupterm() error (#47851)
* [Bugfix] curses.setupterm() error

When run playbook in celery task, curses.setupterm()  will be failed

```
  File "<frozen importlib._bootstrap>", line 675, in _load
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/Users/guang/.virtualenvs/ansible/lib/python3.6/site-packages/ansible/plugins/action/pause.py", line 45, in <module>
    curses.setupterm()
TypeError: argument must be an int, or have a fileno() method.

```

* Add changelog

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Martin Krizek 0cd07eb3fd
hash filter - fail when unsupported type is passed as an argument (#70292)
Fixes #70258
5 years ago
Abhijeet Kasurde 29169ae847
stat: Handle colon in filename (#70259)
Handle colon appearing in filename while parsing the mimetype and charset
using file command.

Fixes: #70256

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Adam Miller 3c9be97e16
fix runtime.yml splunk.enterprise_security -> splunk.es (#70305)
Signed-off-by: Adam Miller <admiller@redhat.com>
5 years ago
Pavel Březina 45e0f74702
display: use stdout for column width (#70199)
stdout may differ from stdin so it should be used to determine the column
width, especially since it is the target file descriptor.
5 years ago
Abhijeet Kasurde 3fe48ecba2
Add collection path in CLI version info (#68633)
This will provide user default path of collection

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Matt Clay 74494a1908 Code cleanup for ansible-test coverage support. 5 years ago
Matt Clay 8d92df4537 Test netconf plugins in ansible-doc sanity test. 5 years ago
Sam Doran a3c1ea886f
Revert commits related to delegated facts changes (#70272)
* Revert "Fix storing delegate_to facts (#70231)"

This reverts commit 88bb76f248.

* Revert "Fix storing local task_vars facts for the retry (#70171)"

This reverts commit eaf6086eea.
5 years ago
Rick Elrod 2531d54880
Throw AnsibleError instead of OSError, py3.9 crypt (#70246)
Change:
- On Python 3.9, `crypt.crypt` will throw instead of returning `None`
  when the algorithm isn't supported. So we catch that and handle it
  the same way we handled the algorithm not being supported on 3.8: by
  throwing AnsibleError.

Test Plan:
- CI for <=3.8.
- Local for 3.9b3:
  ansible -m debug -a "msg=\"{{ 'changeme' | password_hash('bcrypt') }}\"" localhost

  Before:
  localhost | FAILED! => {
      "msg": "Unexpected failure during module execution.",
      "stdout": ""
  }

  After:
  localhost | FAILED! => {
      "msg": "crypt.crypt does not support 'bcrypt' algorithm"
  }

Tickets:
- Fixes #69930

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Felix Fontein ebd20ddca6
Include changelogs/changelog.yaml in distribution. (#70260) 5 years ago
Matt Clay 262f93e22c Code cleanup in the import sanity test. 5 years ago
David Shrewsbury dd07bdf22d
Fix galaxy role info bug to support multiple roles (#70148)
* Add changelog fragment
* Update changelog
5 years ago
Abhijeet Kasurde 4885ebad27
default_callback: Move 'check_mode_markers' in doc_fragments (#70228)
Callback plugin dense, yaml, and debug implement 'check_mode_markers'
so moving documentation to default callback doc_fragments.

Fixes: https://github.com/ansible-collections/community.general/issues/565

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Matt Clay 70c59423fc Pin ansible-test requirements for RHEL.
The `packaging` and `pyparsing` packages are now installed by `ansible-test` during provisioning of RHEL instances to match the downstream vendored versions.
5 years ago
Rick Elrod 0073ab5fa6
Nuke strategy.SharedPluginLoaderObj, depr. 2.11 (#70235)
* Nuke strategy.SharedPluginLoaderObj, depr. 2.11

Change:
- Nuke SharedPluginLoaderObj class
- Update tests (which seemingly didn't use it anyway)
- Changelog

Test Plan:
CI, grep

Signed-off-by: Rick Elrod <rick@elrod.me>

* Nuke from ignore.txt

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod 7584e145a9
Nuke _get_item() from callback, deprecated 2.11 (#70233)
Change:
- Remove _get_item() alias as it has been deprecated
- Update tests
- Remove relevant sanity curtailment
- Add changelog

Test Plan:
CI, grep

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod c9edb35652
Nuke systemd module user option, 2.11 deprecation (#70211)
Change:
Remove all references to the 'user' param in systemd module.

Test Plan:
CI and grep.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod 9cfc76a221
Nuke with_ squashing, deprecated for 2.11 (#70209)
Change:
Removes with_* loop squashing and tests for 2.11

Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay fa48678a08
Rename pylint plugin and add tests. (#70225)
* Renamed custom pylint plugin for unwanted names.
* Add integration tests for sanity test failures.
5 years ago
Matt Clay 234994fc07 Fix boilerplate in setup.py and lib/ansible/ dir. 5 years ago
Matt Martz 1fedb95e4b
Use libc wcwidth to calculate print width in display (#66214)
* Use libc wcwidth to calculate print width in display. Fixes #63105

* Remove errantly added blank lines

* Fixes

* Move setlocale, adjust tests to work around py2 oddity with characters following null

* Don't change cli stub

* emojis

* Remove to_text call

* Special accounting for deletions

* Add initialization function, expand tests, ensure fallback to len

* get_text_width requires text, ensure banner deals with it

* Handle setlocale errors

* Move variable decrement

* Remove unused import
5 years ago
Martin Krizek eaf6086eea
Fix storing local task_vars facts for the retry (#70171)
* Fix storing local task_vars facts for the retry

Fixes #70168
5 years ago
Jordan Borean 8b24a4c5ed
Fix up the redirection of the win setup module (#70158) 5 years ago
Sam Doran bc05415109
Only pass kwargs to our string checker not callable checkers (#70151)
Since only check_type_str() accepts extra param, only pass to our checker and
do not pass kwargs to custom checkers.

* Add unit tests
5 years ago
Matt Clay bccf6b85f2 Change comprehension to `list()` in plugin loader. 5 years ago
Matt Clay 1a0d8a51cd Fix yamllint sanity test line numbers.
Parse errors from libyaml now compensate for the offset of the documentation within a module.
5 years ago
Matt Clay e3f3b4002e
Clean up ansible-test pylint configs. (#70129)
* Sort disabled pylint codes in config.
* Fix comments in pylint config.
* Add comments to pylint config.
* Prepare to sort pylint config.
* Fix pylint config indentation.
* Sort pylint good/bad names in config.
* Add changelog fragment.
5 years ago
Rick Elrod cc071cdea6
Bump devel to 2.11.0 (#70121)
Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay 8152d8bc1a Disabled inconsistent pylint checks. 5 years ago
Matt Clay 598786e16d Update ansible-test default test containers.
The main change is the upgrade to Python 3.9.0b3.
5 years ago
David Shrewsbury fc448ea54f
Clarify blockinfile docs for insertafter/insertbefore (#69396)
* Clarify blockinfile docs for insertafter/insertbefore

It's not clear from the docs that these options take effect
only when no marker lines are found in the document.

* Add changelog fragment
5 years ago
Matt Clay f69b1846f9 Add pyparsing constraint to ansible-test.
The upcoming pyparsing 3 release will require Python 3.5 or later, see:

https://github.com/pypa/packaging/issues/313

Unfortunately pip 8.x and earlier versions do not support python version requirements, which is why this constraint is needed.
5 years ago
Matt Davis de63cba7e8
transparent downstream vendoring (#69850)
* builtin downstream vendoring support

* allows downstream packagers to install packages to `ansible/_vendor` that will automatically be added to head of sys.path during `ansible` package load
* tests

* sort conflicting package names in warning text

* sanity fixes

* skip unnecessary comparison
5 years ago
Matt Clay 7641d32f8e
Fix ansible-test import sanity test issues. (#70084) 5 years ago
Martin Krizek 843751a00d
yum/dnf: check type of elements in a name (#70072) 5 years ago
Matt Clay 5b3719805e Update ansible-test test containers. 5 years ago
John R Barker 37785255f2
validate ansible-base's and collections runtime.yml (#69742)
* Validate ansible-base & collection's runtime.yml

Add new test `runtime-metadata`

* Schema validation of file
* Error if a a legacy meta/routing.yml exist in a collection
* removal_date OR removal_version

* Add tombstone validation.

* Allow both ISO 8601 date strings and datetime.date objects (from YAML dates).

* Address review comments.

* Add metadata to test collection.

* Add requirements file.

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay 9f49db1f99 Avoid use of deprecated junit-xml method. 5 years ago
Brian Coca 4c9d9dbb56
fix discovery on loop with delegation (#70013)
* fix discovery on loop with delegation

fixes #69963
5 years ago
Matt Clay 8ffaed00f8 Add Azure Pipelines support to ansible-test. 5 years ago
Brian Coca 8c3eb7ae4a
fix configurable pipelining (#69920)
* fix configurable pipelining

Co-authored-by: Sloane Hertel <shertel@redhat.com>
5 years ago
Matt Martz 4a4a11d282
Add mccabe complexity testing (#64623)
* Add mccabe complexity testing

* Make mccabe complexity an optional error

* Add mccabe to new sanity pylint requirements

* Add a changelog fragment.

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
David Shrewsbury 0ae4dac65a
Fix copy module file perms with remote_src (#69993)
When using 'remote_src: yes' and 'mode: preserve', the code handling
the file modes has to be handled on the remote node because it's
the one that has access to the source files. This means that the
copy module itself must handle this, rather than the copy action
plugin (which is where all that logic exists). The copy module
handles this when we copy a single file over. But when it is a
directory as the src parameter value, the mode of the files
beneath it are not considered. Subdirectories are copied with
shutil.copytree() which will preserve permissions automatically.
Individual files are copied with shutil.copyfile() which does NOT
preserve permissions. We need to add some calls to shutil.copymode()
to correct that.

Note: This *always* retains individial file permissions. Specifying
a 'mode' other than 'preserve' when giving a source directory for
the 'src' param does not make sense so will be ignored in that case
only.

Fixes #69783

* Add changelog and test
5 years ago
James Cassell 47d14a33bd
config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
5 years ago
Felix Fontein a114da80ee
Use antsibull-changelog instead of packaged changelog generator (#69313)
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
5 years ago
Felix Fontein 8d93ba9120
Plugin/module docs: parse return values, add collection names in them (version_added_collection), and format them nicely in ansible-doc (#69796)
* Tag return value docs if they are a dict (and not str/None).

* Try to parse return docs as YAML.

* Properly dump return values in ansible-doc.

* Adjust plugin formatter.

* Add changelog fragment.

* Don't add 'default' for return values.

* Fix plugin_formatter.

* Only try to parse return docs if they are still a string.

* Add tests.

* Warn if RETURN cannot be parsed.

* Adjust tests. Also test for warning.

* if -> elif (otherwise EXAMPLE will be parsed too).

* Always parse return documentation, and fail if it is invalid YAML.

* Polishing.

* Mostly re-enable ansible-doc tests.

Listing from the local collection seems to be somewhat broken. I assume this
is why the test was disabled.

* Lint and make tests work with Python 2.

* Keep FQCNs in plugins (not modules), i.e. restore previous state.
5 years ago
Alicia Cozine f509a22f9d
add changelog categories, update CVE fragments to use security_fix category (#69968)
* use security_fix category in changelogs for CVEs

* these fragments do not say CVE but are security fixes

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
5 years ago
Nilashish Chakraborty 9d6b0f2b03
Support `removed_at_date` in ansible-doc (#70002)
* Support removed_at_date in ansible-doc

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

Changes:
  * ansible-doc does not support `removed_at_date` and assumes that
    deprecated dict will either have `removed_in` or `version`. This
    results in ansible-doc (and hence "sanity --test=ansible-doc")
    failing for modules having only `removed_at_date`.

  * This patch adds support for `removed_at_date` and also gives it
    precedence over `removed_in` or `version`.

* Add tests and changelog

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
5 years ago
Brian Coca ebb22655e4
implemented 'prefix' for file based cache (#69872)
* implemented 'prefix' for file based cache

Co-authored-by: s-hertel <shertel@redhat.com>
5 years ago
Jordan Borean d30fc6c0b3
galaxy - preserve symlinks on build/install (#69959)
* galaxy - preserve symlinks on build/install

* Handle directory symlinks

* py2 compat change

* Updated changelog fragment
5 years ago
Brian Coca 37e7191304
added more ways to config new options in url (#69950)
* added more ways to config new options in url
5 years ago
Martin Krizek ac20466375
Do not keep empty blocks after tag filtering (#69987)
This prevents PlayIterator having to go through empty blocks
that were created in filter_tagged_tasks. This should
be a performance improvement for playbooks that mostly skip
tasks with tags.

ci_complete
5 years ago
Brian Coca 026bf4a9a8
do we really need update vars from pc? (#69952)
* don't we really need update vars from pc
5 years ago
Brian Coca 805dff4129
fixed missing default (#69972)
* fixed missing default

 also deprecated so new options wont have to go through this
5 years ago
Matt Clay a81dd4f06a
Update ansible-test default containers. (#69819)
* Move ansible-test completion code.

* Fix a few type hints.

* Change docker completion based on context.

Collections now use version 2.0 of the default-test-container.
This is an updated version of the earlier 1.x default-test-container with ansible-base and cloud specific requirements removed.

Testing of ansible-base now uses version 1.0 of a new ansible-base-test-container.
This container is similar to the earlier 1.x default-test-container, but with unnecessary duplication of requirements across Python versions removed.

Collections which have tests that depend on requirements no longer present in the default test container should specify them in their test requirements files:

* tests/integration/requirements.txt
* tests/unit/requirements.txt

* Bump test container versions

Co-authored-by: Jordan Borean <jborean93@gmail.com>
5 years ago
Sloane Hertel 51f6d129cb
support hard coded module_defaults.yml groups for collections (#69919)
* Only allow groups which were hardcoded in module_defaults.yml

only load action groups from the collection if module_defaults contains a potential group for the action

* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml

Third party modules should not be using group/ - use the action name instead

* add externalized module_defaults tests

add the missing group and collections

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

ci_complete

* Fix import in tests

ci_complete

* Update with requested changes

ci_complete

* don't traceback since we don't validate the contents of module_defaults

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>
5 years ago
Felix Fontein a862ff2d43
Deprecation revisited (#69926)
* Allow to specify collection_name separately for deprecation.

* Use new functionality in Ansible.

* Use new functionality in tests.

* Update tagging/untagging functions.

* Update pylint deprecated sanity test.

* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).

* Improve version validation. Re-add version_added validation.

* Make sure collection names are added to return docs before schema validation.

* Extra checks to avoid crashes on bad data.

* Make C# module utils code work, and update/extend tests.

* Add changelog fragment.

* Stop extracting collection name from potentially tagged versions/dates.

* Simplify C# code.

* Update Windows modules docs.

* Forgot semicolons.
5 years ago
Matt Clay 7bff3d312f
Split out and install sanity test requirements. (#69971)
* Split out sanity test requirements.

* Run each --venv test separately.

This provides verification that the requirements for each test are properly specified.

* Use a separate requirements file per sanity test.

* Skip setuptools/cryptography setup for sanity.

* Eliminate pyyaml missing warning.

* Eliminate more pip noise.

* Fix conflicting generate_pip_install commands.

* Add changelog fragment.
5 years ago
Brian Coca ce199ef0e1
Add intg test to repro #36045 (add_host traceback) (#69912)
* Add intg test to repro #36045 (add_host traceback)

* fix raw_params usage in add_host

Co-authored-by: Adrian Likins <alikins@redhat.com>
5 years ago
Sam Doran 6065638e00
Display parameter name in string conversion warning (#57145)
Add prefix to error message for nested options. This is helpful if a subelement key has the same name as a top level key.
5 years ago
Brian Coca a64418c2b3
avoid constant checking controlpersist (#69910)
* avoid constant checking controlpersist
5 years ago
Sam Doran c41a160951
Update test images to version 1.16.0 (#66771) 5 years ago
Matt Martz bfff95393c
Add custom globals to the environment, and not per template (#69278)
* Add custom globals to the environment, and not per template

* Add changelog fragment
5 years ago
Matt Martz 9667f221a5
Make AnsibleVaultEncryptedUnicode work more like a string (#67823)
* Make AnsibleVaultEncryptedUnicode work more like a string. Fixes #24425

* Remove debugging

* Wrap some things

* Reduce diff

* data should always result in text

* add tests

* Don't just copy and paste, kids

* Add eq and ne back

* Go full UserString copy/paste

* Various version related fixes

* Remove trailing newline

* py2v3

* Add a test that can evaluate whether a variable is vault encrypted

* map was introduces in jinja2 2.7

* moar jinja

* type fix

Co-Authored-By: Sam Doran <sdoran@redhat.com>

* Remove duplicate __hash__

* Fix typo

* Add changelog fragment

* ci_complete

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Sam Doran 8dd0356719
Update bundled version of distro from 1.4.0 to 1.5.0 (#69709)
Includes changes we made in our version that were merged upstream

ci_complete
5 years ago
Matt Martz c1c6f61a18
Auto unroll generators produced by jinja filters (#68014)
* Auto unroll generators produced by jinja filters

* Unroll for native in finalize

* Fix indentation

Co-authored-by: Sam Doran <sdoran@redhat.com>

* Add changelog fragment

* ci_complete

* Always unroll regardless of jinja2

* ci_complete

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Felix Fontein 840d3a9dd7
ansible-doc: properly handle suboptions (#69795)
* Specifically handle suboptions, contains, etc in ansible-doc.
5 years ago
Matt Clay 06efc41fb0
Reduce pip output noise in ansible-test. (#69925) 5 years ago
Rick Elrod efe103cdff
Make add_host be idempotent/show changed status (#69897)
Change:
- The `add_host` action now shows an accurate change status.

Test Plan:
- Added a plethora of integration tests.

Tickets:
Fixes #69881

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Sam Doran f5718a354c
lineinfile - Add alternative backrefs example to docs (#69844)
* lineinfile - Add alternative backrefs example to docs

* Add changelog
5 years ago
the-antz 66bd385f6f
Fix documentation for the assemble modules remote_src property. (#69845) 5 years ago
Ganesh Nalawade 646536643f
Use fqcr from command module invocation using shell module (#69790)
* Use fqcr from command module invocation using shell module

Fixes https://github.com/ansible/ansible/issues/69788

Use fully qualified collection reference while invoking
command module from shell module

* Fox review comment
5 years ago
Alex Schultz 247e43b252
Fix IncludedFile equality check (#69524)
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from.  Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.

This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.

    - hosts: all
      strategy: free
      gather_facts: false
      tasks:
        - include_role:
            name: random_sleep
        - block:
          - name: set a fact (1)
            include_role:
              name: set_a_fact
              tasks_from: fact1.yml
          - name: set a fact (2)
            include_role:
              name: set_a_fact
              tasks_from: fact2.yml
        - name: include didn't run
          fail:
            msg: >
              set_a_fact didn't run
              fact1: {{ fact1 | default('not defined')}}
              fact2: {{ fact2 | default('not defined') }}"
          when: (fact1 is not defined or fact2 is not defined)

Closes #69521
5 years ago
Brian Coca 22da903e9c
return changed for group_by (#32057)
* return changed for group_by

* added tests and fixed 'early registeration'
5 years ago
David Shrewsbury 2d394382cf
Reap collections changelog fragments - 6 (#69491) 5 years ago
Andrew Klychkov 00ead98595
systemd: should fail in check_mode when service not found on host (#68136)
* systemd: should fail in check_mode when service not found on host
5 years ago
Abhijeet Kasurde d757995bb6
service_facts: Update documentation about service status (#69757)
Update documentation about undocumented service status provided
by systemd

Fixes: #69752

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Dongsu Park 598e3392a9
Discover Flatcar Linux properly for hostname (#69627)
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.

See also https://github.com/ansible/ansible/issues/69516
5 years ago
Jack 80f09efd03
do not return the body even if it failed (#69706)
* do not return the body even if it failed

* add some tests for this and rebase

* import test task

* ignore_errors when fails

Co-authored-by: Jack Zhang <jack.zhang@aspiraconnect.com>
5 years ago
Rick Elrod 2abaf320d7
[ansiballz] ensure that '' is not in sys.path (#69342)
Change:
On OpenBSD when using pipelining, we do not set cwd which results in a
permissions fatal. Ensure that `''` - cwd - is not in `sys.path`.

Test Plan:
Tested against local OpenBSD VM

Tickets:
Fixes #69320

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Hideki Saito 79ab798427 Enable logging using setup_loggers() API in dnf-4.2.17-6 or later
- Fixed issue #28061
- https://bugzilla.redhat.com/show_bug.cgi?id=1788212

Signed-off-by: Hideki Saito <saito@fgrep.org>
5 years ago
Martin Krizek 3606dcfe65
Ensure --version works with non-ascii project path (#66624)
Fixes #66617
5 years ago
Sloane Hertel e40889e711
Add support to install collections from git repositories (#69154)
* Enable installing collections from git repositories

* Add tests for installing individual and multiple collections from git repositories

* Test to make sure recursive dependencies with different syntax are deduplicated

* Add documentation

* add a changelog

* Skip Python 2.6

* Only fail if no collections are located in a git repository

Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.

Add a note that the collection(s) in the repo must contain a galaxy.yml

* Add a warning about embedding credentials in SCM URLs

* Update with review suggestions

* suppress sanity compile failure for Python 2.6
5 years ago
Felix Fontein 0b82d4499e
ansible-test ansible-doc sanity test: also run with --json parameter (#69288) 5 years ago
Abhijeet Kasurde 7b3d9cbcb8
distribution: Added support for openEuler OS (#69324)
Detect os_family for openEuler OS as 'RedHat', instead of 'openEuler'.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Jordan Borean f81f5da20e
Ansible.Basic added generic fragment merger for module options (#69719) 5 years ago
Felix Fontein 40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
5 years ago
Felix Fontein 31bf3a5622
Deprecate module in collection: allow removal date in documentation, make validate-modules ensure version and date match (#69727)
* Allow to deprecate module by date in documentation.

* Make sure deprecation date/version match between module docs and meta/runtime.yml.

* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.

* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.

* Linting.

* Allow to deprecate plugin options by date.

* Add changelog fragment for deprecation by date (also covers #68177).
5 years ago
Martin Krizek 061c6c7c6f
Prevent losing error info by including both loop and cond error msgs (#68485)
Fixes #66529
5 years ago
Ganesh Biradar 187de7a8aa
Updated CLI help for 'ansible-galaxy' for 'collection' subcommand (#69458)
* Added YAML header line and link to existing github issue
5 years ago
Sam Doran d7618712fd
blockinfile - fix line end regression (#69734)
PR #66461 introduced a regression that resulted in an in correct block in the file
if the block to be inserted did not end with a line separator. Fix this bug and add
tests to cover this scenario.
5 years ago
Matt Clay 65c57a7f5b Update default-test-container to version 1.14. 5 years ago
Riyad Preukschas e5cc12a64f
blockinfile - Preserve line endings (#66461)
Fixes #64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
5 years ago
Abhijeet Kasurde 564907d8ac
distribution: Refactor test_distribution_version testcases (#69300) 5 years ago
Rick Elrod 304c3e57e8
[ssh] Add new sshpass_prompt option (#68874)
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.

Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.

Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.

Tickets:
Fixes #34722, fixes #54743, refs #11565.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Davis f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
5 years ago
Brian Coca de3f7c7739
fix delegated interpreter discovery (#69604)
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
 as it stands we rerun discovery for the delegated host
unless its saving facts to itself
 * fixed test lacking delegate_to mock
5 years ago
James Cammarata a4072ad0e9
Split regular and handler results into their own queues (#69498)
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.

Fixes #69457
5 years ago
Sam Doran cd8920af99
file - return 'state': 'absent' when a file does not exist (#66503)
This was changed in ansible/ansible#51350.
5 years ago
Matt Martz e0f25a2b1f
Strip no log values from module response keys. Fixes #68400 (#69653) 5 years ago
Brian Coca 9281148b62
correctly merge multiple facts results (#68987)
* correctly merge multiple facts results

  fixes #68532
5 years ago
Matt Martz b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
5 years ago
Martin Krizek ecd986006e
Properly handle unicode in safe_eval (#68576)
* Properly handle unicode in safe_eval

Fixes #66943

* Update lib/ansible/template/safe_eval.py

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Sam Doran f01de15d25
ansible-galaxy - remove warning during collection install (#69541)
* ansible-galaxy - remove warning during collection install

If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.

A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.

* Update unit tests
5 years ago
Felix Fontein 43acd61901
ansible-test local change detection: use --base-branch if specified (#69508) 5 years ago
Jordan Borean ecea15c508
Unify ansible-galaxy install -r (#67843)
* Unify ansible-galaxy install -r

* Minor nit fixes for docs

* Re-align warnings

* Fix up integration test

* Fix up test where no roles/collections were in file
5 years ago
Matt Clay 86743292ec Remove incorrect function args in unarchive. 5 years ago
Matt Clay c495c92a6e Code cleanup in ansible-test. 5 years ago
Matt Clay ed4fd9be67 Code cleanup in `ansible-inventory`. 5 years ago
Matt Clay 159d79995c Freeze pycodestyle in ansible-test. 5 years ago
Matt Clay e7c2eb519b
Add Shippable request signing to ansible-test. (#69526) 5 years ago
Sam Doran b8469d5c7a
lineinfile - Use ANSIBLE_REMOTE_TMP for temporary file (#69543) 5 years ago
Lukas Pirl 34db57a47f
introduce fact "ansible_processor_nproc": number of usable vcpus (#66569)
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.

* Add fixture data and update unit tests

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Matt Clay 83381cd912 Update the default-test-container in ansible-test.
The updated container includes fewer requirements now that the collection migration has completed.

Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
5 years ago
Matt Clay d8e0aadc0d Update ansible-test support for CI providers.
Refactored CI provider code to simplify multiple provider support and addition of new providers.
5 years ago
Richard Lau 9d8cf1ed21
Add path for yum on IBM i to PKG_MGRS (#69484)
* Add path for yum on IBM i to PKG_MGRS

On IBM i, yum is installed under the `/QOpenSys/pkgs` prefix, see:
https://bitbucket.org/ibmi/opensource/src/master/docs/yum/#markdown-header-must-know-usage-notes-read-this-after-you-install

* Add changelog fragment
5 years ago
Chris Holland dfa2863dee
Remove FIXME where no change is needed (#69466)
* Added exception

* Added changelog fragment

* Remove FIXME comment

* Edit changelog fragment
5 years ago
Bob Weinand f200487414
Fix filedescriptor out of range in select() when running commands (#65058)
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
5 years ago
Brian Coca 87d9b49de2
Fix listing of colleciton plugins with symlinks (#69305)
* Fix listing of colleciton plugins with symlinks
5 years ago
Ted Pearson c13b040e67
systemd: set scope default to "system" (#67006)
- Make "system" default scope for systemd
- Remove extra None checks since there is now a default
5 years ago
Matt Martz 4c4406b2df
Flatten the directory hierarchy of modules (#68966)
* Flatten the directory hierarchy of modules

* Update ignore.txt, flatten units

* Update imports

* Completely flatten the modules directory

* Update sanity ignore

* Fix some sanity test ignores

* Fix relative import

* Fix docs builds without category

* ci_complete

* Clean up docs. ci_complete

* Adjust needs/file alias

* ci_complete

* fix hardcoded ping module paths

Co-authored-by: Matt Davis <mrd@redhat.com>
5 years ago
Felix Fontein 0e15375ffe
Add deprecated removed_in_version and deprecated_aliases version tests (#66920) 5 years ago
Brian Coca f0b6f76bc6
only show_vars when showing vars (#69365)
* only show_vars when showing vars

avoid processing function params that can be very expensive
and might not be used at all in called function.

fixes #69357

* Update changelogs/fragments/69357_optimize_inventory_graph_wo_vars.yml

Co-authored-by: Sloane Hertel <shertel@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
5 years ago
Brian Coca 0aa76503dc
avoid fatal tb on bad fqcn callback name (#69440) 5 years ago
Martin Krizek 6086ea62ee
Remove deprecation for TRANSFORM_INVALID_GROUP_CHARS (#66650)
Fixes #61889
5 years ago
Matt Davis 776f4840fc
fix `ansible-test units` to work(ish) under podman (#69462)
* ignore the missing `Networks` key, issue a warning that network disconnection won't function
5 years ago
Mark Goddard aa36b02ede
Fix fileglob plugin with non-existent subdirectory (#69451)
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:

lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL

The exact error depends on Python 2 or 3, but here is the error on
Python 2:

    AttributeError: 'NoneType' object has no attribute 'endswith'

And on Python 3:

    TypeError: expected str, bytes or os.PathLike object, not NoneType

This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().

Fixes: #69450
5 years ago
Martin Krizek 9645304da3
Validate args for includes in handlers too (#57537) 5 years ago
Rylan Polster eb40ecc843
Check for correct version of systemd Python library (#60692)
Fix issue where some versions of systemd don't have journal.sendv
5 years ago
Brian Coca 96f504cd11 added missing clog for 58461 5 years ago
Adrian Likins cd8dd4a272
Fix galaxy publish sha256 value format. (#67942)
* Fix galaxy publish sha256 value format.

The multipart/form content used for the body
of the POST to /api/automation-hub/v3/collections
was missing a newline before the line with the value
of the sha256.

automation-hub/galaxy/django skips the field entirely in
that case and automation-hub code will use None for default
to indicate that no sha256 is provided (an available option).

Fixes ansible/galaxy-dev#246

* Add changelog fragment

Co-authored-by: Matt Martz <matt@sivel.net>
5 years ago
Rick Elrod de59b17c7f
Add Fedora 32 to CI (#69222)
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.

Test Plan:
CI

Tickets:
Fixes #69230

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Abhijeet Kasurde e28e86a428
hostname: PopOS support (#69295)
Added support for PopOS in hostname module.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
John R Barker 084587913a
validate-modules: deprecated modules in collections (#68646)
* validate-modules: deprecated modules in collections

In Collections a module is marked as deprecated via meta/routing.yml

Use this file, rather than the leading `_` as part of the deprecated
test.

* Correct variable

* review comments

* indentation

* Read routing.yml only once

* pep8

* Apply suggestions from code review

Co-authored-by: Matt Clay <matt@mystile.com>

* review: remove duplicated conditional

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Brian Coca 4e72ce805d
Order my dicts (#58000)
* use orderdict for yaml dictionaries

* clog

* SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

* allow user to toggle odict off

* removed config, since requored to load config

* remove unused import

* Update changelogs/fragments/yaml_orderd_mappings.yml

Co-authored-by: Matt Clay <matt@mystile.com>

* Update lib/ansible/parsing/yaml/objects.py

Co-authored-by: Matt Clay <matt@mystile.com>

* Update lib/ansible/parsing/yaml/objects.py

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
flowerysong d446a4f70d
yum: avoid running non-yum modules via the action (#69296)
If the system's detected package manager is not yum or dnf it should
refuse to run without an explicit backend, instead of running the
non-yum module.
5 years ago
Abhijeet Kasurde 794d269a4d
Distribution: Add PopOS as Debian OS_FAMILY (#69294)
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'

Fixes: #69286

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Felix Fontein 947fa3bad3
ansible-test validate-modules: don't allow arbitrary lists and dicts for 'default', 'sample' and 'example' (#69287)
* Don't allow arbitrary lists and dicts for 'default', 'sample' and 'example'.

* Add changelog.

* Make compile with Python 2.
5 years ago
Toshio Kuratomi 049800c063 Fix metadata defaults
* Metadata defaults were not being set if only a few fields were missing.

* ansible-doc with no documentation and no status in metadata should
  return empty, just like if there was no documentation and no metadata
  at all.
5 years ago
Chris Holland 6d8cfcf539
Addressed FIXME's in hurd.py (Extracted functionality and exit early) (#69226)
* Addressed FIXME's in hurd.py

* Added changelog fragment

* Fix function order

* Added self reference

* Changed to 'is None'
5 years ago
Abhijeet Kasurde 8d43d79191
galaxy: Handle empty roles and collections (#69199)
Galaxy collection install command raised indexError,
when requirements.yml contains empty roles and collections.

This fix handles empty roles and/or empty collections.

Fixes: #68186

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Chris Holland 70219df905
Address FIXME's in sysctl.py (#69175)
* Address FIXME's in sysctl.py

* Added changelog fragment

* Updated check

* Update conditions

* if not instead of is None

* Restore and delete FIXME comments
5 years ago
Hideki Saito a67d5dbcb7
Add example setting for collections_paths parameter to examples/ansible.cfg (#69018)
Signed-off-by: Hideki Saito <saito@fgrep.org>
5 years ago
David Shrewsbury 1142faa213
Do not pass file mode during recursive copy on symlink files. (#69011)
* Do not pass file mode during recursive copy on symlink files.

The 'file' module cannot deal with mode=preserve. Do not pass that
mode to the module when 'preserve' is used.

* Fix changelog fragment filename
5 years ago
Adam Miller dba17aeb13
dnf allowerasing (#48319)
Add the ability to pass allowerasing to alter the dnf transaction
behavior.

Fixes #24161

This is effectively a port of the original pull request from
poettler-ric who has since abandoned the PR

    https://github.com/ansible/ansible/pull/34111

Signed-off-by: Adam Miller <admiller@redhat.com>
5 years ago
Rick Elrod 1c5c89baa6 Add RHEL 8.2 to CI
Change:
RHEL 8.2 GA was released this week, test it in CI instead of 8.1.

Test Plan:
CI

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod f6cfcba346 [dnf] Make behavior/errors compatible for new DNF
Change:
Extend the logic for custom error handling in the dnf module, so that on
newer DNF (such as DNF that ships with modern Fedora 31 container
images, and ships with RHEL 8.2) we report errors consistently with
older DNF.

Test Plan:
Ran dnf integration tests against an old Fedora 31 container image and a
brand new Fedora 32 container image; tess passed on both.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Abhijeet Kasurde b437236633
Redirect inventory script links (#69143)
With inventory script migrated to their respective collection,
redirect links in documentation to their respective collection
location.

Fixes: #69139

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Matt Martz bc41dd4514
Handle non-ascii paths during role installation. Fixes #69133 (#69213) 5 years ago
Brian Coca babac66f9c
Be strict about what is a boolean for keywords (#67625)
* be strict about what is a boolean for keywords

 - found and fixed typo in test , 'yes' != 'yes.'
5 years ago
David Shrewsbury ff47d3f766
Make sure collection is a list if a str is given (#69081)
* Make sure collection is a list if a str is given

* Call field validation early on collections

Because we are doing work on modifying the collections value before
it is actually validated, we can validate it ourselves early to make
sure the user supplies either a string or list. Dicts are not valid.

The new validation allows us to simplify the _ensure_default_collection()
function. And since the field is now static, we no longer need to specify
a default for it, which also allows us to simplify the function. Since
the default is now removed, we can also remove the sanity/ignore.txt entry
for collectionsearch.py.

New unit tests are added (and the existing one modified) that allow us to
make sure that we throw a parser error if a user specifies something other
than a string or list for the collections value everywhere it can be specified.

* Revert removing the collection default

The default is actually used, so restore it.

* Fix unit tests in test_helpers.py affected by early collection validation
5 years ago
Alexandre Chouinard 91bb5af688
Update ansible-galaxy to handle role requirements (#68288)
* Update galaxy role object to handle requirements

Co-Authored-By: Sandra McCann <samccann@redhat.com>
5 years ago
Chris Holland d42151e676
Removed redundant conditional in paramiko_ssh.py (#69164)
* Removed redundant conditional

* Added changelog fragment

* Removed trailing whitespace
5 years ago
Chris Holland 339c442250
Added missing parameter (#69160)
The call to daemonize() in sysvinit.py was missing the module parameter included in the function definition in service.py.

This pull request simply adds that parameter, as the module is used for error handling in daemonize().
5 years ago
Matt Clay a050d892d8 Remove obsolete vcenter setup from ansible-test. 5 years ago
Chih-Hsuan Yen bd4fdb1ca2
service_facts: fix for systemd 245 (#68211)
* service_facts: fix for systemd 245

Since systemd 245, `systemctl list-unit-files` comes with a new column
"VENDOR PRESET" [1] and breaks the service_facts module:

$ ansible localhost -m service_facts
localhost | FAILED! => {
    "changed": false,
    "msg": "Malformed output discovered from systemd list-unit-files: auditd.service                             disabled        disabled     "
}

This patch drops the third column to make it work with old and new
systemd. With the new slice operation, IndexError instead of ValueError
is raised if the output contains less than 2 columns.

Test plan: running `ansible-test integration -v service_facts` on
up-to-date Arch Linux

[1] https://github.com/systemd/systemd/pull/14445

* add changelog

Signed-off-by: Rick Elrod <rick@elrod.me>

Co-authored-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay d7da1d9bd0 Fix ansible-test coverage constraints. 5 years ago
Abhijeet Kasurde 572fe9e480
Adjust Openstack Inventory script path (#69058) 5 years ago
Abhijeet Kasurde fce66b9707
Openstack inventory script moved to openstack.cloud (#68908)
Openstack dynamic inventory script is moved from community.general to
openstack.cloud, adjust the bot meta accordingly.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Sylvia van Os 3f47610d94
Don't crash ansible-vault create when no arguments (#68667)
* Don't crash ansible-vault create when no arguments

* Add changelog entry
5 years ago
Jesse Pretorius (odyssey4me) da390b297e [pip] Enable virtualenv_command to have arguments
Currently if virtualenv_command has arguments, then the
search for the binary in the path does not work so the
user has to specify the full path to it.

To allow arguments to be used without having to specify
the path to the binary, we split the module argument into
the command and anything after the first space.

This makes using this module argument more flexible and
user friendly.

Fixes: #52275
5 years ago
Matt Martz f27c417fc4
Use Templar for galaxy skeletons (#69106)
* Use Templar for galaxy skeletons. Fixes #69104

* Update checksum, our templar doesn't remove trailing newline, jinja2 seems to remove it
5 years ago
Adam Miller 8b30360ca9
fixed handling of releasever for all known scenarios (#69057)
Signed-off-by: Adam Miller <admiller@redhat.com>
5 years ago
Rick Elrod cace616aab
Filter BLACKLIST_EXTS in PluginLoader (#69029)
Change:
Rather than hardcoding .pyo and .pyc, filter on all BLACKLIST_EXTS in
the non-legacy logic of PluginLoader (_find_fq_plugin). The two harcoded
extensions are part of BLACKLIST_EXTS already and this simply adds the
rest of the blacklisted extensions to the check.

In addition, check .endswith() instead of an exact match of the suffix,
like everywhere else that uses BLACKLIST_EXTS. This allows for
blacklisting, for example, emacs's backup files which can appear after
any extension, leading to things like `foo.py~`.

Test Plan:
Ran `ansible-playbook` against a collection where a `foo.py~` module was
getting executed instead of `foo.py` which also appeared in the same
directory. `foo.py~` is no longer executed.

Tickets:
Fixes #22268
Refs #27235

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Martz da98fc267a
Allow a collection role to call a standalone role by default (#69102)
* Allow a collection role to call a standalone role by default. Fixes #69101

* tweaked changelog text

* Guard against NoneType

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
5 years ago
rwagnergit 977b58740b
update ActionBase._low_level_execute_command to honor executable (#68315)
* update ActionBase._low_level_execute_command to honor executable

* adding changelog fragment

* renaming changelog fragment to .yml

* noop change to bump shippable

* adding raw_executable integration test

* copying aliases from raw

* removing blank lines

* skipping aix and freebsd

* noop to bump shippable

* moving tests to raw/

* removing become_method: sudo ; it doesn't work on AIX

* removing trailing blank line

* forcing become_method: su to try to get AIX to work

Co-authored-by: Rob Wagner <rob.wagner@sas.com>
5 years ago
Rick Elrod 2a90e9b615 Fix incorrect CVE reference in changelog fragment
Change:
This corrects an incorrect CVE identifier in the changelog entry for
CVE-2020-1735.

Test Plan:
N/A

Tickets:
Refs #67793, #68720

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
David Shrewsbury af44bd4ddd
Fix --start-at-task when skipping tasks with no name (#68951)
Using --start-at-task on a playbook with tasks with no name would fail
if those unnamed tasks were encountered before the targetted start task.
5 years ago