Commit Graph

30977 Commits (662d34b9a7a1b3ab1d4847eeaef201a005826aef)

Author SHA1 Message Date
Samer Deeb 7ef6b174e8
Migrate onyx modules from community.network to mellanox.onyx (#70213)
Signed-off-by: Samer Deeb <samerd@mellanox.com>
4 years ago
Rick Elrod 7584e145a9
Nuke _get_item() from callback, deprecated 2.11 (#70233)
Change:
- Remove _get_item() alias as it has been deprecated
- Update tests
- Remove relevant sanity curtailment
- Add changelog

Test Plan:
CI, grep

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Martin Krizek 88bb76f248
Fix storing delegate_to facts (#70231)
* Fix storing delegate_to facts
4 years ago
Rick Elrod c9edb35652
Nuke systemd module user option, 2.11 deprecation (#70211)
Change:
Remove all references to the 'user' param in systemd module.

Test Plan:
CI and grep.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 9cfc76a221
Nuke with_ squashing, deprecated for 2.11 (#70209)
Change:
Removes with_* loop squashing and tests for 2.11

Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Abhijeet Kasurde 40a42de081
async_status: Update documentation (#70196)
Fixes: #38164

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Clay 234994fc07 Fix boilerplate in setup.py and lib/ansible/ dir. 4 years ago
Matt Martz 1fedb95e4b
Use libc wcwidth to calculate print width in display (#66214)
* Use libc wcwidth to calculate print width in display. Fixes #63105

* Remove errantly added blank lines

* Fixes

* Move setlocale, adjust tests to work around py2 oddity with characters following null

* Don't change cli stub

* emojis

* Remove to_text call

* Special accounting for deletions

* Add initialization function, expand tests, ensure fallback to len

* get_text_width requires text, ensure banner deals with it

* Handle setlocale errors

* Move variable decrement

* Remove unused import
4 years ago
Martin Krizek eaf6086eea
Fix storing local task_vars facts for the retry (#70171)
* Fix storing local task_vars facts for the retry

Fixes #70168
4 years ago
Jordan Borean 8b24a4c5ed
Fix up the redirection of the win setup module (#70158) 4 years ago
Sam Doran bc05415109
Only pass kwargs to our string checker not callable checkers (#70151)
Since only check_type_str() accepts extra param, only pass to our checker and
do not pass kwargs to custom checkers.

* Add unit tests
4 years ago
Abhijeet Kasurde 87406890cf
find: Specify elements type in paths (#70139)
Fixes: #35990

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Matt Clay bccf6b85f2 Change comprehension to `list()` in plugin loader. 4 years ago
Matt Clay 9bbde9d085 Avoid file module false positive in newer pylint. 4 years ago
Martin Krizek 0ed5b77377
Detect failure in always block after rescue (#70094)
* Detect failure in always block after rescue

Fixes #70000

ci_complete

* Add more tests
4 years ago
anshulbehl 33e25504ac
redirecting the conjur lookup plugin to correct collection (#70106) 4 years ago
Matt Martz fa1fb2d13b
Test ansible-galaxy against pulp/pulp_ansible (#69605)
* Test galaxy cli against pulp

* linting fix

* Renames and small fixes

* Better handling for resetting pulp

* Clean up some things, add a comment

* I can't spell

* Bump fallaxy, use alternate pulp image

* Only reset pulp when we're are executing against pulp

* Update for updated pulp container

* Update some comments with correct URLs and typos

* Linting fix

* Pin pulp-fedora31 to a digest

* Address review comments for documentation
4 years ago
Rick Elrod cc071cdea6
Bump devel to 2.11.0 (#70121)
Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Davis 29c6aae2fc
try to load unqualified plugins from whitelist (#70086)
* try to load unqualified plugins from whitelist

* necessary for backcompat loading of unqualified collectionized callback plugins redirected from <= 2.9 core
* also added de-duping from actual loaded name

* add tests

* add warning test

* group test script entries by topic

* shorten warning text grep because wrapping is dumb

* fix adhoc callback loading behavior

* collections pass over whitelist wasn't respecting `_run_additional_callbacks`
* adds regression tests for same

* avoid `grep -L` in tests since it breaks the world
4 years ago
David Shrewsbury fc448ea54f
Clarify blockinfile docs for insertafter/insertbefore (#69396)
* Clarify blockinfile docs for insertafter/insertbefore

It's not clear from the docs that these options take effect
only when no marker lines are found in the document.

* Add changelog fragment
4 years ago
Matthew Davis 7ef0fd1c08
document return values of file module (#69560)
* document return values of file module
* Clarify whether path or dest input is used as dest output in file module

Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
4 years ago
Vincent Bernat ccf90a62c3
Enhance documentation around add_host bypassing the play host loop (#69692)
Signed-off-by: Vincent Bernat <vincent@bernat.ch>
4 years ago
Jordan Borean d1dcd5bcb3
remove new Windows modules not present in 2.9 (#70087) 4 years ago
Matt Davis de63cba7e8
transparent downstream vendoring (#69850)
* builtin downstream vendoring support

* allows downstream packagers to install packages to `ansible/_vendor` that will automatically be added to head of sys.path during `ansible` package load
* tests

* sort conflicting package names in warning text

* sanity fixes

* skip unnecessary comparison
4 years ago
Sam Doran d45cb01b84
ansible-galaxy - fix collection installation with trailing slashes (#70016)
If we fail to find a member when extracting a directory, try adding a trailing
slash to the member name. In certain cases, the member in the tarfile will
contain a trailing slash but the file name in FILES.json will never contain
the trailing slash.

If unable to find the member, handle the KeyError and print a nicer error.

Also check if a directory exists before creating it since it may have been
extracted from the archive.

Fixes #70009

* Add unit tests
* Use loop for trying to get members
4 years ago
Martin Krizek 843751a00d
yum/dnf: check type of elements in a name (#70072) 4 years ago
Felix Fontein 8a88b845ce
Fix aci_intf_policy_* redirects. (#70051) 4 years ago
Matt Clay 9f49db1f99 Avoid use of deprecated junit-xml method. 4 years ago
Brian Coca 4c9d9dbb56
fix discovery on loop with delegation (#70013)
* fix discovery on loop with delegation

fixes #69963
4 years ago
Falcon Taylor-Carter f5df702b55
Fix Include_vars example (#69966)
* Remove incorrect var in first_found docs
* Fix include_vars example using lookup
4 years ago
Brian Coca 8c3eb7ae4a
fix configurable pipelining (#69920)
* fix configurable pipelining

Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
David Shrewsbury 0ae4dac65a
Fix copy module file perms with remote_src (#69993)
When using 'remote_src: yes' and 'mode: preserve', the code handling
the file modes has to be handled on the remote node because it's
the one that has access to the source files. This means that the
copy module itself must handle this, rather than the copy action
plugin (which is where all that logic exists). The copy module
handles this when we copy a single file over. But when it is a
directory as the src parameter value, the mode of the files
beneath it are not considered. Subdirectories are copied with
shutil.copytree() which will preserve permissions automatically.
Individual files are copied with shutil.copyfile() which does NOT
preserve permissions. We need to add some calls to shutil.copymode()
to correct that.

Note: This *always* retains individial file permissions. Specifying
a 'mode' other than 'preserve' when giving a source directory for
the 'src' param does not make sense so will be ignored in that case
only.

Fixes #69783

* Add changelog and test
4 years ago
James Cassell 47d14a33bd
config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
4 years ago
Felix Fontein 8d93ba9120
Plugin/module docs: parse return values, add collection names in them (version_added_collection), and format them nicely in ansible-doc (#69796)
* Tag return value docs if they are a dict (and not str/None).

* Try to parse return docs as YAML.

* Properly dump return values in ansible-doc.

* Adjust plugin formatter.

* Add changelog fragment.

* Don't add 'default' for return values.

* Fix plugin_formatter.

* Only try to parse return docs if they are still a string.

* Add tests.

* Warn if RETURN cannot be parsed.

* Adjust tests. Also test for warning.

* if -> elif (otherwise EXAMPLE will be parsed too).

* Always parse return documentation, and fail if it is invalid YAML.

* Polishing.

* Mostly re-enable ansible-doc tests.

Listing from the local collection seems to be somewhat broken. I assume this
is why the test was disabled.

* Lint and make tests work with Python 2.

* Keep FQCNs in plugins (not modules), i.e. restore previous state.
4 years ago
Nilashish Chakraborty 9d6b0f2b03
Support `removed_at_date` in ansible-doc (#70002)
* Support removed_at_date in ansible-doc

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

Changes:
  * ansible-doc does not support `removed_at_date` and assumes that
    deprecated dict will either have `removed_in` or `version`. This
    results in ansible-doc (and hence "sanity --test=ansible-doc")
    failing for modules having only `removed_at_date`.

  * This patch adds support for `removed_at_date` and also gives it
    precedence over `removed_in` or `version`.

* Add tests and changelog

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
4 years ago
Brian Coca ebb22655e4
implemented 'prefix' for file based cache (#69872)
* implemented 'prefix' for file based cache

Co-authored-by: s-hertel <shertel@redhat.com>
4 years ago
Jordan Borean d30fc6c0b3
galaxy - preserve symlinks on build/install (#69959)
* galaxy - preserve symlinks on build/install

* Handle directory symlinks

* py2 compat change

* Updated changelog fragment
4 years ago
Brian Coca 37e7191304
added more ways to config new options in url (#69950)
* added more ways to config new options in url
4 years ago
Sam Doran 3815ed67d5
ansible-galaxy - Fix role info when role is not installed (#69924)
* ansible-galaxy - Fix role info when role is not installed

Only report the role not found if in offline mode, otherwise query the galaxy API
to get role information.

Fixes #69867

* Improve error message when role is not found in Ansible Galaxy
4 years ago
Kshitij Chawla 32c392e622
Raise Warning when import_playbook receives additional parameters (#64156)
* Extra whitespace around imported playbook filename are stripped.
* Corrected call for display.deprecated
* Changed warning type & added test
* Added auto verification of raised warning
* More accurate warning message
4 years ago
Martin Krizek ac20466375
Do not keep empty blocks after tag filtering (#69987)
This prevents PlayIterator having to go through empty blocks
that were created in filter_tagged_tasks. This should
be a performance improvement for playbooks that mostly skip
tasks with tags.

ci_complete
4 years ago
Matt Martz aff6b1ff21
Address linting errors in default callback (#69989) 4 years ago
Brian Coca 026bf4a9a8
do we really need update vars from pc? (#69952)
* don't we really need update vars from pc
4 years ago
Brian Coca 805dff4129
fixed missing default (#69972)
* fixed missing default

 also deprecated so new options wont have to go through this
4 years ago
Jordan Borean 15f07e7a6c
Fix up schema for collection deprecation (#69977)
* Fix up schema for collection deprecation

* Fix up modules using wrong name

* Another fix
4 years ago
Sloane Hertel 51f6d129cb
support hard coded module_defaults.yml groups for collections (#69919)
* Only allow groups which were hardcoded in module_defaults.yml

only load action groups from the collection if module_defaults contains a potential group for the action

* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml

Third party modules should not be using group/ - use the action name instead

* add externalized module_defaults tests

add the missing group and collections

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

ci_complete

* Fix import in tests

ci_complete

* Update with requested changes

ci_complete

* don't traceback since we don't validate the contents of module_defaults

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Felix Fontein a862ff2d43
Deprecation revisited (#69926)
* Allow to specify collection_name separately for deprecation.

* Use new functionality in Ansible.

* Use new functionality in tests.

* Update tagging/untagging functions.

* Update pylint deprecated sanity test.

* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).

* Improve version validation. Re-add version_added validation.

* Make sure collection names are added to return docs before schema validation.

* Extra checks to avoid crashes on bad data.

* Make C# module utils code work, and update/extend tests.

* Add changelog fragment.

* Stop extracting collection name from potentially tagged versions/dates.

* Simplify C# code.

* Update Windows modules docs.

* Forgot semicolons.
4 years ago
Brian Coca ce199ef0e1
Add intg test to repro #36045 (add_host traceback) (#69912)
* Add intg test to repro #36045 (add_host traceback)

* fix raw_params usage in add_host

Co-authored-by: Adrian Likins <alikins@redhat.com>
4 years ago
Sam Doran 6065638e00
Display parameter name in string conversion warning (#57145)
Add prefix to error message for nested options. This is helpful if a subelement key has the same name as a top level key.
4 years ago
Brian Coca a64418c2b3
avoid constant checking controlpersist (#69910)
* avoid constant checking controlpersist
4 years ago
Matt Martz bfff95393c
Add custom globals to the environment, and not per template (#69278)
* Add custom globals to the environment, and not per template

* Add changelog fragment
4 years ago
Matt Martz 9667f221a5
Make AnsibleVaultEncryptedUnicode work more like a string (#67823)
* Make AnsibleVaultEncryptedUnicode work more like a string. Fixes #24425

* Remove debugging

* Wrap some things

* Reduce diff

* data should always result in text

* add tests

* Don't just copy and paste, kids

* Add eq and ne back

* Go full UserString copy/paste

* Various version related fixes

* Remove trailing newline

* py2v3

* Add a test that can evaluate whether a variable is vault encrypted

* map was introduces in jinja2 2.7

* moar jinja

* type fix

Co-Authored-By: Sam Doran <sdoran@redhat.com>

* Remove duplicate __hash__

* Fix typo

* Add changelog fragment

* ci_complete

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Sam Doran 8dd0356719
Update bundled version of distro from 1.4.0 to 1.5.0 (#69709)
Includes changes we made in our version that were merged upstream

ci_complete
4 years ago
Matt Martz c1c6f61a18
Auto unroll generators produced by jinja filters (#68014)
* Auto unroll generators produced by jinja filters

* Unroll for native in finalize

* Fix indentation

Co-authored-by: Sam Doran <sdoran@redhat.com>

* Add changelog fragment

* ci_complete

* Always unroll regardless of jinja2

* ci_complete

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Felix Fontein 840d3a9dd7
ansible-doc: properly handle suboptions (#69795)
* Specifically handle suboptions, contains, etc in ansible-doc.
4 years ago
Matt Clay 5194965a24 Revert "Revert "fixes f5 modules specific redirects (#69865)""
This reverts commit a69c23fd5b.
4 years ago
Sloane Hertel 4882471bc9
fix unvault lookup name in example (#69913) 4 years ago
Matt Davis 984216f52e
various deprecation, display, warning, error fixes for collections redirection (#69822)
* various deprecation, display, warning, error fixes

* Update lib/ansible/utils/display.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/utils/display.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update lib/ansible/utils/display.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* cleanup, test fixes

* add collection name to deprecated() calls

* clean up redirect entries from uncommitted tests

* fix dep warning/error header text to match previous

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Matt Davis d79b23910a
fix sys.modules cleanup and blacklist behavior (#69898)
* fix sys.modules cleanup and blacklist behavior

* fix map-as-generator py2/py3 issue
* clear path_importer_cache between runs

* sanity fix

* don't be stupid with moving target generators
4 years ago
Rick Elrod efe103cdff
Make add_host be idempotent/show changed status (#69897)
Change:
- The `add_host` action now shows an accurate change status.

Test Plan:
- Added a plethora of integration tests.

Tickets:
Fixes #69881

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 062e780a68
starting metadata sunset (#69454)
* starting metadata sunset

 - purged metadata from any requirements
 - fix indent in generic handler for yaml content (whey metadata display was off)
 - make more resilient against bad formed docs
 - removed all metadata from docs template
 - remove metadata from schemas
 - removed mdata tests and from unrelated tests

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Sam Doran f5718a354c
lineinfile - Add alternative backrefs example to docs (#69844)
* lineinfile - Add alternative backrefs example to docs

* Add changelog
4 years ago
Rick Elrod 3b00c161dc
Re-add documentation for yum "install_repoquery" (#69871)
Change:
- This was removed in 2014 in 122a7021bc.
- The option still exists and is enabled by default and can lead to user
  confusion when people aren't expecting packages (or updated
  dependencies for it) to get installed and they do.
- Add the option documentation back with a few notes to make it clear what
  is happening.

Test Plan:
N/A, no code change, just documentation

Tickets:
- Refs #69497

Signed-off-by: Rick Elrod <rick@elrod.me>

* Fix sanity errors

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Nathaniel Case b69e5c55f4
Only show one deprecation warning if version and date are both defined. (#69808)
This prefers date to version on the basis that date will have been added
in 2.10, and version retained for 2.9 compatibility
4 years ago
the-antz 66bd385f6f
Fix documentation for the assemble modules remote_src property. (#69845) 4 years ago
Ganesh Nalawade 646536643f
Use fqcr from command module invocation using shell module (#69790)
* Use fqcr from command module invocation using shell module

Fixes https://github.com/ansible/ansible/issues/69788

Use fully qualified collection reference while invoking
command module from shell module

* Fox review comment
4 years ago
Alex Schultz 247e43b252
Fix IncludedFile equality check (#69524)
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from.  Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.

This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.

    - hosts: all
      strategy: free
      gather_facts: false
      tasks:
        - include_role:
            name: random_sleep
        - block:
          - name: set a fact (1)
            include_role:
              name: set_a_fact
              tasks_from: fact1.yml
          - name: set a fact (2)
            include_role:
              name: set_a_fact
              tasks_from: fact2.yml
        - name: include didn't run
          fail:
            msg: >
              set_a_fact didn't run
              fact1: {{ fact1 | default('not defined')}}
              fact2: {{ fact2 | default('not defined') }}"
          when: (fact1 is not defined or fact2 is not defined)

Closes #69521
4 years ago
Toshio Kuratomi 53b95b865f __eq__ should be redefined if __hash__ is defined. 4 years ago
Matt Clay 4ecf22db9a Revert "fixes f5 modules specific redirects (#69865)"
This reverts commit 16f5dbf985.
4 years ago
Wojciech Wypior 16f5dbf985
fixes f5 modules specific redirects (#69865)
* fixes f5 modules specific redirects

* adds removed entry
4 years ago
Brian Coca 22da903e9c
return changed for group_by (#32057)
* return changed for group_by

* added tests and fixed 'early registeration'
4 years ago
Andrew Klychkov 00ead98595
systemd: should fail in check_mode when service not found on host (#68136)
* systemd: should fail in check_mode when service not found on host
4 years ago
Abhijeet Kasurde d757995bb6
service_facts: Update documentation about service status (#69757)
Update documentation about undocumented service status provided
by systemd

Fixes: #69752

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Dongsu Park 598e3392a9
Discover Flatcar Linux properly for hostname (#69627)
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.

See also https://github.com/ansible/ansible/issues/69516
4 years ago
Evgeni Golov e10e5fc4ad
only ever catch ImportError (#69792)
ModuleNotFoundError is a subclass of ImportError but only exists in
Python 3.6 or newer. Instead of doing hacks to be able to catch that on
older Pythons, just always only catch ImportError, which will also catch
ModuleNotFoundError on Python 3.6 or later
4 years ago
Jack 80f09efd03
do not return the body even if it failed (#69706)
* do not return the body even if it failed

* add some tests for this and rebase

* import test task

* ignore_errors when fails

Co-authored-by: Jack Zhang <jack.zhang@aspiraconnect.com>
4 years ago
Abhijeet Kasurde 723a904f4e
Handle disabled service units (#69349)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Rick Elrod 2abaf320d7
[ansiballz] ensure that '' is not in sys.path (#69342)
Change:
On OpenBSD when using pipelining, we do not set cwd which results in a
permissions fatal. Ensure that `''` - cwd - is not in `sys.path`.

Test Plan:
Tested against local OpenBSD VM

Tickets:
Fixes #69320

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Hideki Saito 79ab798427 Enable logging using setup_loggers() API in dnf-4.2.17-6 or later
- Fixed issue #28061
- https://bugzilla.redhat.com/show_bug.cgi?id=1788212

Signed-off-by: Hideki Saito <saito@fgrep.org>
4 years ago
Felix Fontein b1b79be08b
Fix return value interpretation of PluginLoader.find_plugin_with_context(). (#69772) 4 years ago
Evgeni Golov 53f9822e75
make plugins loader Python 3.5 compatible again (#69754)
ModuleNotFoundError was only introduced in Python 3.6
4 years ago
Martin Krizek 3606dcfe65
Ensure --version works with non-ascii project path (#66624)
Fixes #66617
4 years ago
Sloane Hertel e40889e711
Add support to install collections from git repositories (#69154)
* Enable installing collections from git repositories

* Add tests for installing individual and multiple collections from git repositories

* Test to make sure recursive dependencies with different syntax are deduplicated

* Add documentation

* add a changelog

* Skip Python 2.6

* Only fail if no collections are located in a git repository

Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.

Add a note that the collection(s) in the repo must contain a galaxy.yml

* Add a warning about embedding credentials in SCM URLs

* Update with review suggestions

* suppress sanity compile failure for Python 2.6
4 years ago
Karl Fogel 225ae65b0f
Make Python path warning say what it means to say (#69669) 4 years ago
Nilashish Chakraborty c3dff34c15
Add entries for missing network modules (#69760)
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
4 years ago
Rick Elrod 8e19ab178a
connection plugins: try config, then play_context (#69751)
Change:
Rather than only using config, have base connection plugins fall back to
play_context.

Test Plan:
- Tested ansible-connection logic against an IOS device
- Tested -k against a VM
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
John R Barker eea84b6fdd
routing.yml: update collections (#69699)
* routing.yml: update collections

routing.yml had gotten out of sync since `migrate.py` was run.
Update based on where files are today (more updates are needed)
Also validate routing.yml's schema

* Update lib/ansible/config/routing.yml

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>

* rename scripts

* docker_service

* updated ansible_builtin_runtime schema

* Correct schema for plugin_router and import_redirect

* Correct schema for plugin_router and import_redirect

* validation moved to #69742

* netap, slxos, checkpoint

* test moved to separate PR

* even more

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
4 years ago
Abhijeet Kasurde 7b3d9cbcb8
distribution: Added support for openEuler OS (#69324)
Detect os_family for openEuler OS as 'RedHat', instead of 'openEuler'.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Jordan Borean f81f5da20e
Ansible.Basic added generic fragment merger for module options (#69719) 4 years ago
Felix Fontein 40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
4 years ago
Jordan Borean f5f3ba7ab5
ansible-test - fix up relative util import for powershell validate-modules (#69753)
* ansible-test - fix up relative util import for powershell validate-modules

* Use different tactic for generic group

* Use python 2 and 3
4 years ago
Felix Fontein 31bf3a5622
Deprecate module in collection: allow removal date in documentation, make validate-modules ensure version and date match (#69727)
* Allow to deprecate module by date in documentation.

* Make sure deprecation date/version match between module docs and meta/runtime.yml.

* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.

* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.

* Linting.

* Allow to deprecate plugin options by date.

* Add changelog fragment for deprecation by date (also covers #68177).
4 years ago
Martin Krizek 061c6c7c6f
Prevent losing error info by including both loop and cond error msgs (#68485)
Fixes #66529
4 years ago
Nick Busey 0894ea1b1d
Adding nested variable example (#69518)
* Adding nested variable example to lib/ansible/plugins/lookup/vars.py
Co-authored-by: Matt Martz <matt@sivel.net>
4 years ago
Andrew Klychkov 323d2adfcc
Fix example format in Ansible modules (#69509)
* Fix example format in several Ansible modules
* Update lib/ansible/modules/fail.py
* Update lib/ansible/modules/pip.py

Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Toshio Kuratomi 412af6ae91
Fix docs return formatting. (#69632)
Gets rid of the unknown field names in the return data.
Allows the plugin return docs to format under the new docs pipeline.
* Expect that package_facts will pass return-syntax-error now.
4 years ago
Ganesh Biradar 187de7a8aa
Updated CLI help for 'ansible-galaxy' for 'collection' subcommand (#69458)
* Added YAML header line and link to existing github issue
4 years ago
Sam Doran d7618712fd
blockinfile - fix line end regression (#69734)
PR #66461 introduced a regression that resulted in an in correct block in the file
if the block to be inserted did not end with a line separator. Fix this bug and add
tests to cover this scenario.
4 years ago
Riyad Preukschas e5cc12a64f
blockinfile - Preserve line endings (#66461)
Fixes #64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
4 years ago
Rick Elrod 8b6c02fc69
Make connection plugins use password from config (#69670)
Change:
Rather than connection plugins directly accessing play_context and pulling
the password from there, have them pull it from the config system, and
have TaskExecutor store it there for now.

Internally, it still routes through play_context for now, but this is
the first step away from that.

Test Plan:
- Local test with `ansible -c ssh`
- grep -R play_context.pass lib/ansible/plugins/connection/
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Felix Fontein cb8f645e0f
Allow to deprecate / remove plugins by version. (#69712) 4 years ago
Rick Elrod 304c3e57e8
[ssh] Add new sshpass_prompt option (#68874)
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.

Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.

Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.

Tickets:
Fixes #34722, fixes #54743, refs #11565.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Felix Fontein ea04e0048d
Allow to deprecate options and aliases by date (#68177)
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in #67684.

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
4 years ago
Matt Davis fa81cc6a0c
fix delegated loading when path is not a directory (#69713)
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
4 years ago
Brian Coca c987c0c8a1
provides per host/plugin world readable temp opt (#69393)
fixes #46174
4 years ago
Rick Elrod 4aff87770e
[yum] Make package removal confirmation strict (#69592)
* [yum] Make package removal confirmation strict

Change:
After removing packages, the yum module does a final check to ensure the
packages are really installed. The check would include packages that
were RPM `Provides:` values of another package.

This means that, for example, if a third-party kernel RPM spec had
`Provides: kernel` in it, removing the stock kernel would be successful
but the check to see if it was really removed would fail and cause
Ansible to report a failure.

Test Plan:
Tested on local CentOS 7 VM with kernel from elrepo which is known to
`Provides: kernel`.

Tickets:
Fixes #69237
Refs #35672
Refs #40723

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Davis f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
4 years ago
Helder Garcia fdfa6fec75
Module Find returns empty list when setting path to "/" and depth to 1 (#69650)
* Fix bug on find module when path is / and depth 1.
4 years ago
Nilashish Chakraborty dae3ba71a8
Add FQCN mappings for network facts modules (#69601)
* Add FQCN mappings for network facts modules

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add unit tests

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* update entry for ironware

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
4 years ago
Rick Elrod fe9696be52
Fix `ansible -K` become_pass regression (#69629)
* Fix `ansible -K` become_pass regression

Change:
- This fixes a breaking change introduced in
  2165f9ac40

Test Plan:
- Local VM for now, with plans to add an integration test for -K going
  forward.

Tickets:
Refs #69244
4 years ago
Brian Coca de3f7c7739
fix delegated interpreter discovery (#69604)
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
 as it stands we rerun discovery for the delegated host
unless its saving facts to itself
 * fixed test lacking delegate_to mock
4 years ago
Rick Elrod dc63b36501
Spelling: dependant -> dependent (#69661) 4 years ago
James Cammarata a4072ad0e9
Split regular and handler results into their own queues (#69498)
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.

Fixes #69457
4 years ago
Sam Doran cd8920af99
file - return 'state': 'absent' when a file does not exist (#66503)
This was changed in ansible/ansible#51350.
4 years ago
Matt Martz e0f25a2b1f
Strip no log values from module response keys. Fixes #68400 (#69653) 4 years ago
David Soper eb5147e57b
intersight_info and intersight_facts routed to cisco.intersight (#69648)
Co-authored-by: “dsoper2” <“dsoper@cisco.com”>
4 years ago
Martin Krizek 1ee70fc272
ansible.utils.vars.isidentifier improvements (#58278)
ci_complete
4 years ago
Brian Coca 9281148b62
correctly merge multiple facts results (#68987)
* correctly merge multiple facts results

  fixes #68532
4 years ago
Matt Martz b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
4 years ago
Martin Krizek ecd986006e
Properly handle unicode in safe_eval (#68576)
* Properly handle unicode in safe_eval

Fixes #66943

* Update lib/ansible/template/safe_eval.py

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Toshio Kuratomi 9780d3db18 Remove "keywords" field in DOCUMENTATION.
This doesn't seem to be used by config although it came in with the
config PR.  It's not displayed in docs.  Removing it as this is the only
place where it's present.
4 years ago
Brian Coca df4e83deda
added 'task timeout' feature (#69284)
* added 'task timeout' feature


Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Felix Fontein ac533403e3
Google gcp_*_facts modules were symlinks to gcp_*_info modules in 2.9. (#69566)
Since it doesn't make sense to have extra copies of the _info modules in community.general only so that the _facts
symlinks are not in google.cloud, let's simply redirect the _facts modules to their google.cloud _info counterpart.

See also ansible-collections/community.general#351.
4 years ago
Sam Doran f01de15d25
ansible-galaxy - remove warning during collection install (#69541)
* ansible-galaxy - remove warning during collection install

If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.

A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.

* Update unit tests
4 years ago
Monty Taylor dd4219184b
Update openstack module redirections (#69562)
The openstack related modules moved to the openstack cloud
collection. Update the target pointers.
4 years ago
Jordan Borean ecea15c508
Unify ansible-galaxy install -r (#67843)
* Unify ansible-galaxy install -r

* Minor nit fixes for docs

* Re-align warnings

* Fix up integration test

* Fix up test where no roles/collections were in file
4 years ago
Matt Clay 86743292ec Remove incorrect function args in unarchive. 4 years ago
Matt Clay ed4fd9be67 Code cleanup in `ansible-inventory`. 4 years ago
Sam Doran b8469d5c7a
lineinfile - Use ANSIBLE_REMOTE_TMP for temporary file (#69543) 4 years ago
Lukas Pirl 34db57a47f
introduce fact "ansible_processor_nproc": number of usable vcpus (#66569)
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.

* Add fixture data and update unit tests

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Jeff Geerling d63a71e3f8
typofix - globbing instead of globing. (#69515)
Fix typo introduced in PR #69087
4 years ago
Matt Clay d0484fbf74 Cleanup for Python 3.9 and pycodestyle compat. 4 years ago
Richard Lau 9d8cf1ed21
Add path for yum on IBM i to PKG_MGRS (#69484)
* Add path for yum on IBM i to PKG_MGRS

On IBM i, yum is installed under the `/QOpenSys/pkgs` prefix, see:
https://bitbucket.org/ibmi/opensource/src/master/docs/yum/#markdown-header-must-know-usage-notes-read-this-after-you-install

* Add changelog fragment
4 years ago
Brian Coca 2165f9ac40
fix delegation vars usage (debug still shows inventory_hostname) (#69244)
* fix delegation vars usage and reporting

 - just pass delegated host vars + task vars to plugins
   and avoid poluting with original host vars
 - updated tests
4 years ago
Chris Holland dfa2863dee
Remove FIXME where no change is needed (#69466)
* Added exception

* Added changelog fragment

* Remove FIXME comment

* Edit changelog fragment
4 years ago
Felix Fontein 9767d592a8
Add resource_from_fqcr utility function. (#68474) 4 years ago
Sloane Hertel 55e29a1464
ansible-galaxy - fall back to galaxy.yml when listing collections (#68925)
Fallback to galaxy.yml for listing collections that don't have a FILES.json or MANIFEST.json

Co-authored-by: Sam Doran <sdoran@redhat.com>
4 years ago
Sloane Hertel 343ffaa18b
better error for "ansible-galaxy collection verify" if there's no MANIFEST.json (#67498)
* Add a better error for "ansible-galaxy verify" if the MANIFEST.json has been deleted from the installed collection or if the collection hasn't been installed via normal means

* Fix unit tests for the remote collection

If there's something wrong with the local collection's version it will fail before the remote collection is sought

* Add a test for the new error msg

* Prevent the duplicate warning

Mock the new isfile call where needed

* Update lib/ansible/galaxy/collection.py

Co-Authored-By: Martin Krizek <martin.krizek@gmail.com>

Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
4 years ago
Bob Weinand f200487414
Fix filedescriptor out of range in select() when running commands (#65058)
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
4 years ago
Brian Coca 87d9b49de2
Fix listing of colleciton plugins with symlinks (#69305)
* Fix listing of colleciton plugins with symlinks
4 years ago
Ted Pearson c13b040e67
systemd: set scope default to "system" (#67006)
- Make "system" default scope for systemd
- Remove extra None checks since there is now a default
4 years ago
Brian Coca cedfe34619
added unvault lookup plugin (#69087)
* added unvault lookup plugin
4 years ago
Matt Martz 4c4406b2df
Flatten the directory hierarchy of modules (#68966)
* Flatten the directory hierarchy of modules

* Update ignore.txt, flatten units

* Update imports

* Completely flatten the modules directory

* Update sanity ignore

* Fix some sanity test ignores

* Fix relative import

* Fix docs builds without category

* ci_complete

* Clean up docs. ci_complete

* Adjust needs/file alias

* ci_complete

* fix hardcoded ping module paths

Co-authored-by: Matt Davis <mrd@redhat.com>
4 years ago
Brian Coca c43f275c7e
added missing headers + note (#69500) 4 years ago
Brian Coca f0b6f76bc6
only show_vars when showing vars (#69365)
* only show_vars when showing vars

avoid processing function params that can be very expensive
and might not be used at all in called function.

fixes #69357

* Update changelogs/fragments/69357_optimize_inventory_graph_wo_vars.yml

Co-authored-by: Sloane Hertel <shertel@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
Toshio Kuratomi 3b3c811be5
Separate the galaxy lib from the cli (#69473)
The galaxy lib knew about the cli args in context.  This shouldn't be
the case as it makes it hard to use the lib in other contexts.  Moved
the context knowledge into cli/galaxy.py.
4 years ago
Brian Coca 0aa76503dc
avoid fatal tb on bad fqcn callback name (#69440) 4 years ago
Toshio Kuratomi eb3e4b3a7b cyberark appears to have renamed their collection
Trying to build docs for Ansible-2.10 and ran into error #69475
in ansible-doc.  To work around that I need to get a newer cyberark
collection which fixes the issues.  Unfortunately, it looks like the
cyberark team has renamed their collection from bizdev to pas is not
responding to issues on github.  I'm putting in this PR for them to
get this updated before 2.10 so that the modules will work out of the
box.
4 years ago
Martin Krizek 6086ea62ee
Remove deprecation for TRANSFORM_INVALID_GROUP_CHARS (#66650)
Fixes #61889
4 years ago
Mark Goddard aa36b02ede
Fix fileglob plugin with non-existent subdirectory (#69451)
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:

lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL

The exact error depends on Python 2 or 3, but here is the error on
Python 2:

    AttributeError: 'NoneType' object has no attribute 'endswith'

And on Python 3:

    TypeError: expected str, bytes or os.PathLike object, not NoneType

This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().

Fixes: #69450
4 years ago
Toby Foster 01258580b9
Fix formatting in docs for `--become-method` (#68152)
The default value isn't displayed currently: <https://docs.ansible.com/ansible/latest/cli/ansible.html#cmdoption-ansible-become-method>

+label: docsite_pr
4 years ago
Martin Krizek 9645304da3
Validate args for includes in handlers too (#57537) 4 years ago
Rylan Polster eb40ecc843
Check for correct version of systemd Python library (#60692)
Fix issue where some versions of systemd don't have journal.sendv
4 years ago
Adrian Likins cd8dd4a272
Fix galaxy publish sha256 value format. (#67942)
* Fix galaxy publish sha256 value format.

The multipart/form content used for the body
of the POST to /api/automation-hub/v3/collections
was missing a newline before the line with the value
of the sha256.

automation-hub/galaxy/django skips the field entirely in
that case and automation-hub code will use None for default
to indicate that no sha256 is provided (an available option).

Fixes ansible/galaxy-dev#246

* Add changelog fragment

Co-authored-by: Matt Martz <matt@sivel.net>
4 years ago
Felix Fontein 5b9418c06c
Fix SemanticVersion comparison (#69395)
* Fix SemanticVersion comparison.

* Complete tests for _Alpha and _Numeric comparators.

* Linting, and add comment.
4 years ago
Rick Elrod 9579113941 Remove some no-op code from InventoryManager
Change:
- Remove some no-op code
- Split up a somewhat complex line into two lines
- Nuke an incorrect comment

Test Plan:
CI

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Gonéri Le Bouder c62ff861d3
git: don't import get_module_path (#69378)
The module don't use the `get_module_path()` function. No need to import
it.
4 years ago
rchincholkar bc1364952b
changing $HOME to $HOSTNAME in Synopsis (#69273)
$HOME variable does work with command module. This may be a bug. $HOSTNAME never works. Changing the example to use $HOSTNAME in the Synopsis of command module documentation page.
4 years ago
Abhijeet Kasurde e28e86a428
hostname: PopOS support (#69295)
Added support for PopOS in hostname module.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Toshio Kuratomi 39b3942048 Remove left hand side slicing
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances.  There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code.  (The most likely case of
this is when it's a global variable and some other code might import
that variable name).

Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement.  At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
4 years ago
Eric L 852906fe7f
Make description of removes option of command module clearer (hopefully) (#68097)
* Improves description of removes option in command module

Co-authored-by: Eric L <android@lavar.de>
4 years ago
Brian Coca 4e72ce805d
Order my dicts (#58000)
* use orderdict for yaml dictionaries

* clog

* SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

* allow user to toggle odict off

* removed config, since requored to load config

* remove unused import

* Update changelogs/fragments/yaml_orderd_mappings.yml

Co-authored-by: Matt Clay <matt@mystile.com>

* Update lib/ansible/parsing/yaml/objects.py

Co-authored-by: Matt Clay <matt@mystile.com>

* Update lib/ansible/parsing/yaml/objects.py

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Kenyon Ralph f6860a7a89
generator inventory plugin: remove "strict" from example (#69178)
This "strict" option appears to be unused. It's probably a copy-paste error from the constructed plugin.
4 years ago
Matt Davis 47da0f7a4f
restore jsonfile cache plugin (#69337)
* restore jsonfile

* update botmeta for jsonfile restoration
4 years ago
flowerysong d446a4f70d
yum: avoid running non-yum modules via the action (#69296)
If the system's detected package manager is not yum or dnf it should
refuse to run without an explicit backend, instead of running the
non-yum module.
4 years ago
Abhijeet Kasurde 794d269a4d
Distribution: Add PopOS as Debian OS_FAMILY (#69294)
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'

Fixes: #69286

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Toshio Kuratomi 049800c063 Fix metadata defaults
* Metadata defaults were not being set if only a few fields were missing.

* ansible-doc with no documentation and no status in metadata should
  return empty, just like if there was no documentation and no metadata
  at all.
4 years ago
Chris Holland 6d8cfcf539
Addressed FIXME's in hurd.py (Extracted functionality and exit early) (#69226)
* Addressed FIXME's in hurd.py

* Added changelog fragment

* Fix function order

* Added self reference

* Changed to 'is None'
4 years ago
Abhijeet Kasurde 8d43d79191
galaxy: Handle empty roles and collections (#69199)
Galaxy collection install command raised indexError,
when requirements.yml contains empty roles and collections.

This fix handles empty roles and/or empty collections.

Fixes: #68186

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Clément Notin 9d48884e36
Fix support for Kali Linux detection (#69194)
* Fix support for Kali Linux detection

* Add test for Kali Linux detection

* Improve path matching with "in list"

Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Chris Holland 70219df905
Address FIXME's in sysctl.py (#69175)
* Address FIXME's in sysctl.py

* Added changelog fragment

* Updated check

* Update conditions

* if not instead of is None

* Restore and delete FIXME comments
4 years ago
Kevin Gentile 5e74ada06b
fix expires usage sentence flow (#69306) 4 years ago
Jordan Borean 722d89478b
Remove Ansible.Service from routing (#69317) 4 years ago
Brian Coca bbdf77a59f
preserve json parsing error (#58461)
* preserve json parsing error
* added test
4 years ago
Brian Coca 23ab8f37df
fix a-doc listing plugins and add tests (#68600)
* add docs listing tests
* added collection module docs test
* always safe_load
* force 'type consistency' for uniquing paths
* bytified
* use our json encoder

Co-Authored-By: Matt Clay <matt@mystile.com>
4 years ago
David Shrewsbury 1142faa213
Do not pass file mode during recursive copy on symlink files. (#69011)
* Do not pass file mode during recursive copy on symlink files.

The 'file' module cannot deal with mode=preserve. Do not pass that
mode to the module when 'preserve' is used.

* Fix changelog fragment filename
4 years ago
Adam Miller dba17aeb13
dnf allowerasing (#48319)
Add the ability to pass allowerasing to alter the dnf transaction
behavior.

Fixes #24161

This is effectively a port of the original pull request from
poettler-ric who has since abandoned the PR

    https://github.com/ansible/ansible/pull/34111

Signed-off-by: Adam Miller <admiller@redhat.com>
4 years ago
Rick Elrod f6cfcba346 [dnf] Make behavior/errors compatible for new DNF
Change:
Extend the logic for custom error handling in the dnf module, so that on
newer DNF (such as DNF that ships with modern Fedora 31 container
images, and ships with RHEL 8.2) we report errors consistently with
older DNF.

Test Plan:
Ran dnf integration tests against an old Fedora 31 container image and a
brand new Fedora 32 container image; tess passed on both.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz bc41dd4514
Handle non-ascii paths during role installation. Fixes #69133 (#69213) 4 years ago
Brian Coca babac66f9c
Be strict about what is a boolean for keywords (#67625)
* be strict about what is a boolean for keywords

 - found and fixed typo in test , 'yes' != 'yes.'
4 years ago
David Shrewsbury ff47d3f766
Make sure collection is a list if a str is given (#69081)
* Make sure collection is a list if a str is given

* Call field validation early on collections

Because we are doing work on modifying the collections value before
it is actually validated, we can validate it ourselves early to make
sure the user supplies either a string or list. Dicts are not valid.

The new validation allows us to simplify the _ensure_default_collection()
function. And since the field is now static, we no longer need to specify
a default for it, which also allows us to simplify the function. Since
the default is now removed, we can also remove the sanity/ignore.txt entry
for collectionsearch.py.

New unit tests are added (and the existing one modified) that allow us to
make sure that we throw a parser error if a user specifies something other
than a string or list for the collections value everywhere it can be specified.

* Revert removing the collection default

The default is actually used, so restore it.

* Fix unit tests in test_helpers.py affected by early collection validation
4 years ago
Alexandre Chouinard 91bb5af688
Update ansible-galaxy to handle role requirements (#68288)
* Update galaxy role object to handle requirements

Co-Authored-By: Sandra McCann <samccann@redhat.com>
4 years ago
Chris Holland d42151e676
Removed redundant conditional in paramiko_ssh.py (#69164)
* Removed redundant conditional

* Added changelog fragment

* Removed trailing whitespace
4 years ago
Chris Holland 339c442250
Added missing parameter (#69160)
The call to daemonize() in sysvinit.py was missing the module parameter included in the function definition in service.py.

This pull request simply adds that parameter, as the module is used for error handling in daemonize().
4 years ago
Daniel Mellado 9217aeeac1
Revert "Fix missing persistent connection messages (#68496)" (#69147)
This reverts commit 5f6427b1fc.
as it breaks netconf connection. This will be a temporary measure
for unlocking CI until a proper fix is shipped.
4 years ago
Kenyon Ralph 91d02e1c1f
constructed inventory plugin: correct example (#69165)
Fixed 'intersect' filter name in constructed inventory plugin example.
4 years ago
Chih-Hsuan Yen bd4fdb1ca2
service_facts: fix for systemd 245 (#68211)
* service_facts: fix for systemd 245

Since systemd 245, `systemctl list-unit-files` comes with a new column
"VENDOR PRESET" [1] and breaks the service_facts module:

$ ansible localhost -m service_facts
localhost | FAILED! => {
    "changed": false,
    "msg": "Malformed output discovered from systemd list-unit-files: auditd.service                             disabled        disabled     "
}

This patch drops the third column to make it work with old and new
systemd. With the new slice operation, IndexError instead of ValueError
is raised if the output contains less than 2 columns.

Test plan: running `ansible-test integration -v service_facts` on
up-to-date Arch Linux

[1] https://github.com/systemd/systemd/pull/14445

* add changelog

Signed-off-by: Rick Elrod <rick@elrod.me>

Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
anshulbehl 42cc3fa923
collection build should ignore .git files (#68750) 4 years ago
Sylvia van Os 3f47610d94
Don't crash ansible-vault create when no arguments (#68667)
* Don't crash ansible-vault create when no arguments

* Add changelog entry
4 years ago
Jordan Borean 813ea48fcf
Bring back more objects that were exposes in _text (#69090) 4 years ago
Jesse Pretorius (odyssey4me) da390b297e [pip] Enable virtualenv_command to have arguments
Currently if virtualenv_command has arguments, then the
search for the binary in the path does not work so the
user has to specify the full path to it.

To allow arguments to be used without having to specify
the path to the binary, we split the module argument into
the command and anything after the first space.

This makes using this module argument more flexible and
user friendly.

Fixes: #52275
4 years ago
Matt Martz f27c417fc4
Use Templar for galaxy skeletons (#69106)
* Use Templar for galaxy skeletons. Fixes #69104

* Update checksum, our templar doesn't remove trailing newline, jinja2 seems to remove it
4 years ago
Adam Miller 8b30360ca9
fixed handling of releasever for all known scenarios (#69057)
Signed-off-by: Adam Miller <admiller@redhat.com>
4 years ago
Rick Elrod cace616aab
Filter BLACKLIST_EXTS in PluginLoader (#69029)
Change:
Rather than hardcoding .pyo and .pyc, filter on all BLACKLIST_EXTS in
the non-legacy logic of PluginLoader (_find_fq_plugin). The two harcoded
extensions are part of BLACKLIST_EXTS already and this simply adds the
rest of the blacklisted extensions to the check.

In addition, check .endswith() instead of an exact match of the suffix,
like everywhere else that uses BLACKLIST_EXTS. This allows for
blacklisting, for example, emacs's backup files which can appear after
any extension, leading to things like `foo.py~`.

Test Plan:
Ran `ansible-playbook` against a collection where a `foo.py~` module was
getting executed instead of `foo.py` which also appeared in the same
directory. `foo.py~` is no longer executed.

Tickets:
Fixes #22268
Refs #27235

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz da98fc267a
Allow a collection role to call a standalone role by default (#69102)
* Allow a collection role to call a standalone role by default. Fixes #69101

* tweaked changelog text

* Guard against NoneType

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
4 years ago
rwagnergit 977b58740b
update ActionBase._low_level_execute_command to honor executable (#68315)
* update ActionBase._low_level_execute_command to honor executable

* adding changelog fragment

* renaming changelog fragment to .yml

* noop change to bump shippable

* adding raw_executable integration test

* copying aliases from raw

* removing blank lines

* skipping aix and freebsd

* noop to bump shippable

* moving tests to raw/

* removing become_method: sudo ; it doesn't work on AIX

* removing trailing blank line

* forcing become_method: su to try to get AIX to work

Co-authored-by: Rob Wagner <rob.wagner@sas.com>
4 years ago
David Shrewsbury af44bd4ddd
Fix --start-at-task when skipping tasks with no name (#68951)
Using --start-at-task on a playbook with tasks with no name would fail
if those unnamed tasks were encountered before the targetted start task.
4 years ago
Martin Krizek cb389f6c31
Prevent rewriting nested Block's data in filter_tagged_tasks (#68337)
This patch fixes the issue where nested Block copies were created from
incorrect Block object. This resulted in nested Blocks data like ``name``
or ``_uuid`` to contain values from the Block the filter_tagged_tasks
method was called on.
4 years ago
Robert de Bock fd8b874273
Update docs, pointing to an existing chapter. (#69026)
The chapter that described more information about become to an unprivileged user has moved. This commit points to an existing chapter

Relates to #69021
4 years ago
Brian Coca 561725bace
corrected comment (#69006) 4 years ago