Commit Graph

28 Commits (b02168d644dfaff1f76f52931d7b41485c2753a3)

Author SHA1 Message Date
Matt Clay 4ea8d9a782
ansible-test - split controller/target testing (#75605) 3 years ago
Sloane Hertel 865bda3a11
Add a resolved_action task attribute (#74709)
* The resolved_action is the formatted version of the final plugin in the PluginLoadContext's redirect_list

* Collection plugins are represented as FQCN

* Legacy plugins are represented with only the plugin name

* Add tests

* Changelog
3 years ago
Sloane Hertel 48c0fbd1cb
Fix a bug adding unrelated candidates to the plugin loader redirect_list (#73863)
* Fix a bug adding unrelated candidates to the plugin loader redirect_list

* Add tests for the redirect list

  * test redirect list for builtin module
  * test redirect list for redirected builtin module
  * test redirect list for collection module
  * test redirect list for redirected collection module
  * test redirect list for legacy module

* changelog
4 years ago
Brian Coca 1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
4 years ago
Brian Coca d22804c4fb
saner path dir management (#72648)
* saner path dir management

   fixes #72628

   ensure we always store paths w/o a_c

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
4 years ago
Felix Fontein 6b30efa454 Fix fallout from #70028. 4 years ago
Brian Coca 4b673484f0
rethink wording (#70028)
* rethink wording
* removed unrequired requirement
* fix tests
* fixed versions
Co-authored-by: Sloane Hertel <shertel@redhat.com>
4 years ago
Brian Coca e05c62547b
run playbook from collections (#67435)
* fixes for collection playbooks

 - add fqcn invocation, also handles extensions
 - bring import_playbook into new normal
 - avoid adding collection playbooks to adjacency
 - deal with extensions or lack of em
 - added tests
 - fix bugs with multiple playbooks supplied
 - unicode/bytes/native fixes
 - fix modulenotfound for < py3.7
 - tests
4 years ago
Matt Martz be5fc4e642
Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
4 years ago
Brian Coca 5ec53f9db8
make collection callbacks follow normal flow (#59932)
make collections whitelist follow normal flow

* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
4 years ago
Matt Davis 29c6aae2fc
try to load unqualified plugins from whitelist (#70086)
* try to load unqualified plugins from whitelist

* necessary for backcompat loading of unqualified collectionized callback plugins redirected from <= 2.9 core
* also added de-duping from actual loaded name

* add tests

* add warning test

* group test script entries by topic

* shorten warning text grep because wrapping is dumb

* fix adhoc callback loading behavior

* collections pass over whitelist wasn't respecting `_run_additional_callbacks`
* adds regression tests for same

* avoid `grep -L` in tests since it breaks the world
4 years ago
James Cassell 47d14a33bd
config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
4 years ago
Brian Coca ebb22655e4
implemented 'prefix' for file based cache (#69872)
* implemented 'prefix' for file based cache

Co-authored-by: s-hertel <shertel@redhat.com>
4 years ago
Matt Davis f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
5 years ago
Brian Coca 87d9b49de2
Fix listing of colleciton plugins with symlinks (#69305)
* Fix listing of colleciton plugins with symlinks
5 years ago
Brian Coca 0aa76503dc
avoid fatal tb on bad fqcn callback name (#69440) 5 years ago
Sloane Hertel 5945415398
fix using inventory and cache plugins in a collection (#56469)
* Allow custom inventory plugins and cache plugins

If _load_name is not set correctly the cache plugin can't load the documentation (which is also the arg spec)

Fix the existing inventory plugin in the collections tests

Add integration tests for using a cache plugin in a collection

* Set the attribute on the instance instead of the class

Deprecate importing custom CacheModules directly - they should use the cache_loader
5 years ago
Brian Coca ecd66a6a6e
Action module xross coll (#68080)
* added test to verify xrs collection action/module

* added to runme

* sanity fixes

* corrected name

* avoid windoez
5 years ago
Sloane Hertel c1f1b2029c
Support vars plugins in collections (#61078)
* Move var plugins handling to a separate file

* Allow var plugins to require whitelisting

* Add global configuration ('demand', 'start') for users to control when they execute

* Add 'stage' configuration ('all', 'task', 'inventory') for users to control on a per-plugin basis when they execute

* Update ansible-inventory and InventoryManager to the global and stage configuration

* Update host_group_vars to use stage configuration and whitelisting

* Add documentation for using new options and to the developer's guide

* Add integration tests to exercise whitelisting and the new configuration options, using vars plugins in collections, and maintain backward compatibility

* Changelog

Co-Authored-By: Brian Coca <brian.coca+git@gmail.com>
Co-Authored-By: Sandra McCann <samccann@redhat.com>
5 years ago
Matt Davis 6d52bdf4db
fix default collection resolution in adhoc (#63298)
* fix default collection resolution in adhoc

* if an adhoc command is run with a playbook-dir under a configured collection, default collection resolution is used to resolve unqualified module/action names

* Set ANSIBLE_PLAYBOOK_DIR in integration tests.

* Fix config conflict in ansible integration test.

* add adhoc default collection test

* text-ify warning string
5 years ago
Matt Davis 7d1a981b61
default collection support (#61415)
* default collection support

* playbooks run from inside a registered collection will set that collection as the first item in the search order (as will all non-collection roles)
* this allows easy migration of runme.sh style playbook/role integration tests to collections without the playbooks/roles needing to know the name of their enclosing collection

* ignore bogus sanity error

* filed #61460

* fixed task unit test failure

* don't append an empty collections list to the ds

* ignore leftover local_action in mod_args ds action parsing

* fix async_extra_data test to not require ssh and bogus locale

* disable default collection test under Windows

* ensure collection location FS code is always bytes

* add changelog
5 years ago
Jordan Borean 66f52b74b1 Change collection PS util import pattern (#61307)
* Change collection PS util import pattern

* Add changes for py2 compat

* fix up regex and doc errors

* fix up import analysis

* Sanity fix for 2.6 CI workers

* Get collection util path for coverage collection
5 years ago
Matt Davis bfa004930a
add subdir support to collection loading (#60682)
* add subdir support to collection loading

* collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py
* centralize parsing/validation in AnsibleCollectionRef class
* fix issues loading Jinja2 plugins from multiple sources 
* resolves #59462, #59890,

* sanity test fixes

* string fixes

* add changelog entry
5 years ago
Matt Clay 60f8483603
Add missing boilerplate to collections test. (#60507)
* Add missing boilerplate to collections test.
* Remove redundant Python version check.
5 years ago
Brian Coca 69b2d7e0f3 fix collection adjacent loading with --playbook-dir (#56194)
* added inventory adjacent test
6 years ago
flowerysong 2ef8b297ff Fix loading namespaced doc_fragments from collections (#55249)
* Fix loading namespaced doc_fragments

The syntax for specifying a different fragment name was already
using '.' as a separator, so the code needed to be tweaked to
avoid choking on names like `testns.testcoll.fragname` and
`testns.testcoll.fragname.altvar`.

`get_plugin_class()` returns 'docfragment' for the fragment loader;
mangling `subdir` provides consistent alignment with the normal plugin
directory names and avoids needing special handling of plugin types
with 'module' in the name.

* Add changelog entry
6 years ago
Matt Davis 1dc8436ed9
module_utils fixes in collections (#55118)
* module_utils fixes in collections

* fixed Windows module_utils in collections
* fixed more Python module_utils cases (from X import module)
* "medium style" Ansiballz modules now work properly with collections (ie, non-replacer but also not using basic.py)
* added more tests
* split Windows/POSIX exec

* sanity
6 years ago
Matt Davis f86345f777
Collection content loading (#52194)
* basic plugin loading working (with many hacks)

* task collections working

* play/block-level collection module/action working

* implement PEP302 loader

* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)

* synthetic package support

* ansible.core.plugins mapping works, others don't

* synthetic collections working for modules/actions

* fix direct-load legacy

* change base package name to ansible_collections

* note

* collection role loading

* expand paths from installed content root vars

* feature complete?

* rename ansible.core to ansible.builtin

* and various sanity fixes

* sanity tweaks

* unittest fixes

* less grabby error handler on has_plugin

* probably need to replace with a or harden callers

* fix win_ping test

* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...

* fix unicode conversion issues on py2

* attempt to keep things working-ish on py2.6

* python2.6 test fun round 2

* rename dirs/configs to "collections"

* add wrapper dir for content-adjacent

* fix pythoncheck to use localhost

* unicode tweaks, native/bytes string prefixing

* rename COLLECTION_PATHS to COLLECTIONS_PATHS

* switch to pathspec

* path handling cleanup

* change expensive `all` back to or chain

* unused import cleanup

* quotes tweak

* use wrapped iter/len in Jinja proxy

* var name expansion

* comment seemingly overcomplicated playbook_paths resolution

* drop unnecessary conditional nesting

* eliminate extraneous local

* zap superfluous validation function

* use slice for rolespec NS assembly

* misc naming/unicode fixes

* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible

* misc fixmes

* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker

* more misc fixmes

* tighten up action dispatch, add unqualified action test

* rename Collection mixin to CollectionSearch

* (attempt to) avoid potential confusion/conflict with builtin collections, etc

* stale fixmes

* tighten up pluginloader collections determination

* sanity test fixes

* ditch regex escape

* clarify comment

* update default collections paths config entry

* use PATH format instead of list

* skip integration tests on Python 2.6

ci_complete
6 years ago