add action_groups support to collections (#74039)

* Canonicalize module_defaults actions and action_groups pre-fork and cache them on the play

* Call get_action_args_with_defaults with the resolved FQCN plugin and don't pass the redirect list

* Add validation for action_group metadata and a toggle to disable the warnings

* Handle groups recursively referring to each other

* Remove special-casing for non-fqcn actions in module_defaults groups

* Error for actions and groups in module_defaults that can't be resolved

* Error for fully templated module_defaults

* Add integration tests for action_groups

* Changelog
pull/75259/head
Sloane Hertel 3 years ago committed by GitHub
parent 9af0d91676
commit 3b861abce1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -0,0 +1,9 @@
bugfixes:
- Fully qualified 'ansible.legacy' and 'ansible.builtin' plugin names work in conjunction with module_defaults.
breaking_changes:
- Action, module, and group names in module_defaults must be static values. Their values can still be templates.
- Unresolvable groups, action plugins, and modules in module_defaults are an error.
- Fully qualified 'ansible.legacy' plugin names are not included implicitly in action_groups.
minor_changes:
- Collections can define action_groups in ``meta/runtime.yml``.
- action_groups can include actions from other groups by using the special ``metadata`` dictionary field.

@ -246,6 +246,24 @@ A collection can store some additional metadata in a ``runtime.yml`` file in the
ansible.module_utils.old_utility:
redirect: ansible_collections.namespace_name.collection_name.plugins.module_utils.new_location
- *action_groups*
A mapping of groups and the list of action plugin and module names they contain. They may also have a special 'metadata' dictionary in the list, which can be used to include actions from other groups.
.. code:: yaml
action_groups:
groupname:
# The special metadata dictionary. All action/module names should be strings.
- metadata:
extend_group:
- another.collection.groupname
- another_group
- my_action
another_group:
- my_module
- another.collection.another_module
.. seealso::
:ref:`distributing_collections`

@ -141,3 +141,25 @@ In a playbook, you can set module defaults for whole groups of modules, such as
ec2_ami_info:
filters:
name: 'RHEL*7.5*'
In ansible-core 2.12, collections can define their own groups in the ``meta/runtime.yml`` file. ``module_defaults`` does not take the ``collections`` keyword into account, so the fully qualified group name must be used for new groups in ``module_defaults``.
Here is an example ``runtime.yml`` file for a collection and a sample playbook using the group.
.. code-block:: YAML
# collections/ansible_collections/ns/coll/meta/runtime.yml
action_groups:
groupname:
- module
- another.collection.module
.. code-block:: YAML
- hosts: localhost
module_defaults:
group/ns.coll.groupname:
option_name: option_value
tasks:
- ns.coll.module:
- another.collection.module

@ -9676,3 +9676,67 @@ import_redirection:
redirect: ansible.module_utils
ansible_collections.ansible.builtin.plugins:
redirect: ansible.plugins
action_groups:
testgroup:
# The list items under a group should always be action/module name strings except
# for a special 'metadata' dictionary.
# The only valid key currently for the metadata dictionary is 'extend_group', which is a
# list of other groups, the actions of which will be included in this group.
# (Note: it's still possible to also have a module/action named 'metadata' in the list)
- metadata:
extend_group:
- testns.testcoll.testgroup
- testns.testcoll.anothergroup
- testns.boguscoll.testgroup
- ping
- legacy_ping # Includes ansible.builtin.legacy_ping, not ansible.legacy.legacy_ping
- formerly_core_ping
testlegacy:
- ansible.legacy.legacy_ping
aws:
- metadata:
extend_group:
- amazon.aws.aws
- community.aws.aws
acme:
- metadata:
extend_group:
- community.crypto.acme
azure:
- metadata:
extend_group:
- azure.azcollection.azure
cpm:
- metadata:
extend_group:
- wti.remote.cpm
docker:
- metadata:
extend_group:
- community.general.docker
- community.docker.docker
gcp:
- metadata:
extend_group:
- google.cloud.gcp
k8s:
- metadata:
extend_group:
- community.kubernetes.k8s
- community.general.k8s
- community.kubevirt.k8s
- community.okd.k8s
- kubernetes.core.k8s
os:
- metadata:
extend_group:
- openstack.cloud.os
ovirt:
- metadata:
extend_group:
- ovirt.ovirt.ovirt
- community.general.ovirt
vmware:
- metadata:
extend_group:
- community.vmware.vmware

@ -1971,6 +1971,17 @@ STRING_CONVERSION_ACTION:
- section: defaults
key: string_conversion_action
type: string
VALIDATE_ACTION_GROUP_METADATA:
version_added: '2.12'
description:
- A toggle to disable validating a collection's 'metadata' entry for a module_defaults action group.
Metadata containing unexpected fields or value types will produce a warning when this is True.
default: True
env: [{name: ANSIBLE_VALIDATE_ACTION_GROUP_METADATA}]
ini:
- section: defaults
key: validate_action_group_metadata
type: bool
VERBOSE_TO_STDERR:
version_added: '2.8'
description:

@ -1378,23 +1378,31 @@ def modify_module(module_name, module_path, module_args, templar, task_vars=None
return (b_module_data, module_style, shebang)
def get_action_args_with_defaults(action, args, defaults, templar, redirected_names=None):
group_collection_map = {
'acme': ['community.crypto'],
'aws': ['amazon.aws', 'community.aws'],
'azure': ['azure.azcollection'],
'cpm': ['wti.remote'],
'docker': ['community.general', 'community.docker'],
'gcp': ['google.cloud'],
'k8s': ['community.kubernetes', 'community.general', 'community.kubevirt', 'community.okd', 'kubernetes.core'],
'os': ['openstack.cloud'],
'ovirt': ['ovirt.ovirt', 'community.general'],
'vmware': ['community.vmware'],
'testgroup': ['testns.testcoll', 'testns.othercoll', 'testns.boguscoll']
}
def get_action_args_with_defaults(action, args, defaults, templar, redirected_names=None, action_groups=None):
if redirected_names:
resolved_action_name = redirected_names[-1]
else:
resolved_action_name = action
if not redirected_names:
redirected_names = [action]
if redirected_names is not None:
msg = (
"Finding module_defaults for the action %s. "
"The caller passed a list of redirected action names, which is deprecated. "
"The task's resolved action should be provided as the first argument instead."
)
display.deprecated(msg % resolved_action_name, version='2.16')
# Get the list of groups that contain this action
if action_groups is None:
msg = (
"Finding module_defaults for action %s. "
"The caller has not passed the action_groups, so any "
"that may include this action will be ignored."
)
display.warning(msg=msg)
group_names = []
else:
group_names = action_groups.get(resolved_action_name, [])
tmp_args = {}
module_defaults = {}
@ -1404,36 +1412,16 @@ def get_action_args_with_defaults(action, args, defaults, templar, redirected_na
for default in defaults:
module_defaults.update(default)
# if I actually have defaults, template and merge
if module_defaults:
# module_defaults keys are static, but the values may be templated
module_defaults = templar.template(module_defaults)
# deal with configured group defaults first
for default in module_defaults:
if not default.startswith('group/'):
continue
if default.startswith('group/'):
group_name = default.split('group/')[-1]
for collection_name in group_collection_map.get(group_name, []):
try:
action_group = _get_collection_metadata(collection_name).get('action_groups', {})
except ValueError:
# The collection may not be installed
continue
if any(name for name in redirected_names if name in action_group):
if group_name in group_names:
tmp_args.update((module_defaults.get('group/%s' % group_name) or {}).copy())
# handle specific action defaults
for redirected_action in redirected_names:
legacy = None
if redirected_action.startswith('ansible.legacy.') and action == redirected_action:
legacy = redirected_action.split('ansible.legacy.')[-1]
if legacy and legacy in module_defaults:
tmp_args.update(module_defaults[legacy].copy())
if redirected_action in module_defaults:
tmp_args.update(module_defaults[redirected_action].copy())
tmp_args.update(module_defaults.get(resolved_action_name, {}).copy())
# direct args override all
tmp_args.update(args)

@ -549,7 +549,8 @@ class TaskExecutor:
# Apply default params for action/module, if present
self._task.args = get_action_args_with_defaults(
self._task.action, self._task.args, self._task.module_defaults, templar, self._task._ansible_internal_redirect_list
self._task.resolved_action, self._task.args, self._task.module_defaults, templar,
action_groups=self._task._parent._play._action_groups
)
# And filter out any fields which were set to default(omit), and got the omit token value

@ -122,7 +122,6 @@ class ModuleArgsParser:
self._task_attrs = frozenset(self._task_attrs)
self.resolved_action = None
self.internal_redirect_list = []
def _split_module_string(self, module_string):
'''
@ -271,8 +270,6 @@ class ModuleArgsParser:
delegate_to = self._task_ds.get('delegate_to', Sentinel)
args = dict()
self.internal_redirect_list = []
# This is the standard YAML form for command-type modules. We grab
# the args and pass them in as additional arguments, which can/will
# be overwritten via dict updates from the other arg sources below
@ -308,15 +305,9 @@ class ModuleArgsParser:
elif skip_action_validation:
is_action_candidate = True
else:
# If the plugin is resolved and redirected smuggle the list of candidate names via the task attribute 'internal_redirect_list'
# TODO: remove self.internal_redirect_list (and Task._ansible_internal_redirect_list) once TE can use the resolved name for module_defaults
context = action_loader.find_plugin_with_context(item, collection_list=self._collection_list)
if not context.resolved:
context = module_loader.find_plugin_with_context(item, collection_list=self._collection_list)
if context.resolved and context.redirect_list:
self.internal_redirect_list = context.redirect_list
elif context.redirect_list:
self.internal_redirect_list = context.redirect_list
is_action_candidate = context.resolved and bool(context.redirect_list)

@ -20,8 +20,10 @@ from ansible.module_utils.six import iteritems, string_types, with_metaclass
from ansible.module_utils.parsing.convert_bool import boolean
from ansible.errors import AnsibleParserError, AnsibleUndefinedVariable, AnsibleAssertionError
from ansible.module_utils._text import to_text, to_native
from ansible.playbook.attribute import Attribute, FieldAttribute
from ansible.parsing.dataloader import DataLoader
from ansible.playbook.attribute import Attribute, FieldAttribute
from ansible.plugins.loader import module_loader, action_loader
from ansible.utils.collection_loader._collection_finder import _get_collection_metadata, AnsibleCollectionRef
from ansible.utils.display import Display
from ansible.utils.sentinel import Sentinel
from ansible.utils.vars import combine_vars, isidentifier, get_unique_id
@ -77,6 +79,45 @@ def _generic_d(prop_name, self):
del self._attributes[prop_name]
def _validate_action_group_metadata(action, found_group_metadata, fq_group_name):
valid_metadata = {
'extend_group': {
'types': (list, string_types,),
'errortype': 'list',
},
}
metadata_warnings = []
validate = C.VALIDATE_ACTION_GROUP_METADATA
metadata_only = isinstance(action, dict) and 'metadata' in action and len(action) == 1
if validate and not metadata_only:
found_keys = ', '.join(sorted(list(action)))
metadata_warnings.append("The only expected key is metadata, but got keys: {keys}".format(keys=found_keys))
elif validate:
if found_group_metadata:
metadata_warnings.append("The group contains multiple metadata entries.")
if not isinstance(action['metadata'], dict):
metadata_warnings.append("The metadata is not a dictionary. Got {metadata}".format(metadata=action['metadata']))
else:
unexpected_keys = set(action['metadata'].keys()) - set(valid_metadata.keys())
if unexpected_keys:
metadata_warnings.append("The metadata contains unexpected keys: {0}".format(', '.join(unexpected_keys)))
unexpected_types = []
for field, requirement in valid_metadata.items():
if field not in action['metadata']:
continue
value = action['metadata'][field]
if not isinstance(value, requirement['types']):
unexpected_types.append("%s is %s (expected type %s)" % (field, value, requirement['errortype']))
if unexpected_types:
metadata_warnings.append("The metadata contains unexpected key types: {0}".format(', '.join(unexpected_types)))
if metadata_warnings:
metadata_warnings.insert(0, "Invalid metadata was found for action_group {0} while loading module_defaults.".format(fq_group_name))
display.warning(" ".join(metadata_warnings))
class BaseMeta(type):
"""
@ -304,6 +345,171 @@ class FieldAttributeBase(with_metaclass(BaseMeta, object)):
self._validated = True
def _load_module_defaults(self, name, value):
if value is None:
return
if not isinstance(value, list):
value = [value]
validated_module_defaults = []
for defaults_dict in value:
if not isinstance(defaults_dict, dict):
raise AnsibleParserError(
"The field 'module_defaults' is supposed to be a dictionary or list of dictionaries, "
"the keys of which must be static action, module, or group names. Only the values may contain "
"templates. For example: {'ping': \"{{ ping_defaults }}\"}"
)
validated_defaults_dict = {}
for defaults_entry, defaults in defaults_dict.items():
# module_defaults do not use the 'collections' keyword, so actions and
# action_groups that are not fully qualified are part of the 'ansible.legacy'
# collection. Update those entries here, so module_defaults contains
# fully qualified entries.
if defaults_entry.startswith('group/'):
group_name = defaults_entry.split('group/')[-1]
# The resolved action_groups cache is associated saved on the current Play
if self.play is not None:
group_name, dummy = self._resolve_group(group_name)
defaults_entry = 'group/' + group_name
validated_defaults_dict[defaults_entry] = defaults
else:
action_names = []
if len(defaults_entry.split('.')) < 3:
defaults_entry = 'ansible.legacy.' + defaults_entry
action_names.append(defaults_entry)
if defaults_entry.startswith('ansible.legacy.'):
action_names.append(defaults_entry.replace('ansible.legacy.', 'ansible.builtin.'))
# Replace the module_defaults action entry with the canonical name,
# so regardless of how the action is called, the defaults will apply
for action_name in action_names:
resolved_action = self._resolve_action(action_name)
if resolved_action:
validated_defaults_dict[resolved_action] = defaults
validated_module_defaults.append(validated_defaults_dict)
return validated_module_defaults
@property
def play(self):
if hasattr(self, '_play'):
play = self._play
elif hasattr(self, '_parent') and hasattr(self._parent, '_play'):
play = self._parent._play
else:
play = self
if play.__class__.__name__ != 'Play':
# Should never happen, but handle gracefully by returning None, just in case
return None
return play
def _resolve_group(self, fq_group_name, mandatory=True):
if not AnsibleCollectionRef.is_valid_fqcr(fq_group_name):
collection_name = 'ansible.builtin'
fq_group_name = collection_name + '.' + fq_group_name
else:
collection_name = '.'.join(fq_group_name.split('.')[0:2])
# Check if the group has already been resolved and cached
if fq_group_name in self.play._group_actions:
return fq_group_name, self.play._group_actions[fq_group_name]
try:
action_groups = _get_collection_metadata(collection_name).get('action_groups', {})
except ValueError:
if not mandatory:
display.vvvvv("Error loading module_defaults: could not resolve the module_defaults group %s" % fq_group_name)
return fq_group_name, []
raise AnsibleParserError("Error loading module_defaults: could not resolve the module_defaults group %s" % fq_group_name)
# The collection may or may not use the fully qualified name
# Don't fail if the group doesn't exist in the collection
resource_name = fq_group_name.split(collection_name + '.')[-1]
action_group = action_groups.get(
fq_group_name,
action_groups.get(resource_name)
)
if action_group is None:
if not mandatory:
display.vvvvv("Error loading module_defaults: could not resolve the module_defaults group %s" % fq_group_name)
return fq_group_name, []
raise AnsibleParserError("Error loading module_defaults: could not resolve the module_defaults group %s" % fq_group_name)
resolved_actions = []
include_groups = []
found_group_metadata = False
for action in action_group:
# Everything should be a string except the metadata entry
if not isinstance(action, string_types):
_validate_action_group_metadata(action, found_group_metadata, fq_group_name)
if isinstance(action['metadata'], dict):
found_group_metadata = True
include_groups = action['metadata'].get('extend_group', [])
if isinstance(include_groups, string_types):
include_groups = [include_groups]
if not isinstance(include_groups, list):
# Bad entries may be a warning above, but prevent tracebacks by setting it back to the acceptable type.
include_groups = []
continue
# The collection may or may not use the fully qualified name.
# If not, it's part of the current collection.
if not AnsibleCollectionRef.is_valid_fqcr(action):
action = collection_name + '.' + action
resolved_action = self._resolve_action(action, mandatory=False)
if resolved_action:
resolved_actions.append(resolved_action)
for action in resolved_actions:
if action not in self.play._action_groups:
self.play._action_groups[action] = []
self.play._action_groups[action].append(fq_group_name)
self.play._group_actions[fq_group_name] = resolved_actions
# Resolve extended groups last, after caching the group in case they recursively refer to each other
for include_group in include_groups:
if not AnsibleCollectionRef.is_valid_fqcr(include_group):
include_group_collection = collection_name
include_group = collection_name + '.' + include_group
else:
include_group_collection = '.'.join(include_group.split('.')[0:2])
dummy, group_actions = self._resolve_group(include_group, mandatory=False)
for action in group_actions:
if action not in self.play._action_groups:
self.play._action_groups[action] = []
self.play._action_groups[action].append(fq_group_name)
self.play._group_actions[fq_group_name].extend(group_actions)
resolved_actions.extend(group_actions)
return fq_group_name, resolved_actions
def _resolve_action(self, action_name, mandatory=True):
context = action_loader.find_plugin_with_context(action_name)
if not context.resolved:
context = module_loader.find_plugin_with_context(action_name)
if context.resolved:
return context.resolved_fqcn
if mandatory:
raise AnsibleParserError("Could not resolve action %s in module_defaults" % action_name)
display.vvvvv("Could not resolve action %s in module_defaults" % action_name)
def squash(self):
'''
Evaluates all attributes and sets them to the evaluated version,

@ -95,6 +95,9 @@ class Play(Base, Taggable, CollectionSearch):
self.only_tags = set(context.CLIARGS.get('tags', [])) or frozenset(('all',))
self.skip_tags = set(context.CLIARGS.get('skip_tags', []))
self._action_groups = {}
self._group_actions = {}
def __repr__(self):
return self.get_name()
@ -339,6 +342,8 @@ class Play(Base, Taggable, CollectionSearch):
roles.append(role.serialize())
data['roles'] = roles
data['included_path'] = self._included_path
data['action_groups'] = self._action_groups
data['group_actions'] = self._group_actions
return data
@ -346,6 +351,8 @@ class Play(Base, Taggable, CollectionSearch):
super(Play, self).deserialize(data)
self._included_path = data.get('included_path', None)
self._action_groups = data.get('action_groups', {})
self._group_actions = data.get('group_actions', {})
if 'roles' in data:
role_data = data.get('roles', [])
roles = []
@ -362,4 +369,6 @@ class Play(Base, Taggable, CollectionSearch):
new_me.ROLE_CACHE = self.ROLE_CACHE.copy()
new_me._included_conditional = self._included_conditional
new_me._included_path = self._included_path
new_me._action_groups = self._action_groups
new_me._group_actions = self._group_actions
return new_me

@ -91,10 +91,6 @@ class Task(Base, Conditional, Taggable, CollectionSearch):
def __init__(self, block=None, role=None, task_include=None):
''' constructors a task, without the Task.load classmethod, it will be pretty blank '''
# This is a reference of all the candidate action names for transparent execution of module_defaults with redirected content
# This isn't a FieldAttribute to prevent it from being set via the playbook
self._ansible_internal_redirect_list = []
self._role = role
self._parent = None
self.implicit = False
@ -227,7 +223,6 @@ class Task(Base, Conditional, Taggable, CollectionSearch):
# But if it wasn't, we can add the yaml object now to get more detail
raise AnsibleParserError(to_native(e), obj=ds, orig_exc=e)
else:
self._ansible_internal_redirect_list = args_parser.internal_redirect_list[:]
self.resolved_action = args_parser.resolved_action
# the command/shell/script modules used to support the `cmd` arg,
@ -393,9 +388,6 @@ class Task(Base, Conditional, Taggable, CollectionSearch):
def copy(self, exclude_parent=False, exclude_tasks=False):
new_me = super(Task, self).copy()
# if the task has an associated list of candidate names, copy it to the new object too
new_me._ansible_internal_redirect_list = self._ansible_internal_redirect_list[:]
new_me._parent = None
if self._parent and not exclude_parent:
new_me._parent = self._parent.copy(exclude_tasks=exclude_tasks)
@ -420,9 +412,6 @@ class Task(Base, Conditional, Taggable, CollectionSearch):
if self._role:
data['role'] = self._role.serialize()
if self._ansible_internal_redirect_list:
data['_ansible_internal_redirect_list'] = self._ansible_internal_redirect_list[:]
data['implicit'] = self.implicit
data['resolved_action'] = self.resolved_action
@ -454,8 +443,6 @@ class Task(Base, Conditional, Taggable, CollectionSearch):
self._role = r
del data['role']
self._ansible_internal_redirect_list = data.get('_ansible_internal_redirect_list', [])
self.implicit = data.get('implicit', False)
self.resolved_action = data.get('resolved_action')

@ -41,12 +41,13 @@ class ActionModule(ActionBase):
mod_args = dict((k, v) for k, v in mod_args.items() if v is not None)
# handle module defaults
redirect_list = self._shared_loader_obj.module_loader.find_plugin_with_context(
resolved_fact_module = self._shared_loader_obj.module_loader.find_plugin_with_context(
fact_module, collection_list=self._task.collections
).redirect_list
).resolved_fqcn
mod_args = get_action_args_with_defaults(
fact_module, mod_args, self._task.module_defaults, self._templar, redirect_list
resolved_fact_module, mod_args, self._task.module_defaults, self._templar,
action_groups=self._task._parent._play._action_groups
)
return mod_args

@ -73,7 +73,8 @@ class ActionModule(ActionBase):
# get defaults for specific module
context = self._shared_loader_obj.module_loader.find_plugin_with_context(module, collection_list=self._task.collections)
new_module_args = get_action_args_with_defaults(
module, new_module_args, self._task.module_defaults, self._templar, context.redirect_list
context.resolved_fqcn, new_module_args, self._task.module_defaults, self._templar,
action_groups=self._task._parent._play._action_groups
)
if module in self.BUILTIN_PKG_MGR_MODULES:

@ -81,7 +81,8 @@ class ActionModule(ActionBase):
# get defaults for specific module
context = self._shared_loader_obj.module_loader.find_plugin_with_context(module, collection_list=self._task.collections)
new_module_args = get_action_args_with_defaults(
module, new_module_args, self._task.module_defaults, self._templar, context.redirect_list
context.resolved_fqcn, new_module_args, self._task.module_defaults, self._templar,
action_groups=self._task._parent._play._action_groups
)
# collection prefix known internal modules to avoid collisions from collections search, while still allowing library/ overrides

@ -582,15 +582,6 @@ class _AnsibleCollectionPkgLoader(_AnsibleCollectionPkgLoaderBase):
# if redirect.startswith('..'):
# redirect = redirect[2:]
action_groups = meta_dict.pop('action_groups', {})
meta_dict['action_groups'] = {}
for group_name in action_groups:
for action_name in action_groups[group_name]:
if action_name in meta_dict['action_groups']:
meta_dict['action_groups'][action_name].append(group_name)
else:
meta_dict['action_groups'][action_name] = [group_name]
return meta_dict

@ -0,0 +1,38 @@
#!/usr/bin/python
DOCUMENTATION = """
---
module: ios_facts
short_description: supporting network facts module
description:
- supporting network facts module for gather_facts + module_defaults tests
options:
gather_subset:
description:
- When supplied, this argument restricts the facts collected
to a given subset.
- Possible values for this argument include
C(all), C(hardware), C(config), and C(interfaces).
- Specify a list of values to include a larger subset.
- Use a value with an initial C(!) to collect all facts except that subset.
required: false
default: '!config'
"""
from ansible.module_utils.basic import AnsibleModule
def main():
"""main entry point for module execution
"""
argument_spec = dict(
gather_subset=dict(default='!config')
)
module = AnsibleModule(argument_spec=argument_spec,
supports_check_mode=True)
module.exit_json(ansible_facts={'gather_subset': module.params['gather_subset'], '_ansible_facts_gathered': True})
if __name__ == '__main__':
main()

@ -23,3 +23,5 @@ ansible-playbook verify_subset.yml "$@"
# ensure we can set defaults for the action plugin and facts module
ansible-playbook test_module_defaults.yml "$@" --tags default_fact_module
ANSIBLE_FACTS_MODULES='ansible.legacy.setup' ansible-playbook test_module_defaults.yml "$@" --tags custom_fact_module
ansible-playbook test_module_defaults.yml "$@" --tags networking

@ -77,3 +77,54 @@
- assert:
that:
- "gather_subset == ['min']"
- hosts: localhost
gather_facts: no
tags:
- networking
tasks:
- name: test that task args aren't used for fqcn network facts
gather_facts:
gather_subset: min
vars:
ansible_network_os: 'cisco.ios.ios'
register: result
- assert:
that:
- "ansible_facts.gather_subset == '!config'"
- name: test that module_defaults are used for fqcn network facts
gather_facts:
vars:
ansible_network_os: 'cisco.ios.ios'
module_defaults:
'cisco.ios.ios_facts': {'gather_subset': 'min'}
register: result
- assert:
that:
- "ansible_facts.gather_subset == 'min'"
- name: test that task args aren't used for legacy network facts
gather_facts:
gather_subset: min
vars:
ansible_network_os: 'ios'
register: result
- assert:
that:
- "ansible_facts.gather_subset == '!config'"
- name: test that module_defaults are used for legacy network facts
gather_facts:
vars:
ansible_network_os: 'ios'
module_defaults:
'ios_facts': {'gather_subset': 'min'}
register: result
- assert:
that:
- "ansible_facts.gather_subset == 'min'"

@ -1,5 +1,10 @@
action_groups:
testgroup:
# Test metadata 'extend_group' feature does not get stuck in a recursive loop
- metadata:
extend_group: othergroup
- metadata
- ping
- testns.testcoll.echo1
- testns.testcoll.echo2
# note we can define defaults for an action
@ -7,3 +12,28 @@ action_groups:
# note we can define defaults in this group for actions/modules in another collection
- testns.othercoll.other_echoaction
- testns.othercoll.other_echo1
othergroup:
- metadata:
extend_group:
- testgroup
empty_metadata:
- metadata: {}
bad_metadata_format:
- unexpected_key:
key: value
metadata:
extend_group: testgroup
multiple_metadata:
- metadata:
extend_group: testgroup
- metadata:
extend_group: othergroup
bad_metadata_options:
- metadata:
unexpected_key: testgroup
bad_metadata_type:
- metadata: [testgroup]
bad_metadata_option_type:
- metadata:
extend_group:
name: testgroup

@ -0,0 +1,45 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = '''
---
module: metadata
version_added: 2.12
short_description: Test module with a specific name
description: Test module with a specific name
options:
data:
description: Required option to test module_defaults work
required: True
type: str
author:
- Ansible Core Team
'''
EXAMPLES = '''
'''
RETURN = '''
'''
from ansible.module_utils.basic import AnsibleModule
def main():
module = AnsibleModule(
argument_spec=dict(
data=dict(type='str', required=True),
),
)
module.exit_json()
if __name__ == '__main__':
main()

@ -0,0 +1,83 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
# (c) 2016, Toshio Kuratomi <tkuratomi@ansible.com>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = '''
---
module: ping
version_added: historical
short_description: Try to connect to host, verify a usable python and return C(pong) on success
description:
- A trivial test module, this module always returns C(pong) on successful
contact. It does not make sense in playbooks, but it is useful from
C(/usr/bin/ansible) to verify the ability to login and that a usable Python is configured.
- This is NOT ICMP ping, this is just a trivial test module that requires Python on the remote-node.
- For Windows targets, use the M(ansible.windows.win_ping) module instead.
- For Network targets, use the M(ansible.netcommon.net_ping) module instead.
options:
data:
description:
- Data to return for the C(ping) return value.
- If this parameter is set to C(crash), the module will cause an exception.
type: str
default: pong
seealso:
- module: ansible.netcommon.net_ping
- module: ansible.windows.win_ping
author:
- Ansible Core Team
- Michael DeHaan
notes:
- Supports C(check_mode).
'''
EXAMPLES = '''
# Test we can logon to 'webservers' and execute python with json lib.
# ansible webservers -m ping
- name: Example from an Ansible Playbook
ansible.builtin.ping:
- name: Induce an exception to see what happens
ansible.builtin.ping:
data: crash
'''
RETURN = '''
ping:
description: Value provided with the data parameter.
returned: success
type: str
sample: pong
'''
from ansible.module_utils.basic import AnsibleModule
def main():
module = AnsibleModule(
argument_spec=dict(
data=dict(type='str', default='pong'),
),
supports_check_mode=True
)
if module.params['data'] == 'crash':
raise Exception("boom")
result = dict(
ping=module.params['data'],
)
module.exit_json(**result)
if __name__ == '__main__':
main()

@ -0,0 +1,83 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
# (c) 2016, Toshio Kuratomi <tkuratomi@ansible.com>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
DOCUMENTATION = '''
---
module: ping
version_added: historical
short_description: Try to connect to host, verify a usable python and return C(pong) on success
description:
- A trivial test module, this module always returns C(pong) on successful
contact. It does not make sense in playbooks, but it is useful from
C(/usr/bin/ansible) to verify the ability to login and that a usable Python is configured.
- This is NOT ICMP ping, this is just a trivial test module that requires Python on the remote-node.
- For Windows targets, use the M(ansible.windows.win_ping) module instead.
- For Network targets, use the M(ansible.netcommon.net_ping) module instead.
options:
data:
description:
- Data to return for the C(ping) return value.
- If this parameter is set to C(crash), the module will cause an exception.
type: str
default: pong
seealso:
- module: ansible.netcommon.net_ping
- module: ansible.windows.win_ping
author:
- Ansible Core Team
- Michael DeHaan
notes:
- Supports C(check_mode).
'''
EXAMPLES = '''
# Test we can logon to 'webservers' and execute python with json lib.
# ansible webservers -m ping
- name: Example from an Ansible Playbook
ansible.builtin.ping:
- name: Induce an exception to see what happens
ansible.builtin.ping:
data: crash
'''
RETURN = '''
ping:
description: Value provided with the data parameter.
returned: success
type: str
sample: pong
'''
from ansible.module_utils.basic import AnsibleModule
def main():
module = AnsibleModule(
argument_spec=dict(
data=dict(type='str', default='pong'),
),
supports_check_mode=True
)
if module.params['data'] == 'crash':
raise Exception("boom")
result = dict(
ping=module.params['data'],
)
module.exit_json(**result)
if __name__ == '__main__':
main()

@ -3,3 +3,7 @@
set -eux
ansible-playbook test_defaults.yml "$@"
ansible-playbook test_action_groups.yml "$@"
ansible-playbook test_action_group_metadata.yml "$@"

@ -39,7 +39,7 @@
module_defaults:
# Meaningless values to make sure that 'module_defaults' gets
# evaluated for this block
foo:
ping:
bar: baz
block:
- debug:

@ -0,0 +1,8 @@
---
- hosts: localhost
gather_facts: no
module_defaults:
group/{{ group_name }}:
data: value
tasks:
- ping:

@ -0,0 +1,123 @@
---
- hosts: localhost
gather_facts: no
vars:
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
tasks:
- template:
src: test_metadata_warning.yml.j2
dest: test_metadata_warning.yml
vars:
group_name: testns.testcoll.empty_metadata
- command: ansible-playbook test_metadata_warning.yml
register: result
- assert:
that: metadata_warning not in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: "Invalid metadata was found"
- template:
src: test_metadata_warning.yml.j2
dest: test_metadata_warning.yml
vars:
group_name: testns.testcoll.bad_metadata_format
- command: ansible-playbook test_metadata_warning.yml
register: result
- assert:
that: metadata_warning in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: >-
Invalid metadata was found for action_group testns.testcoll.bad_metadata_format while loading module_defaults.
The only expected key is metadata, but got keys: metadata, unexpected_key
- template:
src: test_metadata_warning.yml.j2
dest: test_metadata_warning.yml
vars:
group_name: testns.testcoll.multiple_metadata
- command: ansible-playbook test_metadata_warning.yml
register: result
- assert:
that: metadata_warning in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: >-
Invalid metadata was found for action_group testns.testcoll.multiple_metadata while loading module_defaults.
The group contains multiple metadata entries.
- template:
src: test_metadata_warning.yml.j2
dest: test_metadata_warning.yml
vars:
group_name: testns.testcoll.bad_metadata_options
- command: 'ansible-playbook test_metadata_warning.yml'
register: result
- assert:
that: metadata_warning in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: >-
Invalid metadata was found for action_group testns.testcoll.bad_metadata_options while loading module_defaults.
The metadata contains unexpected keys: unexpected_key
- template:
src: test_metadata_warning.yml.j2
dest: test_metadata_warning.yml
vars:
group_name: testns.testcoll.bad_metadata_type
- command: ansible-playbook test_metadata_warning.yml
register: result
- assert:
that: metadata_warning in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: >-
Invalid metadata was found for action_group testns.testcoll.bad_metadata_type while loading module_defaults.
The metadata is not a dictionary. Got ['testgroup']
- template:
src: test_metadata_warning.yml.j2
dest: test_metadata_warning.yml
vars:
group_name: testns.testcoll.bad_metadata_option_type
- command: ansible-playbook test_metadata_warning.yml
register: result
- assert:
that: metadata_warning in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: >-
Invalid metadata was found for action_group testns.testcoll.bad_metadata_option_type while loading module_defaults.
The metadata contains unexpected key types: extend_group is {'name': 'testgroup'} (expected type list)
- name: test disabling action_group metadata validation
command: ansible-playbook test_metadata_warning.yml
environment:
ANSIBLE_VALIDATE_ACTION_GROUP_METADATA: False
register: result
- assert:
that: metadata_warning not in warnings
vars:
warnings: "{{ result.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
metadata_warning: "Invalid metadata was found for action_group"
- file:
path: test_metadata_warning.yml
state: absent

@ -0,0 +1,132 @@
---
- hosts: localhost
gather_facts: no
tasks:
- name: test ansible.legacy short group name
module_defaults:
group/testgroup:
data: test
block:
- legacy_ping:
register: result
- assert:
that: "result.ping == 'pong'"
- ansible.legacy.legacy_ping:
register: result
- assert:
that: "result.ping == 'pong'"
- ping:
register: result
- assert:
that: "result.ping == 'test'"
- ansible.legacy.ping: # resolves to ansible.builtin.ping
register: result
- assert:
that: "result.ping == 'test'"
- ansible.builtin.ping:
register: result
- assert:
that: "result.ping == 'test'"
- formerly_core_ping:
register: result
- assert:
that: "result.ping == 'test'"
- ansible.builtin.formerly_core_ping:
register: result
- assert:
that: "result.ping == 'test'"
- name: test group that includes a legacy action
module_defaults:
# As of 2.12, legacy actions must be included in the action group definition
group/testlegacy:
data: test
block:
- legacy_ping:
register: result
- assert:
that: "result.ping == 'test'"
- ansible.legacy.legacy_ping:
register: result
- assert:
that: "result.ping == 'test'"
- name: test ansible.builtin fully qualified group name
module_defaults:
group/ansible.builtin.testgroup:
data: test
block:
# ansible.builtin does not contain ansible.legacy
- legacy_ping:
register: result
- assert:
that: "result.ping != 'test'"
# ansible.builtin does not contain ansible.legacy
- ansible.legacy.legacy_ping:
register: result
- assert:
that: "result.ping != 'test'"
- ping:
register: result
- assert:
that: "result.ping == 'test'"
# Resolves to ansible.builtin.ping
- ansible.legacy.ping:
register: result
- assert:
that: "result.ping == 'test'"
- ansible.builtin.ping:
register: result
- assert:
that: "result.ping == 'test'"
- formerly_core_ping:
register: result
- assert:
that: "result.ping == 'test'"
- ansible.builtin.formerly_core_ping:
register: result
- assert:
that: "result.ping == 'test'"
- name: test collection group name
module_defaults:
group/testns.testcoll.testgroup:
data: test
block:
# Plugin resolving to a different collection does not get the default
- ping:
register: result
- assert:
that: "result.ping != 'test'"
- formerly_core_ping:
register: result
- assert:
that: "result.ping == 'test'"
- ansible.builtin.formerly_core_ping:
register: result
- assert:
that: "result.ping == 'test'"
- testns.testcoll.ping:
register: result
- assert:
that: "result.ping == 'test'"
- metadata:
collections:
- testns.testcoll

@ -582,15 +582,6 @@ class _AnsibleCollectionPkgLoader(_AnsibleCollectionPkgLoaderBase):
# if redirect.startswith('..'):
# redirect = redirect[2:]
action_groups = meta_dict.pop('action_groups', {})
meta_dict['action_groups'] = {}
for group_name in action_groups:
for action_name in action_groups[group_name]:
if action_name in meta_dict['action_groups']:
meta_dict['action_groups'][action_name].append(group_name)
else:
meta_dict['action_groups'][action_name] = [group_name]
return meta_dict

@ -97,36 +97,3 @@ class TestNetworkFacts(unittest.TestCase):
get_module_args.call_args.args,
('cisco.ios.ios_facts', {'ansible_network_os': 'cisco.ios.ios'},)
)
def test_network_gather_facts(self):
self.task_vars = {'ansible_network_os': 'ios'}
self.task.action = 'gather_facts'
self.task.async_val = False
self.task.args = {'gather_subset': 'min'}
self.task.module_defaults = [{'ios_facts': {'gather_subset': 'min'}}]
plugin = GatherFactsAction(self.task, self.connection, self.play_context, loader=None, templar=self.templar, shared_loader_obj=plugin_loader)
plugin._execute_module = MagicMock()
res = plugin.run(task_vars=self.task_vars)
self.assertEqual(res['ansible_facts']['_ansible_facts_gathered'], True)
mod_args = plugin._get_module_args('ios_facts', task_vars=self.task_vars)
self.assertEqual(mod_args['gather_subset'], 'min')
@patch.object(module_common, '_get_collection_metadata', return_value={})
def test_network_gather_facts_fqcn(self, mock_collection_metadata):
self.fqcn_task_vars = {'ansible_network_os': 'cisco.ios.ios'}
self.task.action = 'gather_facts'
self.task.async_val = False
self.task.args = {'gather_subset': 'min'}
self.task.module_defaults = [{'cisco.ios.ios_facts': {'gather_subset': 'min'}}]
plugin = GatherFactsAction(self.task, self.connection, self.play_context, loader=None, templar=self.templar, shared_loader_obj=plugin_loader)
plugin._execute_module = MagicMock()
res = plugin.run(task_vars=self.fqcn_task_vars)
self.assertEqual(res['ansible_facts']['_ansible_facts_gathered'], True)
mod_args = plugin._get_module_args('cisco.ios.ios_facts', task_vars=self.fqcn_task_vars)
self.assertEqual(mod_args['gather_subset'], 'min')

Loading…
Cancel
Save