You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ansible/lib/ansible/modules/setup.py

230 lines
11 KiB
Python

# -*- coding: utf-8 -*-
# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import annotations
DOCUMENTATION = '''
---
module: setup
version_added: historical
short_description: Gathers facts about remote hosts
options:
gather_subset:
version_added: "2.1"
description:
- "If supplied, restrict the additional facts collected to the given subset.
Possible values: V(all), V(all_ipv4_addresses), V(all_ipv6_addresses), V(apparmor), V(architecture),
V(caps), V(chroot),V(cmdline), V(date_time), V(default_ipv4), V(default_ipv6), V(devices),
V(distribution), V(distribution_major_version), V(distribution_release), V(distribution_version),
V(dns), V(effective_group_ids), V(effective_user_id), V(env), V(facter), V(fips), V(hardware),
V(interfaces), V(is_chroot), V(iscsi), V(kernel), V(local), V(lsb), V(machine), V(machine_id),
V(mounts), V(network), V(ohai), V(os_family), V(pkg_mgr), V(platform), V(processor), V(processor_cores),
V(processor_count), V(python), V(python_version), V(real_user_id), V(selinux), V(service_mgr),
V(ssh_host_key_dsa_public), V(ssh_host_key_ecdsa_public), V(ssh_host_key_ed25519_public),
V(ssh_host_key_rsa_public), V(ssh_host_pub_keys), V(ssh_pub_keys), V(system), V(system_capabilities),
V(system_capabilities_enforced), V(systemd), V(user), V(user_dir), V(user_gecos), V(user_gid), V(user_id),
V(user_shell), V(user_uid), V(virtual), V(virtualization_role), V(virtualization_type).
Can specify a list of values to specify a larger subset.
Values can also be used with an initial C(!) to specify that
that specific subset should not be collected. For instance:
V(!hardware,!network,!virtual,!ohai,!facter). If V(!all) is specified
then only the min subset is collected. To avoid collecting even the
min subset, specify V(!all,!min). To collect only specific facts,
use V(!all,!min), and specify the particular fact subsets.
Use the filter parameter if you do not want to display some collected
facts."
type: list
elements: str
default: "all"
gather_timeout:
version_added: "2.2"
description:
- Set the default timeout in seconds for individual fact gathering.
type: int
default: 10
filter:
version_added: "1.1"
description:
- If supplied, only return facts that match one of the shell-style
(fnmatch) pattern. An empty list basically means 'no filter'.
As of Ansible 2.11, the type has changed from string to list
and the default has became an empty list. A simple string is
still accepted and works as a single pattern. The behaviour
prior to Ansible 2.11 remains.
type: list
elements: str
default: []
fact_path:
version_added: "1.3"
description:
- Path used for local ansible facts (C(*.fact)) - files in this dir
will be run (if executable) and their results be added to C(ansible_local) facts.
If a file is not executable it is read instead.
File/results format can be JSON or INI-format. The default O(fact_path) can be
specified in C(ansible.cfg) for when setup is automatically called as part of
C(gather_facts).
NOTE - For windows clients, the results will be added to a variable named after the
local file (without extension suffix), rather than C(ansible_local).
- Since Ansible 2.1, Windows hosts can use O(fact_path). Make sure that this path
exists on the target host. Files in this path MUST be PowerShell scripts C(.ps1)
which outputs an object. This object will be formatted by Ansible as json so the
script should be outputting a raw hashtable, array, or other primitive object.
type: path
default: /etc/ansible/facts.d
description:
- This module is automatically called by playbooks to gather useful
variables about remote hosts that can be used in playbooks. It can also be
executed directly by C(/usr/bin/ansible) to check what variables are
available to a host. Ansible provides many I(facts) about the system,
automatically.
- This module is also supported for Windows targets.
extends_documentation_fragment:
- action_common_attributes
- action_common_attributes.facts
attributes:
check_mode:
support: full
diff_mode:
support: none
facts:
support: full
platform:
platforms: posix, windows
notes:
- More ansible facts will be added with successive releases. If I(facter) or
I(ohai) are installed, variables from these programs will also be snapshotted
into the JSON file for usage in templating. These variables are prefixed
with C(facter_) and C(ohai_) so it's easy to tell their source. All variables are
bubbled up to the caller. Using the ansible facts and choosing to not
install I(facter) and I(ohai) means you can avoid Ruby-dependencies on your
remote systems. (See also M(community.general.facter) and M(community.general.ohai).)
- The filter option filters only the first level subkey below ansible_facts.
- If the target host is Windows, you will not currently have the ability to use
O(filter) as this is provided by a simpler implementation of the module.
- This module should be run with elevated privileges on BSD systems to gather facts like ansible_product_version.
- For more information about delegated facts,
please check U(https://docs.ansible.com/ansible/latest/user_guide/playbooks_delegation.html#delegating-facts).
author:
- "Ansible Core Team"
- "Michael DeHaan"
'''
EXAMPLES = r"""
# Display facts from all hosts and store them indexed by `hostname` at `/tmp/facts`.
# ansible all -m ansible.builtin.setup --tree /tmp/facts
# Display only facts regarding memory found by ansible on all hosts and output them.
# ansible all -m ansible.builtin.setup -a 'filter=ansible_*_mb'
# Display only facts returned by facter.
# ansible all -m ansible.builtin.setup -a 'filter=facter_*'
# Collect only facts returned by facter.
# ansible all -m ansible.builtin.setup -a 'gather_subset=!all,facter'
- name: Collect only facts returned by facter
ansible.builtin.setup:
gather_subset:
- '!all'
- '!<any valid subset>'
- facter
- name: Filter and return only selected facts
ansible.builtin.setup:
filter:
- 'ansible_distribution'
- 'ansible_machine_id'
- 'ansible_*_mb'
# Display only facts about certain interfaces.
# ansible all -m ansible.builtin.setup -a 'filter=ansible_eth[0-2]'
# Restrict additional gathered facts to network and virtual (includes default minimum facts)
# ansible all -m ansible.builtin.setup -a 'gather_subset=network,virtual'
# Collect only network and virtual (excludes default minimum facts)
# ansible all -m ansible.builtin.setup -a 'gather_subset=!all,network,virtual'
# Do not call puppet facter or ohai even if present.
# ansible all -m ansible.builtin.setup -a 'gather_subset=!facter,!ohai'
# Only collect the default minimum amount of facts:
# ansible all -m ansible.builtin.setup -a 'gather_subset=!all'
# Collect no facts, even the default minimum subset of facts:
# ansible all -m ansible.builtin.setup -a 'gather_subset=!all,!min'
# Display facts from Windows hosts with custom facts stored in C:\custom_facts.
# ansible windows -m ansible.builtin.setup -a "fact_path='c:\custom_facts'"
# Gathers facts for the machines in the dbservers group (a.k.a Delegating facts)
- hosts: app_servers
tasks:
- name: Gather facts from db servers
ansible.builtin.setup:
delegate_to: "{{ item }}"
delegate_facts: true
loop: "{{ groups['dbservers'] }}"
"""
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
# import module snippets
from ..module_utils.basic import AnsibleModule
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
from ansible.module_utils.common.text.converters import to_text
from ansible.module_utils.facts import ansible_collector, default_collectors
from ansible.module_utils.facts.collector import CollectorNotFoundError, CycleFoundInFactDeps, UnresolvedFactDep
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
from ansible.module_utils.facts.namespace import PrefixFactNamespace
def main():
module = AnsibleModule(
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
argument_spec=dict(
gather_subset=dict(default=["all"], required=False, type='list', elements='str'),
gather_timeout=dict(default=10, required=False, type='int'),
filter=dict(default=[], required=False, type='list', elements='str'),
fact_path=dict(default='/etc/ansible/facts.d', required=False, type='path'),
),
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
supports_check_mode=True,
)
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
gather_subset = module.params['gather_subset']
gather_timeout = module.params['gather_timeout']
filter_spec = module.params['filter']
# TODO: this mimics existing behavior where gather_subset=["!all"] actually means
# to collect nothing except for the below list
# TODO: decide what '!all' means, I lean towards making it mean none, but likely needs
# some tweaking on how gather_subset operations are performed
minimal_gather_subset = frozenset(['apparmor', 'caps', 'cmdline', 'date_time',
'distribution', 'dns', 'env', 'fips', 'local',
'lsb', 'pkg_mgr', 'platform', 'python', 'selinux',
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
'service_mgr', 'ssh_pub_keys', 'user'])
all_collector_classes = default_collectors.collectors
# rename namespace_name to root_key?
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
namespace = PrefixFactNamespace(namespace_name='ansible',
prefix='ansible_')
try:
fact_collector = ansible_collector.get_ansible_collector(all_collector_classes=all_collector_classes,
namespace=namespace,
filter_spec=filter_spec,
gather_subset=gather_subset,
gather_timeout=gather_timeout,
minimal_gather_subset=minimal_gather_subset)
except (TypeError, CollectorNotFoundError, CycleFoundInFactDeps, UnresolvedFactDep) as e:
# bad subset given, collector, idk, deps declared but not found
module.fail_json(msg=to_text(e))
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
facts_dict = fact_collector.collect(module=module)
module.exit_json(ansible_facts=facts_dict)
Facts Refresh (2.4 roadmap) (#23012) Facts Refresh (2.4 roadmap) This commit implements most of the 2.4 roadmap 'Facts Refresh' - move facts.py to facts/__init__.py - move facts Distribution() to its own class - add a facts/utils.py - move get_file_content and get_uname_version to facts/utils.py - move Facts() class from facts/__init__ to facts/facts.py - mv get_file_lines to facts/utils.py - mv Ohai()/Facter() class to facts/ohai.py and facter.py - Start moving fact Hardware() classes to facts/hardware/*.py - mv HPUX() hardware class to facts/hardware/hpux.py - move SunOSHardware() fact class to facts/hardware/sunos.py - move OpenBSDHardware() class to facts/hardware/openbsd.py - mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/ - mv NetBSDHardware() to facts/hardware/netbsd.py - mv Darwin() hardware class to facts/hardware/darwin.py - pep8/etc cleanups on facts/hardware/*.py - Mv network facts classes to facts/network/*.py - mv Virtual fact classes to facts/virtual - mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl - Also mv get_uname_version from facts/utils.py -> distribution.py since distribution.py is the only thing using it. - add collector.py with new BaseFactCollector - add a subclass for AnsibleFactCollector - hook up dict key munging FactNamespaces - add some test cases for testing the names of facts - mv timeout stuff to facts.timeout - rm ansible_facts()/get_all_facts() etc - Instead of calling facts.ansible_facts(), fact collection api used by setup.py is now to create an AnsibleFactCollector() and call it's collect method. - replace Facts.get_user_facts with UserFactCollector - add a 'systems' facts package, mv UserFactCollector there - mv get_dns_facts to DnsFactCollector - mv get_env_facts to EnvFactCollector - include the timeout length in exception message - modules and module_utils that use AnsibleFactCollector can now theoretically set the 'valid_subsets' May be useful for network facts module that currently have to reimplement a good chunk of facts.py to get gather_subsets to work. - get_local_facts -> system/LocalFactCollector - get_date_time -> system/date_time.py - get_fips_facts -> system/fips.py - get_caps_facts() -> system/caps.py - get_apparmor_facts -> system/apparmor.py - get_selinux_facts -> system/selinux.py - get_lsb_facts -> system/lsb.py - get_service_mgr_facts -> system/service_mgr.py - Facts.is_systemd_managed -> system/service_mgr.py - get_pkg_mgr_facts -> system/pkg_mgr.py - Facts()._get_mount_size_facts() -> facts.utils.get_mount_size() - add unit test for EnvFactCollector - add a test case for minimal gather_subsets - add test case for collect_ids - Make gather_subset match existing behavior or '!all' If 'gather_subset' is provided as '!all', the existing behavior (in 2.2/2.3) is that means 'dont collect any facts except those from the Facts() class'. So 'skip everything except 'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb', 'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc. The new facts setup was making '!all' mean no facts at all, since it can add/exclude at a finer granularity. Since that makes more sense for the ansible collector, and the set of minimal facts to collect is really more up to setup.py to decide we do just that. So if setup.py needs to always collect some gather_subset, even on !all, setup.py needs to have the that subset added to the list it passes as minimal_gather_subset. This should fix some intg tests that assume '!all' means that some facts are still collected (user info and env for example). If we want to make setup.py collect a more minimal set, we can do that. - force facts_dicts.keys() to a list so py3 works - split fact collector tests to test_collectors.py - convert Facter(Facts) -> other/facter.py:FacterFactCollector - add FactCollector.collect_with_namespace() regular .collect() will return a dict with the key names using the base names ('ip_address', 'service_mgr' etc) .collect_with_namespace() will return a dict where the key names have been transformed with the collectors namespace, if there is one. For most, this means a namespace that adds 'ansible_' to the start of the key name. For 'FacterFactCollector', the namespace transforms the key to 'facter_*'. - add test cases for collect_with_namespace - move all the concrete 'which facts does setup.py' stuff to setup.py The caller of AnsibleFactCollector.from_gather_subset() needs to pass in the list of collector classes now. - update system/setup.py to import all of the fact classes and pass in that list. - split the Distribution fact class up a bit extracted the 'distro release' file handling (ie, linux boxes with /etc/release, /etc/os-release etc) into its own class. - extract get_cmdline_facts -> cmdline.py - extract get_public_ssh_host_keys -> system/ssh_pub_keys.py - extract get_platform_facts -> system/platform.py platform.py may be a good candidate for further splitting. - rm test for plain Facts() base class - let the base class for Collector unit tests provide collected_facts some Collectors and/or their migrated Facts() subsclasses need to look at facts collected by other modules ('ansible_architecture' the main one...). Collector.collect() has the collected_facts arg for this, so add a class variable to BaseFactsTest so we can specify it. - mv Ohai to other/ohai.py and convert to Collector - update hardware/*.py to return facts (no side effects) - mv AnsibleFactCollector to setup.py - extra collector class gathering to module method in facts/__init__.py (collector_classes_from_gather_subset) - add a CollectorMetaDataCollector collector used to provide the 'gather_setup' fact - add unit test module for 'setup' module (test/units/modules/system/setup.py) - Collector init now doesnt need a module, but collect does An instance of a FactCollector() isnt tied to a AnsibleModule instance, but the collect() method can be, so optionally pass in module to FactCollector.collect() (everywhere) - add a default_collectors for list of default collectors import and use it from setup.py module eventually, would like to replace this with a plugin loader style class finder/loader - unit tests for module_utils/facts/__init__.py - add unit tests for ohai facts collector - remove self.facts side effect on populate() in hardware/sunos.py - convert OpenBSDHardware() to rm side effects on self.facts - try to rm some self.facts side effects in Network() plumb in collected_facts from populate() where it is needed. stop passing collected_facts into Network() [via cached_facts=, where it eventually becomes self.facts] - nothing provides Fact() cached_facts arg now, rm it Facts() should be internal only implementation so nothing should be using it. Of course, now someone will. - add a Collector.name attr to build a map of name->_fact_ids To properly exclude a gather_subset spec like '!hardware', we need to know that 'hardware' also means 'devices', 'dmi', etc. Before, '!hardware' would remove the 'hardware' collector name but not 'devices'. Since both would end up in id_collector_map, we would still end up with the HardwareCollector in the collector list. End result being that '!hardware' wouldn't stop hardware from being collected. So we need to be able to build that map, so add the Collector.name attribute that is the primary name (like 'hardware') and let Collector._fact_ids be the other fact ids that a collector is responsible for. Construct the aliases_map of Collector.name -> set of _fact_ids in fact/__init__.py get_collector_names, and use it when we are populating the exclude set. - refactor of distribution.py make the big OS_FAMILY literal a little easier to read Also keys can now be any string instead of python literals 99% sure the test for 'KDE Neon' was wrong I don't see how/where it should or could get 'Neon' instead of 'KDE Neon' as provided in os-release NAME= Use 'distribution' string for key to OS_MAP ie, we dont need to make it a valid python label anymore so dont. move _has_dist_file to module as _file_exists easier to mock without mucking with os.path mv platform.system() calls to within get_distribution_facts() instead of Distribution() init. - remove _json compat module The code in here was to support: -a 'json' python module that was not the standard one included with python since 2.6. - potentially fallback to simplejson if 'json' was not available. 'json' is available for all supported python versions now so no longer needed. - mv get_collector_names -> facts.collector - mv collector_classes_from_gather_subset -> facts.collector - mv collector tests from test_facts -> test_collector - Use six's reduce() in sunos/netbsd hardware facts - rm extraneous get_uname_version in utils only system/distribution.py uses it - Remove Facts() subclass metaclass usage - using fact_id and a platform id for matching collectors gut most of Facts() subclasses rm Facts() subclasses with weird metaclass only add collectors that match the fact_ids and the platform_info to the list of collectors used. atm, a collectors platform_id will default to 'Generic', and any platform matches 'Generic' goal is to select collector classes including matching the systems platform in collector.py, instead of relying on metaclasses in hardware/*. To finish this, the various Facts() subclasses will need to be replaced entirely with Collector() subclasses. use collector classmethod platform_match() to match the platform This lets the particular class decide if it is compatible with a given platform_info. platform_info is a dict like obj, so it could be expanded in the future. Add a default platform_match to BaseFactCollector that matches platform_info['system'] == cls._platform They were needed previously to trigger a module load on all the collector classes when we import facts/hardare so that the Hardware() and related classes that used __new__ and find_all_subclasses() would work. Now that is done in collectors based on platform matching at runtime we dont need to do it py module import/parse time. So the non empty __init__.pys are no longer needed and their is a more flexible mechanism for selection platform specific stuff. facts/facts.py is no longer used, rm'ed - if we dont find an implement class for gather spec.. just ignore it. Would be useful to add a warn to warn about this case. - Fix SD-UX typo (should be HP-UX) - Port fix for #21893 (0 sockets) to this branch This readds the change from 8ad182059d5a085e4f8ecfe8ae63bad14a7dc01c that got lost in merge/rebase Fixes #21893 - port sunos fact locale fix for #24542 to this branch based on e558ec19cd1fd9d00e299cf1938d4ca3cec85cda Fixes #24542 Solaris fact fix (#24793) ensure locale for solaris fact gathering fixes issue with locale interfering with proper reading of decimals - raise exceptions in the air like we just dont care. Pretty much ignore any not exit exception in facts collection. And add some test cases. - added new selinux fact to clarify python lib the selinux fact is boolean false when the library is not installed, a dictionary/hash otherwise, but this is ambigous added new fact so we can eventually remove the type dichtomy and normalize it as a dict Re-add of devel commit 85c7a7b844bf429fc9cc0ffce5dd9bf05ed47b5a to the new code layout, since it got removed in merge/rebase
7 years ago
if __name__ == '__main__':
main()