Commit Graph

28 Commits (3a4b69096f68cf73ed8fc45cbd4b8be429871170)

Author SHA1 Message Date
Toshio Kuratomi cd36164239
Porting tests to pytest (#33387)
* Porting tests to pytest

* Achievement Get: No longer need mock/generator.py
  * Now done via pytest's parametrization
  * Port safe_eval to pytest
  * Port text tests to pytest
  * Port test_set_mode_if_different to pytest

* Change conftest AnsibleModule fixtures to be more flexible
  * Move the AnsibleModules fixtures to module_utils/conftest.py for sharing
  * Testing the argspec code requires:
    * injecting both the argspec and the arguments.
    * Patching the arguments into sys.stdin at a different level

* More porting to obsolete mock/procenv.py
  * Port run_command to pytest
  * Port known_hosts tests to pytest
  * Port safe_eval to pytest
  * Port test_distribution_version.py to pytest
  * Port test_log to pytest
  * Port test__log_invocation to pytest
  * Remove unneeded import of procenv in test_postgresql

* Port test_pip to pytest style
  * As part of this, create a pytest ansiblemodule fixture in
    modules/conftest.py.  This is slightly different than the
    approach taken in module_utils because here we need to override the
    AnsibleModule that the modules will inherit from instead of one that
    we're instantiating ourselves.

* Fixup usage of parametrization in test_deprecate_warn

* Check that the pip module failed in our test
7 years ago
Adrian Likins 65393e4747 Fix ansible_distribution on Ubuntu 10.04 (#31108)
The /etc/os-release based distro detection doesn't
seem to work for Ubuntu 10.04 (no /etc/os-release?).

So it was testing the next case which was /etc/lsb-release to
see if it is 'Mandriva'. Since the check for existence of
(/etc/lsb-release, Mandrive) was the first non-empty dist
file match, 'ansible_distribution' was being set to 'Mandriva'
expecting to be corrected by the data from the dist file content.

But since the dist file parsing for Mandriva didn't match for
Ubuntu 10.04 /etc/lsb-release _and_ there is no Debian specific
lsb-release check, 'ansible_distribution' stayed at 'Mandriva'
and the dist file checking loop keeps going and eventually off
the end of the list before finding a better match.

Adding a debian/ubuntu specific check for /etc/lsb-release after
the debian os-release sets the info correctly and stops further
checking of dist files.

Fixes #30693
7 years ago
Adrian Likins 3eab636b3f Fix 'distribution' fact for ArchLinux (#30723)
* Fix 'distribution' fact for ArchLinux

Allow empty wasn't breaking out of the process_dist_files
loop, so a empty /etc/arch-release would continue searching
and eventually try /etc/os-release. The os-release parsing
works, but the distro name there is 'Arch Linux' which does
not match the 2.3 behavior of 'Archlinux'

Add a OS_RELEASE_ALIAS map for the cases where we need to get
the distro name from os-release but use an alias.

We can't include 'Archlinux' in SEARCH_STRING because a name match on its keys
but without a match on the content causes a fallback to using the first
whitespace seperated item from the file content as the name.
For os-release, that is in form 'NAME=Arch Linux'

With os-release returning the right name, this also supports the
case where there is no /etc/arch-release, but there is a /etc/os-release

Fixes #30600

* pep8 and comment cleanup
7 years ago
Adrian Likins 45a9f96774 Facts Refresh (2.4 roadmap) (#23012)
Facts Refresh (2.4 roadmap)

This commit implements most of the 2.4 roadmap 'Facts Refresh'
- move facts.py to facts/__init__.py
- move facts Distribution() to its own class
- add a facts/utils.py
- move get_file_content and get_uname_version to facts/utils.py
- move Facts() class from facts/__init__ to facts/facts.py
- mv get_file_lines to facts/utils.py
- mv Ohai()/Facter() class to facts/ohai.py and facter.py
- Start moving fact Hardware() classes to facts/hardware/*.py
- mv HPUX() hardware class to facts/hardware/hpux.py
- move SunOSHardware() fact class to facts/hardware/sunos.py
- move OpenBSDHardware() class to facts/hardware/openbsd.py
- mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/
- mv NetBSDHardware() to facts/hardware/netbsd.py
- mv Darwin() hardware class to facts/hardware/darwin.py
- pep8/etc cleanups on facts/hardware/*.py
- Mv network facts classes to facts/network/*.py
- mv Virtual fact classes to facts/virtual
- mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl

- Also mv get_uname_version from facts/utils.py -> distribution.py
  since distribution.py is the only thing using it.

- add collector.py with new BaseFactCollector
- add a subclass for AnsibleFactCollector
- hook up dict key munging FactNamespaces
- add some test cases for testing the names of facts
- mv timeout stuff to facts.timeout

- rm ansible_facts()/get_all_facts() etc

- Instead of calling facts.ansible_facts(), fact collection
  api used by setup.py is now to create an AnsibleFactCollector()
  and call it's collect method.

- replace Facts.get_user_facts with UserFactCollector
- add a 'systems' facts package, mv UserFactCollector there
- mv get_dns_facts to DnsFactCollector
- mv get_env_facts to EnvFactCollector
- include the timeout length in exception message

- modules and module_utils that use AnsibleFactCollector
  can now theoretically set the 'valid_subsets'

  May be useful for network facts module that currently have
  to reimplement a good chunk of facts.py to get gather_subsets
  to work.

- get_local_facts -> system/LocalFactCollector
- get_date_time -> system/date_time.py
- get_fips_facts -> system/fips.py
- get_caps_facts() -> system/caps.py
- get_apparmor_facts -> system/apparmor.py
- get_selinux_facts -> system/selinux.py
- get_lsb_facts -> system/lsb.py
- get_service_mgr_facts -> system/service_mgr.py
- Facts.is_systemd_managed ->  system/service_mgr.py
- get_pkg_mgr_facts -> system/pkg_mgr.py
- Facts()._get_mount_size_facts() -> facts.utils.get_mount_size()

- add unit test for EnvFactCollector
- add a test case for minimal gather_subsets
- add test case for collect_ids
- Make gather_subset match existing behavior or '!all'

    If 'gather_subset' is provided as '!all', the existing behavior
    (in 2.2/2.3) is that means 'dont collect any facts except those
    from the Facts() class'. So 'skip everything except
    'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb',
    'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc.

    The new facts setup was making '!all' mean no facts at all, since
    it can add/exclude at a finer granularity. Since that makes more
    sense for the ansible collector, and the set of minimal facts to
    collect is really more up to setup.py to decide we do just that.

    So if setup.py needs to always collect some gather_subset, even
    on !all, setup.py needs to have the that subset added to the
    list it passes as minimal_gather_subset.

    This should fix some intg tests that assume '!all' means that
    some facts are still collected (user info and env for example).

    If we want to make setup.py collect a more minimal set, we can do that.

- force facts_dicts.keys() to a list so py3 works
- split fact collector tests to test_collectors.py

- convert Facter(Facts) -> other/facter.py:FacterFactCollector

- add FactCollector.collect_with_namespace()

    regular .collect() will return a dict with the key names
    using the base names ('ip_address', 'service_mgr' etc)

    .collect_with_namespace() will return a dict where the key names
    have been transformed with the collectors namespace, if there is
    one. For most, this means a namespace that adds 'ansible_' to the
    start of the key name.

    For 'FacterFactCollector', the namespace transforms the key to
    'facter_*'.

- add test cases for collect_with_namespace

- move all the concrete 'which facts does setup.py' stuff to setup.py

    The caller of AnsibleFactCollector.from_gather_subset() needs to
    pass in the list of collector classes now.

- update system/setup.py to import all of the fact classes and pass
  in that list.
- split the Distribution fact class up a bit

    extracted the 'distro release' file handling (ie, linux
    boxes with /etc/release, /etc/os-release etc) into its
    own class.
- extract get_cmdline_facts -> cmdline.py
- extract get_public_ssh_host_keys -> system/ssh_pub_keys.py
- extract get_platform_facts -> system/platform.py

  platform.py may be a good candidate for further splitting.

- rm test for plain Facts() base class
- let the base class for Collector unit tests provide collected_facts

    some Collectors and/or their migrated Facts() subsclasses need
    to look at facts collected by other modules ('ansible_architecture'
    the main one...).

    Collector.collect() has the collected_facts arg for this, so add
    a class variable to BaseFactsTest so we can specify it.

- mv Ohai to other/ohai.py and convert to Collector
- update hardware/*.py to return facts (no side effects)

- mv AnsibleFactCollector to setup.py
- extra collector class gathering to module method in
  facts/__init__.py (collector_classes_from_gather_subset)
- add a CollectorMetaDataCollector collector used to provide
  the 'gather_setup' fact
- add unit test module for 'setup' module
  (test/units/modules/system/setup.py)

- Collector init now doesnt need a module, but collect does

    An instance of a FactCollector() isnt tied to a AnsibleModule
    instance, but the collect() method can be, so optionally pass
    in module to FactCollector.collect() (everywhere)

- add a default_collectors for list of default collectors

  import and use it from setup.py module

  eventually, would like to replace this with a plugin loader
  style class finder/loader

- unit tests for module_utils/facts/__init__.py
- add unit tests for ohai facts collector
- remove self.facts side effect on populate() in hardware/sunos.py
- convert OpenBSDHardware() to rm side effects on self.facts
- try to rm some self.facts side effects in Network()

    plumb in collected_facts from populate() where it is needed.

    stop passing collected_facts into Network() [via cached_facts=,
    where it eventually becomes self.facts]

- nothing provides Fact() cached_facts arg now, rm it

    Facts() should be internal only implementation so nothing
    should be using it.

    Of course, now someone will.

- add a Collector.name attr to build a map of name->_fact_ids

    To properly exclude a gather_subset spec like '!hardware', we
    need to know that 'hardware' also means 'devices', 'dmi', etc.
    Before, '!hardware' would remove the 'hardware' collector name
    but not 'devices'. Since both would end up in id_collector_map,
    we would still end up with the HardwareCollector in the collector
    list. End result being that '!hardware' wouldn't stop hardware
    from being collected.

    So we need to be able to build that map, so add the Collector.name
    attribute that is the primary name (like 'hardware') and let
    Collector._fact_ids be the other fact ids that a collector is
    responsible for.

    Construct the aliases_map of Collector.name -> set of _fact_ids
    in fact/__init__.py get_collector_names, and use it when we are
    populating the exclude set.

- refactor of distribution.py

    make the big OS_FAMILY literal a little easier to read
    Also keys can now be any string instead of python literals

    99% sure the test for 'KDE Neon' was wrong
    I don't see how/where it should or could get 'Neon' instead
    of 'KDE Neon' as provided in os-release NAME=

    Use 'distribution' string for key to OS_MAP

    ie, we dont need to make it a valid python label anymore so dont.

    move _has_dist_file to module as _file_exists
    easier to mock without mucking with os.path

    mv platform.system() calls to within get_distribution_facts() instead
    of Distribution() init.

- remove _json compat module

    The code in here was to support:

      -a 'json' python module that was not the standard one included
      with python since 2.6.

      - potentially fallback to simplejson if 'json' was not available.

    'json' is available for all supported python versions now so
    no longer needed.

- mv get_collector_names -> facts.collector
- mv collector_classes_from_gather_subset -> facts.collector
- mv collector tests from test_facts -> test_collector

- Use six's reduce() in sunos/netbsd hardware facts

- rm extraneous get_uname_version in utils

  only system/distribution.py uses it

- Remove Facts() subclass metaclass usage

  - using fact_id and a platform id for matching collectors

    gut most of Facts() subclasses

    rm Facts() subclasses with weird metaclass

    only add collectors that match the fact_ids and the platform_info
    to the list of collectors used.

    atm, a collectors platform_id will default to 'Generic', and
    any platform matches 'Generic'

    goal is to select collector classes including matching the
    systems platform in collector.py, instead of relying on
    metaclasses in hardware/*. To finish this, the various
    Facts() subclasses will need to be replaced entirely with
    Collector() subclasses.

    use collector classmethod platform_match() to match the platform

    This lets the particular class decide if it is compatible with
    a given platform_info. platform_info is a dict like obj, so it could be
    expanded in the future.

    Add a default platform_match to BaseFactCollector that matches
    platform_info['system'] == cls._platform

    They were needed previously to trigger a module
    load on all the collector classes when we import
    facts/hardare so that the Hardware() and related
    classes that used __new__ and find_all_subclasses()
    would work.

    Now that is done in collectors based on platform matching
    at runtime we dont need to do it py module import/parse
    time. So the non empty __init__.pys are no longer needed
    and their is a more flexible mechanism for selection
    platform specific stuff.

    facts/facts.py is no longer used, rm'ed

- if we dont find an implement class for gather spec.. just ignore it.

  Would be useful to add a warn to warn about this case.

- Fix SD-UX typo (should be HP-UX)

- Port fix for #21893 (0 sockets) to this branch

    This readds the change from 8ad182059d
    that got lost in merge/rebase

    Fixes #21893

- port sunos fact locale fix for #24542 to this branch

    based on e558ec19cd

    Fixes #24542

    Solaris fact fix (#24793)

    ensure locale for solaris fact gathering

    fixes issue with locale interfering with proper reading of decimals

- raise exceptions in the air like we just dont care.

    Pretty much ignore any not exit exception in facts
    collection. And add some test cases.

- added new selinux fact to clarify python lib

    the selinux fact is boolean false when the library is not installed,
    a dictionary/hash otherwise, but this is ambigous
    added new fact so we can eventually remove the type dichtomy and normalize it as a dict

    Re-add of devel commit 85c7a7b844 to
    the new code layout, since it got removed in merge/rebase
8 years ago
Dag Wieers 4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
8 years ago
Pavel Glushchak 097173c6f5 Added Virtuozzo distribution support
Virtuozzo Linux is based on CentOS sources. Thus OS family
should be recognized as 'RedHat'.

Signed-off-by: Pavel Glushchak <pglushchak@virtuozzo.com>
8 years ago
Wittmer, Christian 52fbfa00f9 fix for #21729, use VERSION_ID instead of PRETTY_NAME to get release (#21730)
* fix for #21729, use VERSION_ID instead of PRETTY_NAME to get release
8 years ago
Matt Martz 87aa59af79 Legacy pep8 updates for setup.py and tests 8 years ago
Matt Clay 1293ec85dd Convert unit test yield to pytest parametrize. 8 years ago
Matt Clay 272ff10fa1 Fix @contextmanager leak on exception. (#21031)
* Fix @contextmanager leak on exception.
* Fix test leaks of global module args cache.
8 years ago
Vlad Glagolev 039d207d24 Add Source Mage GNU/Linux facts (#19671)
* Add Source Mage GNU/Linux facts

* Add unit test for OS-detection
8 years ago
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
8 years ago
Matt Clay e2c0b375d3 PEP 8 cleanup. (#20790)
* PEP 8 E115 cleanup.
* PEP 8 E131 cleanup.
8 years ago
Matt Clay d913f69ba1 PEP 8 W291 whitespace cleanup. 8 years ago
Matt Robinson 4ff8890ec1 Set ansible_os_family correctly under KDE neon
As neon is derived from Ubuntu, ansible_os_family should have the value
"Debian" instead of "Neon".  Add a test case for KDE neon and set
os_family correctly for it.
8 years ago
stephane 77868a4104 Set Suse family for openSUSE Tumbleweed & Leap
On openSUSE Tumbleweed, lsb-release -a currently reports
the distributor ID as "openSUSE Tumbleweed". On openSUSE
Leap, the distributor ID is "SUSE LINUX".

Add them to the OS_FAMILY dict as Suse family systems.

Also add an entry to TESTSETS in test_distribution_version.py
for openSUSE Tumbleweed.
8 years ago
cinerama70 02cec7dca9 Add os_family to test_distribution_version (#17620)
As suggested in feedback on
https://github.com/ansible/ansible/pull/17575, add
os_family to test_distribution_version. Add the
correct os_family to the existing testcase data
entries.

Also add os_family to the output of
gen_distribution_version_testcase.py so any new
generated entries will contain this data.
8 years ago
Toshio Kuratomi 44d979c8f5 Enable most unittests on python3 (just some vault unittests and a logging one left) (#17240)
Make some python3 fixes to make the unittests pass:

* galaxy imports
* dictionary iteration in role requirements
* swap_stdout helper for unittests
* Normalize to text string in a facts.py function
8 years ago
Robin Roth 1f04130c00 add more distribution version tests (#16563)
* ubuntu 16.04
* solaris 10, 11
* fedora 22, 25

Thanks Peter Oliver for providing the test data
9 years ago
Nate Coraor 706778ee7f Fix distro detection for Solaris derivatives 9 years ago
Toshio Kuratomi 186337db28 Ship constants to the modules via internal module params rather than a secondary dict. 9 years ago
Robin Roth 1d6608e84f Dist version fix for Red Hat and more tests (#15663)
* add tests for centos6, rhel6 and rhel7

* gen_distribution_version_testcase with python2.6

* remove unused imports

* fix redhat/vmware/... parsing

* add centos7 test case
9 years ago
nitzmahone 5b336832af add _load_params debug overrides for module args/file passed on cmdline
Updated python module wrapper explode method to drop 'args' file next to module.
Both execute() and excommunicate() debug methods now pass the module args via file to enable debuggers that are picky about stdin.
Updated unit tests to use a context manager for masking/restoring default streams and argv.
9 years ago
Robin Roth 49cdc565c5 remove nose.tools use 9 years ago
Robin Roth 4088aa2b4c get rid of assert_in 9 years ago
Robin Roth 37188ea336 cleanup tests
* use nose test generator
* more comments
* move facts import inside the skipped function, fix python3 warning
9 years ago
Robin Roth 2b104fe6ad fix tests for SLES and CoreOS
* include #15230
9 years ago
Robin Roth 3c883d8a6d add testcases from refactor PR 9 years ago