* Create get_exception and wildcard import code-smell tests
* Add more detail to boilerplate and no-basestring descriptions
* Remove the no-list-cmp test as the pylint undefined-variable test covers it
Fix adds missing imports and boilerplate for proxysql.
It also remove get_exception calls in-favor of native exception.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Enable specific tests (this lets us disable a group and then
enable a particular test inside of it)
* Comment out tests in the enable and disable files
Made the following changes:
* Removed wildcard imports
* Replaced long form of GPL header with short form
* Removed get_exception usage
* Added from __future__ boilerplate
* Adjust division operator to // where necessary
For the following files:
* web_infrastructure modules
* system modules
* linode, lxc, lxd, atomic, cloudscale, dimensiondata, ovh, packet,
profitbricks, pubnub, smartos, softlayer, univention modules
* compat dirs (disabled as its used intentionally)
We do want to allow certain from __future__ imports in modules that make
it easier to code compatible python2 and python3. Note that
unicode_literals is specifically left out and should never be allowed.
Now that python-3.4+ allows u"" there's no good reason to use
unicode_literals.
Also switch tables in the validate_modules documentation to simple table format
* Enable the pylint no-name-in-module check. Checks that identifiers in
imports actually exist. When we do this, we also have to ignore
_MovedItems used in our bundled six. This means pylint won't check
for bad imports below ansible.module_utils.six.moves but that's
something that pylint punts on with a system copy of six so this is
still an improvement.
* Remove automatic use of system six. The exec in the six code which
tried to use a system library if available destroyed pylint's ability
to check for imports of identifiers which did not exist (the
no-name-in-module check). That test is important enough that we
should sacrifice the bundling detection in favour of the test.
Distributions that want to unbundle six can replace the bundled six in
ansible/module_utils/six/__init__.py to unbundle. however, be aware
that six is tricky to unbundle. They may want to base their efforts
off the code we were using:
2fff690caa/lib/ansible/module_utils/six/__init__.py
* Update tests for new location of bundled six Several code-smell tests
whitelist the bundled six library. Update the path to the library so
that they work.
* Also check for basestring in modules as the enabled pylint tests will
also point out basestring usage for us.
The OpenSSLObject class has been merged[1]. This commit makes the
openssl_privatekey rely on this class and standardize the way openssl
module should be written.
Co-Authored-By: Christian Pointner <cpointner@mgit.at>
[1] https://github.com/ansible/ansible/pull/26945
* Add example of templating inline using copy module
The **copy** module documentation implies that `content:` only works
for 'simple values' and for complex stuff you need the **template**
module, but that is an understatement. You can use **copy** to template
anything you desire.
So I changed the wording, added an example, and also added a note
to the template module that the **copy** module could be used for
'inline templating'.
This fixes#19741.
* Revert change to docker_common as it's not as good as the try: except fix
* limit docker_volume fix to ImportErrors
* fix docker_secret i nthe same way
* Remove docker_secret from import tests
* adds more intelligent save logic and diff to network config modules
* adds sha1 property to NetworkConfig
* adds new argument save_when to argument_spec
* adds new argument diff_against to argument_spec
* adds new argument intended_config to argument_spec
* renames config argument to running_config with alias to config
* deprecates the use of the save argument
* before and after now work with src argument
* misc module clean
Modules updated
* nxos_config
* ios_config
* eos_config
Most notably this makes the save mechanism more intelligent for config
modules for devices that need to copy the ephemeral config to
non-volatile storage.
The diff_against argument allows the playbook task to control what the
device's running-config is diff'ed against. By default it will return
the diff of the startup-config.
* removes ios_config from pep8/legacy_files.txt
* extends the ignore lines argument to the module
* clean up CI errors
* add missing list brackets
* fixes typo
* fixes unit test cases
* remove last line break when returning config contents
* encode config string to bytes before hashing
* fix typo
* addresses feedback in PR
* update unit test cases
Module was importing '*' from facts to get to TimeoutError
but that has moved to facts.timeout, so import is updated.
Also rm old style imports to new style imports at the start
of the module.
'signal' py module was used and referenced but never imported,
presumably it was using the 'signal' previously imported into
module_utils.facts. Now imported directly.
'AnsibleModule' was also from a * import, so now imported directly.
A ref to 'module' was in _delete_disks_when_detached(), so now it
is updated to raise an AzureException() with its message, and
let its caller catch it and call module.fail_json()
This PR includes:
- Checkmode improvements
- Integration tests
- A fix for python3
- PEP8 fixes
This backports improvements from the win_wakeonlan module.
* Add junos_system declartive module and other related change
* junos_system declartive module
* integration test for junos_system
* integration test for net_system (junos platform)
* pep8 fixes for junos modules
* move to lxml from elementree for xml parsing as it support
complete set of xpath api's
* other minor changes
* Fix CI and doc changes
* Fix unit test failures
* Fix typo in import
* Fix import issue for py2.6
* Add missed Element in import
* Allow EC2-VPC instances to update SG
make ec2 pep8
* use sets instead of loop and a break
* bring things in an indentation level
* Use to_text instead of str, text_type instead of basestring, - instead of difference
* basestrings not unicode
* simplifying syntax
* Start of ansible config project
moved configuration definitions to external yaml file vs hardcoded
* updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
* prints green for default/unchanged and yellow for those that have been overriden
* added list action to show all configurable settings and their associated ini and env var names
* allows specifying config file to see what result would look like
* TBD update, edit and view options
removed test for functions that have been removed
env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling
updated as per feedback
* pep8
* Various fixes to VM customizations (from template)
This patch implements:
- New find_obj() function from vmware.py replacing get_obj()
- Implement proper resource_pool selection
- Fix productId implementation (was not working)
- Ensure that we are not changing anything that is not mandatory (hostName, orgName, fullName)
This is an alternative proposal to #24283
This does not fix#19860 yet though.
For our use-case, we do not want to customize the network information (or any information in fact).
What is used in the template should remain intact.
* Added find_obj() function
* Fix the returned object-list (unused yet)
* Small improvement
* Support DHCP type and fix customizations
* Small fix
* Support resource_pool also for reconfiguring VM
* Remove redundant
* Fix short hostname, specific resource_pool, PEP8
* Improve docs and examples
* Fix missing hostsystem
* Make folder absolute path
* Improve docs, add missing 'mac'
* ec2_asg and ec2_asg_facts module improvements
Return target group information for both ec2_asg and ec2_asg_facts
modules
Provide RETURN documentation for ec2_asg module
PEP8 fixes for ec2_asg_facts
* ec2_asg: use pagination when describing target groups
In case an ASG has 100s of target groups, ensure that
we get the full result using build_full_result
* Factorize tests related to no_password_change using an include task
* Refactor: deduplicate tasks
* postgresql_user: test 'expires' parameter
* Change 'valid until' even it's the only updated field
* value is changed when another value is provided
* value isn't returned when unset
* Remove unused variable
* psycopg2.extras.DictRow is able to handle comparison
* postgresql_user: simplify helper method
* postgresql_user: define variable just before using it
* Fix comparison between user input and applied configuration
* new test: adding an invalid attribute
* Refactor, add cleaning task
* Check that using same attribute a 2nd time does nothing
* Always try to remove created user
* postgresql_user: fix pep8
PR #5165 at https://github.com/ansible/ansible-modules-core/pull/5165
adds redirection and capture of stdout during execution of
docker-compose.
This doesn't necessarily catch all errors, since some are printed to
stderr and lost.
This extends the redirection to include stderr, and does minor string
processing to attempt to find a 'useful' message to present as the
final Ansible error.
* Support check_mode in ec2_vpc_nacl
Ensure that all API calls that make changes are guarded by
`if not module.check_mode`.
* Update ec2_vpc_nacl_facts to latest pep8 standards
* Add support to fetch old style junos facts
Fixes#25050
Add support to fetch old style facts supported in Ansible <= 2.2
To fetch old style facts value `gather_subset` value shoule be `all`
and `junos-eznc` is required to be installed on control node as a
prerequisite.
* Remove unwanted import
Facts Refresh (2.4 roadmap)
This commit implements most of the 2.4 roadmap 'Facts Refresh'
- move facts.py to facts/__init__.py
- move facts Distribution() to its own class
- add a facts/utils.py
- move get_file_content and get_uname_version to facts/utils.py
- move Facts() class from facts/__init__ to facts/facts.py
- mv get_file_lines to facts/utils.py
- mv Ohai()/Facter() class to facts/ohai.py and facter.py
- Start moving fact Hardware() classes to facts/hardware/*.py
- mv HPUX() hardware class to facts/hardware/hpux.py
- move SunOSHardware() fact class to facts/hardware/sunos.py
- move OpenBSDHardware() class to facts/hardware/openbsd.py
- mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/
- mv NetBSDHardware() to facts/hardware/netbsd.py
- mv Darwin() hardware class to facts/hardware/darwin.py
- pep8/etc cleanups on facts/hardware/*.py
- Mv network facts classes to facts/network/*.py
- mv Virtual fact classes to facts/virtual
- mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl
- Also mv get_uname_version from facts/utils.py -> distribution.py
since distribution.py is the only thing using it.
- add collector.py with new BaseFactCollector
- add a subclass for AnsibleFactCollector
- hook up dict key munging FactNamespaces
- add some test cases for testing the names of facts
- mv timeout stuff to facts.timeout
- rm ansible_facts()/get_all_facts() etc
- Instead of calling facts.ansible_facts(), fact collection
api used by setup.py is now to create an AnsibleFactCollector()
and call it's collect method.
- replace Facts.get_user_facts with UserFactCollector
- add a 'systems' facts package, mv UserFactCollector there
- mv get_dns_facts to DnsFactCollector
- mv get_env_facts to EnvFactCollector
- include the timeout length in exception message
- modules and module_utils that use AnsibleFactCollector
can now theoretically set the 'valid_subsets'
May be useful for network facts module that currently have
to reimplement a good chunk of facts.py to get gather_subsets
to work.
- get_local_facts -> system/LocalFactCollector
- get_date_time -> system/date_time.py
- get_fips_facts -> system/fips.py
- get_caps_facts() -> system/caps.py
- get_apparmor_facts -> system/apparmor.py
- get_selinux_facts -> system/selinux.py
- get_lsb_facts -> system/lsb.py
- get_service_mgr_facts -> system/service_mgr.py
- Facts.is_systemd_managed -> system/service_mgr.py
- get_pkg_mgr_facts -> system/pkg_mgr.py
- Facts()._get_mount_size_facts() -> facts.utils.get_mount_size()
- add unit test for EnvFactCollector
- add a test case for minimal gather_subsets
- add test case for collect_ids
- Make gather_subset match existing behavior or '!all'
If 'gather_subset' is provided as '!all', the existing behavior
(in 2.2/2.3) is that means 'dont collect any facts except those
from the Facts() class'. So 'skip everything except
'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb',
'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc.
The new facts setup was making '!all' mean no facts at all, since
it can add/exclude at a finer granularity. Since that makes more
sense for the ansible collector, and the set of minimal facts to
collect is really more up to setup.py to decide we do just that.
So if setup.py needs to always collect some gather_subset, even
on !all, setup.py needs to have the that subset added to the
list it passes as minimal_gather_subset.
This should fix some intg tests that assume '!all' means that
some facts are still collected (user info and env for example).
If we want to make setup.py collect a more minimal set, we can do that.
- force facts_dicts.keys() to a list so py3 works
- split fact collector tests to test_collectors.py
- convert Facter(Facts) -> other/facter.py:FacterFactCollector
- add FactCollector.collect_with_namespace()
regular .collect() will return a dict with the key names
using the base names ('ip_address', 'service_mgr' etc)
.collect_with_namespace() will return a dict where the key names
have been transformed with the collectors namespace, if there is
one. For most, this means a namespace that adds 'ansible_' to the
start of the key name.
For 'FacterFactCollector', the namespace transforms the key to
'facter_*'.
- add test cases for collect_with_namespace
- move all the concrete 'which facts does setup.py' stuff to setup.py
The caller of AnsibleFactCollector.from_gather_subset() needs to
pass in the list of collector classes now.
- update system/setup.py to import all of the fact classes and pass
in that list.
- split the Distribution fact class up a bit
extracted the 'distro release' file handling (ie, linux
boxes with /etc/release, /etc/os-release etc) into its
own class.
- extract get_cmdline_facts -> cmdline.py
- extract get_public_ssh_host_keys -> system/ssh_pub_keys.py
- extract get_platform_facts -> system/platform.py
platform.py may be a good candidate for further splitting.
- rm test for plain Facts() base class
- let the base class for Collector unit tests provide collected_facts
some Collectors and/or their migrated Facts() subsclasses need
to look at facts collected by other modules ('ansible_architecture'
the main one...).
Collector.collect() has the collected_facts arg for this, so add
a class variable to BaseFactsTest so we can specify it.
- mv Ohai to other/ohai.py and convert to Collector
- update hardware/*.py to return facts (no side effects)
- mv AnsibleFactCollector to setup.py
- extra collector class gathering to module method in
facts/__init__.py (collector_classes_from_gather_subset)
- add a CollectorMetaDataCollector collector used to provide
the 'gather_setup' fact
- add unit test module for 'setup' module
(test/units/modules/system/setup.py)
- Collector init now doesnt need a module, but collect does
An instance of a FactCollector() isnt tied to a AnsibleModule
instance, but the collect() method can be, so optionally pass
in module to FactCollector.collect() (everywhere)
- add a default_collectors for list of default collectors
import and use it from setup.py module
eventually, would like to replace this with a plugin loader
style class finder/loader
- unit tests for module_utils/facts/__init__.py
- add unit tests for ohai facts collector
- remove self.facts side effect on populate() in hardware/sunos.py
- convert OpenBSDHardware() to rm side effects on self.facts
- try to rm some self.facts side effects in Network()
plumb in collected_facts from populate() where it is needed.
stop passing collected_facts into Network() [via cached_facts=,
where it eventually becomes self.facts]
- nothing provides Fact() cached_facts arg now, rm it
Facts() should be internal only implementation so nothing
should be using it.
Of course, now someone will.
- add a Collector.name attr to build a map of name->_fact_ids
To properly exclude a gather_subset spec like '!hardware', we
need to know that 'hardware' also means 'devices', 'dmi', etc.
Before, '!hardware' would remove the 'hardware' collector name
but not 'devices'. Since both would end up in id_collector_map,
we would still end up with the HardwareCollector in the collector
list. End result being that '!hardware' wouldn't stop hardware
from being collected.
So we need to be able to build that map, so add the Collector.name
attribute that is the primary name (like 'hardware') and let
Collector._fact_ids be the other fact ids that a collector is
responsible for.
Construct the aliases_map of Collector.name -> set of _fact_ids
in fact/__init__.py get_collector_names, and use it when we are
populating the exclude set.
- refactor of distribution.py
make the big OS_FAMILY literal a little easier to read
Also keys can now be any string instead of python literals
99% sure the test for 'KDE Neon' was wrong
I don't see how/where it should or could get 'Neon' instead
of 'KDE Neon' as provided in os-release NAME=
Use 'distribution' string for key to OS_MAP
ie, we dont need to make it a valid python label anymore so dont.
move _has_dist_file to module as _file_exists
easier to mock without mucking with os.path
mv platform.system() calls to within get_distribution_facts() instead
of Distribution() init.
- remove _json compat module
The code in here was to support:
-a 'json' python module that was not the standard one included
with python since 2.6.
- potentially fallback to simplejson if 'json' was not available.
'json' is available for all supported python versions now so
no longer needed.
- mv get_collector_names -> facts.collector
- mv collector_classes_from_gather_subset -> facts.collector
- mv collector tests from test_facts -> test_collector
- Use six's reduce() in sunos/netbsd hardware facts
- rm extraneous get_uname_version in utils
only system/distribution.py uses it
- Remove Facts() subclass metaclass usage
- using fact_id and a platform id for matching collectors
gut most of Facts() subclasses
rm Facts() subclasses with weird metaclass
only add collectors that match the fact_ids and the platform_info
to the list of collectors used.
atm, a collectors platform_id will default to 'Generic', and
any platform matches 'Generic'
goal is to select collector classes including matching the
systems platform in collector.py, instead of relying on
metaclasses in hardware/*. To finish this, the various
Facts() subclasses will need to be replaced entirely with
Collector() subclasses.
use collector classmethod platform_match() to match the platform
This lets the particular class decide if it is compatible with
a given platform_info. platform_info is a dict like obj, so it could be
expanded in the future.
Add a default platform_match to BaseFactCollector that matches
platform_info['system'] == cls._platform
They were needed previously to trigger a module
load on all the collector classes when we import
facts/hardare so that the Hardware() and related
classes that used __new__ and find_all_subclasses()
would work.
Now that is done in collectors based on platform matching
at runtime we dont need to do it py module import/parse
time. So the non empty __init__.pys are no longer needed
and their is a more flexible mechanism for selection
platform specific stuff.
facts/facts.py is no longer used, rm'ed
- if we dont find an implement class for gather spec.. just ignore it.
Would be useful to add a warn to warn about this case.
- Fix SD-UX typo (should be HP-UX)
- Port fix for #21893 (0 sockets) to this branch
This readds the change from 8ad182059d
that got lost in merge/rebase
Fixes#21893
- port sunos fact locale fix for #24542 to this branch
based on e558ec19cdFixes#24542
Solaris fact fix (#24793)
ensure locale for solaris fact gathering
fixes issue with locale interfering with proper reading of decimals
- raise exceptions in the air like we just dont care.
Pretty much ignore any not exit exception in facts
collection. And add some test cases.
- added new selinux fact to clarify python lib
the selinux fact is boolean false when the library is not installed,
a dictionary/hash otherwise, but this is ambigous
added new fact so we can eventually remove the type dichtomy and normalize it as a dict
Re-add of devel commit 85c7a7b844 to
the new code layout, since it got removed in merge/rebase
Dag Wieers