Commit Graph

185 Commits (1d640182c61af2d59cc1005f1ab5b217a56dcf15)

Author SHA1 Message Date
Matt Martz 1663b64e18
Allow subspec defaults to be processed when the parent argument is not supplied (#38967)
* Allow subspec defaults to be processed when the parent argument is not supplied

* Allow this to be configurable via apply_defaults on the parent

* Document attributes of arguments in argument_spec

* Switch manageiq_connection to use apply_defaults

* add choices to api_version in argument_spec
6 years ago
Matthew Stone d030032b47 Extreme Networks SLXOS Config Module (#38607)
* Adding slxos_config module and supporing util functions.

* Adding slxos module_utils load_config test

* Adding slxos_config module tests

* Removing unneeded required false statements from slxos_config module

* Removing version_aded from slxos_config module

* Removing force and save from slxos config module

* Removing save test
6 years ago
Alex Stephen 9de9633cac [GCP] New module GCP DNS Resource Record Set (#35920) 6 years ago
Matthew Stone 5b1a8ee1e8 Adding slx_command module and supporting module_utils. (#38235)
* Adding slx_command module and supporting module_utils.

This commit adds the slx_command module and tests as well as the
required slxos module_utils.

* Update copyright in header

* Adding missing module init

* Cleaning up shebangs/licensing.

* Incorporating feedback

Removing reference to `waitfor` alias in `slxos_command` module.
Adding `Extreme Networks` to `short_description` of `slxos_command` module.

* Adding cliconf tests

* Fixing 3.X tests

* Adding docstrings to test methods for slxos cliconf tests

* Adding slxos terminal tests

* Adding slxos module_utils tests

* Adding Extreme Networks team members to BOTMETA.yml
6 years ago
Matt Martz 450cfa8776
Handle duplicate headers in the uri module (#33792)
* Handle duplicate headers, and make it easier for users to use cookies, by providing a pre-built string

* Ensure proper cookie ordering, make key plural

* Add note about cookie sort order

* Add tests for duplicate headers and cookies_string

* Extend tests, normalize headers between py2 and py3

* Add some notes in test code

* Don't use AttributeError, use six.PY3. Use better names.
6 years ago
Matt Martz 6332beef65
Add unit tests for ansible.module_utils.urls (#38059)
* Start of tests for ansible.module_utils.urls

* Start adding file for generic functions throughout urls

* Add tests for maybe_add_ssl_handler

* Remove commented out line

* Improve coverage of maybe_add_ssl_handler, test basic_auth_header

* Start tests for open_url

* pep8 and ignore urlopen in test_url_open.py tests

* Extend auth tests, add test for validate_certs=False

* Finish tests for open_url

* Add tests for fetch_url

* Add fetch_url tests to replace-urlopen ignore

* dummy instead of _

* Add BadStatusLine test

* Reorganize/rename tests

* Add tests for RedirectHandlerFactory

* Add POST test to confirm behavior is to convert to GET

* Update tests to handle recent changes to RedirectHandlerFactory

* Special test, just to confirm that aliasing http_error_308 to http_error_307 does not cause issues with urllib2 type redirects
6 years ago
Alexander Bethke eebe728ae8 Fix SteamOS ansible_distribution fact (Fixes #33628) (#37576)
Fixes #33628
6 years ago
Claes Nästén 458a07d5a4 add timeout parameter and increase default timeout for nso modules (#37391)
NSO operations can take much longer than 10 seconds as they operate on
real network equipment, set default timeout to 5 minutes and allow for
user override.
6 years ago
Claes Nästén 042c111563 nso_config break cycles in dependency sorting (#36828)
False assumption that values can not have cyclic dependencies. Fix by
removing dependency on self and look for cycles, if found remove
dependency to get a partial sort done.
6 years ago
Claes Nästén e75989ec88 nso_config work around ordering issues (#36774)
Include dependencies when sorting entries to avoid issues with certain
versions of NSO.
6 years ago
Claes Nästén 2789cc5c09 NSO ValueBuilder improvements. 4.5 leaf-list compatability. (#36583)
Fix issues in ValueBuilder used in nso_config and nso_verify so that it
can handle leaf-list in NSO 4.5 and detect identityref types from
unions.

Fail gracefully if a type is not found.
6 years ago
Peter Sprygada 93b795baf0
allows ib_spec attrs to be filtered in update (#36673)
* allows ib_spec attrs to be filtered in update

This change will allow the ib_spec entries to be be filtered on a change
object by setting the update keyword to false.  The default value for
update is true.  When the update keyword is set to false, the keyed
entry will be removed from the update object before it is sent to the
api endpoint.

fixes #36563

* fix up pep8 issues
6 years ago
Martin Krizek 3a5263a92f facts: fix device uuid's on el6 (#36128)
* facts: fix device uuid's on el6

Fixes #36077
6 years ago
René Moser 2f36b9e5ce basic: allow one or more when param list having choices (#34537)
* basic: allow one or more when param list having choices

* add unit tests

* optimize a bit

* re-add get_exception import

* a number of existing modules expect to be able to get it from basic.py
6 years ago
Dag Wieers bee765fa6b
ACI: Change RETURN output as discussed (#35617)
* ACI: Change result output as discussed

* Update all modules to use new aci.exit_json()

* Update output_level spec and docs

* Fix integration tests

* Small PEP8 fix

* Asorted fixes to tests and aci_rest

* More test fixes and support for ANSIBLE_DEBUG

* Fix another PEP8 issues

* Move response handling inside ACI module

* Reform of ACI error handling and error output

* Diff multiline json output

* Fix a few more tests

* Revert aci_bd tests

* Small correction

* UI change: existing->current, original->previous

* UI change: config->sent

* Update all modules with RETURN values

* Fix a few more tests

* Improve docstring and add 'raw' return value

* Fix thinko

* Fix sanity/pep8 issues

* Rewrite unit tests to comply with new design
6 years ago
Clement Trebuchet 287f3a609e Fixes #35468: ADD max_results to the nios api (#35473)
* Fixes #35468: ADD max_results to the nios api

* Fixes #35468: fix nios unittest

* Fixes #35468: document the module docs fragments for nios
7 years ago
Peter Sprygada c2d3b9cbd5
refactors nios api shared code to handle provider better (#35393)
* refactors nios api shared code to handle provider better

This change refactors the shared code to be easily shared between
modules, plugins and dynamic inventory scripts.  All parts now implement
the provider arguments uniformly.

This also provides a centralized fix to suppress urllib3 warnings coming
from the requests library implemented by infoblox_client

* fix up pep8 errors

* fix missing var name
7 years ago
Claes Nästén 0582521f96 NSO: handle types for leaf-lists in nso_config (ValueBuilder) (#34986)
Fixes: #34985
7 years ago
Claes Nästén b7f815ba89 nso_config handle data.not_found in exists for nested lists (#35267)
Recent NSO can return data.not_found error on exists when parent list
entries does not exist. Handle this and return as false.

Fixes: #35264
7 years ago
Adrian Likins 6e585bdf24
Fact collector ordering deps (#31362)
Add deps/requires for fact collectors

Fact collectors can now set a required_facts
class attribute that will be a set of the names
of fact collectors they require to be run first.

ie, if a collector needs to know the ansible_distribution,
it should set it's required_facts to include 'distribution'

        required_facts = set(['distribution'])

If a collector requires another collector, it gets added
to the selected collector names.

We then topological sort the ordering of the collectors
so that deps work out (ie, 'distribution' will run before
'service_mgr')

required_facts were added to the collectors for:

        - network (requires 'distribution', 'platform')
        - hardware (requires 'platform')
        - service_mgr (requires 'distribution', 'platform')

Fix name references for facts (need 'ansible_' prefix)
is service_mgr

Fixes #30753
7 years ago
Peter Sprygada 8d775a332e Add support for Infoblox NIOS (#35097)
* WIP: initial commit

* update documentation in modules

* fix up sanity check issues

* add unit tests for nios api

* fix up sanity check failures

* fix up EXAMPLES in modules

* mock infoblox_client.connector

* remove unsupported assert statement
7 years ago
Adrian Likins 08f92a9f0f
Fix fact deps when 'filter=ansible_fact' is used. (#33441)
The accumulated collected_facts was being update
with new facts _after_ filtering them. So only
facts that pass the filter would ever be passed
to other fact collectors.

For 'filter=ansible_service_mgr', even though it requires
the platform and distribution facts and even collects them,
they would get filtered out and never passed to the other
collectors that need them (service_mgr for ex).

Fix is just to add the unfiltered facts to collected_facts.

Adds unit tests for fact filter and collected_facts.

Fixes #32286
7 years ago
Adrian Likins ec9769c82f
Facts distribution clear linux 31501 (#32453)
The search string used to look for Clear Linux
was changed in 45a9f96774 to
be more specific, but was too specific. Now finding
a substring match for 'Clear Linux' in /usr/lib/os-release
is enough to consider a match.

Since the details of the full name in os-release varies
('Clear Linux Software for Intel Architecture',
 'Clear Linux OS for Intel Architecture', etc) the
search string match was failing and would fall back to the
'first word in the release file' method resulting in
ansible_distribution='NAME="Clear'

Also add a meta fact indicating which search string
was matched.

Test case info from:
        https://github.com/ansible/ansible/issues/31501#issuecomment-340861535

Fixes #31501
7 years ago
Matt Clay 30093dc4ed
Add Python 3.7 to CI unit test matrix. (#34680)
* Add Python 3.7 to CI unit test matrix.
* Fix `os.errno` reference to be `errno`.
* Update test_aci unit test for Python 3.7.
7 years ago
Will Thames a685b621cd Route table boto3 (#32059)
* Allow protection of certain keys during camel_to_snake

Create an `ignore_list` parameter that preserves the case
of the contents of certain dictionaries. Most valuable
for `tags` but other uses might arise.

* Port ec2_vpc_route_table to boto3

Update tests to reflect fixes in boto3.

* Add RETURN documentation to ec2_vpc_route_table

* Update DOCUMENTATION to be valid yaml

* Add check mode tests
7 years ago
Pilou e9df2083a3 If check mode enabled and file missing set changed to true 32676 (#33967)
* basic.py: add mock to os.path.exists

* set_*_if_different: if check_mode enabled & file missing: set changed to True

Fixes #32676
Thanks to mscherer and Spredzy for the distributed triplet programming
session!
7 years ago
Claes Nästén d6fd9703f1 Add nso_show module for Cisco NSO (#34142)
* update version parsing and move requirements to nso_* modules

prepare for introduction of nso_show module that has other version
requirements than the existing nso_* modules.

* Add nso_show module for retreiving config from Cisco NSO

New module that supports getting configuration and operational data
from Cisco NSO.
7 years ago
Sloane Hertel d877c146ab [cloud] ec2_group fix CIDR with host bits set - fixes #25403 (#29605)
* WIP adds network subnetting functions

* adds functions to convert between netmask and masklen
* adds functions to verify netmask and masklen
* adds function to dtermine network and subnet from address / mask pair

* network_common: add a function to get the first 48 bits in a IPv6 address.

ec2_group: only use network bits of a CIDR.

* Add tests for CIDRs with host bits set.

* ec2_group: add warning if CIDR isn't the networking address.

* Fix pep8.

* Improve wording.

* fix import for network utils

* Update tests to use pytest instead of unittest

* add test for to_ipv6_network()

* Fix PEP8
7 years ago
Pierre-Louis Bonicoli b94630a081 AnsibleModule.get_file_attributes: add unit test 7 years ago
Toshio Kuratomi 370a7ace4b
Split basic units (#33510)
Split the one monolithic test for basic.py into several files

* Split test_basic.py along categories.
  This is preliminary to get a handle on things.  Eventually we may want
  to further split it so each file is only testing a single function.
* Cleanup unused imports from splitting test_basic.py
* Port atomic_move test to pytest.
  Working on getting rid of need to maintain procenv
* Split a test of symbolic_mode_to_octal to follow unittest best practices
  Each test should only invoke the function under test once
* Port test_argument_spec to pytest.
* Fix suboptions failure
7 years ago
Will Thames 15a58d498d [cloud] Improve snake <-> camel conversion for AWS utils (#31400)
Allow CamelCase version of snake_dict_to_camel_dict
(currently only dromedaryCase is supported)

Add reversible option to camel_dict_to_snake_dict

Add tests for both of these options
7 years ago
Ganesh Nalawade ea18b9021a
Revert "Deprecate Entity, EntityCollection and use subspec in network modules (#33575)" (#33849)
This reverts commit 4349b56643.
7 years ago
Ganesh Nalawade 4349b56643
Deprecate Entity, EntityCollection and use subspec in network modules (#33575)
* Deprecate Entity, EntityCollection and use subspec in network modules

*  As per proposal https://github.com/ansible/proposals/issues/76
   deprecate use of Entity, EntityCollection, ComplexDict, ComplexList
   and use subspec instead.
*  Refactor ios modules
*  Refactor eos modules
*  Refactor vyos modules
*  Refactor nxos modules
*  Refactor iosxr modules
*  Add support for key in suboptions handling

* Fix CI issues
7 years ago
Claes Nästén c8a5e689e3 Fix issue with setting values in choices (#33574) 7 years ago
Toshio Kuratomi cd36164239
Porting tests to pytest (#33387)
* Porting tests to pytest

* Achievement Get: No longer need mock/generator.py
  * Now done via pytest's parametrization
  * Port safe_eval to pytest
  * Port text tests to pytest
  * Port test_set_mode_if_different to pytest

* Change conftest AnsibleModule fixtures to be more flexible
  * Move the AnsibleModules fixtures to module_utils/conftest.py for sharing
  * Testing the argspec code requires:
    * injecting both the argspec and the arguments.
    * Patching the arguments into sys.stdin at a different level

* More porting to obsolete mock/procenv.py
  * Port run_command to pytest
  * Port known_hosts tests to pytest
  * Port safe_eval to pytest
  * Port test_distribution_version.py to pytest
  * Port test_log to pytest
  * Port test__log_invocation to pytest
  * Remove unneeded import of procenv in test_postgresql

* Port test_pip to pytest style
  * As part of this, create a pytest ansiblemodule fixture in
    modules/conftest.py.  This is slightly different than the
    approach taken in module_utils because here we need to override the
    AnsibleModule that the modules will inherit from instead of one that
    we're instantiating ourselves.

* Fixup usage of parametrization in test_deprecate_warn

* Check that the pip module failed in our test
7 years ago
Ganesh Nalawade 11c9ad23d5
Refactor common network shared and platform utils code into package (#33452)
* Refactor common network shared and platform specific code into package (part-1)

As per proposal #76 refactor common network shared and platform specific
code into sub-package.
https://github.com/ansible/proposals/issues/76

*  ansible.module_utils.network.common - command shared functions
*  ansible.module_utils.network.{{ platform }} - where platform is platform specific shared functions

*  Fix review comments

* Fix review comments
7 years ago
Toshio Kuratomi e499bccbaa
Port some tests away from nose as examples (#33437)
We don't need to use both nose and pytest.  Once we get rid of all uses
of nose we can remove the extra dependency
7 years ago
Toshio Kuratomi aa7bd8bc11 Port AnsibleModule deprecate and warning tests to pytest (#33162) 7 years ago
Brian Coca ebd08d2a01 jsonify inventory (#32990)
* jsonify inventory
* smarter import, dont pass kwargs where not needed
* added datetime
* Eventual plan for json utilities to migrate to common/json_utils when we split
  basic.py no need to move jsonify to another file now as we'll do that later.
* json_dict_bytes_to_unicode and json_dict_unicode_to_bytes will also
  change names and move to common/text.py at that time (not to json).
  Their purpose is to recursively change the elements of a container
  (dict, list, set, tuple) into text or bytes, not to json encode or
  decode (they could be a generic precursor to that but are not limited
  to that.)
* Reimplement the private _SetEncoder which changes sets and datetimes
  into objects that are json serializable into a private function
  instead.  Functions are more flexible, less overhead, and simpler than
  an object.
* Remove code that handled simplejson-1.5.x and earlier.  Raise an error
  if that's the case instead.
  * We require python-2.6 or better which has the json module builtin to
    the stdlib.  So this is only an issue if the stdlib json has been
    overridden by a third party module and the simplejson on the system
    is 1.5.x or less.  (1.5 was released on 2007-01-18)
7 years ago
Matt Clay e45c763b64 Fix invalid string escape sequences. 7 years ago
cnasten f1fe467c22 nso_config module for setting configuration in Cisco NSO (#30973) 7 years ago
Sam Doran e7902d888c Make ansible_selinux facts a consistent type (#31065)
* Make ansible_selinux facts a consistent type

Rather than returning a bool if the Python library is missing, return a dict with one key containing a message explaining there is no way to tell the status of SELinux on the system becasue the Python library is not present.

* Fix unit test
7 years ago
Adrian Likins 65393e4747 Fix ansible_distribution on Ubuntu 10.04 (#31108)
The /etc/os-release based distro detection doesn't
seem to work for Ubuntu 10.04 (no /etc/os-release?).

So it was testing the next case which was /etc/lsb-release to
see if it is 'Mandriva'. Since the check for existence of
(/etc/lsb-release, Mandrive) was the first non-empty dist
file match, 'ansible_distribution' was being set to 'Mandriva'
expecting to be corrected by the data from the dist file content.

But since the dist file parsing for Mandriva didn't match for
Ubuntu 10.04 /etc/lsb-release _and_ there is no Debian specific
lsb-release check, 'ansible_distribution' stayed at 'Mandriva'
and the dist file checking loop keeps going and eventually off
the end of the list before finding a better match.

Adding a debian/ubuntu specific check for /etc/lsb-release after
the debian os-release sets the info correctly and stops further
checking of dist files.

Fixes #30693
7 years ago
Adrian Likins 95abc1d82e Fix fact failures cause by ordering of collectors (#30777)
* Fix fact failures cause by ordering of collectors

Some fact collectors need info collected by other facts.
(for ex, service_mgr needs to know 'ansible_system').
This info is passed to the Collector.collect method via
the 'collected_facts' info.

But, the order the fact collectors were running in is
not a set order, so collectors like service_mgr could
run before the PlatformFactCollect ('ansible_system', etc),
so the 'ansible_system' fact would not exist yet. 

Depending on the collector and the deps, this can result
in incorrect behavior and wrong or missing facts.

To make the ordering of the collectors more consistent
and predictable, the code that builds that list is now
driven by the order of collectors in default_collectors.py,
and the rest of the code tries to preserve it.

* Flip the loops when building collector names

iterate over the ordered default_collectors list
selecting them for the final list in order instead
of driving it from the unordered collector_names set.

This lets the list returned by select_collector_classes
to stay in the same order as default_collectors.collectors

For collectors that have implicit deps on other fact collectors,
the default collectors can be ordered to include those early.

* default_collectors.py now uses a handful of sub lists of
collectors that can be ordered in default_collectors.collectors.

fixes #30753
fixes #30623
7 years ago
Adrian Likins 3eab636b3f Fix 'distribution' fact for ArchLinux (#30723)
* Fix 'distribution' fact for ArchLinux

Allow empty wasn't breaking out of the process_dist_files
loop, so a empty /etc/arch-release would continue searching
and eventually try /etc/os-release. The os-release parsing
works, but the distro name there is 'Arch Linux' which does
not match the 2.3 behavior of 'Archlinux'

Add a OS_RELEASE_ALIAS map for the cases where we need to get
the distro name from os-release but use an alias.

We can't include 'Archlinux' in SEARCH_STRING because a name match on its keys
but without a match on the content causes a fallback to using the first
whitespace seperated item from the file content as the name.
For os-release, that is in form 'NAME=Arch Linux'

With os-release returning the right name, this also supports the
case where there is no /etc/arch-release, but there is a /etc/os-release

Fixes #30600

* pep8 and comment cleanup
7 years ago
Adrian Likins 12404f470a Fix pkg_mgr fact on OpenBSD (#30725)
* Fix pkg_mgr fact on OpenBSD

Add a OpenBSDPkgMgrFactCollector that hardcodes pkg_mgr
to 'openbsd_pkg'. The ansible collector will choose the
OpenBSD collector if the system is OpenBSD and the 'Generic'
one otherwise.

This removes PkgMgrFactCollectors depenency on the
'system' fact being in collected_facts, which also
avoids ordering issues (if the pkg mgr fact is collected
before the system fact...)

Fixes #30623
7 years ago
Matt Clay 7714dcd04e Enable more pylint rules and fix reported issues. (#30539)
* Enable pylint unreachable test.
* Enable pylint suppressed-message test.
* Enable pylint redundant-unittest-assert test.
* Enable pylint bad-open-mode test.
* Enable pylint signature-differs test.
* Enable pylint unnecessary-pass test.
* Enable pylint unnecessary-lambda test.
* Enable pylint raising-bad-type test.
* Enable pylint logging-not-lazy test.
* Enable pylint logging-format-interpolation test.
* Enable pylint useless-else-on-loop test.
7 years ago
Dag Wieers 6a6f88d035 Clean up ACI error text (#30501)
* Clean up ACI error text
* Get rid of explicit unicode strings, use PY2
7 years ago
Toshio Kuratomi 1fa3fb45bc Update aci tests for new messages from lxml-4.0+ 7 years ago
Matt Clay 68aeaa58a8 Fix dangerous default args. (#29839) 7 years ago