* Fix unit test parametrize order on Python 3.5.
(cherry picked from commit 53b230ca74)
* Fix ansible-test unit test execution. (#45772)
* Fix ansible-test units requirements install.
* Run unit tests as unprivileged user under Docker.
(cherry picked from commit 379a7f4f5a)
* Run unit tests in parallel. (#45812)
(cherry picked from commit abe8e4c9e8)
* Minor fixes for unit test delegation.
(cherry picked from commit be199cfe90)
* add support for opening shell on remote Windows host (#43919)
* add support for opening shell on remote Windows host
* added arg completion and fix sanity check
* remove uneeded arg
(cherry picked from commit 6ca4ea0c1f)
* Block network access for unit tests in docker.
(cherry picked from commit 99cac99cbc)
* Make ansible-test available in the bin directory. (#45876)
(cherry picked from commit f3d1f9544b)
* Support comments in ansible-test flat files.
(cherry picked from commit 5a3000af19)
* Fix incorrect use of subprocess.CalledProcessError (#45890)
(cherry picked from commit 24dd87bd0a)
* Improve ansible-test match error handling.
(cherry picked from commit 2056c981ae)
* Improve error handling for docs-build test.
(cherry picked from commit 2148999048)
* Bug fixes and cleanup for ansible-test. (#45991)
* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.
(cherry picked from commit ac492476e5)
* Fix integration test library search path.
This prevents tests from loading modules outside the source tree,
which could result in testing the wrong module if a system-wide
install is present, or custom modules exist.
(cherry picked from commit d603cd41fe)
* Update default container to version 1.2.0.
(cherry picked from commit d478a4c3f6)
(cherry picked from commit 21c4eb8db5)
* Fix ansible-test docker python version handling.
This removes the old name based version detection behavior and
uses versions defined in the docker completion file instead, as
the new containers do not follow the old naming scheme.
(cherry picked from commit 54937ba784)
* Reduce noise in docs-build test failures.
(cherry picked from commit 4085d01617)
* Fix ansible-test encoding issues for exceptions.
(cherry picked from commit 0d7a156319)
* Fix ansible-test multi-group smoke test handling. (#46363)
* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.
(cherry picked from commit e53390b3b1)
* Use default-test-container version 1.3.0.
(cherry picked from commit 6d9be66418)
* Add file exists check in integration-aliases test.
(cherry picked from commit 33a8be9109)
* Improve ansible-test environment checking between tests. (#46459)
* Add unified diff output to environment validation.
This makes it easier to see where the environment changed.
* Compare Python interpreters by version to pip shebangs.
This helps expose cases where pip executables use a different
Python interpreter than is expected.
* Query `pip.__version__` instead of using `pip --version`.
This is a much faster way to query the pip version. It also more
closely matches how we invoke pip within ansible-test.
* Remove redundant environment scan between tests.
This reuses the environment scan from the end of the previous test
as the basis for comparison during the next test.
(cherry picked from commit 0dc7f38787)
* Add symlinks sanity test. (#46467)
* Add symlinks sanity test.
* Replace legacy test symlinks with actual content.
* Remove dir symlink from template_jinja2_latest.
* Update import test to use generated library dir.
* Fix copy test symlink setup.
(cherry picked from commit e2b6047514)
* Fix parametrize warning in unit tests.
(cherry picked from commit 1a28898a00)
* Update MANIFEST.in (#46502)
* Update MANIFEST.in:
- Remove unnecessary prune.
- Include files needed by tests.
- Exclude botmeta sanity test.
These changes permit sanity tests to pass on sdist output.
(cherry picked from commit cbb49f66ec)
* Fix unit tests which modify the source tree. (#45763)
* Fix CNOS unit test log usage.
* Use temp dir for Galaxy unit tests.
* Write to temp files in interfaces_file unit test.
* Fix log placement in netapp_e_ldap unit test.
(cherry picked from commit 0686450cae)
* Fix ansible-test custom docker image traceback.
(cherry picked from commit 712ad9ed64)
* ansible-test: Create public key creating Windows targets (#43760)
* ansible-test: Create public key creating Windows targets
* Changed to always set SSH Key for Windows hosts
(cherry picked from commit adc0efe10c)
* Fix and re-enable sts_assume_role integration tests (#46026)
* Fix the STS assume role error message assertion when the role to assume does not exist.
(cherry picked from commit 18dc928e28)
* Fix ACI unit test on Python 3.7.0.
The previous logic was only needed for pre-release versions of 3.7.
(cherry picked from commit c0bf9815c9)
* Remove placeboify from unit tests that are not calling AWS (i.e. creating a recording) (#45754)
(cherry picked from commit 2167ce6cb6)
* Update sanity test ignore entries.
When parsing the distribution files such as /etc/os-release, we extract
the full distribution version but not the major version. As such, the
ansible_distribution_major_version ends up being 'NA' whereas the
ansible_distribution_version contains the full version.
Before this patch we get this on openSUSE Leap 15
ansible -o localhost -m setup -a filter=ansible_distribution_major_version
localhost | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "NA"}, "changed": false}
After this patch we get this
ansible -o localhost -m setup -a filter=ansible_distribution_major_version
localhost | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "15"}, "changed": false}
This also fixes the Tumbleweed distribution test to report a proper
major version and also adds a test for openSUSE Leap 15.0 to avoid
potential future regressions.
Fixes: #41410
(cherry picked from commit 1737b7be3e)
False assumption that values can not have cyclic dependencies. Fix by
removing dependency on self and look for cycles, if found remove
dependency to get a partial sort done.
(cherry picked from commit 042c111563)
Fix issues in ValueBuilder used in nso_config and nso_verify so that it
can handle leaf-list in NSO 4.5 and detect identityref types from
unions.
Fail gracefully if a type is not found.
(cherry picked from commit 2789cc5c09)
* allows ib_spec attrs to be filtered in update
This change will allow the ib_spec entries to be be filtered on a change
object by setting the update keyword to false. The default value for
update is true. When the update keyword is set to false, the keyed
entry will be removed from the update object before it is sent to the
api endpoint.
fixes#36563
* fix up pep8 issues
(cherry picked from commit 93b795baf0)
* basic: allow one or more when param list having choices
* add unit tests
* optimize a bit
* re-add get_exception import
* a number of existing modules expect to be able to get it from basic.py
* ACI: Change result output as discussed
* Update all modules to use new aci.exit_json()
* Update output_level spec and docs
* Fix integration tests
* Small PEP8 fix
* Asorted fixes to tests and aci_rest
* More test fixes and support for ANSIBLE_DEBUG
* Fix another PEP8 issues
* Move response handling inside ACI module
* Reform of ACI error handling and error output
* Diff multiline json output
* Fix a few more tests
* Revert aci_bd tests
* Small correction
* UI change: existing->current, original->previous
* UI change: config->sent
* Update all modules with RETURN values
* Fix a few more tests
* Improve docstring and add 'raw' return value
* Fix thinko
* Fix sanity/pep8 issues
* Rewrite unit tests to comply with new design
* refactors nios api shared code to handle provider better
This change refactors the shared code to be easily shared between
modules, plugins and dynamic inventory scripts. All parts now implement
the provider arguments uniformly.
This also provides a centralized fix to suppress urllib3 warnings coming
from the requests library implemented by infoblox_client
* fix up pep8 errors
* fix missing var name
Add deps/requires for fact collectors
Fact collectors can now set a required_facts
class attribute that will be a set of the names
of fact collectors they require to be run first.
ie, if a collector needs to know the ansible_distribution,
it should set it's required_facts to include 'distribution'
required_facts = set(['distribution'])
If a collector requires another collector, it gets added
to the selected collector names.
We then topological sort the ordering of the collectors
so that deps work out (ie, 'distribution' will run before
'service_mgr')
required_facts were added to the collectors for:
- network (requires 'distribution', 'platform')
- hardware (requires 'platform')
- service_mgr (requires 'distribution', 'platform')
Fix name references for facts (need 'ansible_' prefix)
is service_mgr
Fixes#30753
The accumulated collected_facts was being update
with new facts _after_ filtering them. So only
facts that pass the filter would ever be passed
to other fact collectors.
For 'filter=ansible_service_mgr', even though it requires
the platform and distribution facts and even collects them,
they would get filtered out and never passed to the other
collectors that need them (service_mgr for ex).
Fix is just to add the unfiltered facts to collected_facts.
Adds unit tests for fact filter and collected_facts.
Fixes#32286
The search string used to look for Clear Linux
was changed in 45a9f96774 to
be more specific, but was too specific. Now finding
a substring match for 'Clear Linux' in /usr/lib/os-release
is enough to consider a match.
Since the details of the full name in os-release varies
('Clear Linux Software for Intel Architecture',
'Clear Linux OS for Intel Architecture', etc) the
search string match was failing and would fall back to the
'first word in the release file' method resulting in
ansible_distribution='NAME="Clear'
Also add a meta fact indicating which search string
was matched.
Test case info from:
https://github.com/ansible/ansible/issues/31501#issuecomment-340861535Fixes#31501
* Allow protection of certain keys during camel_to_snake
Create an `ignore_list` parameter that preserves the case
of the contents of certain dictionaries. Most valuable
for `tags` but other uses might arise.
* Port ec2_vpc_route_table to boto3
Update tests to reflect fixes in boto3.
* Add RETURN documentation to ec2_vpc_route_table
* Update DOCUMENTATION to be valid yaml
* Add check mode tests
* basic.py: add mock to os.path.exists
* set_*_if_different: if check_mode enabled & file missing: set changed to True
Fixes#32676
Thanks to mscherer and Spredzy for the distributed triplet programming
session!
* update version parsing and move requirements to nso_* modules
prepare for introduction of nso_show module that has other version
requirements than the existing nso_* modules.
* Add nso_show module for retreiving config from Cisco NSO
New module that supports getting configuration and operational data
from Cisco NSO.
* WIP adds network subnetting functions
* adds functions to convert between netmask and masklen
* adds functions to verify netmask and masklen
* adds function to dtermine network and subnet from address / mask pair
* network_common: add a function to get the first 48 bits in a IPv6 address.
ec2_group: only use network bits of a CIDR.
* Add tests for CIDRs with host bits set.
* ec2_group: add warning if CIDR isn't the networking address.
* Fix pep8.
* Improve wording.
* fix import for network utils
* Update tests to use pytest instead of unittest
* add test for to_ipv6_network()
* Fix PEP8
Split the one monolithic test for basic.py into several files
* Split test_basic.py along categories.
This is preliminary to get a handle on things. Eventually we may want
to further split it so each file is only testing a single function.
* Cleanup unused imports from splitting test_basic.py
* Port atomic_move test to pytest.
Working on getting rid of need to maintain procenv
* Split a test of symbolic_mode_to_octal to follow unittest best practices
Each test should only invoke the function under test once
* Port test_argument_spec to pytest.
* Fix suboptions failure
Allow CamelCase version of snake_dict_to_camel_dict
(currently only dromedaryCase is supported)
Add reversible option to camel_dict_to_snake_dict
Add tests for both of these options
* Deprecate Entity, EntityCollection and use subspec in network modules
* As per proposal https://github.com/ansible/proposals/issues/76
deprecate use of Entity, EntityCollection, ComplexDict, ComplexList
and use subspec instead.
* Refactor ios modules
* Refactor eos modules
* Refactor vyos modules
* Refactor nxos modules
* Refactor iosxr modules
* Add support for key in suboptions handling
* Fix CI issues
* Porting tests to pytest
* Achievement Get: No longer need mock/generator.py
* Now done via pytest's parametrization
* Port safe_eval to pytest
* Port text tests to pytest
* Port test_set_mode_if_different to pytest
* Change conftest AnsibleModule fixtures to be more flexible
* Move the AnsibleModules fixtures to module_utils/conftest.py for sharing
* Testing the argspec code requires:
* injecting both the argspec and the arguments.
* Patching the arguments into sys.stdin at a different level
* More porting to obsolete mock/procenv.py
* Port run_command to pytest
* Port known_hosts tests to pytest
* Port safe_eval to pytest
* Port test_distribution_version.py to pytest
* Port test_log to pytest
* Port test__log_invocation to pytest
* Remove unneeded import of procenv in test_postgresql
* Port test_pip to pytest style
* As part of this, create a pytest ansiblemodule fixture in
modules/conftest.py. This is slightly different than the
approach taken in module_utils because here we need to override the
AnsibleModule that the modules will inherit from instead of one that
we're instantiating ourselves.
* Fixup usage of parametrization in test_deprecate_warn
* Check that the pip module failed in our test
* Refactor common network shared and platform specific code into package (part-1)
As per proposal #76 refactor common network shared and platform specific
code into sub-package.
https://github.com/ansible/proposals/issues/76
* ansible.module_utils.network.common - command shared functions
* ansible.module_utils.network.{{ platform }} - where platform is platform specific shared functions
* Fix review comments
* Fix review comments
* jsonify inventory
* smarter import, dont pass kwargs where not needed
* added datetime
* Eventual plan for json utilities to migrate to common/json_utils when we split
basic.py no need to move jsonify to another file now as we'll do that later.
* json_dict_bytes_to_unicode and json_dict_unicode_to_bytes will also
change names and move to common/text.py at that time (not to json).
Their purpose is to recursively change the elements of a container
(dict, list, set, tuple) into text or bytes, not to json encode or
decode (they could be a generic precursor to that but are not limited
to that.)
* Reimplement the private _SetEncoder which changes sets and datetimes
into objects that are json serializable into a private function
instead. Functions are more flexible, less overhead, and simpler than
an object.
* Remove code that handled simplejson-1.5.x and earlier. Raise an error
if that's the case instead.
* We require python-2.6 or better which has the json module builtin to
the stdlib. So this is only an issue if the stdlib json has been
overridden by a third party module and the simplejson on the system
is 1.5.x or less. (1.5 was released on 2007-01-18)
* Make ansible_selinux facts a consistent type
Rather than returning a bool if the Python library is missing, return a dict with one key containing a message explaining there is no way to tell the status of SELinux on the system becasue the Python library is not present.
* Fix unit test
The /etc/os-release based distro detection doesn't
seem to work for Ubuntu 10.04 (no /etc/os-release?).
So it was testing the next case which was /etc/lsb-release to
see if it is 'Mandriva'. Since the check for existence of
(/etc/lsb-release, Mandrive) was the first non-empty dist
file match, 'ansible_distribution' was being set to 'Mandriva'
expecting to be corrected by the data from the dist file content.
But since the dist file parsing for Mandriva didn't match for
Ubuntu 10.04 /etc/lsb-release _and_ there is no Debian specific
lsb-release check, 'ansible_distribution' stayed at 'Mandriva'
and the dist file checking loop keeps going and eventually off
the end of the list before finding a better match.
Adding a debian/ubuntu specific check for /etc/lsb-release after
the debian os-release sets the info correctly and stops further
checking of dist files.
Fixes#30693
* Fix fact failures cause by ordering of collectors
Some fact collectors need info collected by other facts.
(for ex, service_mgr needs to know 'ansible_system').
This info is passed to the Collector.collect method via
the 'collected_facts' info.
But, the order the fact collectors were running in is
not a set order, so collectors like service_mgr could
run before the PlatformFactCollect ('ansible_system', etc),
so the 'ansible_system' fact would not exist yet.
Depending on the collector and the deps, this can result
in incorrect behavior and wrong or missing facts.
To make the ordering of the collectors more consistent
and predictable, the code that builds that list is now
driven by the order of collectors in default_collectors.py,
and the rest of the code tries to preserve it.
* Flip the loops when building collector names
iterate over the ordered default_collectors list
selecting them for the final list in order instead
of driving it from the unordered collector_names set.
This lets the list returned by select_collector_classes
to stay in the same order as default_collectors.collectors
For collectors that have implicit deps on other fact collectors,
the default collectors can be ordered to include those early.
* default_collectors.py now uses a handful of sub lists of
collectors that can be ordered in default_collectors.collectors.
fixes#30753fixes#30623
* Fix 'distribution' fact for ArchLinux
Allow empty wasn't breaking out of the process_dist_files
loop, so a empty /etc/arch-release would continue searching
and eventually try /etc/os-release. The os-release parsing
works, but the distro name there is 'Arch Linux' which does
not match the 2.3 behavior of 'Archlinux'
Add a OS_RELEASE_ALIAS map for the cases where we need to get
the distro name from os-release but use an alias.
We can't include 'Archlinux' in SEARCH_STRING because a name match on its keys
but without a match on the content causes a fallback to using the first
whitespace seperated item from the file content as the name.
For os-release, that is in form 'NAME=Arch Linux'
With os-release returning the right name, this also supports the
case where there is no /etc/arch-release, but there is a /etc/os-release
Fixes#30600
* pep8 and comment cleanup
* Fix pkg_mgr fact on OpenBSD
Add a OpenBSDPkgMgrFactCollector that hardcodes pkg_mgr
to 'openbsd_pkg'. The ansible collector will choose the
OpenBSD collector if the system is OpenBSD and the 'Generic'
one otherwise.
This removes PkgMgrFactCollectors depenency on the
'system' fact being in collected_facts, which also
avoids ordering issues (if the pkg mgr fact is collected
before the system fact...)
Fixes#30623
* Added the ability to extend the exception list in CloudRetry
* AWSRetry boto and boto compatible
* Updated tests to reflect boto/boto3
* Added boto to shippable requirements
* Have base_class and added_exceptions default to None in CloudRetry
AWSRetry - only retry on boto3 exceptions and remove boto requirement from tests
* Make requested changes.
* Fix for issue ansible/ansible#27715
* Also fixing mutually exclusive check
* Updating subspec checks
These changes take into account a spec with all features enabled and do
the following tests for subspecs:
1. Test proper specs
2. Test Alias
3. Test missing required param
4. Test mutually exclusive params
5. Test required if params
6. Test required one of params
7. Test required together params
8. Test required if params with a default value
9. Test basis subspec params
10. Test invalid subsec params
* adds new filter plugins for network use cases
* adds parse_cli filter
* adds parse_cli_textfsm filter
* adds Template class to network_common
* adds conditional function to network_common
* fix up PEP8 issues
previously gather_subset=['!all'] would still gather the
min set of facts, and there was no way to collect no facts.
The 'min' specifier in gather_subset is equilivent to
exclude the minimal_gather_subset facts as well.
gather_subset=['!all', '!min'] will collect no facts
This also lets explicitly added gather_subsets override excludes.
gather_subset=['pkg_mgr', '!all', '!min'] will collect only the pkg_mgr
fact.
Matt Clay