* Making tests more uniform
Removing unnecessary GalaxyCLI arguments/patching of the command line since parsing of the CLI args has been modified.
Run GalaxyCLI.parse() without saving the returned value to be uniform with the rest of the code.
Fix test_execute_remove to use the correct path
Use GalaxyCLI.run() instead of super(GalaxyCLI, gc).run() and GalaxyCLI.api = ansible.galaxy.api.GalaxyAPI(gc.galaxy).
* Refactor so one unit test checks one thing instead of multiple.
Improve readability by using a dict instead of lots of elifs
* Removing import used for debugging
* Fixing PEP 8 issues.
* Fix PEP 8 issues
* Don't check for var._obj in template._clean_data
AnsibleUnsafe or other unsafe vars used to have a
'_obj' slot but no longer do. This was causing attribute
errors if a object was 'unsafe' but not a string.
Add tests for AnsibleUnsafe, lookups, and AnsibleContext
* Fix 'yum skips updates' bug
When the 'yum check-update' output is parsed, the regex used
to stitch wrapped lines together would fail on the first package.
It would fail because there is an empty line before the first
package, and this triggered the regex. To avoid a more complicated
regex, preprocess the check-update output to strip out any
empty lines.
The regex is also updated to include a group on the non whitespace
match to be used in the sub.
Add test cases based on info provided in the bug reports.
Fixes#20608
* known_hosts: support --diff
* known_hosts: support --diff also without --check
* Add unit tests and fix incorrect diff in one corner case
Tests are good!
* Refactor for readability
* Python 3 compat
* More Python 3 compat
* Add an integration test for known_hosts
* Handle ssh-keygen -HF returning non-zero exit code
AFAICT this is a bug in ssh-keygen in some newer OpenSSH versions
(>= 6.4 probably; see commit dd9d5cc670):
when you invoke ssh-keygen with -H and -F <host> options, it always
returns exit code 1. This is because in ssh-keygen.c there's a function
do_known_hosts() which calls
exit (find_host && !ctx.found_key);
at the end, and find_host is 1 (because we passed -F on the command line),
but ctx.found_key is always 0. Why is found_key always 0? Because the
callback passed to hostkeys_foreach(), which is known_hosts_hash(),
never bothers to set found_key to 1.
* This test does not need root
* Avoid ssh-ed25519 keys in sample known_hosts file
Older versions of OpenSSH do not like them and ssh-keygen -HF
aborts with an error when it sees such keys:
line 5 invalid key: example.net...
/root/ansible_testing/known_hosts is not a valid known_hosts file.
* Fix Python 3 errors
Specifically, the default mode of tempfile.NamedTemporaryFile is 'w+b',
which means Python 3 wants us to write bytes objects to it -- but the
keys we have are all unicode strings.
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
* added docs for vault and made trigger shorter: !vault
* added single var valuting
* Update playbooks_vault.rst
Edit pass for spelling and grammar. Ship it!
* Update playbooks_vault.rst
Typo fixes.
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.
* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
select on shippable
Fixes#14143
* returns support for prompt/response over cli
* now sends native dict instead of str command
* fixes issue with run_commands() in ios to jsonify request
* updates unit test cases
* Rename vyos2 over vyos
* Update vyos_config to LocalAnsibleModule
Change result key 'updates' -> 'commands'
vyos_config is supported by core
* vyos_config tests
* Enable bracket config use
* Sanitize config before use
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.
Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).
Rename snippet* variables to module_utils*
Add a small number of unittests for recursive_finder
Add a larger number of integration tests to demonstrate that
module_utils is working.
Whitelist module-style shebang in test target library dirs
Prefix module_data variable with b_ to be clear that it holds bytes data
* Unittests for some of module_common.py
* Port test_run_command to use pytest-mock
The use of addCleanup(patch.stopall) from the unittest idiom was
conflicting with the pytest-mock idiom of closing all patches
automatically. Switching to pytest-mock ensures that the patches are
closed and removing the stopall stops the conflict.
* provides declarative resource for configuring eos system parameters
* supports network_cli connection plugin only
* adds unit tests for module using mock interfaces
* update eos_config to use eapi exclusively and remove cli transport
* add unit test cases for eos_config
* updates action plugin to handle both eapi and network_cli connections
* eos module now uses network_cli connection plugin
* adds unit tests for eos module
* eapi support now provided by eapi module
* updates doc fragment for eapi common properties
* Google Cloud Pubsub Module
The Google Cloud Pubsub module allows the Ansible user to:
* Create/Delete Topics
* Create/Delete Subscriptions
* Change subscription from pull to push (and configure endpoint)
* Publish messages to a topic
* Pull messages from a Subscription
An accessory module, gcpubsub_facts, has been added to list topics and subscriptions.
* Added docs for state field to DOCUMENTATION and RETURN blocks.
Rather than trying to enumerate tasks or track an ever changing cur_role
flag in PlayIterator, this change simply sets a flag on the last block in
the list of blocks returned by Role.compile(). The PlayIterator then checks
for that flag when the cur_block number is incremented, and marks the role
as complete if the given host had any tasks run in that role.
Fixes#20224
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
* Fix a test failure on Python 3.6
tox -e py36 failed with
======================================================================
ERROR: test_action_base__execute_module (units.plugins.action.test_action.TestActionBase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/mg/src/ansible/test/units/plugins/action/test_action.py", line 507, in test_action_base__execute_module
self.assertEqual(action_base._execute_module(), dict(_ansible_parsed=True, rc=0, stdout="ok", stdout_lines=['ok']))
File "/home/mg/src/ansible/lib/ansible/plugins/action/__init__.py", line 596, in _execute_module
remote_module_path = self._connection._shell.join_path(tmp, remote_module_filename)
File "/home/mg/opt/python36/lib/python3.6/unittest/mock.py", line 939, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/home/mg/opt/python36/lib/python3.6/unittest/mock.py", line 1005, in _mock_call
ret_val = effect(*args, **kwargs)
File "/home/mg/src/ansible/.tox/py36/lib/python3.6/posixpath.py", line 92, in join
genericpath._check_arg_types('join', a, *p)
File "/home/mg/src/ansible/.tox/py36/lib/python3.6/genericpath.py", line 149, in _check_arg_types
(funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'MagicMock'
because os.path.join() now checks argument types since Python 3.6 (due
to pathlib support, I expect).
* Use a more realistic module name in test
The behavior now matches GNU diff.
Fixes#14094.
Example of output before this change:
TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
changed: [ranka]
--- before: /etc/cron.hourly/mg-healthchecks-dot-io
+++ after: /tmp/tmpOTvXTw
@@ -1,2 +1,2 @@
#!/bin/sh
-curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null+curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
after this change:
TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
changed: [ranka]
--- before: /etc/cron.hourly/mg-healthchecks-dot-io
+++ after: /tmp/tmpOTvXTw
@@ -1,2 +1,2 @@
#!/bin/sh
-curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
\ No newline at end of file
+curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
The added unit tests contain more examples.
This commit also takes care to avoid "no newline at EOF" warnings when
no_log is in effect, and also when modules return dicts rather than
strings. (It also removes trailing whitespace from using json
serialization when diffing dicts, because I hate trailing whitespace in
Python source files, even if they're test files.)
* updates the ios_config module to use the network_cli plugin
* updates the local action plugin to derive from network
* add unit test cases for ios_config
* updates the deprecated ios_template module to use network_cli
* adds unit test cases for ios_template
* adds check for provider argument and displays warning message
* Add a encode() to AnsibleVaultEncryptedUnicode
Without it, calling encode() on it results in a bytestring
of the encrypted !vault-encrypted string.
ssh connection plugin triggers this if ansible_password
is from a var using !vault-encrypted. That path ends up
calling .encode() instead of using the __str__.
Fixes#19795
* Fix str.encode() errors on py2.6
py2.6 str.encode() does not take keyword arguments.
Support for the Google API and GCloud-Python Clients have been added.
The three libraries:
* GCloud-Python: A new function, get_google_cloud_credentials, should be used. The credentials-object returned can be passed to any gcloud-python client. Using this client library requires in the installation of gcloud-python. This is preferred library for new modules.
* Google API: A new function, gcp_api_auth, should be used to take advantage of services requiring this client. This client library should be used if the desired functionality is not available in GCloud-Python. Using this library requires the installation of google-api-python-client.
* libcloud: Existing function, gcp_connect, should be used. The interface and return values have not changed and existing modules (such as gce, gce_pd and gce_net) should work without modification. Note that the credentials-fetching code has been refactored out of gcp_connect so that can be reused by all connection functions. To use this function, apache-libcloud must be installed.
Import guards have been added and will only be trigger if a user tries to use a function that is missing dependencies.
Credential-specifying mechanisms (i.e, ansible module params, env vars and libcloud secrets.py) have not changed. They have been refactored and unit tests have been added to allow for changes going forward. We are deprecating (and removing in a subsequent release) the ability to specify credentials via the libcloud secrets file. Also, we have deprecated (and also plan to remove in a subsequent release) the ability to use a p12 pem file for a key - the JSON format is strongly preferred. Deprecation warnings have been added for both of these issues (see the Ansible docs on how to disable deprecation warnings).
The gce_tag module can support updating tags on multiple instances via an instance_pattern field. Full Python regex is supported in the instance_pattern field.
'instance_pattern' and 'instance_name' are mutually exclusive and one must be specified.
The integration test for the gce_tag module has been updated to support the instance_pattern parameter. Unit tests have been added to test the list-manipulation functionality.
Run the integration test with:
TEST_FLAGS='--tags "test_gce_tag"' make gce
Run the unit tests with:
python test/units/modules/cloud/google/test_gce_tag.py
This plugin can be used with the lpass cli interface for lastpass.
[lastpass-cli](https://github.com/lastpass/lastpass-cli)
Example:
Add a lookup to your playbooks/variables somewhere:
```
some_variable: "{{ lookup('lastpass','Some Lastpass entry name or ID', field='username') }}"
```
Usage:
* start a lpass session prior to using ansible
* run ansible
* logout when finished
```
lpass login user@domain.com
ansible-playbook foo.yml
lpass logout
```
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.
This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.
Fixes#18289
* Fix synchronize retries
The synchronize module munges its task args on every invocation of
run(). This was problematic because the munged data was not fit for use
by a second pass of the synchronize module. Correct this by using a copy
of the task args on every invocation of run() so that the original args
are not affected.
Local testing using this playbook seems to confirm that things work as
expected:
- hosts: all
tasks:
- delay: 2
register: task_result
retries: 1
until: task_result.rc == 0
synchronize:
dest: /tmp/out
mode: pull
src: /tmp/nonexistent/
fixes#18281
* Update synchroncization fixture assertions
When we started operating on a copy of the task args the test assertions
were no longer asserting things about the munged state but of the
pristine state. Convert the copy of task args to a class member so that
it can be compared against later in testing and update the assertions to
check this munged copy.
* Shuffle objects around for cleaner testing
Attach the temporary args dict to the task rather than the action as
this makes updating the existing tests cleaner.
This adds back the change to the network_cli plugin. Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.
The issue was due to the addition of the decorator that will call
_connect() when there is no connection. The _connect() method should
have been mocked in the test case. This commit fixes the test
case as well
Change was originally reverted in c414ded69a
* make hash_params more robust in the face of many corner cases
Fixes#18680
Alternative fix to #18681
* add test case for role.hash_params
* Add role.hash_params test for more types
A set, a generator/iterable, and a Container that
is not Iterable.
* wip: add a unit test for playbook/base.py
This commit include a failing test
TestBaseSubClass.test_attr_class_post_validate
It fails with the error:
Traceback (most recent call last):
File "/home/adrian/src/ansible/test/units/playbook/test_base.py", line 264, in test_attr_class_post_validate
bsc = self._base_validate(ds)
File "/home/adrian/src/ansible/test/units/playbook/test_base.py", line 206, in _base_validate
bsc.post_validate(templar)
File "/home/adrian/src/ansible/lib/ansible/playbook/base.py", line 450, in post_validate
" Error was: %s" % (name, value, attribute.isa, e), obj=self.get_ds())
AnsibleParserError: the field 'test_attr_class_post_validate' has an invalid value (<class 'units.playbook.test_base.ExampleSubClass'>), and could not be converted to an class. Error was: test_attr_class_post_validate is not a valid <class 'units.playbook.test_base.ExampleSubClass'> (got a <class 'ansible.playbook.base.BaseMeta'> instead)
* wip, test refactoring
* wip, trying to add a parent->child
* wip, fix isa=class.
the ds the base using needs an instance of the class
(ie, whats normally created by the yaml loaders)
* wip, theres no need to argue, I just dont understand parents
* stub a _preprocess_data for coverage
* cleanup, required, parent, etc
* Make sure include_role inherit variables from parent role
Setting the parent of task blocks generated by include_role after they
have been produced is not sufficient - it means the tasks don't have the
correct dependency chain set afterwards, and therefore, don't properly
inherit variables from outer roles.
In addition to manually setting the parents, pass the dep_chain when
compiling the role, such that variables are correctly imported.
Fixes#18540.
* Add tests for include_role
* Fix include_role variable inheritance for multiple parent levels
* Replace pipes.quote for shlex_quote
* More migration of pipes.quote to shlex_quote
Note that we cannot yet move module code over. Modules have six-1.4
bundled which does not have shlex_quote. This shouldn't be a problem as
the function is still importable from pipes.quote. It's just that this
has become an implementation detail that makes us want to import from
shlex instead.
Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
If the current ansible enviroment has a config setup
that doesn't use 'smart' as the configured transport
test_play_context would fail when it assumes the
transport will be 'smart'.
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.
This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.
Fixes#17983
- Use assertRaisesRegexp to make sure correct exceptions are raised.
- Set docker_command to avoid docker dependency (skips find_executable).
- Use a fake path for docker_command to make sure mock.patch is working.
* Fix bug (#18355) where encrypted inventories fail
This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes
The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).
So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().
The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.
Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.
_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.
Fixes#18355
- Remove shebangs from:
- ini files
- unit tests
- module_utils
- plugins
- module_docs_fragments
- non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.
Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.
A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
- Update import for relocated tests.
- Fix test to expect changed from update_tags.
- Add checks for boto3 and botocore to tests.
- Set check mode with kwarg.
- Python 3 fixes for unit tests.
- Python 2.6 fix for unit tests.
- Add missing meta value for test_create_server
- Add .gitignore for pytest .cache directory
Exclude test_os_server from nose test runs since it was designed
for pytest. The test will work correctly when run using pytest.
This is a temporary issue, as we'll be moving to pytest soon.
Matt Clay