* Fix synchronize retries
The synchronize module munges its task args on every invocation of
run(). This was problematic because the munged data was not fit for use
by a second pass of the synchronize module. Correct this by using a copy
of the task args on every invocation of run() so that the original args
are not affected.
Local testing using this playbook seems to confirm that things work as
expected:
- hosts: all
tasks:
- delay: 2
register: task_result
retries: 1
until: task_result.rc == 0
synchronize:
dest: /tmp/out
mode: pull
src: /tmp/nonexistent/
fixes#18281
* Update synchroncization fixture assertions
When we started operating on a copy of the task args the test assertions
were no longer asserting things about the munged state but of the
pristine state. Convert the copy of task args to a class member so that
it can be compared against later in testing and update the assertions to
check this munged copy.
* Shuffle objects around for cleaner testing
Attach the temporary args dict to the task rather than the action as
this makes updating the existing tests cleaner.
This adds back the change to the network_cli plugin. Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.
The issue was due to the addition of the decorator that will call
_connect() when there is no connection. The _connect() method should
have been mocked in the test case. This commit fixes the test
case as well
Change was originally reverted in c414ded69a
* make hash_params more robust in the face of many corner cases
Fixes#18680
Alternative fix to #18681
* add test case for role.hash_params
* Add role.hash_params test for more types
A set, a generator/iterable, and a Container that
is not Iterable.
* wip: add a unit test for playbook/base.py
This commit include a failing test
TestBaseSubClass.test_attr_class_post_validate
It fails with the error:
Traceback (most recent call last):
File "/home/adrian/src/ansible/test/units/playbook/test_base.py", line 264, in test_attr_class_post_validate
bsc = self._base_validate(ds)
File "/home/adrian/src/ansible/test/units/playbook/test_base.py", line 206, in _base_validate
bsc.post_validate(templar)
File "/home/adrian/src/ansible/lib/ansible/playbook/base.py", line 450, in post_validate
" Error was: %s" % (name, value, attribute.isa, e), obj=self.get_ds())
AnsibleParserError: the field 'test_attr_class_post_validate' has an invalid value (<class 'units.playbook.test_base.ExampleSubClass'>), and could not be converted to an class. Error was: test_attr_class_post_validate is not a valid <class 'units.playbook.test_base.ExampleSubClass'> (got a <class 'ansible.playbook.base.BaseMeta'> instead)
* wip, test refactoring
* wip, trying to add a parent->child
* wip, fix isa=class.
the ds the base using needs an instance of the class
(ie, whats normally created by the yaml loaders)
* wip, theres no need to argue, I just dont understand parents
* stub a _preprocess_data for coverage
* cleanup, required, parent, etc
* Make sure include_role inherit variables from parent role
Setting the parent of task blocks generated by include_role after they
have been produced is not sufficient - it means the tasks don't have the
correct dependency chain set afterwards, and therefore, don't properly
inherit variables from outer roles.
In addition to manually setting the parents, pass the dep_chain when
compiling the role, such that variables are correctly imported.
Fixes#18540.
* Add tests for include_role
* Fix include_role variable inheritance for multiple parent levels
* Replace pipes.quote for shlex_quote
* More migration of pipes.quote to shlex_quote
Note that we cannot yet move module code over. Modules have six-1.4
bundled which does not have shlex_quote. This shouldn't be a problem as
the function is still importable from pipes.quote. It's just that this
has become an implementation detail that makes us want to import from
shlex instead.
Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
If the current ansible enviroment has a config setup
that doesn't use 'smart' as the configured transport
test_play_context would fail when it assumes the
transport will be 'smart'.
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.
This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.
Fixes#17983
- Use assertRaisesRegexp to make sure correct exceptions are raised.
- Set docker_command to avoid docker dependency (skips find_executable).
- Use a fake path for docker_command to make sure mock.patch is working.
* Fix bug (#18355) where encrypted inventories fail
This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes
The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).
So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().
The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.
Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.
_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.
Fixes#18355
- Remove shebangs from:
- ini files
- unit tests
- module_utils
- plugins
- module_docs_fragments
- non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.
Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.
A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
- Update import for relocated tests.
- Fix test to expect changed from update_tags.
- Add checks for boto3 and botocore to tests.
- Set check mode with kwarg.
- Python 3 fixes for unit tests.
- Python 2.6 fix for unit tests.
- Add missing meta value for test_create_server
- Add .gitignore for pytest .cache directory
Exclude test_os_server from nose test runs since it was designed
for pytest. The test will work correctly when run using pytest.
This is a temporary issue, as we'll be moving to pytest soon.
- Correct directory name in test/README.md
- Move code-smell tests to test/sanity/code-smell
- Update code-smell.sh to use new script paths
- Add test/integration/target-prefixes.win for ansible-test
- Move module unit tests to match module directory layout
In order to support legacy plugins, the following two method signatures
are allowed for `CallbackBase.v2_playbook_on_start`:
def v2_playbook_on_start(self):
def v2_playbook_on_start(self, playbook):
Previously, the logic to handle this divergence checked to see if the
callback plugin being called supported an argument named `playbook`
in its `v2_playbook_on_start` method. This was fragile in a few ways:
- if a plugin author did not use the literal `playbook` to name their
method argument, their plugin would not be called correctly
- if a plugin author wrapped their `v2_playbook_on_start` method and
by doing so changed the argspec to no longer expose an argument
with that literal name, their plugin would not be called correctly
In order to continue to support both types of callback for backwards
compatibility while making the call more robust for plugin authors,
the logic can be reversed in order to have a positive check for the old
method signature instead of a positive check for the new one.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
As neon is derived from Ubuntu, ansible_os_family should have the value
"Debian" instead of "Neon". Add a test case for KDE neon and set
os_family correctly for it.
Nothing seems to use this now.
Was added originally added in2d11cfab92f9d26448461b4bc81f466d1910a15e
but the code that used it was removed in
e02b98274b
On openSUSE Tumbleweed, lsb-release -a currently reports
the distributor ID as "openSUSE Tumbleweed". On openSUSE
Leap, the distributor ID is "SUSE LINUX".
Add them to the OS_FAMILY dict as Suse family systems.
Also add an entry to TESTSETS in test_distribution_version.py
for openSUSE Tumbleweed.
Implement tag and skip_tag handling in the CLI() class. Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.
* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
default will change in 2.4 and go away in 2.5.
* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
Prior to this commit, the ini parser would fail if the inventory was
not 100% utf-8. This commit makes this slightly more robust by
omitting full line comments from that requirement.
Fixes#17593
If the sftp fails, roll over to scp by default. This saves users
from having to know about the scp_if_ssh method when sftp is broken
on the remote host.
Python2 seems to allow any integer. Python3.5 on Linux seems to allow
a 32 bit unsigned int. Python3.5 on El Capitan seems to limit it to
a smaller size... perhaps a 16 bit int.
This adds some test data to test_facts.py that
includes mnt points that have a single quote in
the path.
Ala, https://github.com/ansible/ansible/issues/16855
The bug was already fixed via other changes, but this is
for regression testing.
A parameter of type int should accept int and string, but not float.
A parameter of type float should accept float, int, and string.
Also reset the arguments in another test so that it runs cleanly. This
agrees with what all the other tests are doing.
* Improve unit testing of 'password' lookup
The tests showed some UnicodeErrors for the
cases where the 'chars' param include unicode,
causing the 'getattr(string, c, c)' to fail.
So the candidate char generation code try/excepts
UnicodeErrors there now.
Some refactoring of the password.py module to make
it easier to test, and some new tests that cover more
of the password and salt generation.
* More refactoring and fixes.
* manual merge of text enc fixes from pr17475
* moving methods to module scope
* more refactoring
* A few more text encoding fixes/merges
* remove now unused code
* Add test cases and data for _gen_candidate_chars
* more test coverage for password lookup
* wip
* More text encoding fixes and test coverage
* cleanups
* reenable text_type assert
* Remove unneeded conditional in _random_password
* Add docstring for _gen_candidate_chars
* remove redundant to_text and list comphenesion
* Move set of 'chars' default in _random_password
on py2, C.DEFAULT_PASSWORD_CHARS is a regular str
type, so the assert here fails. Move setting the
default into the method and to_text(DEFAULT_PASSWORD_CHARS)
if it's needed.
* combine _random_password and _gen_password
* s/_create_password_file/_create_password_file_dir
* native strings for exception msgs
* move password to_text to _read_password_file
* move to_bytes(content) to _write_password_file
* add more test assertions about genned pw's
* Some cleanups to alikins and abadger's password lookup refactoring:
* Make DEFAULT_PASSWORD_CHARS into a text string in constants.py
- Move this into the nonconfigurable section of constants.
* Make utils.encrypt.do_encrypt() return a text string because all the
hashes in passlib should be returning ascii-only strings and they are
text strings in python3.
* Make the split up of functions more sane:
- Don't split such that conditionals have to occur in two separate functions.
- Don't go overboard: Good to split file system manipulation from parsing
but we don't need to do every file manipulation in a separate
function.
- Don't split so that creation of the password store happens in two
parts.
- Don't split in such a way that no decisions are made in run.
* Organize functions by when it gets called from run().
* Run all potential characters through the gen_candidate_chars function
because it does both normalization and validation.
* docstrings for functions
* Change when we store salt slightly. Store it whenever it was already
present in the file as well as when encrypt is requested. This will
head of potential idempotence bugs where a user has two playbook tasks
using the same password and in one they need it encrypted but in the
other they need it plaintext.
* Reorganize tests to follow the order of the functions so it's easier
to figure out if/where a function has been tested.
* Add tests for the functions that read and write the password file.
* Add tests of run() when the password has already been created.
* Test coverage currently at 100%
As suggested in feedback on
https://github.com/ansible/ansible/pull/17575, add
os_family to test_distribution_version. Add the
correct os_family to the existing testcase data
entries.
Also add os_family to the output of
gen_distribution_version_testcase.py so any new
generated entries will contain this data.
Clark Boylan