Per a change in jinja2 2.9, local variables no longer are prefixed
with l_, so this updates AnsibleJ2Vars to pull in all locals (while
excluding some) regardless of name.
Fixes#20063
(cherry picked from commit 4d49b317929b86e1fc1b0cbace825ff73b372dc7)
* Fix `iterators` test for osx.
* Enable `git` test for osx.
* Enable `get_url` test for osx and freebsd.
* Remove unused SNI_URI from uri test.
* Use ansible hosted SNI host oustside docker.
* Enable `uri` test for osx and freebsd.
* Enable `gathering_facts` test for osx.
* Skip single task on osx until test is fixed.
* Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes#20098
* Address some of the requested changes
* Quoting
* Print the python path and version
* Be less explicitly verbose, rely on implicit verbosity
* Adding support for Amazon ECR
This patch adds a new module named ecr, which can create, update or
destroy Amazon EC2 Container Registries. It also handles the management
of ECR policies.
* ecs_ecr: addressed review feeback
* Renaming ecr to ecs_ecr
* Fixed docs
* Removed bad doc about empty string handling
* Added example of `delete_policy`
* Removed `policy_text` option; switched policy to `json` type so
it can accept string or dict
* Added support for specifying registry_id
* Added explicit else after returned if clauses
* Added `force_set_policy` option
* Improved `set_repository_policy` error handling
* Fixed policy comparisons when AWS doesn't keep the ordering stable
* Moved `boto_exception` into the module
* Conditional include on ansible_network_os
* copy & paste error
* More tests
* More tests
* junos tests (based on vyos)
* remove excessive whitespace
* Pass in ansible_network_os
* net_command for ios
* consistent debug
* wrapp line
* ansible-test changes made in another PR
* ansible-test changes made in another PR
map + extract is the usual way to use it but map isn't available on
older versions of jinja2 that we still work with. Test extract even on
those versions.
* Only add Content-Type if not specified in headers. Fixes#20046
* Update documentation to indicate body_format will not override Content-Type if specified in headers
When using AWS we have to use the full domain name in the inventory file, which
we rather than the short name. This change avoids that ending up being
set in the tests.
* Cleanup git tests
* Split git tests in seperate files
* Remove use of repo_depth_url
* Use native yaml
* Remove unnecessary remote/local clones
* Fix newlines for yamllint
* If the hash is valid (full-length) but doesn't exist, git returns 128 instead of 1.
* Ensure git doesn't use hardlinks for shallow clones
* Reenable yum install root tests
No need for sos to test installroot. Something with less deps works
just as well.
* Fix yum installroot.
Fix module import to use fail_json when the modules aren't installed.
Remove wildcard imports
* Lsat task is supposed to remove sos so make that happen
* Add --installroot to YUM and DNF modules, issue #11310
This continues ansible-modules-core#1558, and
ansible-modules-core#1669
Allow specifying installroot for the yum and dnf modules
to install and remove packages in a location other than /.
* Remove empty aliases
* Simpler installroot set default logic
* Refactoring: split readkeys() into readfile() and parsekeys()
* Refactoring: split writekeys() into writefile() and serialize()
* authorized_key: support --diff
* Refactoring: remove no-longer used readkeys()/writekeys()
* Integration test for authorized_key in check mode
The gce_tag module can support updating tags on multiple instances via an instance_pattern field. Full Python regex is supported in the instance_pattern field.
'instance_pattern' and 'instance_name' are mutually exclusive and one must be specified.
The integration test for the gce_tag module has been updated to support the instance_pattern parameter. Unit tests have been added to test the list-manipulation functionality.
Run the integration test with:
TEST_FLAGS='--tags "test_gce_tag"' make gce
Run the unit tests with:
python test/units/modules/cloud/google/test_gce_tag.py
An inner single-quote pair breaks out of the outer single-quote
pair. Rather than escaping the inner quotes to protect against
this, just use the fact that `str()` is equivalent to `""`.
* Fix UnboundLocalError remote_head in git
Fixes#5505
The use of remote_head was a leftover of #4562.
remote_head is not necessary, since the repo is unchanged anyway and
after is set correctly.
Further changes:
* Set changed=True and msg once local_mods are detected and reset.
* Remove need_fetch that is always True (due to previous if) to improve
clarity
* Don't exit early for local_mods but run submodules update and
switch_version
* Add test for git with local modifications
* Use native yaml for apache2 test
* Test removal of default modules with force
a2enmod on debian has `-f`, but not on SUSE (runs there without force).
Therefore don't test that option on SUSE.
The docs already specify that the option is intended for Debian systems
only.
* Fix regression in jinja2 include search path
Since commit 3c39bb5, the 'ansible_search_path' variable is used to set
jinja2's search path for {% include %} directives. However, this path is
the the proper one because our templates live in 'templates' subdirs in
our search path.
This is a regression because previously, our include search path would
include the dirname of the currently interpreted file, which worked most
of the time.
fixes#18526
* Fix template lookup search path
Improve fix in commit c96c853 so that the search path contain both
template-suffixed paths as well as original paths.
ref PR #18617
* Add integration test for template lookups
Tests regression at #18526
This test fails on current devel branch and succeeds on PR #18617
* Add test cases for VyOS commands that don't honor paging settings
Testing for issue fixed in PR #18546
* Add provider line and fix indentation
For the way we invoke the tests we need to specify the `provider:`
Also fix the indentation on `register:`
This change corrects problems reported by the `yamllint` linter.
Since key duplication problems were removed in 4d48711, this commit
mainly fixes trailing spaces and extra empty lines at beginning/end of
files.
* Added test for sequenced-name instance generation (num_instances)
* Added param-check tags to tests that only do argument checking
Should be merged AFTER ansible/ansible-modules-core#4276
- Correct directory name in test/README.md
- Move code-smell tests to test/sanity/code-smell
- Update code-smell.sh to use new script paths
- Add test/integration/target-prefixes.win for ansible-test
- Move module unit tests to match module directory layout
* Network Test Documentation
Will need improving over time, though this ensure that everything that was in `ansible/test-network-modules` is in `ansible/ansible`
* Update README.md
* Inventory file
* Network module prefixes
In ansible-test we should skip tests for these modules, they will be
tested via another process.
* Update target-prefixes.network
If the facts returned by setup included strings that
had double quotes in them, the asserts in test_gathering_facts.yml
would fail with errors like:
"The conditional check '\"[{u'mounts': {u'options':
u'rw,context=\"system_u:\"'}}]\" != \"UNDEF_HW\"' failed. The error was:
template error while templating string: expected token 'end of statement
block', got 'system_u'. String: {% if \"[{u'mounts': {u'options':
u'rw,context=\"system_u:\"'}}]\" != \"UNDEF_HW\" %} True {% else %}
False {% endif %}"
For one example, if mount facts returned an 'options' field that
included double quoated selinux context ids, the test would fail.
Fix is removing the double quoting in the assert 'that:' lines,
and removing the unneeded double curly brackets.
* Add tag verification test (ansible-modules-core PR 2654)
* Fix typo
* Use smaller repo for testing, add dependency control
* Test is gpg exists before running git signing tasks
* Correct the test conditionals so that gpg1 is tested
Implement tag and skip_tag handling in the CLI() class. Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.
* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
default will change in 2.4 and go away in 2.5.
* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
* Turn mount test back on
* Mount tests need PRIVILEGED so turn that back on
Revert "Revert "Set PRIVILEGED=true for non_destructive tests. (#17733)" (#17738)"
This reverts commit dc0fb1c212.
* Add a needs_privileged tag so that we can skip mount tests on centos6
Some containers timeout on shippable tests when run with privileged.
Unfortunately, some tests require that in order to run. Tagging those
allows us to skip those tests on the platforms that timeout when we get
ready to run the integration test in shippable.
* Centos6 times out with PRIVILEGED set so remove that (will disable the mount tests on centos6)
* Remove false start
* Add tests for the mount module
* Switch from unmounted to absent...
the code for mounting always modifies fstab so we need to always modify
fstab to cleanup as well.
* Fix comments and copyright
Includes:
* Tests to create, delete, resize, recreate and configure an Autoscaler
* Tests for parameters only. Use TEST_FLAGS="--tags 'param-check'" to execute only these tests.
Requires:
* An instance template to be configured in your project.
* Fixes to the controller text model
* Change command line args to text type
* Make display replace undecodable bytes with replacement chars. This
is only a problem on pyhton3 where surrogates can enter into the msg
but sys.stdout doesn't know how to handle them.
* Remove a deprecated playbook syntax in unicode.yml
* Fix up run_cmd to change its parameters to byte string at appropriate times.
The test_async test target was updated to accommodate changes in
output buffering behavior in python 3. This change in behavior
may need to be addressed in the future.
* New features for include_vars
include_vars.py now allows you to include an entire directory and its nested directories of variable files.
Added Features..
* Ignore by default *.md, *.py, and *.pyc
* Ignore any list of files.
* Only include files nested by depth (default=unlimited)
* Match only files matching (valid regex)
* Sort files alphabetically and load in that order.
* Sort directories alphabetically and load in that order.
```
- include_vars: 'vars/all.yml'
- name: include all.yml
include_vars:
file: 'vars/all.yml'
- name: include all yml files in vars/all and all nested directories
include_vars:
dir: 'vars/all'
- name: include all yml files in vars/all and all nested directories and save the output in test.
include_vars:
dir: 'vars/all'
name: test
- name: include all yml files in vars/services
include_vars:
dir: 'vars/services'
depth: 1
- name: include only bastion.yml files
include_vars:
dir: 'vars'
files_matching: 'bastion.yml'
- name: include only all yml files exception bastion.yml
include_vars:
dir: 'vars'
ignore_files: 'bastion.yml'
```
* Added whitelist for file extensisions (yaml, yml, json)
* Removed unit tests in favor of integration tests
While trying to fix the test suite on python3, I noticed
this test fail due to to_json adding more whitespace in
python3 than in python2. So -w should ignored those
differences.
Fixes for these are either rewriting to get rid of the need for the
functions or using six.moves to get equivalent functions for both
python2 and python3
Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.
This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.
Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.
vault.VaultLib api changes:
- Split VaultLib.encrypt to encrypt and encrypt_bytestring
- VaultLib.encrypt() previously accepted the plaintext data
as either a byte string or a unicode string.
Doing the right thing based on the input type would fail
on py3 if given a arg of type 'bytes'. To simplify the
API, vaultlib.encrypt() now assumes input plaintext is a
py2 unicode or py3 str. It will encode to utf-8 then call
the new encrypt_bytestring(). The new methods are less
ambiguous.
- moved VaultLib.is_encrypted logic to vault module scope
and split to is_encrypted() and is_encrypted_file().
Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml
Integration tests added as roles test_vault and test_vault_embedded
* actions/unarchive: fix unarchive from remote url
Currently unarchive from remote url does not work because the core
unarchive module was updated to support 'remote_src' [1], but the
unarchive action plugin was not updated for this. This causes failures
because the action plugin assumes it needs to copy a file to the
remote server, but in the case of downloading a file from a remote
url a local file does not exist, so an error occurs when the file is
not found.
[1] https://github.com/ansible/ansible-modules-core/commit/467516e
* test_unarchive: fix test with wrong remote_src use
The non-ascii filenames test had improperly set remote_src=yes even
though it was actually copying the file from the local machine (i.e.
the file did not already exist remotely). This test was passing
until the remote_src behavior of unarchive was fixed in 276550f.
Rather than repeatedly searching for tasks by uuid via iterating over
all known blocks, cache the tasks when they are added to the PlayIterator
so the lookup becomes a simple key check in a dict.
* Query lookup plugin
* Add license and docstrings
* Add python3-ish imports
* Change query plugin type from lookup to filter
* Switch from dq to jsonpath_rw
* Add integration test for query filter
* Rename query filter to json_query
* Add jsonpath-rw
* Rename query filter to json_query
* Switch query implementation from jsonpath-rw to jmespath
When a task result has an empty results list, the
list should be ignored when determining the results
of `_check_key`. Here the empty list is treated the
same as a non-existent list.
This fixes a bug that manifests itself with squashed
items - namely the task result contains the correct
value for the key, but an empty results list. The
empty results list was treated as zero failures
when deciding which handler to call - so the task
show as a success in the output, but is deemed to
have failed when deciding whether to continue.
This also demonstrates a mismatch between task
result processing and play iteration.
A test is also added for this case, but it would not
have caught the bug - because the bug is really in
the display, and not the success/failure of the
task (visually the test is more accurate).
Fixesansible/ansible-modules-core#4214
Binary module tests now download pre-compiled binaries from S3
instead of downloading go and compiling the modules.
Files downloaded form SourceForge are now downloaded from S3.
* Enabled more tests in test_service on systems with systemd.
* Fix inconsistency in cleaning up test service. The conditions for service setup and service cleanup were different.
* Add check mode run for service enable.
Enable color output from integration tests in Docker containers:
* In run_tests.sh when output is attached to a terminal.
* In shippable/integration.sh using force mode (can be disabled).
Also fix blocks tests to work with or without color output
* add check_mode option for tasks
includes example testcases for the template module
* extend check_mode option
* replace always_run, see also proposal rename_always_run
* rename always_run where used and add deprecation warning
* add some documentation
* have check_mode overwrite always_run
* use unique template name to prevent conflicts
test_check_mode was right before, but failed due to using the same filename as other roles
* still mention always_run in the docs
* set deprecation of always_run to version 2.4
* fix rst style
* expand documentation on per-task check mode
I suspect this problem was masked previously as older versions of pywinrm
where significantly slower, allowing more time for the windows installer service to complete.
6eefc11c converted task.loop_control into an object, but while the other
callers were updated to use .loop_var instead of .get('loop_var'), this
site was overlooked.
This can be reproduced by including with loop_control a file that does
set_fact; a simple regression test along these lines is included.
* Lookup unencrypted password must not include salt
* Integration test lookup: remove previous directory
* Test that lookup password doesn't return salt
* Lookup password: test behavior with empty encrypt parameter
Closes#16189
* add git shallow fetch test
covers https://github.com/ansible/ansible-modules-core/issues/3782
updating a repo with depth=1 fails silently if version==HEAD
* disable git shallow tests for old git versions
Older git versions don't treat the --depth option correctly.
While the git module tried to work around this and introduced subtle
bugs, ansible/ansible-modules-core#3794 falls back to full checkouts.
Don't run the tests then.
* reduce async sleep time in test
* make zypper test less destructive (don't break following uses of zypper)
* fix ca cert on suse
* fix/enable postgres/mysql on opensuseleap
* fix mysql test for mysql versions 5.7.6 and newer
* skip sni_host check on ubuntu1604
* add HTTPTESTER flag for test_uri
ubuntu 16.04 uses dash which drops env variables containing a dot
we work around this by adding an explicit env variable to enable httptester
* alias module is very basic and removing it leads to the suse default
config failing
* future improvements might test different modules and the effect of
them being removed
* When var name is the same as var content, try to template it before reporting that var is not defined.
Add asserts in test_var_blending to check this special corner case.
* Fix integration tests when using debug with list or dict.
* extend zypper integration tests
* fix caching issue of local RPMS on openSUSE 42.1 Leap
* add tests for simultaneous install/remove via prefixes +-
* test fail cases (rm patch or URL)
* test patch install (succes, unchanged second run, fail on wrong name)
* add test for pattern install
* include options refresh and priority introduced in ansible/ansible-modules-extras#1990
* test repo update both based on name and URL
* run on all suse flavors
* test overwriting repos by url or name
* add integration test for apache2_module
* fix assert syntax
* check os_family for this test
* add comment why only debian and suse are tested
* move apache2 test to destructive
* install apache2 package
- get_real_file will decrypt vault encrypted files and return a path to
a temporary file.
- cleanup_real_file will remove a temporary file created previously with
get_real_file
Matt Clay