Commit Graph

89 Commits (df8b2d45bcd43b9b78d14c04c4740dcd6055b2b2)

Author SHA1 Message Date
Brian Coca bd70397e24 always template when called from template (#18490)
* Have template action plugin call do_template

Avoids all the magic done for 'inline templating' for ansible plays.
renamed _do_template to do_template in templar to make externally accessible.
fixes #18192

* added backwards compat as per feedback
8 years ago
Dag Wieers cd784cd345 Remove unnecessary warnings (#18121) 8 years ago
Dag Wieers d52a9cee46 Support late evaluation, suppress warnings on SyntaxError (#17462)
See #14304 comments (https://github.com/ansible/ansible/pull/14304#issuecomment-245453209) for more information.
8 years ago
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
8 years ago
Brian Coca ff34f5548d Dynamic role include (#17401)
* dynamic role_include

* more fixes for dynamic include roles

* set play yfrom iterator when dynamic

* changes from jimi-c

* avoid modules that break ad hoc

TODO: should really be a config
8 years ago
Adrian Likins e396d5d508 Implement vault encrypted yaml variables. (#16274)
Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.

This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.

Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.

vault.VaultLib api changes:
    - Split VaultLib.encrypt to encrypt and encrypt_bytestring

    - VaultLib.encrypt() previously accepted the plaintext data
      as either a byte string or a unicode string.
      Doing the right thing based on the input type would fail
      on py3 if given a arg of type 'bytes'. To simplify the
      API, vaultlib.encrypt() now assumes input plaintext is a
      py2 unicode or py3 str. It will encode to utf-8 then call
      the new encrypt_bytestring(). The new methods are less
      ambiguous.

    - moved VaultLib.is_encrypted logic to vault module scope
      and split to is_encrypted() and is_encrypted_file().

Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml

Integration tests added as roles test_vault and test_vault_embedded
8 years ago
Dag Wieers 56148291e9 Emit warnings when safe_eval() raises a SyntaxError or other Exception (#14304)
This change is related to reported issue #14291 and pull request #14293.

Without the fix from #14293, this change will emit a warning as shown below, on the following playbook:

``yaml
---
- hosts: localhost
  gather_facts: no
  vars:
    works:
      key1: 'string'
      key2: 1234
    fails:
      key1: 'string'
      key2: 1234
      key3: false
  tasks:
  - debug: msg={{ works | to_json }}
  - debug: msg={{ fails | to_json }}
```

On error, this results in a proper warning:

```
[dag@moria ansible.dag]$ ansible-playbook test49.yml

PLAY ***************************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": {
        "key1": "string",
        "key2": 1234
    }
}

TASK [debug] *******************************************************************
 [WARNING]: Error in expression "{"key3": false, "key2": 1234, "key1": "string"}". (name 'false' is not defined)

ok: [localhost] => {
    "msg": "{\"key3\": false, \"key2\": 1234, \"key1\": \"string\"}"
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0
```
8 years ago
Colin Hoglund 5a2b34e159 allow spaces before the filter string (#16312) 9 years ago
Brian Coca 005dc84aa7 moved last utils.debug to display.debug 9 years ago
Brian Coca 0b2f631197 Merge pull request #14504 from bcoca/template_comments
template also when only comments present
9 years ago
Brian Coca b901b15d3b Merge pull request #14537 from Yannig/devel_better_error_handling
Improve message content to get more hint about the raised error.
9 years ago
Brian Coca c022a43a47 fixed str to to_str 9 years ago
Brian Coca 4cfd042839 more complete and informative errors
mostly templating issues now all return data templated and actual error
fixes #14781
9 years ago
Toshio Kuratomi 1dbbd2a80c Remove debugging and fix the place where bytes are being handed to StringIO 9 years ago
Toshio Kuratomi 093b3422ec more information 9 years ago
Toshio Kuratomi 5aef65edcd Testing whether this fixes jenkins 9 years ago
James Cammarata 3247390123 Conditionally import StringIO in template so we only use io.StringIO on py3 9 years ago
Toshio Kuratomi b70bf3b056 Use io.StringIO and io.BytesIO instead of StringIO.StringIO for compat with py3 9 years ago
Matt Martz 439a385215 Catch TypeError on join, and if caught just return a single item, or a list of many 9 years ago
Yannig Perré 588045cc46 Improve message content to get more hint about the raised error. 9 years ago
Brian Coca 6102a4b9b4 template also when only comments present 9 years ago
Marc Poirier 0792983ec8 Display the string which is causing an exception to be raised.
In the ansible template module, when there is an error while
expanding a templated string, displaying the string causing the
exception is very useful.
9 years ago
Brian Coca 6149685652 Merge pull request #14293 from dagwieers/fix-eval-json-booleans
Defined JSON booleans in global context for python eval()
9 years ago
Dag Wieers 2e171610e0 Also add 'null' as a possible JSON value 9 years ago
Dag Wieers dc48d27dd2 Defined JSON booleans in global context for python eval()
We define 'false' and 'true' as variables so that python eval() recognizes them as False and True.

This fixes #14291.
9 years ago
Brian Coca dc15eb806e deprecate all bare variable conversions (not debug)
now deprecation message appears with variable name in all spots where this occurs
debug's var= option is excluded as this is only place where bare variables shold actually
be accepted.
9 years ago
Brian Coca d82d65ee7b keep string type filters as strings
now we don't try to convert types if using a filter that outputs a specifically formated string
made list of filters configurable
9 years ago
James Cammarata f926e81782 Re-implement lookup wantlist
Fixes #13285
9 years ago
Yannig Perré 0c360d17cb New parameter for template method.
We do not compute hash when we know that the result does not need to be cached (like with_items loop).
It also result in a small speed improvement.
9 years ago
James Cammarata 7e04947599 Be more selective about what variables we cache during templating
Fixes #13087
9 years ago
James Cammarata 5040abaaf2 Merge branch 'devel_cache_for_do_template_call' of https://github.com/Yannig/ansible into Yannig-devel_cache_for_do_template_call 9 years ago
James Cammarata 1027abd596 Cleaning up bugs related to perf improvement changes 9 years ago
James Cammarata 63c47fb271 Fixing up performance 9 years ago
Yannig Perre ccbdd6229a Use static vars when computing host vars known to be static (inventory_hostname, inventory_dir etc.). 9 years ago
Yannig Perré 30f827d92d Fix python3 test. 9 years ago
Yannig Perré 130139dc80 Fix unicode issue introduced by previous commit. 9 years ago
Yannig Perré 4a8d1703d4 New patch against hostvars.py. With this patch, Ansible run lose 50% of time.
Little rewrite of previous patch to use sha1 signature.
Use fail_on_undefined to compute sha1 signature.
9 years ago
Brian Coca 79bf66336b switched clean_data from hardcoded delimiters to actual currently set jinja2 environment delimiters 9 years ago
Yannig Perré 805f768dab Clear cache when updating template variables. 9 years ago
Yannig Perré 87a9485b2f Cache for _do_template call. May result in nice speed improvement (4-5 times faster). 9 years ago
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 9 years ago
James Cammarata 45b803efb4 Properly use test_loader from SharedPluginLoaderObj 9 years ago
Marius Gedminas 9b39af8e9a Python 3: the StringIO module is gone
Fixes 17 failing tests on Python 3.4, restoring the 8 failures we had
last Friday.
9 years ago
James Cammarata cdc6c5208e Clean string data run through the template engine
Also strip UnsafeProxy off of low level srings and objects to ensure
they don't cause issues later down the road

Fixes #12513
9 years ago
Marius Gedminas 56f2a25bff Python 3: there's no 'unicode' 9 years ago
Marius Gedminas f91b28ef23 Fix safe_eval() of set literals
There was code to support set literals (on Python 2.7 and newer), but it
was buggy: SAFE_NODES.union() doesn't modify SAFE_NODES in place,
instead it returns a new set object that is then silently discarded.

I added a unit test and fixed the code.  I also changed the version
check to use sys.version_tuple instead of a string comparison, for
consistency with the subsequent Python 3.4 version check that I added in
the previous commit.
9 years ago
Marius Gedminas 2b73002044 Python 3: make ansible.template.safe_eval() work
Two things changed in Python 3.4:

- 'basestring' is no longer defined, so use six.string_types

- True/False are now special AST node types (NamedConstant) rather than
  just names

(Good thing we had tests, or I wouldn't have noticed the 2nd thing!)

I found only one place where safe_eval() is called inside the ansible
codebase: in lib/template/__init__.py.  The call to safe_eval(result,
...) is protected by result.startswith('...'), which means result cannot
possibly be a byte string on Python 3 (or startswith() would raise, so
six.string_types (which excludes byte strings on Python 3) is fine here.
9 years ago
Toshio Kuratomi 46bf3d1bc1 Remove plugin._basedirs as we now use different data structures to achieve that functionality. 9 years ago
Marius Gedminas ba0cf42ce9 Python 3: use six.string_types instead of basestring
Reduces the number of failing tests from 35 to 17.
9 years ago
James Cammarata 30399edada Use UnsafeProxy for lookup results too
Also fixes a couple of bugs that popped up when using the proxy class
9 years ago