* InventoryManager start of perf improvements
* 0 not 1
* More startswith to [0] improvements
* Remove unused var
* The hash doesn't need to be a string, start as a list, make it into a tuple
* set actually appears faster than frozenset, and these don't need to be frozen
* Cache hosts lists, to avoid extra get_hosts calls, pass to get_vars too
* negligible perf improvement, it could help with memory later
* Try the fast way, fallback to the safe way
* Revert to previous logic, linting fix
* Extend pre-caching to free
* Address test failures
* Hosts are strings
* Fix unit test
* host is a string
* update test assumption
* drop SharedPluginLoaderObj, pre-create a set, instead of 2 comparisons in the list comprehension
* Dedupe code
* Change to _hosts and _hosts_all in get_vars
* Add backwards compat for strategies that don't do set host caches
* Add deprecation message to SharedPluginLoaderObj
* Remove unused SharedPluginLoaderObj import
* Update docs/comments
* Remove debugging
* Indicate what patterh_hash is
* That won't work
* Re-fix tests
* Update _set_hosts_cache to accept the play directly, use without refresh in get_hosts_remaining and get_failed_hosts for backwards compat
* Rename variable to avoid confusion
* On add_host only manipulate _hosts_cache_all
* Add warning docs around _hosts and _hosts_all args
* move extravars and option vars loading into VM
also safedir setting, all these are intrinsic to VM
avoid uneeded and inefectual shallow copy
remove setters/getters as VM now does most of the work in init
updated and added tests
* feedback + fixes
* keep extra_vars property for vars_prompt
* pass values not objects
* Fix encoding issues with file paths.
Discovered while testing with ANSIBLE_CONFIG env var set to a path
that contained unicode characters while LC_ALL=C.
* Fix unit tests.
* Fix another path encoding issue.
* Fix FactCache to conform to the dict API
* update needs to take a dict rather than a key and a value
* __init__ needs to allow for setting the intial dictionary
* Remove unneeded _display and _cache attributes
* Move ansible.plugins.cache.FactCache to
ansible.vars.fact_cache.FactCache because this isn't part of the cache
plugin API.
* Add backwards compatibility when calling update on the new FactCache
* Remove code for calling old FactCache. There's no way to call the old
FactCache so there's no need for backwards compatible code for calling
code. Backwards compatibility is handling things which are calling
the new FactCache.
* Port our code to the new FactCache location.
* -Change: Include dependency role names in `role_names`.
-Add: `play_role_names` magic variable to include only explicitly named roles (formerly `role_names`).
-Add: `dependent_role_names` magic variable to include all dependency names for all roles.
* -Change: use the ansible_ prefix for new magic variables.
-Change: keep `role_names` as a deprecated variable, using the old functionality.
* -Add: changelog fragment for the role_names rework.
* -Add: Tests for the role_names (and ansible_*role_names) special variables
* -Fix: resolve erroneous documentation snippet that was introduced after rebasing.
* -Fix: explicitly sort to ensure list comparison works in test.
Maintain one Templar for the lifetime of VariableManager, calling
set_available_variables() prior to each use, enabling _get_filter()'s
cache to function correctly.
It does not seem possible for concurrent calls into one (non-copied)
VariableManager instance, and so it need not be reentrant. If that
became a requirement, serializing its or Templar's entry points would be
fine, as it's so CPU-heavy other threads will only fight with it for the
GIL anyway.
Reduces _get_filters() runtime 91%, get_vars() runtime 19%, function
call count 16%, overall runtime 10%.
Tested aginst dummy load comprised of the 12 disabled steps of
debops.auth with an inventory of 80 hosts, which stresses variable
processing and task setup. Before:
7447296 function calls (7253994 primitive calls) in 32.611 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 32.762 32.762 ansible-playbook:3(<module>)
1 0.007 0.007 31.733 31.733 ansible-playbook:21(<module>)
...
1371/971 0.671 0.000 21.332 0.022 manager.py:154(get_vars)
...
3044 0.315 0.000 5.166 0.002 __init__.py:295(_get_filters)
After:
6252978 function calls (6059638 primitive calls) in 29.055 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 29.218 29.218 ansible-playbook:3(<module>)
1 0.007 0.007 28.159 28.159 ansible-playbook:21(<module>)
...
1371/971 0.675 0.000 17.211 0.018 manager.py:154(get_vars)
...
3044 0.028 0.000 0.441 0.000 __init__.py:295(_get_filters)
* Don't use copy.deepcopy in high workload areas, use deepishcopy. ci_complete
* Add tests
* Add changelog fragment
* rename to naive_deepcopy and add extra docs
* Rename to module_response_deepcopy and move to vars/clean
delegate_to parameter in task only accepts string,
this fix will error out if other datatypes are provided instead of
string.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* If we evaluate task.loop/with_items when calculating delegate_to vars, cache the items. Fixes#28231
* Add comments about caching loop items
* Add test for delegate_to+loop+random
* Be more careful about where we update task.loop
* validate vars_files entries are string_type or list
Fixes#17594
Signed-off-by: Adam Miller <admiller@redhat.com>
* use collections.Sequence to simplify if check
Signed-off-by: Adam Miller <admiller@redhat.com>
avoids some repetitive loading
- read config file only once
- now cache the ini parser per file
- optimize shell plugin loading
tried to 'optimize' vars_plugins loading but it creates issues with precedence,
probalby due to iterator not being reset, will look into it in subsequent fix/PR
* Don't skip an inventory source just because it has a comma, make sure it's also doesn't exist as a path. Fixes#34931
* Add integration test for inventory path with commas
* Remove uses of assert in production code
* Fix assertion
* Add code smell test for assertions, currently limited to lib/ansible
* Fix assertion
* Add docs for no-assert
* Remove new assert from enos
* Fix assert in module_utils.connection
* better cleanup on task results display
callbacks get 'clean' copy of result objects
moved cleanup into result object itself
removed now redundant callback cleanup
moved no_log tests
* moved import as per feedback
- old functionality is still available direct lookup use, the following are equivalent
with_nested: [[1,2,3], ['a','b','c']]
loop: "{{lookup('nested', [1,2,3], ['a','b','c'])}}"
- avoid squashing with 'loop:'
- fixed test to use new intenal attributes
- removed most of 'lookup docs' as these now reside in the plugins
this should allow user to control how they want the playbook dirs inspected
for additional vars, default now reverts to 2.3 behaviour (top).
corrected paths order
minor doc reword
* Properly handle user selection of `None` as vars_files
In a playbook, if a user has a playbook like:
```
- hosts: localhost
connection: local
vars_files:
tasks:
- ....
```
Then `vars_files` will be none, and cause a `TypeError` in vars-manager when it
tries to iterate over them. To avoid this, I changed the getter to either send
back the vars files from the user, or an empty list when the user passed
`None`.
* Only replace None with an empty list, not all falsey values
* Catch error when vars_files isn't iterable
* Move whole `for` loop into try/except and catch TypeError
* Line length
since we want to make namespaced facts drop ansible_ prefix but don't have the
time before release to perfect this feature, we are going to postpone it for now
until we have the resources to fix this issue. That way we won't have people relying
on the 'incorrect' names for a release.
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
* make groups magic var dependant on inventory
it was overtly restricted by 'host'
minor fixes to test_var_manager.py, need to test other values also
* pepe hates extra blank line
Matt Martz