Commit Graph

5145 Commits (c83f51b7f2e4ebf318174c320b8861dd985b31e9)

Author SHA1 Message Date
James Cammarata b9afbf0ee4 Reorganizing the way the connection lockfile is created 9 years ago
James Cammarata 7034bbef30 Merge branch 'connection-locking' of https://github.com/amenonsen/ansible into amenonsen-connection-locking 9 years ago
Abhijit Menon-Sen 9378c8e2da Make the paramiko plugin use locking 9 years ago
Abhijit Menon-Sen 5887e96b27 Introduce a connection locking infrastructure
The lock file is (a temporary file) opened in the parent process, whose
open fd is inherited by the workers after fork, and passed down through
the PlayContext. Connection grows lock/unlock methods which can be used
by individual connection plugins.
9 years ago
Abhijit Menon-Sen 12d8f9132e Make lock_host_keys a real noop, instead of half-a-noop
Right now, we don't do any locking, but we still scan known_hosts files
twice per connection. That's completely unnecessary, and the proposed
solutions to the locking problem wouldn't need known_hosts scanning
anyway, so this code can go away.
9 years ago
Brian Coca a006e85947 removed printing module args as it violates no_log settings 9 years ago
Toshio Kuratomi 1998eddf8d Keep newlines by default now.
In v1, a trailing newline was kept if the parameter was passed as key=value.  If
the parameter was passed as yaml dict the trailing newline was
discarded.  Since key-value and yaml dict were unified in v2 we have to
make a choice as to which behaviour we want.  Decided that keeping trailing
newlines by default made the most sense.

Fixes #12200
Fixes #12199
9 years ago
Toshio Kuratomi 41da8de094 Speedup for counting newlines 9 years ago
James Cammarata 843138e7d9 Merge pull request #12202 from mgedmin/py3k
Use rich comparisons for Attribute and FieldAttribute
9 years ago
James Cammarata 5434ed973b Make sure locals in safe_eval is a plain dict when running eval
Fixes #12206
9 years ago
Toshio Kuratomi 417bf1c805 Unittest the _count_trailing_newlines function 9 years ago
Toshio Kuratomi 7ed746ad45 Fix preserve_trailing_newlines (broken by 7f5080f64a )
Fix for one half of hte bug reported in #12198
9 years ago
Brian Coca a7231c2203 actually implemented flags correctly for all priv escalation methods 9 years ago
Brian Coca b9642585bd added missing become_exe from config 9 years ago
Brian Coca e156d9b677 fixed and generalized privilege escalation exe settings 9 years ago
Marius Gedminas 33a374edde Use rich comparisons for Attribute and FieldAttribute
Because __cmp__ is not supported on Python 3.
9 years ago
Marius Gedminas ca908f2e12 Fix indentation to be a multiple of 4 9 years ago
Abhijit Menon-Sen 147b04b682 Remove duplicated function definition (fallout from earlier repeated merge) 9 years ago
James Cammarata f162990cb3 Properly assign the searchpath for templates to the environment loader
dbd755e0 previously assigned the value to self._templar.environment.searchpath,
which is incorrect - it needs to be assigned to the environment.loader.searchpath
value instead.

Fixes #11931
9 years ago
James Cammarata 2761df232e Small tweak to 7551b75 to correct the ordering of tests 9 years ago
James Cammarata 7551b75e61 Add ispath type for constants and make sure all local paths are ispath=True
Fixes #12180
9 years ago
Toshio Kuratomi aeff960d02 Cleanup combine_vars
* Dedupe combine_vars() code (removed from VariableManager)
* Fix merge_hash algorithm to preserve the type
* unittest combine_vars and merge_hash
9 years ago
James Cammarata 7fe495d619 Merge pull request #12187 from caphrim007/add-utf-8-encoding-to-f5-module-utils
Adds utf-8 encoding to f5 module_utils
9 years ago
James Cammarata d00c445d2c Merge pull request #12188 from amenonsen/easy-debug
Show module name and args with -vvv
9 years ago
Abhijit Menon-Sen d6d523bcae Show module name and args with -vvv
This information was earlier shown only with ANSIBLE_DEBUG, but it's
extremely useful in a user context, especially with module invocations
with deeply nested args like the ec2_vpc/ec2 modules.

Closes #11680
9 years ago
James Cammarata 3e1a774ba5 Don't use iteritems() in templar to avoid (hostvars) dict size change errors 9 years ago
James Cammarata 974b69d236 looped includes need unique references to prevent information bleeding
Fixes #12155
9 years ago
James Cammarata a341a8a093 Make sure tasks from a role see their defaults above all others
Fixes #12081
9 years ago
Tim Rupp 6f41a72158 Adds utf-8 encoding to f5 module_utils
The contributor's name on line 10 (originally line 7) includes a character
that the default Python encoding (ASCII) raises an error on when interpreting
the file.

Specifying the utf-8 encoding, as is done in other modules, resolves
the error.

The error being raised is

SyntaxError: Non-ASCII character '\xc3' in file /.../lib/ansible/module_utils/f5.py
on line 7, but no encoding declared; see http://www.python.org/peps/pep-0263.html
for details
9 years ago
James Cammarata a7da25d48b Submodule update 9 years ago
James Cammarata 3d282cd1b0 Also always post_validate Handler class as well
Fixes #12154
9 years ago
Toshio Kuratomi 12edbfeb0b Merge pull request #12177 from mgedmin/py3k
Make combine_vars() compatible with Python 3
9 years ago
Brian Coca 66e2eb2231 removed as it does not fool setup.py to include data and it must be specified in setup.py 9 years ago
Marius Gedminas 54dbfba8f8 Make combine_vars() compatible with Python 3
Fixes

  TypeError: unsupported operand type(s) for +: 'dict_items' and 'dict_items'

on Python 3.
9 years ago
James Cammarata 5adcd7054b Allow setup to run when using --start-at-task
Fixes #12147
9 years ago
Brian Coca 7458331539 package and service now check that module exists before trying to execute it 9 years ago
Toshio Kuratomi 9d193d8fb4 Merge pull request #12173 from ansible/pr/10204
Rebase of 10204 - Add host key for ssh url only.
9 years ago
James Cammarata fca27c29f4 Properly fail if slurp fails during fetch 9 years ago
Desmond O. Chang 58c3539196 Add host key for ssh url only.
Rewrite function `get_fqdn`.  It returns fqdn for all kinds of urls now.

`add_git_host_key` determines whether a url is ssh and whether its host
key should be added.
9 years ago
James Cammarata 66a2f2923e Look for proper values in slurp result during fetch with become enabled 9 years ago
Toshio Kuratomi 7f5080f64a Fix backslash escaping inside of jinja2 expressions
Fixes #11891
9 years ago
Toshio Kuratomi 5ae300a084 Update module refs 9 years ago
Brian Coca 6b73cac35a corrected skipped_hosts constant usage, thanks @benjixx 9 years ago
Brian Coca 5a65dce6fb fix issues with csh redirection 9 years ago
Brian Coca b525eedb8e less misleading message for when someone looks up vars for a host not in inventory 9 years ago
Brian Coca 6154ed1dda Merge pull request #12146 from thekarlbrown/patch-1
Fixed Spelling Error on Line 67
9 years ago
James Cammarata f0257683b0 Fixed missing constants import in default callback
Fixes #12145
9 years ago
James Cammarata 057712c129 Also move action/connection plugins to shared plugin loader code
Fixes #12099
9 years ago
James Cammarata 0859ba7726 Adjust field loading order for vars and tweak post_validate exclusion logic
FieldAttributes will now by default not be post_validated unless a flag
is set on them in the class, as a large number of fields are really there
simply to be inherited by Task/PlayContext and shouldn't be templated too
early.

The other (unrelated to the base issue) in #12084 is also fixed here, where
the roles field is loaded before vars/vars_files, meaning there are no vars
yet loaded in the play when the templating occurs.

Fixes #12084
9 years ago
Karl Brown 9aae1d5810 Fixed Spelling Error on Line 67
Noticed while running Ansible from Tip.
Throws: NameError: global name 'tself' is not defined
9 years ago
James Cammarata 4c65306e73 Merge branch 'hostrange' of https://github.com/amenonsen/ansible into amenonsen-hostrange 9 years ago
Toshio Kuratomi 120243d36d Fix python3.4 syntax 9 years ago
James Cammarata 2043fcd5db Merge pull request #12083 from resmo/for-ansible
cloudstack: implement general api_region support, update docs
9 years ago
Brian Coca b6c6ed91fe implemented not showing skipped hosts config 9 years ago
Brian Coca 514fa73fcd galaxy fixes 9 years ago
James Cammarata 66ed397360 Fix redis fact_caching_timeout=0 bug
Fixes #12018
9 years ago
James Cammarata 5a5b9f211b Validate variable names when loading 'vars:' blocks
TODO: add this to VariableManager to validate vars loaded from files too

Fixes #12022
9 years ago
James Cammarata 266a069a73 Merge pull request #12107 from TravisPaul/devel
Update "smart" transport to handle Sun_SSH_1.5 on SmartOS
9 years ago
Brian Coca 48aa0dd1c7 now acknowledges ask_pass setting from ansible.cfg
fixes #12111
9 years ago
James Cammarata 399d215f5a Merge pull request #12122 from maty0609/devel
Adding support for Archlinux and Slackware in fallback
9 years ago
Marius Gedminas eb99aa8c68 Fix to_bytes(None) on Python 3
You cannot call bytes(obj) to get a simple representation of obj on
Python 3!  E.g. bytes(42) returns a byte string with 42 NUL characters
instead of b'42'.
9 years ago
Marius Gedminas df1b41d3d3 Avoid types.NoneType
types.NoneType was removed in Python 3.

None is a singleton in Python, so 'x is None' is equivalent to
'isinstance(x, NoneType)'.
9 years ago
Marius Gedminas da1e611b26 Support print as a function
I neglected the __future__ import because

  print(one_thing)

works the same way even when print is a statement.
9 years ago
Marius Gedminas 39196ec91e Drop the L suffix on numerical constants
Python has had automatic int-to-long promotion for a long long time now.
Even Python 2.4 does that automatically.

Python 3 drops support for the L suffix altogether.
9 years ago
Marius Gedminas 0eb538df03 Use 0oNNN octal syntax
This syntax is valid in Python 2.6+ and 3.x.
9 years ago
Marius Gedminas 47b088504d Don't mix tabs and spaces
It's not allowed in Python 3 and merely a bad idea in Python 2.
9 years ago
Marius Gedminas 0c6ce31f76 Use 'except ... as' syntax
This syntax works on Python 2.6 through 3.x.  lib/ansible/module_utils
(and lib/ansible/modules) need to support Python 2.4, so I didn't touch
those.
9 years ago
Brian Coca 2ac931d6c9 ported module_docs to use display 9 years ago
Brian Coca 2b28cdc0dd be more tolerant with non list descriptions 9 years ago
Toshio Kuratomi 86b2982005 Merge pull request #12112 from amenonsen/vault-stdio
Implement cat-like filtering behaviour for encrypt/decrypt
9 years ago
Toshio Kuratomi db4a96a8d6 Merge pull request #12078 from amenonsen/hash_merge
Add hash_merge and hash_merge_recursive filters with documentation
9 years ago
Abhijit Menon-Sen b328bc023d Add a combine filter with documentation
This is based on some code from (closed) PR #7872, but reworked based on
suggestions by @abadger and the other core team members.

Closes #7872 by @darkk (hash_merge/hash_replace filters)
Closes #11153 by @telbizov (merged_dicts lookup plugin)
9 years ago
James Cammarata 2df6513f8d Version bump for v2.0.0-0.1.alpha1 and submodule updates 9 years ago
Brian Coca 6c9dc78d8c Merge pull request #12126 from amenonsen/vault-aes-deprecate
Remove deprecated and unused VaultAES encryption code
9 years ago
James Cammarata 1170a453c8 Merge pull request #12114 from ilya-epifanov/devel
fixed hostvars access in conjunction with --limit usage
9 years ago
Abhijit Menon-Sen 090cfc9e03 More helpful prompts from ansible-vault encrypt/decrypt
Now we issue a "Reading … from stdin" prompt if our input isatty(), as
gpg does. We also suppress the "x successful" confirmation message at
the end if we're part of a pipeline.

(The latter requires that we not close sys.stdout in VaultEditor, and
for symmetry we do the same for sys.stdin, though it doesn't matter in
that case.)
9 years ago
Abhijit Menon-Sen b6de6e69a6 Also support output to stdout with no arguments
This allows "cat plaintext|ansible-vault encrypt > ciphertext".
9 years ago
Abhijit Menon-Sen e7eebb6954 Implement cat-like filtering behaviour for encrypt/decrypt
This allows the following invocations:

    # Interactive use, like gpg
    ansible-vault encrypt --output x

    # Non-interactive, for scripting
    echo plaintext|ansible-vault encrypt --output x

    # Separate input and output files
    ansible-vault encrypt input.yml --output output.yml

    # Existing usage (in-place encryption) unchanged
    ansible-vault encrypt inout.yml

…and the analogous cases for ansible-vault decrypt as well.

In all cases, the input and output files can be '-' to read from stdin
or write to stdout. This permits sensitive data to be encrypted and
decrypted without ever hitting disk.
9 years ago
Abhijit Menon-Sen 32b38d4e29 Fix add_option indentation for consistency before adding another option 9 years ago
Abhijit Menon-Sen 8fc8bf9439 Simplify VaultEditor methods
We don't need to keep creating VaultLibs everywhere, and we don't need
to keep checking for errors because VaultLib does it already.
9 years ago
Abhijit Menon-Sen e99395f0c0 Don't create a VaultLib in each method; do it in __init__ instead 9 years ago
Abhijit Menon-Sen 159887a6c9 Remove deprecated and unused VaultAES encryption code
Now that VaultLib always decides to use AES256 to encrypt, we don't need
this broken code any more. We need to be able to decrypt this format for
a while longer, but encryption support can be safely dropped.
9 years ago
maty0609 52e94468c9 Merge remote-tracking branch 'ansible/devel' into devel 9 years ago
maty0609 6f24e6f994 Adding support for Archlinux and Slackware in fallback
In some cases Archlinux and Slackware is not detected by
platform.dist(). This should solve the issue.
9 years ago
Brian Coca b2bfe3502b make sure delimiter is basestring for cvsfile
fixes #12062
9 years ago
Ilya Epifanov 81bf88b6e0 fixed hostvars access in conjunction with --limit usage 9 years ago
James Cammarata 0441a7a217 Finishing off porting of chroot connection plugin 9 years ago
Toshio Kuratomi 017bd7b1cd Fix synchronize lookup of localhost info 9 years ago
James Cammarata 50448d68e1 Implement max_fail_percentage and any_errors_fatal support
Fixes #11997
9 years ago
James Cammarata af41ba929c Add float and percent types for FieldAttributes
Also sets the max_fail_percentage value to the percent type.
9 years ago
Brian Coca b2ae6945c4 always load vars plugins
fixes #12104
9 years ago
Travis Paul 604f825a8e Update "smart" transport to handle Sun_SSH_1.5 on SmartOS 9 years ago
Toshio Kuratomi 111c0cc204 Merge pull request #12106 from amenonsen/vault-cleanups
Vault cleanups, pass #1
9 years ago
Abhijit Menon-Sen b84053019a Make the filename the first argument to rekey_file 9 years ago
Toshio Kuratomi 5df5a14edc Merge pull request #12101 from tquenolle/devel
Synchronize fix error
9 years ago
Abhijit Menon-Sen 20fd9224bb Pass the filename to the individual VaultEditor methods, not __init__
Now we don't have to recreate VaultEditor objects for each file, and so
on. It also paves the way towards specifying separate input and output
files later.
9 years ago
Brian Coca 82603bb2a0 avoids running abspath on None 9 years ago
Abhijit Menon-Sen a27c5741a1 Remove inaccurate outdated comment 9 years ago
Abhijit Menon-Sen f91ad3dabe Don't pass the cipher around so much
It's unused and unnecessary; VaultLib can decide for itself what cipher
to use when encrypting. There's no need (and no provision) for the user
to override the cipher via options, so there's no need for code to see
if that has been done either.
9 years ago
Abhijit Menon-Sen 017566a2d9 Use AES256 if the cipher is not write-whitelisted 9 years ago
Abhijit Menon-Sen 47bcdf5952 Remove incorrect copy-pasted comment 9 years ago
Thomas Quenolle c948af3b1e Synchronize fix error
Fix the error:
 "RuntimeError: dictionary changed size during iteration"
9 years ago
James Cammarata 601a1cc6d9 Multiple fixes for include statements and blocks in general
Fixes #11981
Fixes #11995
Fixes #12039
Fixes #12077
9 years ago
Brian Coca 154754ae50 pushed module_loader to task_queue_manager so all cli's can benefit from it
also normalized -M option across all cli
fixes #12016
9 years ago
Toshio Kuratomi d2c948dd6a Remove decrypted vault temp_file mistakenly left from patch making vault edit idempotent
This bug was introduced in commit f8bf2ba on July 27.  Hasn't gone out
in a release yet.
9 years ago
Toshio Kuratomi 56ae3a032f Merge pull request #12075 from ansible/fix-vault-unicode
Unicode and other fixes for vault
9 years ago
Toshio Kuratomi a3fd4817ef Unicode and other fixes for vault 9 years ago
Toshio Kuratomi 16e8a7dd67 Merge pull request #11767 from amenonsen/vault-new-password-file
add option to ansible-vault to read new password from file for rekey
9 years ago
Toshio Kuratomi 156feec264 Merge pull request #11650 from objectified/feature-docker-connection
allow ansible to connect to docker containers (without using ssh)
9 years ago
Abhijit Menon-Sen 8bf0dbb7a9 Use [x:y] host ranges instead of [x-y]
This commit deprecates the earlier groupname[x-y] syntax in favour of
the inclusive groupname[x:y] syntax. It also makes the subscripting
code simpler and adds explanatory comments.

One problem addressed by the cleanup is that _enumeration_info used to
be called twice, and its results discarded the first time because of the
convoluted control flow.
9 years ago
Abhijit Menon-Sen 73f10de386 Document the behaviour of _match_one_pattern in some detail
The possibilities are complicated enough that I didn't want to make
changes without having a complete description of what it actually
accepts/matches. Note that this text documents current behaviour, not
necessarily the behaviour we want. Some of this is undocumented and may
not be intended.
9 years ago
Abhijit Menon-Sen fa6ffa1dbd Remove & and ! pattern prefixes as early as possible
Now everything under _match_one_pattern can ignore them. This also means
that we can use the cache to return the same results for 'foo' and '!foo'.
9 years ago
Abhijit Menon-Sen 704c3815d3 Reorder functions into a logical sequence based on usage
There are no code changes; this is committed separately so as to make
the subsequent "real" diffs easier to read.
9 years ago
Richard Poole 3090a45891 add option to ansible-vault to read new password from file for rekey
The --new-vault-password-file option works the same as
--vault-password-file but applies only to rekeying (when
--vault-password-file sets the old password). Also update the manpage
to document these options more fully.
9 years ago
Brian Coca ae91cdfc98 fixed environment inheritance 9 years ago
Brian Coca 8aa732e0a4 allow for lists, sets and dicts to default to None, now return empty type in post processing
remove defaults from inhertiable fieldattributes to allow for proper detection and override
9 years ago
Marius Gedminas 44c94328c8 Speed up execution
`if method in dir(self):` is very inefficient:

- it must construct a list object listing all the object attributes & methods
- it must then perform a O(N) linear scan of that list

Replace it with the idiomatic `if hasattr(self, method):`, which is a
O(1) expected time hash lookup.

Should fix #11981.
9 years ago
Rene Moser 3db4039ad1 cloudstack: implement general api_region support, update docs 9 years ago
objectified b1785a0361 replace compare_versions() with distutils.version 9 years ago
objectified c39fb43ad9 added Maintainer comment header 9 years ago
Toshio Kuratomi 1f7b0fee0a Fixes #12076 9 years ago
Brian Coca 17060f9849 remove +1 from size that was cutting off first char of copied files when a recursive dir
fixes #12055
9 years ago
Toshio Kuratomi 6e107d2f22 Comments pointing the way towards substituting cryptography for pycrypto 9 years ago
Brian Coca 16f3f8e244 now does not error out when notes are not included in module 9 years ago
objectified d9723069c5 align exec_command() definition with local.py 9 years ago
objectified 3a5522a22c fake being connected for logging purposes 9 years ago
objectified 8f2a6a9fae use docker cp when docker >=1.8.0 9 years ago
objectified 2de773477f allow ansible to connect to docker containers 9 years ago
James Cammarata 21e421ce53 Validate required list items are not None or empty strings
Fixes #12011
9 years ago
James Cammarata db65503778 Revert "Add PowerShell exception handling and turn on strict mode." 9 years ago
Abhijit Menon-Sen 3aedc0bca9 Don't insist on ansible-vault taking only one filename parameter
Apart from ansible-vault create, every vault subcommand is happy to deal
with multiple filenames, so we can check that there's at least one, and
make create check separately that there aren't any extra.
9 years ago
Brian Coca f1b8323b62 fixed host/group var loading when inventory is a directory 9 years ago
Brian Coca af06a97f17 kept vars as alias to hostvars[inventory_hostname] and avoid subtrees into themselves 9 years ago
Brian Coca 3ccfebc9f7 Merge pull request #12047 from cchurch/powershell_common_cleanup
Add PowerShell exception handling and turn on strict mode.
9 years ago
Brian Coca a4ffa09414 Merge pull request #11880 from bcoca/configurable_squash
made squashable with_ plugin list configurable
9 years ago
Abhijit Menon-Sen 09e4eac2e5 Use rsplit(':',1) for clarity; no functional changes 9 years ago
Ryan Petrello 1886307845 Fix a parsing bug that prevents IPv6 addresses from being used with `add_host`
Closes #8682
9 years ago
Chris Church 4b2cdadc98 Add PowerShell exception handling and turn on strict mode.
* Add exception handling when running PowerShell modules to provide exception message and stack trace.
* Enable strict mode for all PowerShell modules and internal commands.
* Update common PowerShell code to fix strict mode errors.
* Fix an issue with Set-Attr where it would not replace an existing property if already set.
* Add tests for exception handling using modified win_ping modules.
9 years ago
Brian Coca 47d9e7ca93 Merge pull request #11984 from Alphadelta14/devel
Support any Mapping for with_dict lookup.
9 years ago
James Cammarata 0958edfc7c Submodule pointer update 9 years ago
Brian Coca e8157eab19 now output works for both search and info 9 years ago
Brian Coca a6c0661d21 made src more prominent 9 years ago
Brian Coca 6ffd9c3025 draft galaxy cli search
TODO: paging results
9 years ago
James Cammarata e282309f6d Make sure the inventory restriction is not None 9 years ago
James Cammarata 635fa0757b Several var fixes
* Fixes hostvar serialization issue (#12005)
* Fixes regression in include_vars from within a role (#9498), where
  we had the precedence order for vars_cache (include_vars, set_fact)
  incorrectly before role vars.
* Fixes another bug in which vars loaded from files in the format of
  a list instead of dictionary would cause a failure.

Fixes #9498
Fixes #12005
9 years ago
Brian Coca 144da7e7d1 Merge pull request #11765 from ldx/vault_pbkdf2hmac
Use PBKDF2HMAC() from cryptography for vault keys.
9 years ago
Brian Coca f150fe2c23 moved mandatory back to filters as it does not always return a boolean, it does an exception on failure 9 years ago
Brian Coca 2787b3acac ported consul_kv lookup from v1 9 years ago
Brian Coca 74079db8d2 ported dig lookup from v1 9 years ago
Brian Coca 9bb95b5235 Merge pull request #12031 from amenonsen/deadcode
Remove unused (copied) _before_comment method
9 years ago
Brian Coca f6b6ed530b added file tests 9 years ago
Brian Coca cd4a0c70b0 isnotanumber is not needed as jinja2 has builtin number test 9 years ago
Brian Coca fe06577ac2 fixed mandatory test 9 years ago
Abhijit Menon-Sen 599ad9cb51 Remove unused (copied) _before_comment method
This was copied from inventory/ini.py, but the rewritten version doesn't
use it, and shows that it isn't needed.
9 years ago
Toshio Kuratomi b5a078a385 update submodule refs 9 years ago
Abhijit Menon-Sen 745ecd4845 Sanitize IPv6 hostname/port handling
Now we accept IPv6 addresses _with port numbers_ only in the standard
[xxx]:NN notation (though bare IPv6 addresses may be given, as before,
and non-IPv6 addresses may also be placed in square brackets), and any
other host identifiers (IPv4/hostname/host pattern) as before, with an
optional :NN suffix.
9 years ago
Abhijit Menon-Sen 74aab6f726 Use a self._raise_error helper and avoid passing the lineno around
Based on a patch by @Richard2ndQuadrant.
9 years ago
Abhijit Menon-Sen 9133cd409c Make _parse take an array of input lines as an argument
(There's no compelling reason to do this right now, but should be parser
need to be called multiple times in future, this makes it easier.)
9 years ago
Abhijit Menon-Sen 98a1905796 Rename 'section' to 'groupname' to better reflect its purpose 9 years ago
Abhijit Menon-Sen 1284c49bd7 Rewrite the INI InventoryParser
The new code parses INI-format inventory files in a single pass using a
well-documented state machine that reports precise errors and eliminates
the duplications and inconsistencies and outright errors in the earlier
three-phase parsing code (e.g. three ways to skip comments). It is also
much easier now to follow what decisions are being taken on the basis of
the parsed data. The comments point out various potential improvements,
particularly in the area of consistent IPv6 handling.

On the ornate marble tombstone of the old code, the following
inscription is one last baffling memento from a bygone age:

-    def _before_comment(self, msg):
-        ''' what's the part of a string before a comment? '''
-        msg = msg.replace("\#","**NOT_A_COMMENT**")
-        msg = msg.split("#")[0]
-        msg = msg.replace("**NOT_A_COMMENT**","#")
-        return msg
9 years ago
Jon Hawkesworth a46b500851 Add win_splitdrive filter for windows users 9 years ago
Toshio Kuratomi 4f32a61504 Merge pull request #10957 from feanil/feanil/retain_nonetypes
Don't convert nulls to strings.
9 years ago
Brian Coca e8b86f448d fix issue with improper connection override in delegation 9 years ago
Feanil Patel 892e230514 Don't convert nulls to strings.
This change is similar to https://github.com/ansible/ansible/pull/10465

It extends the logic there to also support none types.  Right now if you have
a '!!null' in yaml, and that var gets passed around, it will get converted to
a string.

eg. defaults/main.yml
```
ENABLE_AWESOME_FEATURE: !!null # Yaml Null
OTHER_CONFIG:
  secret1: "so_secret"
  secret2: "even_more_secret"

CONFIG:
  hostname: "some_hostname"
  features:
    awesame_feature: "{{ ENABLE_AWESOME_FEATURE}}"
  secrets: "{{ OTHER_CONFIG }}"
```

If you output `CONFIG` to json or yaml, the feature flag would get represented in the output
as a string instead of as a null, but secrets would get represented as a dictionary.  This is
a mis-match in behaviour where some "types" are retained and others are not.  This change
should fix the issue.

I also updated the template test to test for this and made the changes to v2.

Added a changelog entry specifically for the change from empty string to null as the default.

Made the null representation configurable.

It still defaults to the python NoneType but can be overriden to be an emptystring by updating
the DEFAULT_NULL_REPRESENTATION config.
9 years ago
Brian Coca 22a69e2498 fixes to delegation code 9 years ago
Brian Coca 36cbd771a4 changed fixme to deprecated 9 years ago
Brian Coca 961bee00d5 centralized the definition of 'localhost' 9 years ago
Rene Moser d16429b59c cloudstack: rename returns for consistency 9 years ago
Abhijit Menon-Sen 2641f70e47 Reset self._groups_cache in refresh_inventory 9 years ago
Harald Laabs 73d6da757f Make get_group run in O(n*log(n)) instead of O(n^2) by using a cache 9 years ago
Brian Coca 1b810e3101 Merge pull request #12000 from bcoca/local_action_connect
changed local_action to alias to connection=local vs delegate_to=locahost
9 years ago
Will West b8865314ec improve ansible start up time for very large inventory
first off, we add an oddly slow basic test of 10k item inventory

Before:
```
Ran 229 tests in 13.214s

OK

real    0m13.403s
user    0m12.106s
sys 0m1.155s
```

After:
```
Ran 230 tests in 21.328s

OK

real    0m21.516s
user    0m20.099s
sys 0m1.275s
```

since that seems like a bit long for the test to add to runtime, lets profile

`python -m cProfile -s time ./bin/ansible all -i test/units/inventory_test_data/huge_range --list-hosts`

Before:
```
         1272607 function calls (1259689 primitive calls) in 8.497 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10000    4.393    0.000    4.396    0.000 __init__.py:395(_get_host)
    20000    2.695    0.000    2.697    0.000 __init__.py:341(__append_host_to_results)
    40369    0.113    0.000    0.113    0.000 {posix.lstat}
    50006    0.102    0.000    0.153    0.000 __init__.py:1490(combine_vars)
    40008    0.089    0.000    0.202    0.000 __init__.py:1546(_load_vars_from_path)
    20195    0.088    0.000    0.088    0.000 {posix.stat}
    10011    0.087    0.000    0.087    0.000 {posix.getcwd}
```

The top two lines are promising optimization targets

- populate Inventory's host cache more in _get_host, as we are looping
  over all the groups anyways.

- eliminate duplicate check of whether we've already included a host
  in the construction around __append_host_to_results we can infer
  presence of a host in the results list implies the presence of its
  name in the hostnames set, allowing us to only to the less expensive
  of the two checks

After:
```
         1252610 function calls (1239692 primitive calls) in 1.320 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    40369    0.105    0.000    0.105    0.000 {posix.lstat}
    50006    0.094    0.000    0.141    0.000 __init__.py:1490(combine_vars)
    40008    0.081    0.000    0.184    0.000 __init__.py:1546(_load_vars_from_path)
    10011    0.080    0.000    0.080    0.000 {posix.getcwd}
    20195    0.074    0.000    0.074    0.000 {posix.stat}
    10002    0.069    0.000    0.261    0.000 __init__.py:1517(load_vars)
```
9 years ago
Brian Coca 7a4a156d91 changed local_action to alias to connection=local vs delegate_to=localhost
fixes #11998, but still leaves issue of delegate_to: localhost not working
9 years ago
Brian Coca f2c6785897 Merge pull request #11989 from bcoca/osx_queue_fix
changed check to avoid OS X broken sem_getvalue()
9 years ago
Brian Coca 26ed50ecdf fixed indent 9 years ago
Brian Coca 2198f48f54 changed check to avoid OS X broken sem_getvalue()
fixes #11988
9 years ago
Brian Coca e081a73aa1 make new inventory config take precedence over old hostfile config
fixes #11907
9 years ago
Brian Coca 17b4b1f85c added ability to limit in ansilbe pull
refactored the options a bit, new inventory_opts made sense to always group
fixes #7917
9 years ago
Brian Coca 1acb29ff9b sequence will now run once for start=end, added test to suite
fixes #11979
9 years ago
Brian Coca ee21ebaffc Merge pull request #11975 from mutemule/become_doas
Add become support for OpenBSD doas
9 years ago
Damian Gerow 1c5611100e Add become support for OpenBSD doas 9 years ago
Alphadelta14 77b60a1a67 Support any Mapping for with_dict lookup.
This resolves 3939348286 breaking `with_dict: hostvars`.
9 years ago
Brian Coca c419cacb04 check if host task is none before continuing
fixes #11982
9 years ago
James Cammarata e7b2308b66 Performance improvements
* Speed up serialization of hostvars by simply using the internal
  dictionary used for cached lookups
* Use blocking gets/puts on queues instead of spin locking
* Merge sequential implicitly created blocks
9 years ago
Brian Coca 4068598c15 Merge pull request #11977 from resmo/for-ansible
cloudstack: add more common returns
9 years ago
Rene Moser 2913d2b44d cloudstack: add more common returns 9 years ago
Brian Coca 6058eaa92f removed unused poller 9 years ago
Brian Coca 9b61cf5840 implemented async tasks in adhoc v2 9 years ago
Brian Coca ee2e31b37a now passes the test of skipping list when dict attribute is undefined, added deprecation warning as this seems like bad behaviour 9 years ago
Brian Coca 950622cebd made sure they all look in files/vars/template dirs also when executed in play and not only from task 9 years ago
Brian Coca 861fdfc5ba updated callbacks to properly test diff no matter if dict or list 9 years ago
Brian Coca 79ea6d3a1e copy now passes a list of diffs 9 years ago
Brian Coca c3372936aa diff can now be a list 9 years ago
Brian Coca 13c91ef9d4 actually use the read source contents for the diff 9 years ago
Brian Coca 45cebd283f added remote_src to copy, now allows copying to be done on the target server 9 years ago
Brian Coca 1b7369cc8d added diff to assemble for remote_src=false 9 years ago
Brian Coca f048aca5f6 runas placeholder 9 years ago
Brian Coca 49eb95e2d1 some fixes to become/sudo
* now it uses -n to get immediate error if no password is supplied and one is needed,
   this should fix the issue with sudo hanging waiting for input.
 * made -k configurable, this can break changing become_users in play if left out,
   but opens up the possiblity of OTP support.
9 years ago
James Cammarata 4836641683 Use ansible_python_interpreter value for remote checksums
Fixes #11968
Fixes #11969
9 years ago
Brian Coca 7a51836530 check for failure in fact gathering 9 years ago
Brian Coca 1edae51e05 added generic service module, step #1 to separating services to their own modules
falls back to current service module and requries service mgr facts and new service modules
9 years ago
Brian Coca 172e0c328c fixed adhoc fact gathering 9 years ago
Brian Coca 0381923333 fixed call to setup module 9 years ago
Brian Coca 92e2f54228 fixed issues with utf-8 encoding in docs, moved pager to use display class instad of bare prints 9 years ago
Brian Coca 245e0dd4b7 made returns from next task consistent to avoid typeerrors, also linear now checks for none
fixes #11965
9 years ago
Brian Coca 9f5e4c0ef6 lookup plugin that does no flattening 9 years ago
Abhijit Menon-Sen 8f231ec469 Tweak deprecation warning, and apply it to --limit also 9 years ago
James Cammarata 4714cbeec8 Override post validation of environment to allow bare variables
Also prints a deprecated warning if a bare variable is detected, so
that we can remove this in a future version.

Fixes #11912
9 years ago
James Cammarata 4dba30ccd0 Cleanup missed merge conflict sections 9 years ago
Abhijit Menon-Sen d41db8edd6 Don't accept undocumented "p1;p2" patterns; note in changelog
The correct (and now only) way to say p1 OR p2 is "p1:p2".
9 years ago
Abhijit Menon-Sen 1437b8e120 Introduce _split_pattern to parse an "x:y:z" pattern
This function takes a string like 'foo:bar[1:2]:baz[x:y]-quux' and
returns a list of patterns ['foo', 'bar[1:2]', 'baz[x:y]-quux'], i.e.
splits the string on colons that are not part of a range specification.
9 years ago
James Cammarata 32034feec5 Add deprecation warning regarding host patterns 9 years ago
Abhijit Menon-Sen 6bf6b9fdbb Rename some internal functions, clarify what they do
get_hosts → used externally, not changed
_get_hosts → _evaluate_patterns (takes a list, evaluates ! and &)
__get_hosts → _match_one_pattern (takes one pattern only, ignores !&)
9 years ago
Abhijit Menon-Sen eff33bac49 Don't accept undocumented "p1;p2" patterns; note in changelog
The correct (and now only) way to say p1 OR p2 is "p1:p2".
9 years ago
Abhijit Menon-Sen 8aaa06dd50 Introduce _split_pattern to parse an "x:y:z" pattern
This function takes a string like 'foo:bar[1:2]:baz[x:y]-quux' and
returns a list of patterns ['foo', 'bar[1:2]', 'baz[x:y]-quux'], i.e.
splits the string on colons that are not part of a range specification.
9 years ago
Abhijit Menon-Sen baf637b9ae Remove dead code: Inventory.also_restrict_to/lift_also_restriction
This was used earlier to implement serial, but that's now done using
restrict_to_hosts() (whose docstring is also suitably adjusted here)
and there are no more callers.
9 years ago
James Cammarata 6c036c3633 Template parent include file paths
Fixes #11955
9 years ago
Brian Coca b905f51341 fixed typo 9 years ago
Brian Coca 8af0ff4a58 Merge pull request #11567 from lekum/shelvefile_lookup
Add shelvefile lookup plugin
9 years ago
Brian Coca fbc7224066 Merge pull request #11643 from bcoca/meta_inventory_and_fixes
Meta inventory and fixes
9 years ago
James Cammarata eb381bd522 Add one more search path to path_dwim_relative 9 years ago
Brian Coca c6733cc1a7 Merge pull request #9225 from qvicksilver/aix_unprivileged
AIX unprivileged facts collection
9 years ago
Brian Coca 5027f66562 Merge pull request #11864 from jhawkesworth/win_path_filters
add win_basename and win_dirname filters
9 years ago
James Cammarata a498876bec Make sure playbook includes use unique var dictionaries before modification 9 years ago
Brian Coca 5fec11f513 capture parsing exceptions and give nice error and location 9 years ago
Brian Coca 621b18c6ef fixed source path discovery for unarchive in roles
fixes #11943
9 years ago
Brian Coca 70d024ffce show errors when missing vars file or when there is syntax issue in the file
fixes #11945
9 years ago
Brian Coca 3d9875773b Merge pull request #11947 from resmo/for-ansible
cloudstack: refactor get_result()
9 years ago
Brian Coca 2c34a5bdbd pass list correctly to lookup 9 years ago
James Cammarata dd67b1e4d6 Adjust relative include searches when roles are involved
Fixes #11895
9 years ago
Rene Moser 4f3c863b42 cloudstack: refactor get_result()
* A commen dict of keys has been defined, which we look in results returned from the API.
* self.returns dict can be use in subclass to extend this dict.
* Optionally the key name can be replaced with a new key name, often used to make the return keys identical to the arguments passed.
* Use new style class
9 years ago
James Cammarata d9833f227f Make sure cached data from file loads isn't impacted by modifications
Fixes #11893
9 years ago
Brian Coca 6bceee9a93 now lvs/vg facts are only attempted if binary found 9 years ago
James Cammarata c7dde72aa0 Default listify to converting bare variables again
Since we explicitly set convert_bare=False in the template lookup
code, but still want individual looks that call listify directly to
convert bare variables if needed.
9 years ago
James Cammarata ac66e4001c Fix for variable precedence with tasks/includes
Fixes #11881
9 years ago
Brian Coca 2688edb32b Merge pull request #11940 from alxgu/fix_lvm_facts
bugfix: missing path for lvm facts
9 years ago
Brian Coca 39f81a8fa5 removed unused imports 9 years ago
Brian Coca c27978fa93 minor fixes to pull 9 years ago
Brian Coca 9f29e39dea give more matching options for ansible-pull 9 years ago
Brian Coca e46b1e8d2b adjusted tree to work more like v1, also now skips when dir not supplied 9 years ago
Alexander Gubin 1ea1b9fb13 bugfix: missing path for lvm facts 9 years ago
Brian Coca 49d88cef9c meta: refresh_inventory and several inventory fixes 9 years ago
Brian Coca 8e30e3791e Merge pull request #11811 from amenonsen/fixme-range
FIXME in host range parsing
9 years ago
Brian Coca ae9d264b86 Merge pull request #11762 from amenonsen/9795-rebase
Add LVM facts to setup module
9 years ago
James Cammarata 829a88b906 Disable undefined error failures during conditional templating
This allows us to properly catch the 'is defined' and 'is undefined'
checks and return the correct boolean result.

Fixes #11892
9 years ago
James Cammarata e0b074000e Merge pull request #11764 from lpirl/devel_v2
fixes remote code execution for su/sudo with strict remote umasks
9 years ago
James Cammarata 38eb53de1d Fix bug in vars using correct temp value for vars
Fixes #11877
9 years ago
Brian Coca 4d853a5d3c implemented for v2, missing --tree option for adhoc 9 years ago
Brian Coca 79a1bca010 avoid circular import 9 years ago
Brian Coca a83b8220ef added import back 9 years ago