Commit Graph

5404 Commits (bafcd7becd0c0c72f1182b25f1ff7fcb030c69e8)

Author SHA1 Message Date
soarpenguin 1ea66e234f fix logic judgment error. 10 years ago
soarpenguin 2caa52a981 fix no self pointer out of class. 10 years ago
Brian Coca 37a918438b task logging revamp
* allow global no_log setting, no need to set at play or task level, but can be overriden by them
 * allow turning off syslog only on task execution from target host (manage_syslog), overlaps with no_log functionality
 * created log function for task modules to use, now we can remove all syslog references, will use systemd journal if present
 * added debug flag to modules, so they can make it call new log function conditionally
 * added debug logging in module's run_command
10 years ago
James Cammarata 60e1a1f8eb Fix bug in nested lookup where pop is called indiscriminately 10 years ago
Brian Coca 191ae0831d fixed syntax error 10 years ago
Brian Coca 48412c13b2 fixed copy n paste typo on future code 10 years ago
James Cammarata bb1101498d Clean up typo from cdc6c52 10 years ago
James Cammarata cdc6c5208e Clean string data run through the template engine
Also strip UnsafeProxy off of low level srings and objects to ensure
they don't cause issues later down the road

Fixes #12513
10 years ago
James Cammarata ae9b34b1d9 Fix for equality checking in Host to make sure things are like types 10 years ago
Toshio Kuratomi fa9ea32a86 Fix test of whether a result has a failed host 10 years ago
Brian Coca 1c49e3b842 draft shared service code for modules, part of spliting service module 10 years ago
Toshio Kuratomi 41756be4d3 Update core module ref 10 years ago
Toshio Kuratomi dcdcd9e9c5 Move is_executable to the toplevel of basic.py so we can utilize it from other code 10 years ago
James Cammarata ccddda1ebc Merge pull request #12502 from mgedmin/py3k
Python 3: make test_variable_manager_precedence pass
10 years ago
James Cammarata eb8d7dcd14 Make sure formatted strings don't bomb on tuples
Fixes #12501
10 years ago
James Cammarata 31d5f88a1d Use the task loop to calculate multiple delegated hosts
Due to the way we're now calculating delegate_to, if that value is based
on a loop variable ('item') we need to calculate all of the possible
delegated_to variables for that loop.

Fixes #12499
10 years ago
Toshio Kuratomi a1428d6bed Remove tmp as a parameter to the connection plugins
There doesn't appear to be anything that actually uses tmp_path in the
connection plugins so we don't need to pass that in to exec_command.
That change also means that we don't need to pass tmp_path around in
many places in the action plugins any more.  there may be more cleanup
that can be done there as well (the action plugin's public run() method
takes tmp as a keyword arg but that may not be necessary).

As a sideeffect of this patch, some potential problems with chmod and
the patch, assemble, copy, and template modules has been fixed (those
modules called _remote_chmod() with the wrong order for their
parameters.  Removing the tmp parameter fixed them.)
10 years ago
James Cammarata 95b371dd60 Use AnsibleFileNotFound instead of AnsibleParsingError when YAML files are not found
And update portions of code to expect the proper error.

Fixes #12512
10 years ago
Toshio Kuratomi 0250beb68a Remove compress option from paramiko connection for now
It's not available on older versions of paramiko such as shipped in RHEL6
10 years ago
James Cammarata 12df9f2e31 Make hostvars more dynamic again to improve performance with large inventories
Fixes #12477
10 years ago
Abhijit Menon-Sen 82b33c381f We don't need even a token timeout here; just poll once
The process is already gone, so there's not going to be any new data
showing up on its stderr; we only want to make sure that we haven't
missed something that was already written. So polling once is enough.
10 years ago
Abhijit Menon-Sen 6e82df451a Clarify select() handling for ssh connections
This change is motivated by an ssh oddity: when ControlPersist is
enabled, the first (i.e. master) connection goes into the background; we
see EOF on its stdout and the process exits, but we never see EOF on its
stderr. So if we ran a command like this:

    ANSIBLE_SSH_PIPELINING=1 ansible -T 30 -vvv somehost -u someuser -m command -a whoami

We would first do select([stdout,stderr], timeout) and read the command
module output, then select([stdout,stderr], timeout) again and read EOF
on stdout, then select([stderr], timeout) AGAIN (though the process has
exited), and select() would wait for the full timeout before returning
rfd=[], and then we would exit. The use of a very short timeout in the
code masked the underlying problem (that we don't see EOF on stderr).

It's always preferable to call select() with a long timeout so that the
process doesn't use any CPU until one of the events it's interested in
happens (and then select will return independent of elapsed time).

(A long timeout value means "if nothing happens, sleep for up to <x>";
omitting the timeout value means "if nothing happens, sleep forever";
specifying a zero timeout means "don't sleep at all", i.e. poll for
events and return immediately.)

This commit uses a long timeout, but explicitly detects the condition
where we've seen EOF on stdout and the process has exited, but we have
not seen EOF on stderr. If and only if that happens, it reruns select()
with a short timeout (in practice it could just exit at that point, but
I chose to be extra cautious). As a result, we end up calling select()
far less often, and use less CPU while waiting, but don't sleep for a
long time waiting for something that will never happen.

Note that we don't omit the timeout to select() altogether because if
we're waiting for an escalation prompt, we DO want to give up with an
error after some time. We also don't set exceptfds, because we're not
actually acting on any notifications of exceptional conditions.
10 years ago
Toshio Kuratomi 03127dcfae remove the stdin return value from connection plugin exec_command() methods
The value was useless -- unused by the callers and always hardcoded to
the empty string.
10 years ago
James Cammarata 9d47eabfa4 Merge pull request #12506 from hyperized/devel
Add Weekday (0-6) as a number and add weeknumber (00-52)
10 years ago
Toshio Kuratomi 24b9e2e6d1 Update extras submodule ref 10 years ago
Gerben Geijteman 4c20964475 Add Weekday (0-6) as a number and add weeknumber (00-52) 10 years ago
Marius Gedminas 56f2a25bff Python 3: there's no 'unicode' 10 years ago
Marius Gedminas 6d4618f46f Python 3: there's no dict.iteritems() 10 years ago
Marius Gedminas 0624797375 Bugfix: if you define a custom __eq__, you must define a __hash__ too
Also, on Python 3 the stock object.__hash__ raises an error ("unhashable
type"), and we have code that uses Host instances as dict keys.
10 years ago
Marius Gedminas a2bc6b4b26 Bugfix: if you define __eq__, you should define __ne__ too 10 years ago
Marius Gedminas 5d29a2eabd Python 3: shlex.split() wants unicode
On Python 2, shlex.split() raises if you pass it a unicode object with
non-ASCII characters in it.  The Ansible codebase copes by explicitly
converting the string using to_bytes() before passing it to
shlex.split().

On Python 3, shlex.split() raises ('bytes' object has no attribute 'read')
if you pass a bytes object.  Oops.

This commit introduces a new wrapper function, shlex_split, that
transparently performs the to_bytes/to_unicode conversions only on
Python 2.

Currently I've only converted one call site (the one that was causing a
unit test to fail on Python 3).  If this approach is deemed suitable,
I'll convert them all.
10 years ago
Toshio Kuratomi 5d3d9cfe0d Convert to byte strings to avoid UnicodeErrors
Fixes #12488
10 years ago
Brian Coca de18bcb95f correct typo on error reporting
fixes #12495
10 years ago
Abhijit Menon-Sen 40f608a377 A bit more debugging output
We used to display input chunks earlier anyway, so this isn't making
things more verbose.
10 years ago
Abhijit Menon-Sen 9700d9c04f Fix typo in checking select results
It's possible for more than one fd to be set, so 'elif' is obviously not
the right thing to use.
10 years ago
James Cammarata 1164e83477 Remove unnecessary calls to save inventory restrictions since 81bf88b 10 years ago
Toshio Kuratomi 89a78ba16e Update submodule refs 10 years ago
James Cammarata 9e734df0ec Conditionally poll longer if we're still waiting for an auth prompt 10 years ago
Toshio Kuratomi 5f0f5363b6 Merge pull request #12487 from mgedmin/py3k
Fix one more failing test on Python 3
10 years ago
James Cammarata 2898e000a0 Don't use the connection timeout for the select poll timeout 10 years ago
Abhijit Menon-Sen 587054db2a Send initial data before calling select whenever possible
Without this, we could execute «ssh -q ...» and call select(), which
would timeout after the default 10s, and only then send initial data.
(This is a relic of the earlier change where we always ran ssh with
-vvv, so the situation where it would sit quietly never happened in
practice; but this would have been the right thing to do even then.)
10 years ago
James Cammarata c9a004227e Improve error catching from malformed playbook data
Fixes #12478
10 years ago
James Cammarata e8e1d9f6fb Apply --limit to inventory in adhoc commands
Fixes #12473
10 years ago
Marius Gedminas 95e655eb67 Python 3: there's no basestring
Fixes one failing test.

The long series of module_utils/basic.py fixes were all because
module_utils/basic is imported in ansible/inventory/script.py.
10 years ago
Marius Gedminas 2c4982b58d Python 3: there's no itertools.imap
Because the builtin map() acts like an iterator already.
10 years ago
Marius Gedminas 6708d56a21 Python 3: avoid long integer literals
Even Python 2.4 automatically promotes int to long.
10 years ago
Marius Gedminas f5d4935197 Python 3: treat python as a function in module_utils/basic.py
NB: we can't use 'from __future__ import print_function', but luckily
print(one_thing) works fine on both Python 2 and Python 3 without that.
10 years ago
Marius Gedminas e71a986e16 Python 3: avoid octal constants in module_utils/basic.py 10 years ago
Marius Gedminas d2bec7f81f Python 3: avoid "except ..., e:" in module_utils/basic.py
Make the code compatible with Pythons 2.4 through 3.5 by using
sys.exc_info()[1] instead.

This is necessary but not sufficient for Python 3 compatibility.
10 years ago
James Cammarata 65630d2ce1 Fixing one more bug related to staticmethods in LookupBase 10 years ago
James Cammarata cbbb270761 Cleanup bug from moving base lookup methods to staticmethods 10 years ago
Abhijit Menon-Sen ac98fe9e89 Implement ssh connection handling as a state machine
The event loop (even after it was brought into one place in _run in the
previous commit) was hard to follow. The states and transitions weren't
clear or documented, and the privilege escalation code was non-blocking
while the rest was blocking.

Now we have a state machine with four states: awaiting_prompt,
awaiting_escalation, ready_to_send (initial data), and awaiting_exit.
The actions in each state and the transitions between then are clearly
documented.

The check_incorrect_password() method no longer checks for empty strings
(since they will always match), and check_become_success() uses equality
rather than a substring match to avoid thinking an echoed command is an
indication of successful escalation. Also adds a check_missing_password
connection method to detect the error from sudo -n/doas -n.
10 years ago
Abhijit Menon-Sen 840a32bc08 Reorganise ssh.py to cleanly separate responsibilities
The main exec_command/put_file/fetch_file methods now _build_command and
call _run to handle input from/output to the ssh process. The purpose is
to bring connection handling together in one place so that the locking
doesn't have to be split across functions.

Note that this doesn't change the privilege escalation and connection IO
code at all—just puts it all into one function.

Most of the changes are just moving code from one place to another (e.g.
from _connect to _build_command, from _exec_command and _communicate to
_run), but there are some other notable changes:

1. We test for the existence of sshpass the first time we need to use
   password authentication, and remember the result.
2. We set _persistent in _build_command if we're using ControlPersist,
   for later use in close(). (The detection could be smarter.)
3. Some apparently inadvertent inconsistencies between put_file and
   fetch_file (e.g. argument quoting, sftp -b use) have been removed.

Also reorders functions into a logical sequence, removes unused imports
and functions, etc.

Aside: the high-level EXEC/PUT/FETCH description should really be logged
from ConnectionBase, while individual subclasses log transport-specific
details.
10 years ago
James Cammarata 95c6fe88e4 Fix handling of conditional vars_files which contain variables
Fixes #12484
10 years ago
Toshio Kuratomi c83f51b7f2 Some LookupBase cleanups:
* Make LookupBase an abc with required methods (run()) marked as an
  abstractmethod
* Mark methods that don't use self as @staticmethod
* Document how to implement the run method of a lookup plugin.
10 years ago
Toshio Kuratomi 049952fa50 Update submodule refs. 10 years ago
James Cammarata 1e7fd2196d Fixing synchronize + delegate_to user bug
Fixes #12464
10 years ago
James Cammarata 3ffc2783c4 Don't bomb out on handlers with undefined variables in their names 10 years ago
Toshio Kuratomi 4b0d52d2cb Merge pull request #12420 from ansible/win_prefix_modules
Fix for user defined modules not overriding modules from core.
10 years ago
James Cammarata 1076155d8d When failing because of vars_files templating, try and bubble up the file/line info 10 years ago
Toshio Kuratomi 18e2ee16ef Fix for user defined modules not overriding modules from core.
This fix takes into account that powershell modules are somewhat
different than regular modules and have to be kept separate.
10 years ago
Toshio Kuratomi f61fb9787d Update submodule refs 10 years ago
Rene Moser b43939dfd6 cloudstack: make tags handling idempotence
Credits to @jeffersongirao, who provided the patch.
10 years ago
James Cammarata c30e464388 Additional tweaks to callback output for delegate_to 10 years ago
James Cammarata 513619867a Show delegated-to host in callback message
Fixes #12465
10 years ago
Rene Moser efd122c2f0 cloudstack: add returns_to_int return handling into utils
It is not uncommon that the API returns string for int values e.g. ports in listFirewallRules or listPortForwardings,
10 years ago
Rene Moser c9a3801a25 cloudstack: common argument_spec and requried_together to utils 10 years ago
James Cammarata f563b22446 Merge pull request #12461 from mgedmin/py3k
Python 3: there's no basestring
10 years ago
James Cammarata d2949f5449 Merge pull request #12463 from mgedmin/fix-ansible-doc
Fix option descriptions in ansible-doc output
10 years ago
James Cammarata 0fb4a6a67b Tweak to the way new host variables are created for delegated hosts 10 years ago
Marius Gedminas 339790adc4 Fix option descriptions in ansible-doc output
Fixes #12462.
10 years ago
James Cammarata 18adfc6d1a Set some default vars on hosts created for delegate_to connections 10 years ago
James Cammarata a22f7b883d Restrict role param vars to tasks within that role
Fixes #12460
10 years ago
Marius Gedminas fc0dcc3947 Python 3: there's no basestring
Fixes one failing test.

Now technically a filename can be a bytestring, even on Python 3.  I
hope this is unlikely for Ansible.
10 years ago
James Cammarata ecf7d8c9ee Fix relative path bug in copy action
Without rebreaking #12055

Fixes #ansible-modules-core/2098
10 years ago
James Cammarata 53794b692c Merge pull request #12457 from ansible/warn-if-core-modules-not-present
Detect if core modules aren't installed and warn if that is the  case
10 years ago
Toshio Kuratomi 627f9d73ba Detect if core modules aren't installed and warn if that is the case
Fixes #11206
10 years ago
Brian Coca d27b73e7b5 fixed case in which boto3 is present but module is not using it yet 10 years ago
James Cammarata cb7060c9fe Allow undefined var errors to bubble up when templating vars_files in certain conditions
Follow up to 8769f03c, which allows the undefined var error to be raised
if we're getting vars with a full context (play/host/task) and the host
has already gathered facts. In this way, vars_files containing variables
that fail to be templated are not silently ignored.
10 years ago
Brian Coca f96255f7fd fixed typo 10 years ago
James Cammarata 86566e691a Merge pull request #12447 from mgedmin/py3k
Python 3: there is no 'basestring'
10 years ago
James Cammarata 8769f03c16 Also catch AnsibleUndefinedVariable errors when templating vars_files names
Fixes #12449
10 years ago
Toshio Kuratomi 2a614c1c0f Close module file once we're done reading from it 10 years ago
Marius Gedminas 9cdb6ebae3 Python 3: there is no 'basestring'
This fixes a failing unit test.

In actual use (which is still quite far), I'm not sure if bytes ->
unicode conversion should be done here (in which case the code will fail
with an AttributeError: 'bytes' object has no attribute 'readlines'), or
inside self._connection.exec_command() (in which case my change is
correct).
10 years ago
Victor Salgado 20bbd66e85 Use dict.get() 10 years ago
James Cammarata 65bf14cbb0 Merge pull request #12431 from hslee16/devel
Stops using _groups_list to check for host when using 'add_host'
10 years ago
James Cammarata 88e1aa94fa Add play context vars to list of variables before post validation
Fixes #12437
10 years ago
James Cammarata 05f6e76407 Template handler names before checking to see if they need to run
Fixes #12427
10 years ago
James Cammarata 72769d1899 Merge pull request #12432 from mgedmin/py3k
Python 3: there's no xrange
10 years ago
James Cammarata 37f2cbc429 Merge pull request #12436 from amenonsen/ranges
Support «hosts: foo[1:]» and add tests for split/apply_subscript
10 years ago
James Cammarata 1f5584aa5b Refactoring delegate_to code
Now, instead of relying on hostvars on the executor side, we compile
the vars for the delegated to host in a special internal variable and
have the PlayContext object look for things there when applying task/
var overrides, which is much cleaner and takes advantage of the code
already dealing with all of the magic variable variations.

Fixes #12127
Fixes #12079
10 years ago
James Cammarata fa69e8ebb8 Fixing some bugs in _compute_delegate() in TaskExecutor
* Clearing interpreter settings from variables, so those set for the
  original host aren't incorrectly applied to the delegated to host
* Fixed incorrect string for remote user in delegated hosts hostvars
* Properly looking for multiple possiblities in the delegated-to hosts
  hostvars (ansible_ssh_host vs. ansible_host)
10 years ago
Peter Sprygada e45b71d6cd added new module object to vca 10 years ago
Abhijit Menon-Sen 2fcdb37e7b Support «hosts: groupname[1:]» notation (~= 'the rest of the group') 10 years ago
Brian Coca aca83b15e5 Merge pull request #12425 from privateip/devel
initial add of vca common module
10 years ago
Brian Coca 11707576a3 fix for AIX network facts
fixes #12434
10 years ago
Brian Coca d7b61db6dc made boto3 non mandatory 10 years ago
Toshio Kuratomi fdb1c14004 Update the submodule pointers 10 years ago
Petr Mifek 27f779a7cc Small clean up and refactor of the rsync target ipv6 observing formatter for synchronize action. 10 years ago
Marius Gedminas baf9320369 Python 3: there's no xrange
Use six.moves.range instead (aliased to xrange on Python 2, aliased to
range on Python 3).

Also I couldn't resist replacing the elaborate chr/ord/randrange dance
with the simpler random.choice(string.ascii_lowercase) that was already
used elsewhere in the Ansible codebase.
10 years ago
Alex Lee cf289191e5 return None if group does not exist in inventory groups 10 years ago
Petr Mifek 6f3f4dff7a Change synchronize module plugin to be backwards compatible with RSync 2.6.9 with regard to handling IPv6 addresses. 10 years ago
James Cammarata cc6627cdd6 Remove custom json encoder cleaner and strip proxy var stuff out before encoding
Fixes #12349
10 years ago
James Cammarata c95d4ca05f Properly assign search path to environment.loader in template lookup
Same fix as was applied in f162990c to the action plugin

Fixes #12355
10 years ago
James Cammarata bde5ed9672 Catch worker errors that may be subclassed on others that lead to incorrect exits 10 years ago
Peter Sprygada 8f43d222c0 initial add of vca common module 10 years ago
James Cammarata 053c41e79d Fix typo in ansible module_utils import from galaxy code 10 years ago
Chrrrles Paul 7b778d8a4b Merge pull request #12424 from amenonsen/ipv6-ranges
Tested.  Thanks for spotting and then fixing this -- Allow hexadecimal ranges in IPv6 addresses, not only 0-9
10 years ago
Abhijit Menon-Sen 2d420a9bb7 Allow hexadecimal ranges in IPv6 addresses, not only 0-9 10 years ago
James Cammarata b386f0c64d Merge pull request #12423 from amenonsen/split-fixup
Remove deprecation warning; now handled in _split_patterns
10 years ago
James Cammarata a1fffff92c Merge branch 'addresses' of https://github.com/amenonsen/ansible into amenonsen-addresses 10 years ago
Abhijit Menon-Sen 62ca6b5e35 Remove deprecation warning; now handled in _split_patterns 10 years ago
Abhijit Menon-Sen 575a8b8c77 Merge Host.ipv[46]_address into .address
The earlier distinction was never used; .ipv6_address was always a copy
of .ipv4_address, and the latter was always used to set the remote_addr
field in the PlayContext.

Also uses the canonical ansible_host/ansible_port names when setting the
address and port from variables.
10 years ago
James Cammarata 951128d7a6 Merge branch 'hostpatterns' of https://github.com/amenonsen/ansible into amenonsen-hostpatterns 10 years ago
Victor Salgado 14fefebaad Modify _split_pattern to use map when working with list input 10 years ago
James Cammarata 46baed864f Submodule update after correcting doc bug 10 years ago
James Cammarata 0beb37a121 Submodule update 10 years ago
James Cammarata ba82e57445 Merge branch 'sudo_on_if_needed' of https://github.com/apollo13/ansible into apollo13-sudo_on_if_needed 10 years ago
Toshio Kuratomi 8532ddd5ed typo in formatting the warning message 10 years ago
Toshio Kuratomi 4ae5512fb0 Restore python3 compat fix and fix AnsibleError undefined before use 10 years ago
James Cammarata c881b1acae Merge pull request #12407 from mgedmin/py3k
Python 3: basestring in play_context.py
10 years ago
James Cammarata 8acbc00853 Fixing groups variable structure to match v1
Fixes #12418
10 years ago
Abhijit Menon-Sen 349eec7855 Fix missing colon (typo) in IPv6 pattern 10 years ago
Abhijit Menon-Sen 2405861a9e Use ',' instead of ':' or ';' to separate host patterns
The earlier-recommended "pat1:pat2:pat3[x:y]" notation doesn't work well
with IPv6 addresses, so we recommend ',' as a separator instead. We know
that commas can't occur within a pattern, so we can just split on it.

We still have to accept the "foo:bar" notation because it's so commonly
used, but we issue a deprecation warning for it.

Fixes #12296
Closes #12404
Closes #12329
10 years ago
Florian Apolloner d9f873495e Ported over #7158 to support SELinux context switches. 10 years ago
Brian Coca ee840f3029 fixed function signature 10 years ago
Florian Apolloner decd9e8710 Only use become if the target user difers from the current remote user. 10 years ago
Marius Gedminas 8243954dcd Python 3: basestring in play_context.py 10 years ago
Toshio Kuratomi 3db8070aa3 Fix urlopen usage to use open_url instead
Add a travis test for urlopen usage
10 years ago
James Cammarata e97d448838 Cleanup one more missed inventory bug 10 years ago
James Cammarata 5db9e38377 Merge pull request #12345 from mgedmin/py3k
Python 3: two more instances of 'basestring'
10 years ago
Chris Church 43b15ab9a4 Merge pull request #12385 from cchurch/winrm_put_empty_file
Enable winrm put_file to upload an empty file.
10 years ago
Chris Church 93af0b327f Merge pull request #12384 from cchurch/powershell_strict_mode
Add PowerShell exception handling and turn on strict mode.
10 years ago
Chris Church 0325f92b6a Merge pull request #12054 from cchurch/winrm_options
Support additional options for WinRM connections via inventory variables
10 years ago
Chris Church c5409ab493 Remove ansible_winrm_ host/port/user/pass options, update exception handling around establishing a winrm connection. 10 years ago
Chris Church 056c6b77d0 Support additional options for WinRM connections via inventory variables. 10 years ago
James Cammarata 11c27078c0 Cleaning up inventory bugs from earlier group change 10 years ago
Chris Church 1d15e8f37a Pass windows command as-is without splitting/rejoining parameters.
* Fixes extra spaces added between parameters from https://github.com/ansible/ansible-modules-core/issues/1929
* Correctly decode PowerShell command encoded as UTF-16-LE so that it displays correctly in debug messages, fixes the other issue from https://github.com/ansible/ansible-modules-core/issues/1929
* Add test to verify that script parameters are passed as-is, so $true is interpreted as a boolean, fixes https://github.com/ansible/ansible/issues/10947
10 years ago
Victor Salgado 54f1560d3e Fix mutable default argument 10 years ago
James Cammarata 33e79203ce Merge branch 'quinot-topic/crash_empty_inventory' into devel 10 years ago
James Cammarata c8f2483d6d Cleaning up default group creation in inventory parsers 10 years ago
Toshio Kuratomi 30552cf7e9 Fix problems with undefined references to self.display and remove method that's no longer called 10 years ago
James Cammarata 3f8e12d1f7 Merge pull request #12359 from cchurch/fetch_no_fail_on_missing
Fix fetch to not fail for missing file when fail_if_missing=False
10 years ago
James Cammarata 4429e7611b Merge pull request #12382 from AxelDelmas/env_async
Added env_string in async_cmd
10 years ago
James Cammarata e899b8e70d Merge branch 'topic/crash_empty_inventory' of https://github.com/quinot/ansible into quinot-topic/crash_empty_inventory 10 years ago
James Cammarata b2848bd25f Merge pull request #12389 from amenonsen/minus-t
Set PlayContext.timeout from the value specified by -T on the command-line
10 years ago
James Cammarata d91ff0db74 Fixing parent block serialization for blocks
Fixes #12395
10 years ago
Thomas Quinot c005f75469 Always create 'all' group
Create 'all' even if host_list is empty, as this group needs to exist
when creating the implicit 'localhost' host.

Fixes #12391
10 years ago
Abhijit Menon-Sen 85bb508f26 Set PlayContext.timeout from the value specified by -T on the command-line
Fixes #12372
10 years ago
James Cammarata 2a50957ad8 Fix galaxy install dep failure
Also fixes issue where force does not force reinstall of deps

Fixes #10425
10 years ago
Toshio Kuratomi 1a39e32a13 Update core modules ref 10 years ago
Chris Church 6ab4cff7db Enable winrm put_file to upload an empty file. 10 years ago
Chris Church 5c65ee7f0c Add PowerShell exception handling and turn on strict mode.
* Add exception handling when running PowerShell modules to provide exception message and stack trace.
* Enable strict mode for all PowerShell modules and internal commands.
* Update common PowerShell code to fix strict mode errors.
* Fix an issue with Set-Attr where it would not replace an existing property if already set.
* Add tests for exception handling using modified win_ping modules.
10 years ago
James Cammarata a1948dd1c1 Merge branch 'bcoca-normalize_plugin_paths' into devel 10 years ago
James Cammarata a7e1113b26 Removing FIXME comment which is no longer required 10 years ago
James Cammarata d818a72375 Merge branch 'normalize_plugin_paths' of https://github.com/bcoca/ansible into bcoca-normalize_plugin_paths 10 years ago
Axel Delmas 4802641de7 Added env_string in async_cmd 10 years ago
James Cammarata 60c139e482 Fix logic for changed/failed_when + retry/until loops
Fixes #11809
10 years ago
James Cammarata 8e664ad226 Fix delegate_to localhost vs. 127.0.0.1 (and ::1) 10 years ago
James Cammarata 38573c55bf Tweak failure condition for field attributes expected to be strings 10 years ago
James Cammarata cfdaec0c22 Make sure string attributes are strings from YAML to avoid bad type conversions
Fixes #12367
10 years ago
Brian Coca b6d6c2e4db corrected all missing paths changes 10 years ago
Brian Coca 4aea1f6568 normalized plugin paths and names and configs 10 years ago
Brian Coca 02372e9761 Merge pull request #12371 from apollo13/patch-1
Remove duplicate SU_PROMPT_LOCALIZATIONS
10 years ago
Florian Apolloner 9334046189 Remove duplicate SU_PROMPT_LOCALIZATIONS 10 years ago
gptech 1c6be7b9ba Replace 'degug' with 'debug'
Tiny typo fix.
10 years ago
Andriy Yurchuk af213241ab Fix typo 10 years ago
Brian Coca e7807c4f2d Merge pull request #12339 from emonty/feature/openstack-no-log
Add no-log parameter to common openstack args
10 years ago
James Cammarata 4f30db8ca5 Check if path is /dev/null when checking if a file is in fact a file 10 years ago
Chris Church 260b9f648c Fix fetch action plugin to not fail if file is missing and fail_if_missing=False (the default). Add tests to test_fetch role to verify it works as expected. 10 years ago
James Cammarata feee4613cd Merge pull request #12357 from apollo13/become_fix
Fixed #12356 -- Restored ansible 1.9.x become behavior.
10 years ago
James Cammarata 49ca0eb797 Track local_action internally to prevent it from being overridden
Fixes #12053
10 years ago
Florian Apolloner 8182eb3787 Fixed #12356 -- Restored ansible 1.9.x become behavior. 10 years ago
Marius Gedminas 9877a5c415 Python 3: two more instances of 'basestring'
Fixes two failing tests on Python 3.4.
10 years ago
Monty Taylor 3bdd6793ca Add no-log parameter to common openstack args
These bad boys can contain secret data, so should be marked no-log.
10 years ago
James Cammarata aa762bb432 Don't split args out unless we're parsing module args using the new style
Fixes #12331
10 years ago
James Cammarata 1cb4e0be1c Fix bug in implicit block squashing where block is not reassigned
Fixes #12333
10 years ago
James Cammarata 7deb8bbd1c Reworking how includes within handlers are run
Fixes #12238
10 years ago
Brian Coca 03758151e4 Merge pull request #12330 from cchurch/revert_find_plugin_changes
Revert "Fix order of loading of modules."
10 years ago
Brian Coca 9d4d7bd74d fixed porting bugs in profile_tasks to v2 10 years ago
Chrrrles Paul ba7734b78b Merge pull request #12165 from amenonsen/address-parsing
Hi @amenonsen - thanks for fixing up the hunting down the unicode bug and expanding test_addresses.  The code looks good, merging!-- Be systematic about parsing and validating hostnames and addresses
10 years ago
James Cammarata 4156bcedf5 Fix typo in 049a48f 10 years ago
Chris Church ed3a9795d4 Revert "Fix order of loading of modules."
This reverts commit 4b895f04e3.
10 years ago
James Cammarata 1243686b90 Submodule update 10 years ago
Abhijit Menon-Sen 049a48f3d3 Test state.tasks_child_state before dereferencing
I hit a traceback in the FAILED_TASKS case, but it seemed prudent to add
guards in the other similar branches.
10 years ago
James Cammarata df89f03627 Fixing precedence order for new nonpersistent vars (set_fact/register)
These used to go in vars_cache, so merging them in after that as they
are "live" variables and the user would most likely want to see these
above anything else.
10 years ago
Abhijit Menon-Sen 7479ab47e0 Be stricter about parsing hostname labels
Labels must start with an alphanumeric character, may contain
alphanumeric characters or hyphens, but must not end with a hyphen.
We enforce those rules, but allow underscores wherever hyphens are
accepted, and allow alphanumeric ranges anywhere.

We relax the definition of "alphanumeric" to include Unicode characters
even though such inventory hostnames cannot be used in practice unless
an ansible_ssh_host is set for each of them.

We still don't enforce length restrictions—the fact that we have to
accept ranges makes it more complex, and it doesn't seem especially
worthwhile.
10 years ago
Abhijit Menon-Sen 065bb52109 Be systematic about parsing and validating hostnames and addresses
This adds a parse_address(pattern) utility function that returns
(host,port), and uses it wherever where we accept IPv4 and IPv6
addresses and hostnames (or host patterns): the inventory parser
the the add_host action plugin.

It also introduces a more extensive set of unit tests that supersedes
the old add_host unit tests (which didn't actually test add_host, but
only the parsing function).
10 years ago
James Cammarata 49803509b4 Make sure include paths are templated before dwim in case they're absolute
Fixes #12327
10 years ago
James Cammarata 4cdd9b2a82 Merge pull request #12318 from mgedmin/py3k
Python 3: make ansible.template.safe_eval() work
10 years ago
Brian Coca 52f587cac3 Merge pull request #12325 from quinot/topic/wrong_interp_test
Test for Python must be against basename of interpreter
10 years ago
James Cammarata c2115c3715 Re-adding part of 926f127 which was actually fixing #12277 10 years ago
James Cammarata 083aef65ad Revert "set_fact should put facts in the fact_cache, not vars_cache"
This reverts commit 926f127245.
10 years ago
Thomas Quinot 0bc9223826 Test for Python must be against basename of interpreter
Followup on issue #12076
10 years ago
James Cammarata 441a8af671 Merge pull request #12267 from h-hirokawa/add_replacer_jsonargs
Add the new module replacer <<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>.
10 years ago
Marius Gedminas f91b28ef23 Fix safe_eval() of set literals
There was code to support set literals (on Python 2.7 and newer), but it
was buggy: SAFE_NODES.union() doesn't modify SAFE_NODES in place,
instead it returns a new set object that is then silently discarded.

I added a unit test and fixed the code.  I also changed the version
check to use sys.version_tuple instead of a string comparison, for
consistency with the subsequent Python 3.4 version check that I added in
the previous commit.
10 years ago
Marius Gedminas 2b73002044 Python 3: make ansible.template.safe_eval() work
Two things changed in Python 3.4:

- 'basestring' is no longer defined, so use six.string_types

- True/False are now special AST node types (NamedConstant) rather than
  just names

(Good thing we had tests, or I wouldn't have noticed the 2nd thing!)

I found only one place where safe_eval() is called inside the ansible
codebase: in lib/template/__init__.py.  The call to safe_eval(result,
...) is protected by result.startswith('...'), which means result cannot
possibly be a byte string on Python 3 (or startswith() would raise, so
six.string_types (which excludes byte strings on Python 3) is fine here.
10 years ago
James Cammarata 0c5bbe3c32 Use the original (un-templated) args for include params
Fixes #12314
10 years ago
James Cammarata 292e2da4e1 Fix some bugs related to facts/nonpersistent-facts cache split
Fixes #12313
10 years ago
Brian Coca 91c9df2154 added verbosity to ansible-doc to make it easier to trace down issues 10 years ago
James Cammarata b35529b09f Submodule pointer update 10 years ago
James Cammarata 95a58d1178 Properly serialize UnsafeProxy objects
Required as part of fix for #12123
10 years ago
James Cammarata a410bf26d3 Wrap registered variables in UnsafeProxy
Fixes #12123
10 years ago
Toshio Kuratomi 46bf3d1bc1 Remove plugin._basedirs as we now use different data structures to achieve that functionality. 10 years ago
Jiri tyr 174f805fb3 Resurrection of the comment filter 10 years ago
Marius Gedminas b95e3d18a7 Python 3: use the right PyYAML SafeRepresenter for unicode
PyYAML has a SafeRepresenter in lib/... that defines

    def represent_unicode(self, data):
        return self.represent_scalar(u'tag:yaml.org,2002:str', data)

and a different SafeRepresenter in lib3/... that defines

    def represent_str(self, data):
        return self.represent_scalar('tag:yaml.org,2002:str', data)

so the right thing to do on Python 3 is to use represent_str.

(AnsibleUnicode is a subclass of six.text_type, i.e. 'str' on Python 3.)
10 years ago
Benno Joy 6de13c3062 Merge pull request #11818 from Java1Guy/module_utils_ec2_for_ecs
change ec2 util to create only requested objects
10 years ago
James Cammarata ee7905c7e1 Compare include params against deprecated param names too
Fixes #12282
10 years ago
James Cammarata 950e067d2f Merge pull request #12295 from napkindrawing/patch-3
Typo in comment: "bsaename" → "basename"
10 years ago
James Cammarata f576e0f1af Merge pull request #12300 from robusto/devel
Vault missing DataLoader for pwd file #12293
10 years ago
James Cammarata 22639a2a22 Fix mistake in fact setting to nonpersistent cache in VariableManager
Fixes #12301
10 years ago
James Cammarata aa3b290efd Fix indent on line in playbook executor 10 years ago
Toshio Kuratomi 4b895f04e3 Fix order of loading of modules.
Allows ANSIBLE_LIBRARY to overload core modules even if the module in
ANSIBLE_LIBRARY doesn't have a .py extension.
10 years ago
James Cammarata e81947c3b4 Fix vars_prompt short form support (and mark deprecated) 10 years ago
Blake Atkinson 6062519986 Vault missing DataLoader for pwd file #12293 10 years ago
James Cammarata be7bd392cb Fix unreachable host detection and have plays continue after unreachable failures
Fixes #12284
10 years ago
Toshio Kuratomi 25c97fff69 Merge pull request #12294 from ansible/fix-password-lookup
Fix problem with "=" in the initial file path.
10 years ago
Brian Coca 0dbebfddaa Revert "remove closing connections after every task, this goes against conneciton caching and was not expected behaviuor nor inhertited from v1"
needed for winrm, disabled closing connections in ssh to avoid issues with that persistance, need to normalize all this in future

This reverts commit 23a22397bf.
10 years ago
Brian Coca 3f0e14c2a0 disabled ssh closing connections 10 years ago
Toshio Kuratomi e2c49b4ef4 Fix problem with "=" in the initial file path. 10 years ago
James Cammarata 56d7f3889d Add new level of vars to deal with set_fact/register precedence rules 10 years ago
napkindrawing 172bbac97a Typo in comment: "bsaename" → "basename" 10 years ago
James Cammarata 926f127245 set_fact should put facts in the fact_cache, not vars_cache
Also fixes a template path lookup issue when using an include

Fixes #12277
10 years ago
Toshio Kuratomi e2e9319f06 Merge pull request #12287 from mgedmin/py3k
Python 3: use six.string_types instead of basestring
10 years ago
Toshio Kuratomi fa2edfa1ef * Make sure we don't sometimes get byte strings instead of unicode strings
* Turn strings into byte strings before passing to shlex and turn them
  back into unicode strings after they are retyurned from there

  Fixes #12257
10 years ago
Toshio Kuratomi e67459df86 Fix docstring of parse_value() to be informative 10 years ago
Marius Gedminas ba0cf42ce9 Python 3: use six.string_types instead of basestring
Reduces the number of failing tests from 35 to 17.
10 years ago
James Cammarata 30399edada Use UnsafeProxy for lookup results too
Also fixes a couple of bugs that popped up when using the proxy class
10 years ago
Brian Coca f1c72a55a5 Merge pull request #12261 from toshywoshy/12169
added Manjaro as an Archlinux family
10 years ago
Victor Costan bd405169fb Fix #12258: don't wrap None in UnsafeProxy. 10 years ago
James Cammarata ac6f61ec7f Properly handle group/host_var files with dots in them
Fixes #12245
10 years ago
James Cammarata 1eb5b42705 Don't double proxy unsafe variables
Potential fix for #12260
10 years ago
Gaurav Jain 6c107258fa Fix rsync connections to IPv6 addresses
Similar to https://github.com/ansible/ansible/pull/11816 we can unconditionally
wrap the host address in square brackets. This is required by rsync for IPv6
addresses.
10 years ago
h-hirokawa cd478a76fc Add the new module replacer <<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>.
Fixes #12266
10 years ago
Toshio Kuratomi 718812d945 update submodule refs for documentation formatting 10 years ago
Toshio Kuratomi 26e109aa13 Update module refs 10 years ago
Toshaan Bharvani 4702d2be09 added Manjaro as an Archlinux family 10 years ago
James Cammarata ff9f5d7dc8 Starting to add additional unit tests for VariableManager
Required some rewiring in inventory code to make sure we're using
the DataLoader class for some data file operations, which makes mocking
them much easier.

Also identified two corner cases not currently handled by the code, related
to inventory variable sources and which one "wins". Also noticed we weren't
properly merging variables from multiple group/host_var file locations
(inventory directory vs. playbook directory locations) so fixed as well.
10 years ago
James Cammarata 87f75a50ad Use proxy class to mark variables unsafe for templating
Fixes #12191
10 years ago
Toshio Kuratomi 2d251cba45 Merge pull request #12256 from ansible/password-lookup-fix
Use parse_kv to correctly parse parameters from password lookup
10 years ago
Toshio Kuratomi c68e04c185 Use parse_kv to correctly parse parameters from password lookup
Fixes https://github.com/ansible/ansible-modules-core/issues/1968
10 years ago
Brian Coca 6e5f3db9de updated submodule refs 10 years ago
Brian Coca c17fbf2f12 simplify become testing and handling, we had drifted and were doulbe checking prompt, become and become_pass
fixed tests to conform to new signature and now tests both with and w/o password
now we are more explicit about self.prompt
10 years ago
Brian Coca eb0e7e198b remove closing connections after every task, this goes against conneciton caching and was not expected behaviuor nor inhertited from v1 10 years ago
Marius Gedminas 37be9539ff Python 3: use six.text_type instead of unicode
Replace 'unicode' with six.text_type, everywhere but in module_utils.
10 years ago
James Cammarata 1840906f74 Merge pull request #12236 from amenonsen/ssh-cpdir
Go back to using ~/.ansible/cp as the ControlPath
10 years ago
Toshio Kuratomi e218bf8da9 Do not escape backslashes when processing a template file.
Fixes #12198
Fixes #12241
10 years ago
Abhijit Menon-Sen 7aa6cd3f63 Go back to using ~/.ansible/cp as the ControlPath
This was commented out earlier because of the lack of interprocess
locking and prepare_writeable_dir in v2.

The locking was not needed: it could only protect against other siblings
of this process (since they were all locking a temporary file that was
opened in the parent), and those would be running as the same user and
with the same umask. Also, os.makedirs() tolerates intermediate paths
being created by other processes. For any other kind of error, both
locking and non-locking code paths would fail in the same way.

So all we really need to do is make sure we have write permissions.

(We also move the cp_dir handling code to where we actually set the
ControlPath ourselves; if the user has set it via ssh_*args already,
we don't need to bother.)
10 years ago
James Cammarata 4ac2bafc4b Set hosts fact gathering flag based on fact cache entries
Fixes #12213
10 years ago
James Cammarata 6650ba7654 Squashed commit of the following:
commit 9921bb9d20
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 20:19:44 2015 +0530

    Document --ssh-extra-args command-line option

commit 8b25595e7b
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Thu Aug 13 13:24:57 2015 +0530

    Don't disable GSSAPI/Pubkey authentication when using --ask-pass

    This commit is based on a bug report and PR by kolbyjack (#6846) which
    was subsequently closed and rebased as #11690. The original problem was:

        «The password on the delegated host is different from the one I
        provided on the command line, so it had to use the pubkey, and the
        main host doesn't have a pubkey on it yet, so it had to use the
        password.»

    (This commit is revised and included here because #11690 would conflict
    with the changes in #11908 otherwise.)

    Closes #11690

commit 119d032389
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Thu Aug 13 11:16:42 2015 +0530

    Be more explicit about why SSH arguments are added

    This adds vvvvv log messages that spell out in detail where each SSH
    command-line argument is obtained from.

    Unfortunately, we can't be sure if, say, self._play_context.remote_user
    is obtained from ANSIBLE_REMOTE_USER in the environment, remote_user in
    ansible.cfg, -u on the command line, or an ansible_ssh_user setting in
    the inventory or on a task or play. In some cases, e.g. timeout, we
    can't even be sure if it was set by the user or just a default.

    Nevertheless, on the theory that at five v's you can use all the hints
    available, I've mentioned the possible sources in the log messages.

    Note that this caveat applies only to the arguments that ssh.py adds by
    itself. In the case of ssh_args and ssh_extra_args, we know where they
    are from, and say so, though we can't say WHERE in the inventory they
    may be set (e.g. in host_vars or group_vars etc.).

commit b605c285ba
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Tue Aug 11 15:19:43 2015 +0530

    Add a FAQ entry about ansible_ssh_extra_args

commit 49f8edd035
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 20:48:50 2015 +0530

    Allow ansible_ssh_args to be set as an inventory variable

    Before this change, ssh_args could be set only in the [ssh_connection]
    section of ansible.cfg, and was applied to all hosts. Now it's possible
    to set ansible_ssh_args as an inventory variable (directly, or through
    group_vars or host_vars) to selectively override the global setting.

    Note that the default ControlPath settings are applied only if ssh_args
    is not set, and this is true of ansible_ssh_args as well. So if you want
    to override ssh_args but continue to set ControlPath, you'll need to
    repeat the appropriate options when setting ansible_ssh_args.

    (If you only need to add options to the default ssh_args, you may be
    able to use the ansible_ssh_extra_args inventory variable instead.)

commit 37c1a5b679
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 19:42:30 2015 +0530

    Allow overriding ansible_ssh_extra_args on the command-line

    This patch makes it possible to do:

        ansible somehost -m setup \
            --ssh-extra-args '-o ProxyCommand="ssh -W %h:%p -q user@bouncer.example.com"'

    This overrides the inventory setting, if any, of ansible_ssh_extra_args.

    Based on a patch originally by @Richard2ndQuadrant.

commit b023ace8a8
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 19:06:19 2015 +0530

    Add an ansible_ssh_extra_args inventory variable

    This can be used to configure a per-host or per-group ProxyCommand to
    connect to hosts through a jumphost, e.g.:

        inventory:
            [gatewayed]
            foo ansible_ssh_host=192.0.2.1

        group_vars/gatewayed.yml:
            ansible_ssh_extra_args: '-o ProxyCommand="ssh -W %h:%p -q bounceuser@gateway.example.com"'

    Note that this variable is used in addition to any ssh_args configured
    in the [ssh_connection] section of ansible.cfg (so you don't need to
    repeat the ControlPath settings in ansible_ssh_extra_args).
10 years ago