Commit Graph

98 Commits (385f319b46cd550a9b1c258e56dc0119dd7ff896)

Author SHA1 Message Date
Peter Sprygada dc23667cc2 add back reverted change to network_cli (#18761)
This adds back the change to the network_cli plugin.  Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.

The issue was due to the addition of the decorator that will call
_connect() when there is no connection.  The _connect() method should
have been mocked in the test case.  This commit fixes the test
case as well

Change was originally reverted in c414ded69a
8 years ago
Peter Sprygada 06f5bf980e adds unit test for network_cli connection plugin (#18729) 8 years ago
Adrian Likins f0c71ad68d More test coverage for plugins/callback/__init__ (#18020) 8 years ago
Toshio Kuratomi ed00741a01 Mcsalgado's change to use shlex.quote instead of pipes.quote (#18534)
* Replace pipes.quote for shlex_quote

* More migration of pipes.quote to shlex_quote

Note that we cannot yet move module code over.  Modules have six-1.4
bundled which does not have shlex_quote.  This shouldn't be a problem as
the function is still importable from pipes.quote.  It's just that this
has become an implementation detail that makes us want to import from
shlex instead.

Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
8 years ago
James Cammarata ca5b361ad8 Reworking iterator logic regarding failed states during always
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.

This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.

Fixes #17983
8 years ago
Matt Clay ac8842eee8 Fix unit test dirs to match code under test. 8 years ago
Matt Clay 8552ad6bf1 Fix docker connection unit tests.
- Use assertRaisesRegexp to make sure correct exceptions are raised.
- Set docker_command to avoid docker dependency (skips find_executable).
- Use a fake path for docker_command to make sure mock.patch is working.
8 years ago
Matt Clay 8c270ac75f Add empty-init code-smell script. (#18406)
Also removed boilerplate from otherwise empty __init__.py files
which should not contain any code (checked by empty-init script).
8 years ago
Matt Clay 0d46805979 Clean up shebangs for various files.
- Remove shebangs from:
  - ini files
  - unit tests
  - module_utils
  - plugins
  - module_docs_fragments
  - non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.

Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.

A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
8 years ago
Adrian Likins c0331d50dc Remove callback.CallbackBase._copy_result_exclude
Nothing seems to use this now.

Was added originally added in2d11cfab92f9d26448461b4bc81f466d1910a15e
but the code that used it was removed in
e02b98274b
8 years ago
Ssawa 8e47b9bc70 Handle 'smart' scp_if_ssh option for fetch (#18125) 8 years ago
Matt Davis aa0ad073b8 bugfixes to JSON junk filter, added unit/integration tests to exercise (#17834) 8 years ago
jctanner fff161f2f6 Smart mode for sftp+scp (#17813)
If the sftp fails, roll over to scp by default. This saves users
from having to know about the scp_if_ssh method when sftp is broken
on the remote host.
8 years ago
James Cammarata 1b54d3b6dc Merge branch 'threaded_receiver' into devel 8 years ago
Toshio Kuratomi 5e9a8d9202 Fix password lookup py3 plus alikins unittest additions refactoring (#17626)
* Improve unit testing of 'password' lookup

The tests showed some UnicodeErrors for the
cases where the 'chars' param include unicode,
causing the 'getattr(string, c, c)' to fail.
So the candidate char generation code try/excepts
UnicodeErrors there now.

Some refactoring of the password.py module to make
it easier to test, and some new tests that cover more
of the password and salt generation.

* More refactoring and fixes.

* manual merge of text enc fixes from pr17475

* moving methods to module scope

* more refactoring

* A few more text encoding fixes/merges

* remove now unused code

* Add test cases and data for _gen_candidate_chars

* more test coverage for password lookup

* wip

* More text encoding fixes and test coverage

* cleanups

* reenable text_type assert

* Remove unneeded conditional in _random_password

* Add docstring for _gen_candidate_chars

* remove redundant to_text and list comphenesion

* Move set of 'chars' default in _random_password

on py2, C.DEFAULT_PASSWORD_CHARS is a regular str
type, so the assert here fails. Move setting the
default into the method and to_text(DEFAULT_PASSWORD_CHARS)
if it's needed.

* combine _random_password and _gen_password

* s/_create_password_file/_create_password_file_dir

* native strings for exception msgs

* move password to_text to _read_password_file

* move to_bytes(content) to _write_password_file

* add more test assertions about genned pw's

* Some cleanups to alikins and abadger's password lookup refactoring:

* Make DEFAULT_PASSWORD_CHARS into a text string in constants.py
  - Move this into the nonconfigurable section of constants.
* Make utils.encrypt.do_encrypt() return a text string because all the
  hashes in passlib should be returning ascii-only strings and they are
  text strings in python3.
* Make the split up of functions more sane:
   - Don't split such that conditionals have to occur in two separate functions.
   - Don't go overboard: Good to split file system manipulation from parsing
     but we don't need to do every file manipulation in a separate
     function.
  - Don't split so that creation of the password store happens in two
    parts.
  - Don't split in such a way that no decisions are made in run.
* Organize functions by when it gets called from run().
* Run all potential characters through the gen_candidate_chars function
  because it does both normalization and validation.
* docstrings for functions
* Change when we store salt slightly.  Store it whenever it was already
  present in the file as well as when encrypt is requested.  This will
  head of potential idempotence bugs where a user has two playbook tasks
  using the same password and in one they need it encrypted but in the
  other they need it plaintext.
* Reorganize tests to follow the order of the functions so it's easier
  to figure out if/where a function has been tested.
* Add tests for the functions that read and write the password file.
* Add tests of run() when the password has already been created.
* Test coverage currently at 100%
8 years ago
James Cammarata 5a57c66e3c Moving result reading to a background thread 8 years ago
Toshio Kuratomi 0139298786 Have unittests import mock from ansible.compat so they work on py3 without mock installed from pypi 8 years ago
James Cammarata dfb1c0647e Revert "Move queuing tasks to a background thread"
This reverts commit b71957d6e6.
8 years ago
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
8 years ago
Matt Clay 94a0d2afb4 Add partially backwards compatible version of _fixup_perms. (#17427)
Also added a deprecation notice for _fixup_perms.

Resolves issue #17352 (assumes custom actions use recursive=False).
8 years ago
Tobias Wolf c23b11d212 [Inventory] Cache the result of enumerating groups and host names
for `VariableManager._get_magic_variables()`.

This saves a lot of time re-iterating the nearly always constant global
list of groups and their members.

Generate once and cache, and invalidate cache in case `add_host:` or
`group_by:` are used.
8 years ago
James Cammarata b71957d6e6 Move queuing tasks to a background thread 8 years ago
Toshio Kuratomi bd68c324ce Get the ssh plugin working with python3 (#17234) 8 years ago
Adrian Likins a4785c2691 Fix docker connection plugin version tests and py2.6 compat (#16841)
* Rm py2.7+ code in docker connection plugin

The docker connection plugin was using subprocess.check_output
which only exists in python 2.7 and later. Connection plugins
need to support python2.6 so this replaces it with Popen/communicate()

* Handle docker ver errors in docker connection

Add unit tests for DockerConnection

Fixes #16971
8 years ago
Adrian Likins 7d41f623dd Move py34 mock_open compat to compat/test/mock (#17157)
test/units/plugins/action/test_action.py had code
for handling a bug in python 3.4's mock_open that
causes errors when reading binary data.

Moved to compat/tests/mock.py so other tests can
use it by default.
8 years ago
Brian Coca b1410fa278 fixed tests to accoutn for new parameter 8 years ago
James Cammarata c669a381d1 Don't immediately return failed for any_errors_fatal tasks
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.

Fixes #16937
8 years ago
James Cammarata d2b3b2c03e Performance improvements 8 years ago
Brian Coca e2f17f8d9b set cwd to task's basedir (#16805)
* switch cwd to basedir of task

This restores previous behaviour in pre 2.0 and allows for 'local type' plugins
and actions to have a more predictable relative path.

fixes #14489

* removed FIXME since prev commit 'fixes' this

* fix tests, now they need a loader (thanks jimi!)
8 years ago
jctanner 238c6461f6 Add a function to check for killed processes in all strategies (#16684)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
9 years ago
James Cammarata 221520cbad Fixing type in 293723f (mock_handler -> mock_handler_task) 9 years ago
James Cammarata 0ea2b2a6af Fix unit test for base strategy in regards to handler changes 9 years ago
nitzmahone de549ad675 update action unit test for powershell shebang behavior 9 years ago
jctanner 4ba60d00c8 Refactor the parameter splitting in ini lookup to handle more path formats (#16477)
Refactor the parameter splitting in ini lookup to handle more path formats.

Fixes #16468
9 years ago
James Cammarata ca6ee4c789 FEATURE: handler listeners
Fixes ansible/proposals#8
9 years ago
James Cammarata 8218591fec Track notified handlers by object rather than simply their name
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.

This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.

Fixes #15084
9 years ago
jctanner 587d3c368b Fix synchronize+vagrant dest assertions that were incorrect. (#16291)
Addresses #16284
9 years ago
Brian Coca a529a60478 raw should not use default executable (#16085)
also removed unused cruft in script
9 years ago
James Cammarata fbec2d9692 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523
9 years ago
Toshio Kuratomi 5a3493be5f Port urls.py to python3 and other byte vs text fixes (#16124)
* Port urls.py to python3

Fixes (largely normalizing byte vs text strings) for python3

* Rework what we do with attributes that aren't set already.

* Comments
9 years ago
Matt Clay 7a533406f9 Merge pull request #13145 from Mic92/devel
lxc connection plugin
9 years ago
Saran Ahluwalia fda927a14b added failing test for testing if executable is not empty string 9 years ago
jctanner c52cea8db3 Add a framework, fixtures and test for common synchronize scenarios. (#15983)
* Add a framework, fixtures and test for common synchronize scenarios.

Addresses #15905
9 years ago
Jörg Thalheim 88482234e6
lxc connection plugin 9 years ago
Toshio Kuratomi b41b6b2ec5 Update test off make_become for ssh pty race mitigation (#15931) 9 years ago
Matt Martz 0faddfa168 Move binary module detection into executor/module_common.py 9 years ago
Matt Martz 3466e73c50 Resolve test failures 9 years ago
Toshio Kuratomi 8a184381af Strip junk after JSON return. (#15822)
Fixes #15601
9 years ago
Toshio Kuratomi 487e6562ca Fix ziploader for the cornercase of ansible invoking ansible.
* Make ziploader's ansible and ansible.module_utils libraries into
  namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
  ansible/release.py.  This is because namespace packages only load one
  __init__.py.  If that is not the __init__.py with the author and
  version info then those won't be available.
* In ziplaoder, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
  when namespace packages are present.
9 years ago
Toshio Kuratomi dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
9 years ago