Commit Graph

424 Commits (dbb0ef40c09f4abc653664f3f493f758bfde713f)

Author SHA1 Message Date
James Cammarata b71957d6e6 Move queuing tasks to a background thread 8 years ago
Toshio Kuratomi fa804125b5 Python3 fixes and porting (#17271)
* Fix to_native call in selinux_context and selinux_default_context to
  use the error handler correctly.
* Port set_mode_if_different to work on python3
* Port atomic_move to work on python3
* Fix check_password_prompt variable which wasn't renamed properly
8 years ago
Alexander Stock 238cccf166 Fix "Text file busy" exception in atomic_move (#9526) (#17204)
tempfile.NamedTemporaryFile keeps a file handle causing os.rename() to fail with windows based vboxfs: [Errno 26] Text file busy.
Changed NamedTemporaryFile to mkstemp() and added a finally block to unlink the temp file in each and every case.
8 years ago
Toshio Kuratomi bd68c324ce Get the ssh plugin working with python3 (#17234) 8 years ago
Toshio Kuratomi 44d979c8f5 Enable most unittests on python3 (just some vault unittests and a logging one left) (#17240)
Make some python3 fixes to make the unittests pass:

* galaxy imports
* dictionary iteration in role requirements
* swap_stdout helper for unittests
* Normalize to text string in a facts.py function
8 years ago
Adrian Likins e396d5d508 Implement vault encrypted yaml variables. (#16274)
Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.

This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.

Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.

vault.VaultLib api changes:
    - Split VaultLib.encrypt to encrypt and encrypt_bytestring

    - VaultLib.encrypt() previously accepted the plaintext data
      as either a byte string or a unicode string.
      Doing the right thing based on the input type would fail
      on py3 if given a arg of type 'bytes'. To simplify the
      API, vaultlib.encrypt() now assumes input plaintext is a
      py2 unicode or py3 str. It will encode to utf-8 then call
      the new encrypt_bytestring(). The new methods are less
      ambiguous.

    - moved VaultLib.is_encrypted logic to vault module scope
      and split to is_encrypted() and is_encrypted_file().

Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml

Integration tests added as roles test_vault and test_vault_embedded
8 years ago
Toshio Kuratomi a22909c226 Migrate basestring to a python3 compatible type (#17199) 8 years ago
Toshio Kuratomi 57701d7115 Give native strings to selinux library functions. (#17184)
* Give native strings to selinux library functions.

SELinux takes pathnames as native strings.  That means we need to
convert to bytes on python2 and convert to text on python3.

Fixes #17155

* Read kitchen documentation, make module_utils params more like kitchen API

* Remove none nonstring strategy and add strict
* Raise TypeError on invalid nonstring strategy

* Document to_native()

* Make unittests for testing module_utils.text
8 years ago
Adrian Likins a4785c2691 Fix docker connection plugin version tests and py2.6 compat (#16841)
* Rm py2.7+ code in docker connection plugin

The docker connection plugin was using subprocess.check_output
which only exists in python 2.7 and later. Connection plugins
need to support python2.6 so this replaces it with Popen/communicate()

* Handle docker ver errors in docker connection

Add unit tests for DockerConnection

Fixes #16971
8 years ago
Toshio Kuratomi d29a7c55fe Enable some basic.py tests on python3 (#17167) 8 years ago
Adrian Likins 7d41f623dd Move py34 mock_open compat to compat/test/mock (#17157)
test/units/plugins/action/test_action.py had code
for handling a bug in python 3.4's mock_open that
causes errors when reading binary data.

Moved to compat/tests/mock.py so other tests can
use it by default.
8 years ago
Toshio Kuratomi bd31cc096a Fix facts.py for python3 (#17131)
* Fix facts.py for python3

* Update facts unittest to account for filepaths being byte strings
8 years ago
Adrian Likins 7bd57acda4 Linux mount/fs (lsblk) facts fixes and tests. (#17036)
Fixes #10779

Refactor some of the block device, mount point, and
mtab/fstab facts collection for linux for better
performance on systems with lots of block devices.

Instead of invoking 'lsblk' for every entry in mtab,
invoke it once, then map the results to mtab entries.

Change the args used for invoking 'findmnt' since the
previous combination of args conflicts, so this would
always fail on some systems depending on version.

Add test cases for facts Hardware()/Network()/Virtual() classes
__new__ method and verify they create the proper subclass based
on the platform.system() results.

Split out all the 'invoke some command and grab it's output'
bits related to linux mount paths into their own methods so
it is easier to mock them in unit tests.

Fix the DragonFly* classes that did not defined a 'platform'
class attribute. This caused FreeBSD systems to potentially
get the DragonFly* subclasses incorrectly. In practice it
didnt matter much since the DragonFly* subclasses duplicated
the FreeBSD ones. Actual DragonFly systems would end up with
the generic Hardware() etc instead of the DragonFly* classes.

Fix Hardware.__new__() on PY3, passing args to __new__
would cause "object() takes no parameters" errors. So
check for PY3 and just call __new__ without the args

See
https://hg.python.org/cpython/file/44ed0cd3dc6d/Objects/typeobject.c#l2818
for some explaination.
8 years ago
Brian Coca b1410fa278 fixed tests to accoutn for new parameter 8 years ago
James Cammarata c669a381d1 Don't immediately return failed for any_errors_fatal tasks
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.

Fixes #16937
8 years ago
James Cammarata 47acf55fa9 Cache tasks by uuid in PlayIterator for O(1) lookups
Rather than repeatedly searching for tasks by uuid via iterating over
all known blocks, cache the tasks when they are added to the PlayIterator
so the lookup becomes a simple key check in a dict.
8 years ago
James Cammarata e244895174 Avoid copying task parents in TaskExecutor
As Block.copy() is potentially expensive, avoid copying the parent
structure of tasks in TaskExecutor.
8 years ago
James Cammarata 06d4f4ad0e Move tasks/blocks to a single parent model 8 years ago
James Cammarata d2b3b2c03e Performance improvements 8 years ago
James Cammarata 159aa26b36 FEATURE: adding variable serial batches
This feature changes the scalar value of `serial:` to a list, which
allows users to specify a list of values, so batches can be ramped
up (commonly called "canary" setups):

- hosts: all
  serial: [1, 5, 10, "100%"]
  tasks:
  ...
8 years ago
Brian Coca e2f17f8d9b set cwd to task's basedir (#16805)
* switch cwd to basedir of task

This restores previous behaviour in pre 2.0 and allows for 'local type' plugins
and actions to have a more predictable relative path.

fixes #14489

* removed FIXME since prev commit 'fixes' this

* fix tests, now they need a loader (thanks jimi!)
8 years ago
s-hertel 93c5ec42cf Test GalaxyCLI when no actions are provided
Adds a new test for GalaxyCLI. Tests cases when:

- no actions are provided
- invalid actions are provided
- each valid action is provided

When parser() runs successfully the method returns True, creates a
SortedOptParser instance, a Galaxy instance, and updates certain class data.
8 years ago
s-hertel 671f9c3d15 Merge pull request #16783 from s-hertel/test_display_galaxy_info_fix
fixing a logic error in test_galaxy
8 years ago
s-hertel 3cb5747f54 fixing a logic error in test_galaxy 8 years ago
s-hertel aae5e50299 Adding a test for GalaxyCLI. Tests exit_without_ignore method. 8 years ago
Christoph Dittmann d55452d3ed Adding unit tests for role variable precedence
Originally from PR #16735
8 years ago
jctanner 238c6461f6 Add a function to check for killed processes in all strategies (#16684)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
8 years ago
Toshio Kuratomi c8a8f546d4 A little unittest refactoring (#16704)
A little unittest refactoring

* Add a class decorator to generate tests when using a unittest.TestCase base class
* Add a TestCase subclass with setUp() and tearDown() that sets up
  module parameter parsing
* Move test_safe_eval to use the class decorator and ModuleTestCase base
  class
* Move testing of set_mode_if_different into its own file and separate
  some test methods out so we get better errors and more coverage in
  case of errors.
* Naming convention for test cases doesn't need to duplicate information
  that's already in the file path.
8 years ago
James Cammarata 221520cbad Fixing type in 293723f (mock_handler -> mock_handler_task) 8 years ago
James Cammarata 0ea2b2a6af Fix unit test for base strategy in regards to handler changes 8 years ago
s-hertel e84ab95733 Fixing bug in GalaxyCLI test; creates temp dir for role_path instead … (#16689)
* Fixing bug in GalaxyCLI test; creates temp dir for role_path instead of installing to /etc/ansible/roles

* Update test_galaxy.py
8 years ago
nitzmahone de549ad675 update action unit test for powershell shebang behavior 8 years ago
s-hertel 9c925eeeab Adding a test for GalaxyCLI and two methods on which it depends. Tes… (#16651)
* Adding a test for GalaxyCLI and two methods on which it depends. Tests execute_info method.

*  Adding a test for GalaxyCLI and two methods on which it depends. Tests execute_remove method.

* Update test_galaxy.py

* Revising test for GalaxyCLI execute_remove.

* Removing mocks
8 years ago
Robin Roth 1f04130c00 add more distribution version tests (#16563)
* ubuntu 16.04
* solaris 10, 11
* fedora 22, 25

Thanks Peter Oliver for providing the test data
8 years ago
Matt Clay 292785ff2b Parse async response in async action. (#16534)
* Parse async response in async action.
* Add async test for non-JSON data before module output.
* Fix existing async unit test.

Resolves #16156
8 years ago
jctanner 4ba60d00c8 Refactor the parameter splitting in ini lookup to handle more path formats (#16477)
Refactor the parameter splitting in ini lookup to handle more path formats.

Fixes #16468
8 years ago
James Cammarata ca6ee4c789 FEATURE: handler listeners
Fixes ansible/proposals#8
9 years ago
James Cammarata 8218591fec Track notified handlers by object rather than simply their name
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.

This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.

Fixes #15084
9 years ago
Nate Coraor 706778ee7f Fix distro detection for Solaris derivatives 9 years ago
jctanner 587d3c368b Fix synchronize+vagrant dest assertions that were incorrect. (#16291)
Addresses #16284
9 years ago
Dag Wieers d91df20620 Fix unit tests for module_name commit #16087 9 years ago
Brian Coca a529a60478 raw should not use default executable (#16085)
also removed unused cruft in script
9 years ago
James Cammarata fbec2d9692 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523
9 years ago
James Cammarata 61f132a609 Adding unit tests for playbook group/host vars 9 years ago
Toshio Kuratomi 5a3493be5f Port urls.py to python3 and other byte vs text fixes (#16124)
* Port urls.py to python3

Fixes (largely normalizing byte vs text strings) for python3

* Rework what we do with attributes that aren't set already.

* Comments
9 years ago
Matt Clay 7a533406f9 Merge pull request #13145 from Mic92/devel
lxc connection plugin
9 years ago
Saran Ahluwalia fda927a14b added failing test for testing if executable is not empty string 9 years ago
Toshio Kuratomi f86df7c88b Test that task.args are also set properly (#15950) 9 years ago
Adrian Likins 644f0fb190 Merge pull request #15178 from alikins/galaxy_info_output
galaxy info displayed 'galaxy_info' section wrong [fix for #15177]
9 years ago
jctanner c52cea8db3 Add a framework, fixtures and test for common synchronize scenarios. (#15983)
* Add a framework, fixtures and test for common synchronize scenarios.

Addresses #15905
9 years ago