Commit Graph

58 Commits (737a0d601565b5291434afd842c8c17547a64f19)

Author SHA1 Message Date
Brian Coca f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
7 years ago
Toshio Kuratomi 9f7b0dfc30 Remove automatic use of system six
* Enable the pylint no-name-in-module check.  Checks that identifiers in
  imports actually exist.  When we do this, we also have to ignore
  _MovedItems used in our bundled six.  This means pylint won't check
  for bad imports below ansible.module_utils.six.moves but that's
  something that pylint punts on with a system copy of six so this is
  still an improvement.
* Remove automatic use of system six.  The exec in the six code which
  tried to use a system library if available destroyed pylint's ability
  to check for imports of identifiers which did not exist (the
  no-name-in-module check).  That test is important enough that we
  should sacrifice the bundling detection in favour of the test.
  Distributions that want to unbundle six can replace the bundled six in
  ansible/module_utils/six/__init__.py to unbundle.  however, be aware
  that six is tricky to unbundle.  They may want to base their efforts
  off the code we were using:

  2fff690caa/lib/ansible/module_utils/six/__init__.py

* Update tests for new location of bundled six Several code-smell tests
  whitelist the bundled six library.  Update the path to the library so
  that they work.

* Also check for basestring in modules as the enabled pylint tests will
  also point out basestring usage for us.
7 years ago
Dag Wieers fceb71128e Return code is not very useful to assume a task failed
So I thought I fixed it before, but there's still one location where
the `rc` value is influential to decide whether a task failed or not.

We already established in #24867 that it is up to the module to decide
what the return code actually means, not the task executor. We modified
the existing modules to move that logic into the module (eg. for
command, shell, etc.)

This relates to the integration tests of win_robocopy, where different
return codes have different meanings:

  - 0  --  No files copied.
  - 1  --  Files copied successfully! (changed)
  - 2  --  Some Extra files or directories were detected. No files were copied. (warning)
  - 3  --  (2+1) Some files were copied. Additional files were present. (changed)
  - 4  --  Some mismatched files or directories were detected. Housekeeping might be required!  (changed + warning)
  - 5  --  (4+1) Some files were copied. Some files were mismatched.  (changed + warning)
  - 6  --  (4+2) Additional files and mismatched files exist.  No files were copied. (warning)
  - 7  --  (4+1+2) Files were copied, a file mismatch was present, and additional files were present. (changed + warning)
  - 8  --  Some files or directories could not be copied! (changed + failed)
  - 9 - 15  --  Fatal error. Check log message! (failed)
  - 16  --  Serious Error! No files were copied! Do you have permissions to access $src and $dest? (failed)

This also fixes #24652
8 years ago
Dag Wieers 4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
8 years ago
Toshio Kuratomi 500de1f557 Remove hack for backwards compatible v2_playbook_on_start callback
Just after release of 2.0.0 (in 2.0.0.1) we had a change to the API of
callbacks without bumping the API version.  We added the playbook to the
arguments passed to the callbacks.

This wasn't in the Tower callback at the time.  In order to prevent
breaking that callback we added a temporary hack to inspect the
callback's API to decide if we needed to call it with arguments or not.

We scheduled the hack for removal in January 2017.  Since that's now
past, removing the hack.

Change signed off by matburt on the Tower side.
8 years ago
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
8 years ago
Matt Martz 87aa59af79 Legacy pep8 updates for setup.py and tests 8 years ago
Toshio Kuratomi 5c38f3cea2 Combine jimi-c and bcoca's ideas and work on hooking module-utils into PluginLoader.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.

Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).

Rename snippet* variables to module_utils*

Add a small number of unittests for recursive_finder

Add a larger number of integration tests to demonstrate that
module_utils is working.

Whitelist module-style shebang in test target library dirs

Prefix module_data variable with b_ to be clear that it holds bytes data
8 years ago
Toshio Kuratomi 1609afbd12 Unittests for some of module_common.py (#20812)
* Unittests for some of module_common.py
* Port test_run_command to use pytest-mock

The use of addCleanup(patch.stopall) from the unittest idiom was
conflicting with the pytest-mock idiom of closing all patches
automatically.  Switching to pytest-mock ensures that the patches are
closed and removing the stopall stops the conflict.
8 years ago
Matt Clay d913f69ba1 PEP 8 W291 whitespace cleanup. 8 years ago
James Cammarata 9d549c3498 Fixing iterator bug related to reworking of end-of-role detection
Bug was introduced in cae682607
8 years ago
James Cammarata cae682607c Reworking the way end of role detection is done
Rather than trying to enumerate tasks or track an ever changing cur_role
flag in PlayIterator, this change simply sets a flag on the last block in
the list of blocks returned by Role.compile(). The PlayIterator then checks
for that flag when the cur_block number is incremented, and marks the role
as complete if the given host had any tasks run in that role.

Fixes #20224
8 years ago
Matt Clay 8c270ac75f Add empty-init code-smell script. (#18406)
Also removed boilerplate from otherwise empty __init__.py files
which should not contain any code (checked by empty-init script).
8 years ago
Steve Kuznetsov 0bc35354ce Change `v2_playbook_on_start` logic to positively detect legacy plugins
In order to support legacy plugins, the following two method signatures
are allowed for `CallbackBase.v2_playbook_on_start`:

def v2_playbook_on_start(self):
def v2_playbook_on_start(self, playbook):

Previously, the logic to handle this divergence checked to see if the
callback plugin being called supported an argument named `playbook`
in its `v2_playbook_on_start` method. This was fragile in a few ways:
 - if a plugin author did not use the literal `playbook` to name their
   method argument, their plugin would not be called correctly
 - if a plugin author wrapped their `v2_playbook_on_start` method and
   by doing so changed the argspec to no longer expose an argument
   with that literal name, their plugin would not be called correctly

In order to continue to support both types of callback for backwards
compatibility while making the call more robust for plugin authors,
the logic can be reversed in order to have a positive check for the old
method signature instead of a positive check for the new one.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
8 years ago
Adrian Likins 6f9ca7bb83 test name TestPlayIterator->TestPlaybookExecutor 8 years ago
James Cammarata 47acf55fa9 Cache tasks by uuid in PlayIterator for O(1) lookups
Rather than repeatedly searching for tasks by uuid via iterating over
all known blocks, cache the tasks when they are added to the PlayIterator
so the lookup becomes a simple key check in a dict.
8 years ago
James Cammarata e244895174 Avoid copying task parents in TaskExecutor
As Block.copy() is potentially expensive, avoid copying the parent
structure of tasks in TaskExecutor.
8 years ago
James Cammarata 06d4f4ad0e Move tasks/blocks to a single parent model 8 years ago
James Cammarata d2b3b2c03e Performance improvements 8 years ago
James Cammarata 159aa26b36 FEATURE: adding variable serial batches
This feature changes the scalar value of `serial:` to a list, which
allows users to specify a list of values, so batches can be ramped
up (commonly called "canary" setups):

- hosts: all
  serial: [1, 5, 10, "100%"]
  tasks:
  ...
8 years ago
Matt Clay 292785ff2b Parse async response in async action. (#16534)
* Parse async response in async action.
* Add async test for non-JSON data before module output.
* Fix existing async unit test.

Resolves #16156
9 years ago
Toshio Kuratomi f86df7c88b Test that task.args are also set properly (#15950) 9 years ago
jctanner 373b23cc24 Fix mock loader for osx /etc symlinks (#16074)
Fix role based unit tests for osx via mock.patch
9 years ago
Matt Davis bc7405efc9 Merge pull request #15797 from robinro/patch-1
reduce async timeout in unittests
9 years ago
Toshio Kuratomi 292f0ed0d6 If we can't squash for any reason, then simply do not optimize the items loop.
Also add more squashing testcases

Fixes #15649
9 years ago
Robin Roth 254cf9ea68 reduce async timeout
reduce from 3 sec to 0.1 sec; the 3 sec sleep was about half of the total unittest time on my development machine...
9 years ago
Toshio Kuratomi 5c7ad654db Add some more tests for item squashing 9 years ago
nitzmahone 133395db30 add jimi-c's unit test for squashed skip results, tweaked is_skipped() logic to pass 9 years ago
James Cammarata 6eefc11c39 Make the loop variable (item by default) settable per task
Required for include+with* tasks which may include files that also
have tasks containing a with* loop.

Fixes #12736
9 years ago
Jonathon Klobucar 7bee994e1c Fix for serial when percent amount is less than one host (#15396)
Ansible when there was a percentage that was calculated to be less than
1.0 would run all hosts as the value for a rolling update.

The error is due to the fact that Python will round a
float that is under 1.0 to 0, which will trigger the case of
0 hosts. The 0 host case tells ansible to run all hosts.

The fix will see if the percentage calculation after int
conversion is 0 and will else to 1 host.
9 years ago
James Cammarata 81788e627d Adding unit tests for TaskResult 9 years ago
James Cammarata 9d61a6cba8 Fixing PlayIterator bugs
* Unit tests exposed a problem where nested blocks did not correctly
  hit rescue/always portions of parent blocks
* Cleaned up logic in PlayIterator
* Unfortunately fixing the above exposed a potential problem in the
  block integration tests, where a failure in an "always" section may
  always lead to a failed state and the termination of execution
  beyond that point, so certain parts of the block integration test
  were disabled.
9 years ago
James Cammarata 299d93f6e9 Updating unit tests for PlayIterator
This knowingly introduces a broken test, planning to fix that later.
9 years ago
James Cammarata e02b98274b issue callbacks per item and retry fails
- now workers passes queue to task_executor so it can send back events per item and on retry attempt
- updated result class to pass along events to strategy
- base strategy updated to forward new events to callback
- callbacks now remove 'items' on final result but process them directly when invoked per item
- new callback method to deal with retry attempt messages (also now obeys nolog)
- updated tests to match new signature of task_executor

fixes #14558
fixes #14072
9 years ago
Toshio Kuratomi 0e410bbc8a Squashing was occuring even though pkgs didn't have a template that would be affected by squash
This broke other uses of looping (looping for delegate_to in the
reported bug)

Fixes #13980
9 years ago
Toshio Kuratomi 318bfbb207 Migrate cli and dependencies to use global display 9 years ago
Toshio Kuratomi aa034f4fcf Add unit tests for issue #12976 9 years ago
Toshio Kuratomi 37052a983a Fix unittest dailure with the new squash items code 9 years ago
James Cammarata 0e1a5919c8 Adding unit tests for PlaybookExecutor _get_serialized_batches 9 years ago
James Cammarata dfa33d0f23 Tweak variable manager use in role includes to avoid test failures 9 years ago
Marius Gedminas c22f4ee697 Add a comment explaining why async = 1 in the test 9 years ago
Marius Gedminas 1a1c9203e2 Python 3: fix TypeError: unorderable types in test
The full error was

======================================================================
ERROR: test_task_executor_execute (units.executor.test_task_executor.TestTaskExecutor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mg/src/ansible/test/units/executor/test_task_executor.py", line 252, in test_task_executor_execute
    mock_action.run.return_value = dict(ansible_facts=dict())
  File "/home/mg/src/ansible/lib/ansible/executor/task_executor.py", line 317, in _execute
    if self._task.async > 0:
TypeError: unorderable types: MagicMock() > int()

----------------------------------------------------------------------

Experiments show that Python 2 MagicMock() > 0 is true, so I'm setting
the async property on mock_task to 1.  (If I set it to 0, the test fails
anyway.)
9 years ago
James Cammarata b2cb3153f1 Fix unit tests for PlayIterator changes regarding fact gathering 9 years ago
James Cammarata ccaf47bd97 Fix unit tests broken by 057712c1 9 years ago
James Cammarata 9891bcbcae Fix unit tests broken by previous commit 9 years ago
James Cammarata 993ce592b1 Update unit tests for flush_handlers fix 10 years ago
James Cammarata e64989beb4 Moving ConnectionInformation -> PlayContext
Also making PlayContext a child class of the Playbook Base class,
which gives it access to all of the FieldAttribute code to ensure
field values are correctly typed after post_validation

Fixes #11381
10 years ago
James Cammarata 3d77723e3d Cleaning up some stuff related to the connection info/become/executable change 10 years ago
Brian Coca 323362e23a added stdout to test result 10 years ago
Brian Coca aa6486778f fixed become test to match new expected output 10 years ago