* throttle tests: fix detection of parallel execution
The test wasn't able to detect if too many workers were running.
On my laptop:
- without this change, the 'throttle' target takes ~20 seconds
- with this change, the 'throttle' target takes ~70 seconds
- 1 second isn't long enough to encounter the issue
* Fix throttle test when strategy is 'free' based
'free' strategy allows multiple tasks to be executed in parallel: use
one 'throttledir' per task.
Use 'linear' strategy with a dedicated play for cleanup/setup tasks
* throttle: reset worker idx before queuing a new task
* TestStrategyBase: define task.throttle
otherwise '1' will be used instead of the default value due to the
following expression being equal to '1':
int(templar.template(task_mock.throttle))
Co-authored-by: James Cammarata <jimi@sngx.net>
* new connection plugin aws_ssm
Return code may be at the end of the last command output line.
Marking regex.
Ensure command status code is on it's owm line - last 3 lines are not part of command stdout.
* Adding timeout parameter - aws_ssm_timeout
Default 10 second timeout (https://docs.ansible.com/ansible/2.4/intro_configuration.html#timeout) is marginal.
This avoids changing this default and allowing the SSM timeout to be controlled via inventory.
This change wraps commands so commands which may never return do timeout.
* Added integration tests
Added AWS SSM Executor, target and config functions
Fixed more code for integration tests
Improved execution
Added S3 bucket name
Fixed pylint
Reverted lib changes
Reverted few more changes
Improved support for integration test execution
added ansible role for aws_ssm_integration_test setup and teardown and modifiled runme.sh
Reset to 17fa565 commit
inventory file location changed
change inventory file location
deleted meta and handlers folder as it is not required
deleted main.yml inside vars, removed extra space from tasks/main.yml, Added appropriate tags for ec2 and delete test folder as it is not required
deleted main.yml inside vars, removed extra space from tasks/main.yml, Added appropriate tags for ec2 and delete test folder as it is not required
modified task/main.yml
added region variable and fixed pattern for using variable
modified policy for IAM role
moved to first line of scrip set -eux
Updated Session Manager plugin installation
edited custme policy
Included tags for Session Manager plugin installation
Added README.md
Upddated README and added support for ssm-plugin for Amazon-Linux
Added Windows Integration test support
Improved user data for Linux
Added random value generation for the role and policy,delete vars_to_delete.yml
upadte README in vars
fixed typo
update policy
Updated IAM policy file
update playbook
Updated playbook to include ssm-agent userdata
modified jing2 template
modified jing2 template
modified jing2 template and fixed role deletion
fixed role name issue while deleting and task name
Updated playbook to include wait_connection for ec2
Corrected Synatx changes and updated ssm-plugin debian file
Changed region variable to us-east-1
Removed vars file and updated to /tmp dir
fixed typo
Improved setup
Fixed boto3 dependency
Fixed missing tag
Added boto as dependency as well
Improved execution workflow
Trying other way of defining tags
Fixed undefined var
Changed AMI ID to Amazon Linux
Improved Tags
Ok, created different directory for WIndows test execution
Fixed IAM Role Name for Windows
Fixed inventory not found
Improved integration test execution
Fixed Windows Inventory path
Fixed wrong Windows AMI ID
Fixes issue for windows test execution
* Don't attempt to terminate sessions without a session id.
* Added Unit test cases file for AWS SSM Connection plugin
updated test file with close
updated unit test file with start_session
updated test files
* Eliminate AWS CLI dependency for terminal session.
* Removing unused code, cleanup logic.
Reduce mark length - 52^26 should be plenty
Be explicit about subprocess.Popen options
Simplify if/else for mark end
_stdin_readline is not used now
* updated test file
Added exec command and fixed close session unit tests
updated test files
Improved ansible ssm test command
updated file for lint checks
updated for pylint checks
New Unit_testcases for pre-signed URL file
removing additonal spaces and white spaces
remaning error changes
fixed changes
fixed spaces issues
python 2.7 version and whitespaces
python 2.7 version and whitespaces
python 2.7 skip if
space issue with 16:1
Unit test cases for windows and linux
Unit test cases for windows and linux with issues fixed issues
Unit test cases for windows and linux with issues fixed issues1
* Added support for S3 Pre-signed URLs
* Updated documentation and comments
* Documentation and curl dependency removal for controller machine
Fixing lint errors and removing requirements.
* Adding support for Windows remote EC2 instances.
* Added Encoding fixes
* Updating author section and adding obvious requirement for the SSM agent.
* Refactor stdout post processing
Attempt to get real return code on error (test using ansible -m raw -a 'cmd /c exit 99'.
Fixes problem at terminal width (ansible windows -i ./hosts.yml -m setup).
* Refactor back to a single module.
* Fixed fetch file for windows
* ssm usage examples for linux and windows
* Update aws_ssm.py
Service state corrected.
* Strip line continuation when at terminal width - otherwise replace.
Strip ANSI control sequences only for Windows.
Test playbook:
---
- name: test
hosts: windows
gather_facts: False
vars:
small: 'abc'
tasks:
- name:
set_fact:
large: "{{ lookup('password', '/dev/null length=2000 chars=ascii_letters,digits,hexdigits,punctuation') }}"
- name: small fixed
raw: echo '{{ small }}'
register: small_result
changed_when: False
- name: check
assert:
that:
- "(item | length) == (small | length)"
- "item == small"
msg: "'{{ item | length }} must equal '{{ small | length }}' and '{{ item }}' must match '{{ small }}'"
with_items:
- "{{ small_result.stdout_lines[0] }}"
- name: large random
raw: echo '{{ large }}'
register: large_result
changed_when: False
- name: check
assert:
that:
- "(item | length) == (large | length)"
- "item == large"
msg: "'{{ item | length }} must equal '{{ large | length }}' and '{{ item }}' must match '{{ large }}'"
with_items:
- "{{ large_result.stdout_lines[0] }}"
- name: gather facts
setup:
* Correct module parameter names.
* Updated Windows Executable variable.
Updated Windows Executable variable to "ansible_shell_type".
Fixing Examples with raw declaration
Updated the plugin timeout variable.
* Fix to work with dynamic inventory plug-in
* IntegrationTest template updates
* Removing unsupported flag for integration tests.
Fixing unit test.
* Adding shippable group.
* SSM Usage examples with dynamic inventory plugin
* Fixing yamllint errors.
* Fixed Integration tests
* Fixed Integration tests
* Updates for python3.
Removing python3 restriction.
* Remove python3 restriction.
Change block from retry to always
* Fixed Integration tests with Python 3
* Fixed shellcheck
* Fix for Windows which could pick up end mark prematurely
Move debug to _wrap_command and use a single return point
Single-quotes not needed around linux marks
Fix typo in comment
End mark to new command.
* Unit test cases now works on Python2 and Python3
* Skip tests on Python 2.6
* Fix for wait_for_connection module for windows.
* Updated changes as per review comments
* Fixing broken pipe error seen with session-manager-plugin version 1.1.17.0.
Eliminating sleep as this looks to be fixed in session-manager-plugin version 1.1.17.0.
* Adding back delays for Windows with session-manager-plugin 1.1.17.0.
* Updating Windows AMI ID for integration Test
* Upgrading windows ssm agent to the latest
* Adding boilerplate code.
* Windows ami and integration test updates
* Revert "Windows ami and integration test updates"
This reverts commit cd6ca3579b7cda584bd9c065f9c0835bddb23627.
* Updating windows ami for Integration tests
* Integration test suite updates and fixes.
* Updates and fixes
* Eliminate duplicate processing for exit code on failed command.
* Add powershell wrap.
* Refactor windows post_process.
* AMI Lookup, aliases, OSC filter, test suite updates
Co-authored-by: Gaurav Ashtikar <gau1991@gmail.com>
Co-authored-by: Deepak Choudhary <40276333+deepsvc@users.noreply.github.com>
Co-authored-by: Hanumanth <46720371+hanumantharaomvl@users.noreply.github.com>
Co-authored-by: KUMAR MAYANK <mayank@flux7.com>
* ipaddr: add an option to return peer of a point-to-point link
Signed-off-by: Vincent Bernat <vincent@bernat.ch>
* ipaddr: extend "peer" to also work with /30
* enable using any domain in the check point machine
* Update checkpoint.py
* trying to checge `test_chrckpoint` according to `test_ftd` in order to pass the tests
* Update test_checkpoint.py
Let the caller choose a namespace for `to_uuid` and document the
behaviour of both the default case, and the new explicit case.
This PR does not change the existing behaviour of the `to_uuid` UUIDv5
filter.
Improve tests
- add more unit test cases
- add specific integration test with more cases
Testing shows no major downside to calling .strip() twice in a comprehension vs. using a regular for loop and only calling .strip() once. Going with the comprehension for ease of maintenance and because comprehensions are optimized in CPython.
when creating or deleting an object (e.g. via an API), before/after can
be `None` (or at least represented as such by the used library). to
avoid modules havig to do
diff={'before': before or '', 'after': after or ''}
let's just convert `None` to an empty string that can be diffed properly
* Allow the use of _paramiko_conn even if the connection hasn't been started.
I'm not sure what the benefit is of Noneing paramiko_conn on close, but will keep for now
* Fix test
* Try to fix up net_put & net_get
* Add changelog
* play, block, task: New attribute forks
With this it is possible to limit the number of concurrent task runs.
forks can now be used in play, block and task. If forks is set in different
levels in the chain, then the smallest value will be used for the task.
The attribute has been added to the Base class as a list to easily provide
all the values that have been set in the different levels of the chain.
A warning has been added because of the conflict with run_once. forks will
be ignored in this case.
The forks limitation in StrategyBase._queue_task is not used for the free
strategy.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Handle forks in free strategy
The forks attribute for the free strategy is handled in run in the free
StrategyModule. This is dony by counting the amount of tasks where the uuid
is the same as the current task, that should be queued next. If this amount
is bigger or equal to the forks attribute from the chain (task, block,
play), then it will be skipped to the next host. Like it is also done with
blocked_hosts.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Test cases for forks with linear and free strategy
With ansible_python_interpreter defined in inventory file using
ansible_playbook_python.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Changing forks keyword to throttle and adding some more docs
* win_updates: Add flag to only download updates without installing them
* Fix test
* Fixes ansible-test (pep8)
* Fix integration test
* Fix actual fix.
* Add information how to change Python interpreter used by Ansible.
* Update lib/ansible/module_utils/basic.py
Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>
* Make test less dependent on exact message.
* Add support for configurable terminal plugin options
Fixes#59404
* Add terminal options to support platform specific login menu
* Add terminal options to support configurable options for stdout
and stderr regex list
* Fix CI failures
* Fix CI issues
* Fix review comments and add integration test
* Fix sanity test failures
* Fix review comments
* Fix integration test case
* Fix integration test failure
* Add support to configure terminal related options
Fixes https://github.com/ansible/ansible/issues/59404
* Add network_cli configurable options to support platform specific login menu
* Add network_cli configurable options to support configurable options for stdout
and stderr regex list
* Fix review comment
* Fix review comment
* Delay calling connect() until absolutely necessary
* Implement transport_test to enable wait_for_connection
* plugin might be connected already for some reason?
* ensure_connect for httpapi
There's some become shenanigans still needing to be ironed out
* Fix tests for network_cli
* Adding Avi ansible lookup module
(cherry picked from commit 77b8951f68cbc889e6595b2a359ca27b84a43c0d)
* Added description for examples
* Added debug logs and unit tests
* Fix __builtin__ import and restting super
* Fix pep8 errors
* Updated as per review comments on IP address
* InventoryManager start of perf improvements
* 0 not 1
* More startswith to [0] improvements
* Remove unused var
* The hash doesn't need to be a string, start as a list, make it into a tuple
* set actually appears faster than frozenset, and these don't need to be frozen
* Cache hosts lists, to avoid extra get_hosts calls, pass to get_vars too
* negligible perf improvement, it could help with memory later
* Try the fast way, fallback to the safe way
* Revert to previous logic, linting fix
* Extend pre-caching to free
* Address test failures
* Hosts are strings
* Fix unit test
* host is a string
* update test assumption
* drop SharedPluginLoaderObj, pre-create a set, instead of 2 comparisons in the list comprehension
* Dedupe code
* Change to _hosts and _hosts_all in get_vars
* Add backwards compat for strategies that don't do set host caches
* Add deprecation message to SharedPluginLoaderObj
* Remove unused SharedPluginLoaderObj import
* Update docs/comments
* Remove debugging
* Indicate what patterh_hash is
* That won't work
* Re-fix tests
* Update _set_hosts_cache to accept the play directly, use without refresh in get_hosts_remaining and get_failed_hosts for backwards compat
* Rename variable to avoid confusion
* On add_host only manipulate _hosts_cache_all
* Add warning docs around _hosts and _hosts_all args
pytest.raises has two parameters, message and match. message is meant
to be the error message that pytest gives when the tested code does not
raise the expected exception. match is the string that pytest expects
to be a match for the repr of the exception. Unfortunately, it seems
that message is often mistakenly used where match is meant. Fix those
cases.
message is also deprecated so removed our usage of it. Perhaps we
should write a sanity test later that prevents the use of
pytest.raises(message) to avoid this mistake.
seealso: https://docs.pytest.org/en/4.6-maintenance/deprecations.html#message-parameter-of-pytest-raises
Also update the exception message tested for as we're now properly
detecting that the messages have changed.
* Skip gitlab tests if dependencies aren't met
* Skip certain unittests if passlib is not installed
* Fix tests with deps on paramiko to skip if paramiko is not installed
* Use pytest to skip for cloudstack
If either on Python-2.6 or the cs library is not installed we cannot run
this test so skip it
* Fix onepassword lookup plugin crashing on fields with no 'name' or 't' property.
* Fix onepassword_facts module crashing on fields with no 'name' or 't' property.
* Add unit test for onepassword lookup plugin failing on entries without a name.
* Add changelog fragment for onepassword lookup plugin and onepassword_facts module fixes on fields without a name.
* add IAM role assumption to aws_ec2 inventory
* Ensure inventory._options has necessary option keys populated since the plugin docs parser isn't accessible to unit tests yet
* Add latest updates from FTD Ansible downstream repository.
- add a better implementation of the upsert operation;
- add API version lookup functionality;
- add filter which remove duplicated references from the list of references;
- fix minor bugs.
* fix issues outlined by ansibot
* fix argument name for _check_enum_method
* basic plugin loading working (with many hacks)
* task collections working
* play/block-level collection module/action working
* implement PEP302 loader
* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)
* synthetic package support
* ansible.core.plugins mapping works, others don't
* synthetic collections working for modules/actions
* fix direct-load legacy
* change base package name to ansible_collections
* note
* collection role loading
* expand paths from installed content root vars
* feature complete?
* rename ansible.core to ansible.builtin
* and various sanity fixes
* sanity tweaks
* unittest fixes
* less grabby error handler on has_plugin
* probably need to replace with a or harden callers
* fix win_ping test
* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...
* fix unicode conversion issues on py2
* attempt to keep things working-ish on py2.6
* python2.6 test fun round 2
* rename dirs/configs to "collections"
* add wrapper dir for content-adjacent
* fix pythoncheck to use localhost
* unicode tweaks, native/bytes string prefixing
* rename COLLECTION_PATHS to COLLECTIONS_PATHS
* switch to pathspec
* path handling cleanup
* change expensive `all` back to or chain
* unused import cleanup
* quotes tweak
* use wrapped iter/len in Jinja proxy
* var name expansion
* comment seemingly overcomplicated playbook_paths resolution
* drop unnecessary conditional nesting
* eliminate extraneous local
* zap superfluous validation function
* use slice for rolespec NS assembly
* misc naming/unicode fixes
* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible
* misc fixmes
* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker
* more misc fixmes
* tighten up action dispatch, add unqualified action test
* rename Collection mixin to CollectionSearch
* (attempt to) avoid potential confusion/conflict with builtin collections, etc
* stale fixmes
* tighten up pluginloader collections determination
* sanity test fixes
* ditch regex escape
* clarify comment
* update default collections paths config entry
* use PATH format instead of list
* skip integration tests on Python 2.6
ci_complete
Pilou