Change:
- Remove _get_item() alias as it has been deprecated
- Update tests
- Remove relevant sanity curtailment
- Add changelog
Test Plan:
CI, grep
Signed-off-by: Rick Elrod <rick@elrod.me>
* Only allow groups which were hardcoded in module_defaults.yml
only load action groups from the collection if module_defaults contains a potential group for the action
* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml
Third party modules should not be using group/ - use the action name instead
* add externalized module_defaults tests
add the missing group and collections
ci_complete
Co-authored-by: Matt Davis <mrd@redhat.com>
* changelog
ci_complete
* Fix import in tests
ci_complete
* Update with requested changes
ci_complete
* don't traceback since we don't validate the contents of module_defaults
ci_complete
Co-authored-by: Matt Davis <mrd@redhat.com>
* various deprecation, display, warning, error fixes
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* cleanup, test fixes
* add collection name to deprecated() calls
* clean up redirect entries from uncommitted tests
* fix dep warning/error header text to match previous
Co-authored-by: Felix Fontein <felix@fontein.de>
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
as it stands we rerun discovery for the delegated host
unless its saving facts to itself
* fixed test lacking delegate_to mock
* Fix filedescriptor out of range in select() when running commands
* Simplify the run_command() code
Now that we're using selectors in run_command(), we can simplify some of
the code.
* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
instead.
Also use a simpler conditional check of whether the selector map is
empty
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* Split test_play_context_make_become_cmd into files
For NWO migration. Split the become module assertions into distinct test
files and functions. For now, this is done naively - there is probably
room to abstract these tests out and remove some of the duplication
later on.
Signed-off-by: Rick Elrod <rick@elrod.me>
* use default_exe variable instead of hardcoding /bin/bash
Signed-off-by: Rick Elrod <rick@elrod.me>
* Move become plugin tests to their proper directory and rename them accordingly
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix up fixtures and imports.
* Remove stray file.
Co-authored-by: Matt Clay <matt@mystile.com>
* Split out cache plugin unit tests.
* Rename unit tests to match code under test.
* Relocate unit test code to match code under test.
* Another rename.
* Update sanity ignores.
* Fix UNC path joining in the powershell shell plugin, add test
* Remove testy bits and a redundant line
* Fix style nits
* Update to use os.ntpath
* Add changelog for #66604
As per:
https://github.com/pytest-dev/pytest-mock#note-about-usage-as-context-manager
pytest-mock is not meant to be used within a `with` context or as a
decorator. Instead, pytest-mock will automatically unpatch the mocked
methods when each test is complete.
In newer pytest-mock, this use actually throws an exception and causes
the tests to fail.
This hasn't been hit in Ansible's CI yet, because the docker image
that the tests run in uses an older version of pytest-mock. However,
there is no constraint on the upper bound of pytest-mock in
test/lib/ansible_test/_data/requirements/constraints.txt which means
that when running the tests locally, outside of that docker image, the
tests never pass.
This patch removes the `with` context in each such case.
Signed-off-by: Rick Elrod <rick@elrod.me>
* throttle tests: fix detection of parallel execution
The test wasn't able to detect if too many workers were running.
On my laptop:
- without this change, the 'throttle' target takes ~20 seconds
- with this change, the 'throttle' target takes ~70 seconds
- 1 second isn't long enough to encounter the issue
* Fix throttle test when strategy is 'free' based
'free' strategy allows multiple tasks to be executed in parallel: use
one 'throttledir' per task.
Use 'linear' strategy with a dedicated play for cleanup/setup tasks
* throttle: reset worker idx before queuing a new task
* TestStrategyBase: define task.throttle
otherwise '1' will be used instead of the default value due to the
following expression being equal to '1':
int(templar.template(task_mock.throttle))
Co-authored-by: James Cammarata <jimi@sngx.net>
* new connection plugin aws_ssm
Return code may be at the end of the last command output line.
Marking regex.
Ensure command status code is on it's owm line - last 3 lines are not part of command stdout.
* Adding timeout parameter - aws_ssm_timeout
Default 10 second timeout (https://docs.ansible.com/ansible/2.4/intro_configuration.html#timeout) is marginal.
This avoids changing this default and allowing the SSM timeout to be controlled via inventory.
This change wraps commands so commands which may never return do timeout.
* Added integration tests
Added AWS SSM Executor, target and config functions
Fixed more code for integration tests
Improved execution
Added S3 bucket name
Fixed pylint
Reverted lib changes
Reverted few more changes
Improved support for integration test execution
added ansible role for aws_ssm_integration_test setup and teardown and modifiled runme.sh
Reset to 17fa565 commit
inventory file location changed
change inventory file location
deleted meta and handlers folder as it is not required
deleted main.yml inside vars, removed extra space from tasks/main.yml, Added appropriate tags for ec2 and delete test folder as it is not required
deleted main.yml inside vars, removed extra space from tasks/main.yml, Added appropriate tags for ec2 and delete test folder as it is not required
modified task/main.yml
added region variable and fixed pattern for using variable
modified policy for IAM role
moved to first line of scrip set -eux
Updated Session Manager plugin installation
edited custme policy
Included tags for Session Manager plugin installation
Added README.md
Upddated README and added support for ssm-plugin for Amazon-Linux
Added Windows Integration test support
Improved user data for Linux
Added random value generation for the role and policy,delete vars_to_delete.yml
upadte README in vars
fixed typo
update policy
Updated IAM policy file
update playbook
Updated playbook to include ssm-agent userdata
modified jing2 template
modified jing2 template
modified jing2 template and fixed role deletion
fixed role name issue while deleting and task name
Updated playbook to include wait_connection for ec2
Corrected Synatx changes and updated ssm-plugin debian file
Changed region variable to us-east-1
Removed vars file and updated to /tmp dir
fixed typo
Improved setup
Fixed boto3 dependency
Fixed missing tag
Added boto as dependency as well
Improved execution workflow
Trying other way of defining tags
Fixed undefined var
Changed AMI ID to Amazon Linux
Improved Tags
Ok, created different directory for WIndows test execution
Fixed IAM Role Name for Windows
Fixed inventory not found
Improved integration test execution
Fixed Windows Inventory path
Fixed wrong Windows AMI ID
Fixes issue for windows test execution
* Don't attempt to terminate sessions without a session id.
* Added Unit test cases file for AWS SSM Connection plugin
updated test file with close
updated unit test file with start_session
updated test files
* Eliminate AWS CLI dependency for terminal session.
* Removing unused code, cleanup logic.
Reduce mark length - 52^26 should be plenty
Be explicit about subprocess.Popen options
Simplify if/else for mark end
_stdin_readline is not used now
* updated test file
Added exec command and fixed close session unit tests
updated test files
Improved ansible ssm test command
updated file for lint checks
updated for pylint checks
New Unit_testcases for pre-signed URL file
removing additonal spaces and white spaces
remaning error changes
fixed changes
fixed spaces issues
python 2.7 version and whitespaces
python 2.7 version and whitespaces
python 2.7 skip if
space issue with 16:1
Unit test cases for windows and linux
Unit test cases for windows and linux with issues fixed issues
Unit test cases for windows and linux with issues fixed issues1
* Added support for S3 Pre-signed URLs
* Updated documentation and comments
* Documentation and curl dependency removal for controller machine
Fixing lint errors and removing requirements.
* Adding support for Windows remote EC2 instances.
* Added Encoding fixes
* Updating author section and adding obvious requirement for the SSM agent.
* Refactor stdout post processing
Attempt to get real return code on error (test using ansible -m raw -a 'cmd /c exit 99'.
Fixes problem at terminal width (ansible windows -i ./hosts.yml -m setup).
* Refactor back to a single module.
* Fixed fetch file for windows
* ssm usage examples for linux and windows
* Update aws_ssm.py
Service state corrected.
* Strip line continuation when at terminal width - otherwise replace.
Strip ANSI control sequences only for Windows.
Test playbook:
---
- name: test
hosts: windows
gather_facts: False
vars:
small: 'abc'
tasks:
- name:
set_fact:
large: "{{ lookup('password', '/dev/null length=2000 chars=ascii_letters,digits,hexdigits,punctuation') }}"
- name: small fixed
raw: echo '{{ small }}'
register: small_result
changed_when: False
- name: check
assert:
that:
- "(item | length) == (small | length)"
- "item == small"
msg: "'{{ item | length }} must equal '{{ small | length }}' and '{{ item }}' must match '{{ small }}'"
with_items:
- "{{ small_result.stdout_lines[0] }}"
- name: large random
raw: echo '{{ large }}'
register: large_result
changed_when: False
- name: check
assert:
that:
- "(item | length) == (large | length)"
- "item == large"
msg: "'{{ item | length }} must equal '{{ large | length }}' and '{{ item }}' must match '{{ large }}'"
with_items:
- "{{ large_result.stdout_lines[0] }}"
- name: gather facts
setup:
* Correct module parameter names.
* Updated Windows Executable variable.
Updated Windows Executable variable to "ansible_shell_type".
Fixing Examples with raw declaration
Updated the plugin timeout variable.
* Fix to work with dynamic inventory plug-in
* IntegrationTest template updates
* Removing unsupported flag for integration tests.
Fixing unit test.
* Adding shippable group.
* SSM Usage examples with dynamic inventory plugin
* Fixing yamllint errors.
* Fixed Integration tests
* Fixed Integration tests
* Updates for python3.
Removing python3 restriction.
* Remove python3 restriction.
Change block from retry to always
* Fixed Integration tests with Python 3
* Fixed shellcheck
* Fix for Windows which could pick up end mark prematurely
Move debug to _wrap_command and use a single return point
Single-quotes not needed around linux marks
Fix typo in comment
End mark to new command.
* Unit test cases now works on Python2 and Python3
* Skip tests on Python 2.6
* Fix for wait_for_connection module for windows.
* Updated changes as per review comments
* Fixing broken pipe error seen with session-manager-plugin version 1.1.17.0.
Eliminating sleep as this looks to be fixed in session-manager-plugin version 1.1.17.0.
* Adding back delays for Windows with session-manager-plugin 1.1.17.0.
* Updating Windows AMI ID for integration Test
* Upgrading windows ssm agent to the latest
* Adding boilerplate code.
* Windows ami and integration test updates
* Revert "Windows ami and integration test updates"
This reverts commit cd6ca3579b7cda584bd9c065f9c0835bddb23627.
* Updating windows ami for Integration tests
* Integration test suite updates and fixes.
* Updates and fixes
* Eliminate duplicate processing for exit code on failed command.
* Add powershell wrap.
* Refactor windows post_process.
* AMI Lookup, aliases, OSC filter, test suite updates
Co-authored-by: Gaurav Ashtikar <gau1991@gmail.com>
Co-authored-by: Deepak Choudhary <40276333+deepsvc@users.noreply.github.com>
Co-authored-by: Hanumanth <46720371+hanumantharaomvl@users.noreply.github.com>
Co-authored-by: KUMAR MAYANK <mayank@flux7.com>
* ipaddr: add an option to return peer of a point-to-point link
Signed-off-by: Vincent Bernat <vincent@bernat.ch>
* ipaddr: extend "peer" to also work with /30
* enable using any domain in the check point machine
* Update checkpoint.py
* trying to checge `test_chrckpoint` according to `test_ftd` in order to pass the tests
* Update test_checkpoint.py
Let the caller choose a namespace for `to_uuid` and document the
behaviour of both the default case, and the new explicit case.
This PR does not change the existing behaviour of the `to_uuid` UUIDv5
filter.
Improve tests
- add more unit test cases
- add specific integration test with more cases
Testing shows no major downside to calling .strip() twice in a comprehension vs. using a regular for loop and only calling .strip() once. Going with the comprehension for ease of maintenance and because comprehensions are optimized in CPython.
when creating or deleting an object (e.g. via an API), before/after can
be `None` (or at least represented as such by the used library). to
avoid modules havig to do
diff={'before': before or '', 'after': after or ''}
let's just convert `None` to an empty string that can be diffed properly
* Allow the use of _paramiko_conn even if the connection hasn't been started.
I'm not sure what the benefit is of Noneing paramiko_conn on close, but will keep for now
* Fix test
* Try to fix up net_put & net_get
* Add changelog
* play, block, task: New attribute forks
With this it is possible to limit the number of concurrent task runs.
forks can now be used in play, block and task. If forks is set in different
levels in the chain, then the smallest value will be used for the task.
The attribute has been added to the Base class as a list to easily provide
all the values that have been set in the different levels of the chain.
A warning has been added because of the conflict with run_once. forks will
be ignored in this case.
The forks limitation in StrategyBase._queue_task is not used for the free
strategy.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Handle forks in free strategy
The forks attribute for the free strategy is handled in run in the free
StrategyModule. This is dony by counting the amount of tasks where the uuid
is the same as the current task, that should be queued next. If this amount
is bigger or equal to the forks attribute from the chain (task, block,
play), then it will be skipped to the next host. Like it is also done with
blocked_hosts.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Test cases for forks with linear and free strategy
With ansible_python_interpreter defined in inventory file using
ansible_playbook_python.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Changing forks keyword to throttle and adding some more docs
* win_updates: Add flag to only download updates without installing them
* Fix test
* Fixes ansible-test (pep8)
* Fix integration test
* Fix actual fix.
* Add information how to change Python interpreter used by Ansible.
* Update lib/ansible/module_utils/basic.py
Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>
* Make test less dependent on exact message.
* Add support for configurable terminal plugin options
Fixes#59404
* Add terminal options to support platform specific login menu
* Add terminal options to support configurable options for stdout
and stderr regex list
* Fix CI failures
* Fix CI issues
* Fix review comments and add integration test
* Fix sanity test failures
* Fix review comments
* Fix integration test case
* Fix integration test failure
* Add support to configure terminal related options
Fixes https://github.com/ansible/ansible/issues/59404
* Add network_cli configurable options to support platform specific login menu
* Add network_cli configurable options to support configurable options for stdout
and stderr regex list
* Fix review comment
* Fix review comment
* Delay calling connect() until absolutely necessary
* Implement transport_test to enable wait_for_connection
* plugin might be connected already for some reason?
* ensure_connect for httpapi
There's some become shenanigans still needing to be ironed out
* Fix tests for network_cli
* Adding Avi ansible lookup module
(cherry picked from commit 77b8951f68cbc889e6595b2a359ca27b84a43c0d)
* Added description for examples
* Added debug logs and unit tests
* Fix __builtin__ import and restting super
* Fix pep8 errors
* Updated as per review comments on IP address
* InventoryManager start of perf improvements
* 0 not 1
* More startswith to [0] improvements
* Remove unused var
* The hash doesn't need to be a string, start as a list, make it into a tuple
* set actually appears faster than frozenset, and these don't need to be frozen
* Cache hosts lists, to avoid extra get_hosts calls, pass to get_vars too
* negligible perf improvement, it could help with memory later
* Try the fast way, fallback to the safe way
* Revert to previous logic, linting fix
* Extend pre-caching to free
* Address test failures
* Hosts are strings
* Fix unit test
* host is a string
* update test assumption
* drop SharedPluginLoaderObj, pre-create a set, instead of 2 comparisons in the list comprehension
* Dedupe code
* Change to _hosts and _hosts_all in get_vars
* Add backwards compat for strategies that don't do set host caches
* Add deprecation message to SharedPluginLoaderObj
* Remove unused SharedPluginLoaderObj import
* Update docs/comments
* Remove debugging
* Indicate what patterh_hash is
* That won't work
* Re-fix tests
* Update _set_hosts_cache to accept the play directly, use without refresh in get_hosts_remaining and get_failed_hosts for backwards compat
* Rename variable to avoid confusion
* On add_host only manipulate _hosts_cache_all
* Add warning docs around _hosts and _hosts_all args
pytest.raises has two parameters, message and match. message is meant
to be the error message that pytest gives when the tested code does not
raise the expected exception. match is the string that pytest expects
to be a match for the repr of the exception. Unfortunately, it seems
that message is often mistakenly used where match is meant. Fix those
cases.
message is also deprecated so removed our usage of it. Perhaps we
should write a sanity test later that prevents the use of
pytest.raises(message) to avoid this mistake.
seealso: https://docs.pytest.org/en/4.6-maintenance/deprecations.html#message-parameter-of-pytest-raises
Also update the exception message tested for as we're now properly
detecting that the messages have changed.
* Skip gitlab tests if dependencies aren't met
* Skip certain unittests if passlib is not installed
* Fix tests with deps on paramiko to skip if paramiko is not installed
* Use pytest to skip for cloudstack
If either on Python-2.6 or the cs library is not installed we cannot run
this test so skip it
* Fix onepassword lookup plugin crashing on fields with no 'name' or 't' property.
* Fix onepassword_facts module crashing on fields with no 'name' or 't' property.
* Add unit test for onepassword lookup plugin failing on entries without a name.
* Add changelog fragment for onepassword lookup plugin and onepassword_facts module fixes on fields without a name.
* add IAM role assumption to aws_ec2 inventory
* Ensure inventory._options has necessary option keys populated since the plugin docs parser isn't accessible to unit tests yet
* Add latest updates from FTD Ansible downstream repository.
- add a better implementation of the upsert operation;
- add API version lookup functionality;
- add filter which remove duplicated references from the list of references;
- fix minor bugs.
* fix issues outlined by ansibot
* fix argument name for _check_enum_method
* basic plugin loading working (with many hacks)
* task collections working
* play/block-level collection module/action working
* implement PEP302 loader
* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)
* synthetic package support
* ansible.core.plugins mapping works, others don't
* synthetic collections working for modules/actions
* fix direct-load legacy
* change base package name to ansible_collections
* note
* collection role loading
* expand paths from installed content root vars
* feature complete?
* rename ansible.core to ansible.builtin
* and various sanity fixes
* sanity tweaks
* unittest fixes
* less grabby error handler on has_plugin
* probably need to replace with a or harden callers
* fix win_ping test
* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...
* fix unicode conversion issues on py2
* attempt to keep things working-ish on py2.6
* python2.6 test fun round 2
* rename dirs/configs to "collections"
* add wrapper dir for content-adjacent
* fix pythoncheck to use localhost
* unicode tweaks, native/bytes string prefixing
* rename COLLECTION_PATHS to COLLECTIONS_PATHS
* switch to pathspec
* path handling cleanup
* change expensive `all` back to or chain
* unused import cleanup
* quotes tweak
* use wrapped iter/len in Jinja proxy
* var name expansion
* comment seemingly overcomplicated playbook_paths resolution
* drop unnecessary conditional nesting
* eliminate extraneous local
* zap superfluous validation function
* use slice for rolespec NS assembly
* misc naming/unicode fixes
* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible
* misc fixmes
* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker
* more misc fixmes
* tighten up action dispatch, add unqualified action test
* rename Collection mixin to CollectionSearch
* (attempt to) avoid potential confusion/conflict with builtin collections, etc
* stale fixmes
* tighten up pluginloader collections determination
* sanity test fixes
* ditch regex escape
* clarify comment
* update default collections paths config entry
* use PATH format instead of list
* skip integration tests on Python 2.6
ci_complete
* fix AWS plugin credential precedence for environment variables
* Allow aliases in direct plugins options
Consolidate precedence fix just in the doc fragment using aliases for mismatched options
* Access options with the option name rather than alias
* fix indentation
* update unit tests
* Improve readability
* Allow parent groups to be variables or literal, requires {{ }}
* Check strict before failing on templating errors
* Don't add a group if an invalid parent group was provided
* Add support for Windows hosts in the SSH connection plugin
* fix Python 2.6 unit test and sanity issues
* fix up connection tests in CI, disable SCP for now
* ensure we don't pollute the existing environment during the test
* Add connection_windows_ssh to classifier
* use test dir for inventory file
* Required powershell as default shell and fix tests
* Remove exlicit become_methods on connection
* clarify console encoding comment
* ignore recent SCP errors in integration tests
* Add cmd shell type and added more tests
* Fix some doc issues
* revises windows faq
* add anchors for windows links
* revises windows setup page
* Update changelogs/fragments/windows-ssh.yaml
Co-Authored-By: jborean93 <jborean93@gmail.com>
* Replace InventoryFileCacheModule with a better developer-interface
Use new interface for inventory plugins with backwards compatibility
Auto-update the backing cache-plugin if the cache has changed after parsing the inventory plugin
* Update CacheModules to use the config system and add a deprecation warning if they are being imported directly rather than using cache_loader
* Fix foreman inventory caching
* Add tests
* Add integration test to check that fact caching works normally with cache plugins using ansible.constants and inventory caching provides a helpful error for non-compatible cache plugins
* Add some developer documentation for inventory and cache plugins
* Add user documentation for inventory caching
* Add deprecation docs
* Apply suggestions from docs review
* Add changelog
* Python interpreter discovery
* No longer blindly default to only `/usr/bin/python`
* `ansible_python_interpreter` defaults to `auto_legacy`, which will discover the platform Python interpreter on some platforms (but still favor `/usr/bin/python` if present for backward compatibility). Use `auto` to always use the discovered interpreter, append `_silent` to either value to suppress warnings.
* includes new doc utility method `get_versioned_doclink` to generate a major.minor versioned doclink against docs.ansible.com (or some other config-overridden URL)
* docs revisions for python interpreter discovery
(cherry picked from commit 5b53c0012ab7212304c28fdd24cb33fd8ff755c2)
* verify output on some distros, cleanup
* Add laps_password lookup for retrieving a Windows LAPS Password
* Remove python-ldap from requirements and fix doc typos
* enable unit tests without ldap dependency
* use config options for lookup plugin
* Update docs and tests based on further distro testing
* [WIP] become plugins
Move from hardcoded method to plugins for ease of use, expansion and overrides
- load into connection as it is going to be the main consumer
- play_context will also use to keep backwards compat API
- ensure shell is used to construct commands when needed
- migrate settings remove from base config in favor of plugin specific configs
- cleanup ansible-doc
- add become plugin docs
- remove deprecated sudo/su code and keywords
- adjust become options for cli
- set plugin options from context
- ensure config defs are avaialbe before instance
- refactored getting the shell plugin, fixed tests
- changed into regex as they were string matching, which does not work with random string generation
- explicitly set flags for play context tests
- moved plugin loading up front
- now loads for basedir also
- allow pyc/o for non m modules
- fixes to tests and some plugins
- migrate to play objects fro play_context
- simiplify gathering
- added utf8 headers
- moved option setting
- add fail msg to dzdo
- use tuple for multiple options on fail/missing
- fix relative plugin paths
- shift from play context to play
- all tasks already inherit this from play directly
- remove obsolete 'set play'
- correct environment handling
- add wrap_exe option to pfexec
- fix runas to noop
- fixed setting play context
- added password configs
- removed required false
- remove from doc building till they are ready
future development:
- deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems
* cleanup
remove callers to removed func
removed --sudo cli doc refs
remove runas become_exe
ensure keyerorr on plugin
also fix backwards compat, missing method is attributeerror, not ansible error
get remote_user consistently
ignore missing system_tmpdirs on plugin load
correct config precedence
add deprecation
fix networking imports
backwards compat for plugins using BECOME_METHODS
* Port become_plugins to context.CLIARGS
This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
instead
* Refactor make_become_commands as asked for by alikins
* Typo in comment fix
* Stop loading values from the cli in more than one place
Both play and play_context were saving default values from the cli
arguments directly. This changes things so that the default values are
loaded into the play and then play_context takes them from there.
* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH
As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that
should be done all at one time rather than piecemeal.
* One to throw away
This is a set of hacks to get setting FieldAttribute defaults to command
line args to work. It's not fully done yet.
After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.
What we want to be able to do ideally is something like this:
class Base(FieldAttributeBase):
_check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])
class Play(Base):
# lambda so that we have a chance to parse the command line args
# before we get here. In the future we might be able to restructure
# this so that the cli parsing code runs before these classes are
# defined.
class Task(Base):
pass
And still have a playbook like this function:
---
- hosts:
tasks:
- command: whoami
check_mode: True
(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).
There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now. The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)
* Revert "One to throw away"
This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.
* Set FieldAttr defaults directly from CLIARGS
* Remove dead code
* Move timeout directly to PlayContext, it's never needed on Play
* just for backwards compat, add a static version of BECOME_METHODS to constants
* Make the become attr on the connection public, since it's used outside of the connection
* Logic fix
* Nuke connection testing if it supports specific become methods
* Remove unused vars
* Address rebase issues
* Fix path encoding issue
* Remove unused import
* Various cleanups
* Restore network_cli check in _low_level_execute_command
* type improvements for cliargs_deferred_get and swap shallowcopy to default to False
* minor cleanups
* Allow the su plugin to work, since it doesn't define a prompt the same way
* Fix up ksu become plugin
* Only set prompt if build_become_command was called
* Add helper to assist connection plugins in knowing they need to wait for a prompt
* Fix tests and code expectations
* Doc updates
* Various additional minor cleanups
* Make doas functional
* Don't change connection signature, load become plugin from TaskExecutor
* Remove unused imports
* Add comment about setting the become plugin on the playcontext
* Fix up tests for recent changes
* Support 'Password:' natively for the doas plugin
* Make default prompts raw
* wording cleanups. ci_complete
* Remove unrelated changes
* Address spelling mistake
* Restore removed test, and udpate to use new functionality
* Add changelog fragment
* Don't hard fail in set_attributes_from_cli on missing CLI keys
* Remove unrelated change to loader
* Remove internal deprecated FieldAttributes now
* Emit deprecation warnings now
* aws_ec2 Implement the missing 'region discovery'
fixes#45288
tries to use api as documented (which seems to fail in latest boto3 versions)
and fallback to boto3 'hardcoded' list of regions
* fixes and cleanup, add error for worst case scenario
* fix tests, remove more unused code
* add load_name
* acually load the plugin
* set plugin as required
* reverted test changes, removed options tests
* fixes as per feedback and cleanup
* Catch SSH authentication errors and don't retry multiple times to prevent account lock out
Signed-off-by: Sam Doran <sdoran@redhat.com>
* Subclass AnsibleAuthenticationFailure from AnsibleConnectionFailure
Use comparison rather than range() because it's much more efficient.
Signed-off-by: Sam Doran <sdoran@redhat.com>
* Add tests
Signed-off-by: Sam Doran <sdoran@redhat.com>
* Make paramiko_ssh connection plugin behave the same way
Signed-off-by: Sam Doran <sdoran@redhat.com>
* Add changelog
Signed-off-by: Sam Doran <sdoran@redhat.com>
* Add checkpoint httpapi plugin and access rule facts module
* WIP checkpoint_access_rule module
* Add publish and install policy, plus fix empty json object request for publish
* Refactor publish and install_policy onto module_utils
* Add update resource logic
* Add checkpoint_host_facts module
* Return code and response on get_acess_rule function
* Add checkpoint_host module
* Add checkpoint_run_script module
* Add checkpoint_task_facts module
* Show all tasks if no task id is passed
Note, this is only available on v1.3 of Checkpoint WS API
* Add update logic to checkpoint host
* Add full details on get task call
* Add checkpoint httpapi plugin
* Fix pep8
* Use auth instead of sid property and return False on handle_httperror method
* Fix version in docstring
* Remove constructor
* Remove Accept from base headers
* Do not override http error handler and assign Checkpoint sid to connection _auth
There is scaffolding in the base class to autoappend the token, given
it is assigned to connection _send
* Use new connection queue message method instead of display
* Remove unused display
* Catch ValueError, since it's a parent of JSONDecodeError
* Make static methods that are not used outside the class regular methods
* Add missing self to previously static methods
* Fix logout
Was carrying copy pasta from ftd plugin
* Remove send_auth_request
* Use BASE_HEADERS constant
* Simplify copyright header on httpapi plugin
* Remove access rule module
* Remove unused imports
* Add unit test
* Fix pep8
* Add test
* Add test
* Fix pep8
* Once cli args are parsed, they're constant. So, save the parsed args
into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
* Got rid of the private self._options attribute
* Use context.CLIARGS in the individual callback plugins instead.
* Also output positional arguments in default and unixy plugins
* Code has been simplified since we're now dealing with a dict rather
than Optparse.Value
* Fix example in ini.py
* Fix unittest in test_ini.py to pass CI as latest ansible returns list in
different order. To prevent such issues in future results are sorted
* PEP8 E501 styling improvements
Co-Authored-By: Sergii Golovatiuk <sgolovat@redhat.com>
* Consolidate handler tracking
- Remove unused code. ci_complete
- unit test fixes. ci_complete
- Restore previous behavior of matching a single handler
- when notifying a host for a handler, return True if it was added, False otherwise, to reduce copied logic
- rename funcitons for clarity. ci_complete
- Remove handler logic for static includes which was disabled previously
* test for openstack inventory constructed functionality
this adds unit tests for the compose, groups, and keyed_var features
of the openstack inventory plugin
* fix constructed functionality in openstack inventory plugin
The compose, groups, and keyed_groups functionality of the openstack
inventory plugin was broken:
- the plugin was not passing the correct variables to the
Constructable methods for compose and groups
- the plugin was simply never calling the appropriate method for
implementing keyed_groups
This commit fixes both issues.
* FTD modules: bug fixes and upsert functionality
* Fix sanity checks
* Fix unit tests for Python 2.6
* Log status code for login/logout
* Use string formatting in logging
* win_update: Add post search category matching to support product matching
* win_updates: Return categories of each update
* win_updates: Documentation fix-up
* win_updates: Adjusted documentation to reflect regex vs sub-string match of post-cat strings
* win_updates: Sped up post-category checking
* win_updates: Updated documentation to suggest querying post-category strings
* win_updates: Simplified saving and checking post-categories
* fixed some issues and added filtered categories to return value
* win_updates: Moved all category matching to occur after initial search
* win_updates: Adjustments to satisfy PowerShell lint checks
* win_updates: Dropped category validation from action plugin
* win_updates: Documentation updates
* win_updates: Fixed plugin unit tests
This reverts commit c649d0ea32.
The change results in deadlock in network_cli while it is
waiting to check the return value of recv_ready() which
was added in this commit to improve performance
* Fix prompt mismatch issue for ios
Fixes#40884#44463
* If the command prompt is matched check if data is
still pending to be read from buffer.
* This fix adds a new timer `buffer_read_timeout`
which will be trigerred after command prompt
is matched and data is attempted to be read from channel.
If not data is present of channel the timer will expire
and response we be returned to calling function.
* Fix unit test failure
* Update to make buffer timeout float
* Update doc and fix review comment
* Fix CI issues
* Update doc
* Fix review comments
* Fix review comments
* Add support for IOS vlan parsing filter.
Example usage below:
{% set parsed_vlans = vlans | vlan_parser %}
switchport trunk allowed vlan {{ parsed_vlans[0] }}
{% for i in range (1, parsed_vlans | count) %}
switchport trunk allowed vlan add {{ parsed_vlans[i] }}
* Update test_network.py
Add import statement for filter
* Fixed PEP8 issues relating to comments
* Fix PEP8 issues related to blank lines
* Removed magic numbers for line lengths. This should generalize support
to other IOS-like NOS that use similar methods for listing vlans. The
default arguments for line lengths will still be specific to Cisco IOS.
The unit tests for line length are still specific to Cisco IOS.
* Move ansible.compat.tests to test/units/compat/.
* Fix unit test references to ansible.compat.tests.
* Move builtins compat to separate file.
* Fix classification of test/units/compat/ dir.
* allow jinja2 unique filter compat
* detect if unique is provided, fallback with warning
* handle j2 specific params
* now all filters using unique must pass environment
* added env to tests
also normalized on how we normally import and use exceptoins
* Unify login behavior between 1Password lookup plugins and module
- Use the same names for all credential aspects
- Only require the minimal amount of information for each
- Add more examples
* Change parameter terms
- use terms in line with 1Password documentation.
- update examples
- update tests
* Improve error messages in lookup plugin
* Unify onepassword_facts with lookup plugins
- use same methods and logic for signing in or reusing existing session
- unify terms with lookup plugins
* Change rc test for determing login
An rc other than 1 can be returned when a current login session does not exist.
* Create AnsibleModuleError class
ansible.errors is not available to modules, so create an AnsibleModuleError class within the module
Do not user os.path.expanduser since this is already done by virtue of the type being "path" in the argument spec.
* Add note about risk with fact caching sensitive data
* Add note on op version that was used for testing
* Ensure that the src file contents is converted to unicode in diff info. Fixes#45717
* Fix up and cleanup
* The diff functionality in the callback plugins should have the
to_text() calls removed since we're now doing it in ActionBase
* catching of UnicodeError and warnings in the callback diff
functionality from 61d01f549f haven't been
needed since we switched to to_text so remove them.
* Add a note to ActionBase's diff function giving an example of when the
diff function will be inaccurate and how to fix it
* Fix callback get_diff() tests
I believe the unittests of callback's get_diff() were wrong. They were
sending in a list where strings were expected. Because previous code
was transforming the lists into strings via their repr, the previous
tests did not fail but they would have formatted the test cases output
in an odd way if we had looked at it.
* Try to intuit proper plugins to send to ansible-connection
* Move sub-plugins to init so that vars will be populated in executor
* Fix connection unit tests
On all supported Pythons, the io.BytesIO is always a stream
implementation using an in-memory bytes buffer. Makes code slightly more
forward compatible by reducing use of the six module.
* Add check in network_cli to handle all prompts
* Add check_all flag to mandatory handle all the command prompt
in prompts list. By default if any one prompt is handled
remaining prompts are ignored.
* Fix cli_command multiple prompt issue
* If multiple prompt and answers are given as input network_cli
handles only the first prompt that matched by default
* If a command execution results in muliple prompt the fix
add support to set a boolean option C(check_all) to indicate
network_cli to wait till all the prompts and answers are processed.
* Update cli_command
* Update api doc
* Fix unit test failure
* Fix CI failure
* Update network_cli
* Fix review comment
* Add common and Swagger client utils for FTD modules
* Update FTD HTTP API plugin and add unit tests for it
* Add configuration layer handling object idempotency
* Add ftd_configuration module with unit tests
* Add ftd_file_download and ftd_file_upload modules with unit tests
* Validate operation data and parameters
* Fix ansible-doc, boilerplate and import errors
* Fix pip8 sanity errors
* Update object comparison to work recursively
* Add copyright
* Share the implementation of hashing for both vars_prompt and password_hash.
* vars_prompt with encrypt does not require passlib for the algorithms
supported by crypt.
* Additional checks ensure that there is always a result.
This works around issues in the crypt.crypt python function that returns
None for algorithms it does not know.
Some modules (like user module) interprets None as no password at all,
which is misleading.
* The password_hash filter supports all parameters of passlib.
This allows users to provide a rounds parameter, fixing #15326.
* password_hash is not restricted to the subset provided by crypt.crypt,
fixing one half of #17266.
* Updated documentation fixes other half of #17266.
* password_hash does not hard-code the salt-length, which fixes bcrypt
in connection with passlib.
bcrypt requires a salt with length 22, which fixes#25347
* Salts are only generated by ansible when using crypt.crypt.
Otherwise passlib generates them.
* Avoids deprecated functionality of passlib with newer library versions.
* When no rounds are specified for sha256/sha256_crypt and sha512/sha512_crypt
always uses the default values used by crypt, i.e. 5000 rounds.
Before when installed passlibs' defaults were used.
passlib changes its defaults with newer library versions, leading to non
idempotent behavior.
NOTE: This will lead to the recalculation of existing hashes generated
with passlib and without a rounds parameter.
Yet henceforth the hashes will remain the same.
No matter the installed passlib version.
Making these hashes idempotent.
Fixes#15326Fixes#17266Fixes#25347 except bcrypt still uses 2a, instead of the suggested 2b.
* random_salt is solely handled by encrypt.py.
There is no _random_salt function there anymore.
Also the test moved to test_encrypt.py.
* Uses pytest.skip when passlib is not available, instead of a silent return.
* More checks are executed when passlib is not available.
* Moves tests that require passlib into their own test-function.
* Uses the six library to reraise the exception.
* Fixes integration test.
When no rounds are provided the defaults of crypt are used.
In that case the rounds are not part of the resulting MCF output.
NOTE:
1. use os.open() with os.O_CREAT|os.O_EXCL to check existence
and create a lock file if not exists, it's an atomic operation
2. the fastest process will create the lock file and others will
wait until the lock file is removed
3. after the writer finished writing to the password file, all the reading
operations use built-in open so processes can read the file parallel
According to the do_encrypt interface, encrypt arg should be the hash method name used for encrypting returning password. But in the doc and lookup code it's a boolean flag, correct it to string.
* win_reboot: fix 2.6 issues and better handle post reboot reboot
* changed winrm _reset to reset
* Add handler to reset calls when .reset() throws an AnsibleError on older hosts
* Moving back to _reset to get the issue fixed
* Fix tmpdir on non root become
- also avoid exception if tmpdir and remote_tmp are None
- give 'None' on deescalation so tempfile will fallback to it's default behaviour
and use system dirs
- fix issue with bad tempdir (not existing/not createable/not writeable)
i.e nobody and ~/.ansible/tmp
- added tests for blockfile case
* Revert "Temporarily revert c119d54"
This reverts commit 5c614a59a6.
* changes based on PR feedback and changelog fragment
* changes based on the review
* Fix tmpdir when makedirs failed so we just use the system tmp
* Let missing remote_tmp fail
If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side. It's better to
be alerted in this case than to silently ignore it.
jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side. empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
* Create base class for network-style connections
* clean up some differences
* Move NetworkConnectionBase
* Tweak netconf for tests
* Tweak when network_os is checked to avoid failing tests
* Pull back exec_command
* fix minor issues with debug and item labels
- no more `item=None`, we always have a label now
- debug should only show expected information, either msg= or the var in var=
- also fixed method name, deprecated misleading _get_item
* Refactor ios cliconf plugin and ios_config module
* Refactor ios cliconf plugin to support generic network_config module
* Refactor ios_config module to work with cliconf api's
* Enable command and response logging in cliconf pulgin
* cliconf api documentation
* Fix unit test and other minor changes
* Doc update
* Fix CI failure
* Add default flag related changes
* Minor changes
* redact input command logging by default
* All instances of local connection should use _remote_is_local=True. Fixes#40551
* Switch to instance attribute for synchronize
* Add test that shows that synchronize _remote_is_local addresses tmpdir building
* Fix setting the cache when refresh_cache or --flush-cache are used
* Use jsonify function that handles datetime objects in jsonfile cache plugin
* Don't access self._options directly
* Add initial integration tests for aws_ec2 inventory plugin
* Add CI alias
* Fix and add a few more unit tests
* Add integration tests for constructed
* Fix typo
* Use inventory config templates
* Collect all instances that are not terminated by default
* Create separate playbook for setting up the VPC, subnet, security group, and finding an image for the host
Create a separate playbook for removing the resources
* Allow easier grouping by region and add an example
* use a unified json encode/decode that can handle unsafe and vault
* winrm: add better exception handling for krb5 auth with pexpect
* Added changelog fragment
* Added exception handler in case kinit path isn't valid, added test cases
* fixed for Python 2 compatibility
Martin Krizek