* play, block, task: New attribute forks
With this it is possible to limit the number of concurrent task runs.
forks can now be used in play, block and task. If forks is set in different
levels in the chain, then the smallest value will be used for the task.
The attribute has been added to the Base class as a list to easily provide
all the values that have been set in the different levels of the chain.
A warning has been added because of the conflict with run_once. forks will
be ignored in this case.
The forks limitation in StrategyBase._queue_task is not used for the free
strategy.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Handle forks in free strategy
The forks attribute for the free strategy is handled in run in the free
StrategyModule. This is dony by counting the amount of tasks where the uuid
is the same as the current task, that should be queued next. If this amount
is bigger or equal to the forks attribute from the chain (task, block,
play), then it will be skipped to the next host. Like it is also done with
blocked_hosts.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Test cases for forks with linear and free strategy
With ansible_python_interpreter defined in inventory file using
ansible_playbook_python.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
* Changing forks keyword to throttle and adding some more docs
* remove support from Windows pages, add Windows Server 2019
* jborean feedback
* Removed CBT info
Removed the CBT note about what transports Ansible actually supports. We've worked with both NTLM and Kerberos for a while now.
* Fix notifying handlers by using an exact match rather than a string subset if listen is text rather than a list
* Enforce better type checking for listeners
* Share code for validating handler listeners
* Add test for handlers without names
* Add test for templating in handlers
* Add test for include_role
* Add a couple notes about 'listen' for handlers
* changelog
* Add a test for handlers without names
* Test templating in handlers
* changelog
* Add some tests for include_role
* Add a couple notes about 'listen' for handlers
* make more sense
* move local function into a class method
* Update playbooks_strategies.rst
##### SUMMARY
This page is very confusing and may suggest, that there is still a `serial` strategy (like it used to), but now it is just a directive of `linear` strategy. Multiple people reported this to me and it needs change.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* Update docs/docsite/rst/user_guide/playbooks_strategies.rst
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Update docs/docsite/rst/user_guide/playbooks_strategies.rst
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Updates strategies page more broadly
* incorporate samdoran feedback
* fixes broken link
* new page title, focus on user intent
* more sdoran feedback
* adds details on setting the number of forks
* Remove lexers which have been fixed in Pygments 2.4.0.
* Add Pygments >= 2.4.0 to test runner.
* Fix pages that triggered lexer errors.
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Updated the section under "Aborting the play"
Updated "Aborting the play" section in the Error Handling In Playbooks page. Removed the reference to marking all hosts as failed.
+label: docsite_pr
* Update docs/docsite/rst/user_guide/playbooks_error_handling.rst
Co-Authored-By: Sandra McCann <samccann@redhat.com>
* Add a few examples of how to correctly use `regex_replace` filter
because it behaves differently on different Python versions when using
regex qualifiers that can match an empty string (e.g. '*', '?', etc).
Often when I am helping others learn Ansible, they are using nodes spun up on Amazon/Google/etc. where private key files (pem) are needed. Mentioning just a bit more on how to handle those private key files on the intro_getting_started page will help more folks get started faster.
I even reviewed the changes with my learning team and they all agreed this helped clarify immediately for them how to really get started with Ansible from a practical perspective.
Clarify the documentation for `async` and `poll`; describe the behavior when `poll` = 0 and when it does not.
* Update docs/docsite/rst/user_guide/playbooks_async.rst
* Update the default poll value in async doc
Co-Authored-By: tacatac <taca@kadisius.eu>
updates docs/docsite/rst/user_guide/vault.rst
* Add reference to single variable vault encryption
* rST ref to jump to the section
* Clarify the two targets of vault encryption, with notes about advantages and drawbacks of each
* Add the default form for --vault-id
* Add Vault ID examples for normal operations
Co-Authored-By: tacatac <taca@kadisius.eu>
* Start of migration to argparse
* various fixes and improvements
* Linting fixes
* Test fixes
* Fix vault_password_files
* Add PrependAction for argparse
* A bunch of additional tweak/fixes
* Fix ansible-config tests
* Fix man page generation
* linting fix
* More adhoc pattern fixes
* Add changelog fragment
* Add support for argcomplete
* Enable argcomplete global completion
* Rename PrependAction to PrependListAction to better describe what it does
* Add documentation for installing and configuring argcomplete
* Address rebase issues
* Fix display encoding for vault
* Fix line length
* Address rebase issues
* Handle rebase issues
* Use mutually exclusive group instead of handling manually
* Fix rebase issues
* Address rebase issue
* Update version added for argcomplete support
* -e must be given a value
* ci_complete
* Adding more information about blocks and blocks error handling.
* Update docs/docsite/rst/user_guide/playbooks_error_handling.rst and playbooks_blocks.rst
* Removing undefined variables as not rescuable errors.
Signed-off-by: Caio Ramos <caioramos97@gmail.com>
Signed-off-by: Gabriely Pereira <gabriely.pereira@usp.br>
* Apply suggestions from code review
Co-Authored-By: caiohsramos <caioramos97@gmail.com>
* docs: Add example of anchor and aliases in playbook
YAML supports anchors and aliases, add examples for explaining
this functionality with respect to Ansible playbook. Fix headers, fix
local TOC, remove unused section on YAML tags and Python types.
<!--- Your description here -->
`192.168.128.0/20` does not contain `192.168.144.5`, the correct subnet is `192.168.144.0/20` (which is what is rendered on my test template).
+label: docsite_pr
* Allows the use of Private_Keys to be entered as a string instead of just a file. Making it possible to use VAULT to encrypt the key
* Fixed Issues auto check found
* Provide helpful information while avoiding credential exposure
* Restore original variable name :-)
* Fix a few other things
* Influence the default certificate_name in both cases
* Update documentation
* Add contributed docs
* Fix CI issue
* Add a warning about vault leaving secrets in the shell history.
* Add a warning about accidental newlines in vault encrypted strings.
+label: docsite_pr
* modify regex to use implicit charsets this should solve issues in py3 and unicode names
* fix issue with subgroups in yaml inventory
* clarify deprecation message
* separated per name warning from deprecation
* move noise to verbosity, simplify warnings
* fix docs to reflect actual 'good' practice
* change toggle to choice list to give users more options
* Add support for Windows hosts in the SSH connection plugin
* fix Python 2.6 unit test and sanity issues
* fix up connection tests in CI, disable SCP for now
* ensure we don't pollute the existing environment during the test
* Add connection_windows_ssh to classifier
* use test dir for inventory file
* Required powershell as default shell and fix tests
* Remove exlicit become_methods on connection
* clarify console encoding comment
* ignore recent SCP errors in integration tests
* Add cmd shell type and added more tests
* Fix some doc issues
* revises windows faq
* add anchors for windows links
* revises windows setup page
* Update changelogs/fragments/windows-ssh.yaml
Co-Authored-By: jborean93 <jborean93@gmail.com>
* Python interpreter discovery
* No longer blindly default to only `/usr/bin/python`
* `ansible_python_interpreter` defaults to `auto_legacy`, which will discover the platform Python interpreter on some platforms (but still favor `/usr/bin/python` if present for backward compatibility). Use `auto` to always use the discovered interpreter, append `_silent` to either value to suppress warnings.
* includes new doc utility method `get_versioned_doclink` to generate a major.minor versioned doclink against docs.ansible.com (or some other config-overridden URL)
* docs revisions for python interpreter discovery
(cherry picked from commit 5b53c0012ab7212304c28fdd24cb33fd8ff755c2)
* verify output on some distros, cleanup
* Fix and update vault docs, add and clarify examples and options, introduce the concept of labeling a vault for clarity even if only a single password is in use for a given run, rework multi-password section to align to these concepts.
* Document -client scripts for ansible vault.
Co-Authored-By: orthanc <orthanc@users.noreply.github.com>
* Remove misleading statement passwords must be same
Since 2.4 Ansible has supported multiple vault passwords:
<https://docs.ansible.com/ansible/latest/user_guide/vault.html#multiple-vault-passwords>
Meaning lines like the following are misleading:
> The password used with vault currently must be the same for all files you wish
> to use together at the same time.
-- `docs/docsite/rst/user_guide/vault.rst`
To demonstrate this with Ansible 2.7, save the following as `example.yaml`:
```
- name: Display output from two vaults with different passwords
hosts: localhost
connection: local
vars_files: [one.yaml, two.yaml]
tasks:
- name: View secret from one.yaml vault
debug: { var: one }
- name: View secret from two.yaml vault
debug: { var: two }
```
Then run the three following commands choosing two different passwords:
```
$ echo 'one: 1' | ansible-vault encrypt --vault-id id1@prompt --output=one.yaml
$ echo 'two: 2' | ansible-vault encrypt --vault-id id2@prompt --output=two.yaml
$ ansible-playbook --vault-id id1@prompt --vault-id id2@prompt example.yaml
```
`ansible-vault` stores an ID in plain text in the vault file.
* Remove note about default in Ansible 2.1
As requested by gundalow in https://github.com/ansible/ansible/pull/49798
* [WIP] become plugins
Move from hardcoded method to plugins for ease of use, expansion and overrides
- load into connection as it is going to be the main consumer
- play_context will also use to keep backwards compat API
- ensure shell is used to construct commands when needed
- migrate settings remove from base config in favor of plugin specific configs
- cleanup ansible-doc
- add become plugin docs
- remove deprecated sudo/su code and keywords
- adjust become options for cli
- set plugin options from context
- ensure config defs are avaialbe before instance
- refactored getting the shell plugin, fixed tests
- changed into regex as they were string matching, which does not work with random string generation
- explicitly set flags for play context tests
- moved plugin loading up front
- now loads for basedir also
- allow pyc/o for non m modules
- fixes to tests and some plugins
- migrate to play objects fro play_context
- simiplify gathering
- added utf8 headers
- moved option setting
- add fail msg to dzdo
- use tuple for multiple options on fail/missing
- fix relative plugin paths
- shift from play context to play
- all tasks already inherit this from play directly
- remove obsolete 'set play'
- correct environment handling
- add wrap_exe option to pfexec
- fix runas to noop
- fixed setting play context
- added password configs
- removed required false
- remove from doc building till they are ready
future development:
- deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems
* cleanup
remove callers to removed func
removed --sudo cli doc refs
remove runas become_exe
ensure keyerorr on plugin
also fix backwards compat, missing method is attributeerror, not ansible error
get remote_user consistently
ignore missing system_tmpdirs on plugin load
correct config precedence
add deprecation
fix networking imports
backwards compat for plugins using BECOME_METHODS
* Port become_plugins to context.CLIARGS
This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
instead
* Refactor make_become_commands as asked for by alikins
* Typo in comment fix
* Stop loading values from the cli in more than one place
Both play and play_context were saving default values from the cli
arguments directly. This changes things so that the default values are
loaded into the play and then play_context takes them from there.
* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH
As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that
should be done all at one time rather than piecemeal.
* One to throw away
This is a set of hacks to get setting FieldAttribute defaults to command
line args to work. It's not fully done yet.
After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.
What we want to be able to do ideally is something like this:
class Base(FieldAttributeBase):
_check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])
class Play(Base):
# lambda so that we have a chance to parse the command line args
# before we get here. In the future we might be able to restructure
# this so that the cli parsing code runs before these classes are
# defined.
class Task(Base):
pass
And still have a playbook like this function:
---
- hosts:
tasks:
- command: whoami
check_mode: True
(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).
There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now. The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)
* Revert "One to throw away"
This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.
* Set FieldAttr defaults directly from CLIARGS
* Remove dead code
* Move timeout directly to PlayContext, it's never needed on Play
* just for backwards compat, add a static version of BECOME_METHODS to constants
* Make the become attr on the connection public, since it's used outside of the connection
* Logic fix
* Nuke connection testing if it supports specific become methods
* Remove unused vars
* Address rebase issues
* Fix path encoding issue
* Remove unused import
* Various cleanups
* Restore network_cli check in _low_level_execute_command
* type improvements for cliargs_deferred_get and swap shallowcopy to default to False
* minor cleanups
* Allow the su plugin to work, since it doesn't define a prompt the same way
* Fix up ksu become plugin
* Only set prompt if build_become_command was called
* Add helper to assist connection plugins in knowing they need to wait for a prompt
* Fix tests and code expectations
* Doc updates
* Various additional minor cleanups
* Make doas functional
* Don't change connection signature, load become plugin from TaskExecutor
* Remove unused imports
* Add comment about setting the become plugin on the playcontext
* Fix up tests for recent changes
* Support 'Password:' natively for the doas plugin
* Make default prompts raw
* wording cleanups. ci_complete
* Remove unrelated changes
* Address spelling mistake
* Restore removed test, and udpate to use new functionality
* Add changelog fragment
* Don't hard fail in set_attributes_from_cli on missing CLI keys
* Remove unrelated change to loader
* Remove internal deprecated FieldAttributes now
* Emit deprecation warnings now
* standardize user/password connection vars
* docs: use ansible_user and ansible_password
* docs: var precedence for connection vars
* docs: ansible_become_pass -> ansible_become_password etc
* priority setting note in inventory doc
The `ansible_group_priority` variable is also described in *Using Variables* and was given a clarification in issue #41512. The variable is also documented in *Working with Inventory* without the caveat that it needs to be declared on the inventory and not a group_vars file. This change adds that as a note
+label: docsite_pr
* Revert "avoid x2 setting of set_fact when 'cacheable' (#50564)"
This reverts commit 207848f354.
* clarify clear_facts with set_fact cacheable
revert previous 'fix' as it will break playbooks by changing precedence
opted to leave current behaviour but document it on both plugins to mitigate confusion
fixes#50556
also fix grammer, add comment, remove unused e
Make WinRM security warning more explicit.
Currently the warning is "viewed by anyone", this is a much lower risk than arbitrary command injection. Therefore the risk should be phrased appropriately.
+label: docsite_pr
Added examples in playbooks_error_handling doc for handlining
multiple conditions in changed_when and failed_when
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Revert "Fix incorrect examples with random filter (#50137)"
This reverts commit 9a7dbd5213.
The correction is incomplete and also based on a 'fix' that was supposed to have been reverted already
* Revert "Added `+1` to the `end` in `random` filter so that it was inclusive (#27215)"
This reverts commit ea2b89c7ae.
reverted fix as agreed at the time, but missed by maintainers.
* win become: refactor and add support for passwordless become
* make tests more stable
* fix up dep message for Load-CommandUtils
* Add further check for System impersonation token
* re-add support for become with accounts that have no password
* doc fixes and slight code improvements
* fix doc sanity issue
* Add info about loop based on jinja2 loop var
* ansible_loop
* Update test count
* Add extended loop_control that defines whether ansible_loop should be added
* Extended needs to be defaulted
* Revert "Update test count"
This reverts commit f1e93ee469825f4cdcd90fb28667d29aa088275c.
* Add docs about loop_control.extended
* Add revindex and revindex0
* Document ansible_loop in special vars
* Add changelog fragment
* Add tests, change items to allitems so that dot notation works, fix logic error with previtem
* Add append_hash functionality to k8s module
append_hash adds a hash based on the contents of a ConfigMap
or Secret to the name - this enables immutable ConfigMaps and
Secrets.
* Provide k8s_config_resource_name plugin
The k8s_config_resource_name filter plugin provides a means of determining
the name of ConfigMaps and Secrets created with append_hash
* Add changelog fragment
* fix failing tests
* Update openshift version needed for append_hash
* add note regarding {host,group}_vars directory lookup semantics
While the 'ansible-playbook' command will look for {host,group}_vars in
both the playbook and inventory directories, the 'ansible' command
will only look in the inventory directory. This follows from a close
reading of the documentation but isn't explicit and leads to the
perhaps unexpected result that you can run both commands with the same
configuration in the same directory and get different results.
This commit adds a note to the documentation making the difference in
behavior explicit.
Closes#48065
* incorporates bcoca feedback
* Add shorthand debugger method to recreate the task from the original ds, and re-template. Fixes#46321
* Fix rst header
* Indicate update_task was added in 2.8
* add note about named tasks in blocks
Signed-off-by: Adam Miller <admiller@redhat.com>
* adds names to tasks within example block, removes 2nd example
* removes traces of bad rebase
<!--- Your description here -->
+label: docsite_pr
Windows Subsystem for Linux is now officially supported by Microsoft. It is out of beta and will be shipping on Windows Sever 2019.
* Fix spelling of 'separate' throughout.
* Various cleanups in the User Guide for Vault.
- Fix spelling of 'algorithm'
- Fix indentation of nested list in payload format
- Fix mysterious refernce to 'b_pkey1'.
- Fix reference to newline as '\n': the backslash is lost when rendered
to the docs website. Specify the hex value for newline instead of the
backslash escape.
* Fix formatting
* Update vault.rst
* adds support for null values to the ternary filter
This change adds a third optional argument to the ternary filter to
handle a null value. If the third option is specified null and false
are treated differently.
For instance, take the following example:
{{ enabled | ternary('no shutdown', 'shutdown') }}
If enabled == True, then 'no shutdown' is used.
If enabled in (False, None), then 'shutdown' is used.
With this change the following is possible:
{{ enabled | ternary('no shutdown', 'shutdown', omit) }}
If enabled == True, then 'no shutdown'
If enabled == False, then 'shutdown'
If enabled == None, then omit
* update documentation with example of filter
* update filter documentation example per comments
* fix logic error in user_guide example
* Add the key_name/value_name options to dict2items - as with items2dict, allow users to configure the key/value name for dict2items, add "version added" and examples
* orphans testing pages to avoid not-in-toctree errors
* orphans various pages pending reorg
* adds module_utils and special_vars to main TOC
* uses a glob for scenario_guide TOC
* normalize and Sentence-case headings on community pages, typos
* re-orgs community TOC, adds all pages to toctree
* removes scenario guides index page
* adds style guide to community index
* basic update to style guide
* fix typo that created a new error
* removes not-in-toctree from ignore errors list
* leave removing files for future cleanup task
Correct and clarify "set_fact" example, expanding on what is happening
in the easy-to-get-wrong import mode. Add some additional links to "group_by" documentation and the main import/include discussion.
Closes: #31596
* Improve clarity of precedence when command-line parameters are used.
* Add command-line values into the precedence list.
* Several sample config snippets were included without any explanation
of how those snippets would be processed. Added descriptions so that
the reader can understand what each snippet will (or won't) accomplish.
* Don't focus on inventory as much
Expand on the fact that it's the fact that a variable is set that
matters, not the source of the variable.
* Share the implementation of hashing for both vars_prompt and password_hash.
* vars_prompt with encrypt does not require passlib for the algorithms
supported by crypt.
* Additional checks ensure that there is always a result.
This works around issues in the crypt.crypt python function that returns
None for algorithms it does not know.
Some modules (like user module) interprets None as no password at all,
which is misleading.
* The password_hash filter supports all parameters of passlib.
This allows users to provide a rounds parameter, fixing #15326.
* password_hash is not restricted to the subset provided by crypt.crypt,
fixing one half of #17266.
* Updated documentation fixes other half of #17266.
* password_hash does not hard-code the salt-length, which fixes bcrypt
in connection with passlib.
bcrypt requires a salt with length 22, which fixes#25347
* Salts are only generated by ansible when using crypt.crypt.
Otherwise passlib generates them.
* Avoids deprecated functionality of passlib with newer library versions.
* When no rounds are specified for sha256/sha256_crypt and sha512/sha512_crypt
always uses the default values used by crypt, i.e. 5000 rounds.
Before when installed passlibs' defaults were used.
passlib changes its defaults with newer library versions, leading to non
idempotent behavior.
NOTE: This will lead to the recalculation of existing hashes generated
with passlib and without a rounds parameter.
Yet henceforth the hashes will remain the same.
No matter the installed passlib version.
Making these hashes idempotent.
Fixes#15326Fixes#17266Fixes#25347 except bcrypt still uses 2a, instead of the suggested 2b.
* random_salt is solely handled by encrypt.py.
There is no _random_salt function there anymore.
Also the test moved to test_encrypt.py.
* Uses pytest.skip when passlib is not available, instead of a silent return.
* More checks are executed when passlib is not available.
* Moves tests that require passlib into their own test-function.
* Uses the six library to reraise the exception.
* Fixes integration test.
When no rounds are provided the defaults of crypt are used.
In that case the rounds are not part of the resulting MCF output.
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
Wow, this does not seem to be an uncommon misspelling. Might be there
are some left that span over two lines. I noticed the one in the git
module and then used `grep -rw 'the the'` to find some more.
Blocks currently don't support loops such as with_items or sequence. It would be helpful to make this clear in the docs otherwise it's a bit of a gotcha.
+label: docsite_pr
* Remove use of simplejson throughout code base. Fixes#42761
* Address failing tests
* Remove simplejson from contrib and other outlying files
* Add changelog fragment for simplejson removal
* Support multi-doc yaml in the from_yaml filter
* Most automatic method of handling multidoc
* Only use safe_load_all
* Implement separate filter
* Update plugin docs and changelog
<!--- Your description here -->
The example has:
`{{ 'Some DNS servers are 8.8.8.8 and 8.8.4.4' | regex_findall('\b(?:[0-9]{1,3}\.){3}[0-9]{1,3}\b') }}`
It needs be double backslashes to escape the backslashes:
`{{ 'Some DNS servers are 8.8.8.8 and 8.8.4.4' | regex_findall('\\b(?:[0-9]{1,3}\\.){3}[0-9]{1,3}\\b') }}`
+label: docsite_pr
* Update Openstack dynamic inventory link
* Add note for change of script name
* Change name of script to prevent Python module import errors.
Fixes#41562
* runas + async - get working on older hosts
* fixed up sanity issues
* Moved first task to end of test for CI race issues
* Minor change to async test to be more stable, change to runas become to not touch the disk
* moved async test back to normal spot
* Add docs detailing how to convert many with_X style loops to use loop and filters. Fixes#40727
* Switch lookup used in query vs lookup comparison, to not recommend use of nested lookup
* Improve docs based on feedback
* Add a 'machinectl shell' become_method
* docs: add explanations for the machinectl become_method
* docs: machinectl become_method: specify this part is specific to Linux+systemd setups
* Docs - add shared snippet note about password prompts for ssh keys
Signed-off-by: Adam Miller <admiller@redhat.com>
* add note to ssh connection plugin, fix markup, fix typo
Signed-off-by: Adam Miller <admiller@redhat.com>
James Cammarata