Commit Graph

140 Commits (1b8bfd40abfaba139b9ffc76601551ec7720e1a5)

Author SHA1 Message Date
Brian Coca 4fa512406b
loop_control "early exit" feature (#62151)
* add a loop_control break_when directive to break out of a loop after any item

* remove loop var as normal exit would

* example usage:

- name: generate a random password up to 10 times, until it matches the policy
  set_fact:
    password: "{{ lookup('password', '/dev/null', chars=character_set, length=length) }}"
  loop: "{{ range(0, 10) }}"
  loop_control:
    break_when:
      - password is match(password_policy)

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
3 months ago
Lee Garrett 7e3916b767
Typo fixes and other bits and bobs (#83672)
Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
4 months ago
Abhijeet Kasurde d16ad25e4b
Remove extraneous pylint ignore (#83257)
* Pylint issue 511 is closed, remove ignore related to that.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
6 months ago
Matt Martz 5566a655ae
Updates to interpreter discovery for 2.17 (#82420) 9 months ago
Brian Coca 29b2405b84
Revert "Fix older external connection plugins without _sub_plugin declaration (#79372)" (#82473)
This reverts commit ef1e01aa8b.

broken tests
11 months ago
J vanBemmel ef1e01aa8b
Fix older external connection plugins without _sub_plugin declaration (#79372)
fixes https://github.com/ansible/ansible/issues/79371

* Check for sub=={} too
* Add a test case for older plugins that do not define '_sub_plugin'
* Add test for legacy plugins
11 months ago
Matt Clay 5b1b0ce762 Remove Python 2 compat (via six) from unit tests 1 year ago
Matt Clay 18e8401edd
Remove Python 2.x compat from unit tests (#82109) 1 year ago
Sviatoslav Sydorenko d2ba76c117
Remove Python `< 3.5` `selectors` fallbacks (#81872) 1 year ago
Matt Clay 9f899f9492
Require `from __future__ import annotations` (#81902) 1 year ago
Matt Clay b94ee1cefd
Drop Python 2.7 and Python 3.6 support (#81866)
* Drop Python 2.7 and Python 3.6 support

* Remove obsolete _json_compat
1 year ago
Abhijeet Kasurde 86fd7026a8
Handle exceptions in interpreter discovery (#81745)
* Handle exceptions like AnsibleError, AnsibleConnectionFailure
  raise while interpreter discovery.

Fixes: #78264

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
1 year ago
Sloane Hertel ecbbd7e5d1
Remove Python 3.5 support (#80972)
* Remove Python 3.5 support

* changelog

* add requested changes

* Update changelogs/fragments/remove-python3.5.yml

Co-authored-by: Matt Clay <matt@mystile.com>

* remove Python3 < 3.5 error handling

* remove Python3 < 3.5 error handling in TE too

---------

Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Abhijeet Kasurde 652ddc4078
Fix pylint disallowed-name errors (#80854)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2 years ago
Brian Coca 4899d4a4d6
simple skip of connection (#80638)
Avoid 'persistent' connection handling if not needed
removed useless connection passing
added property to action classes as indicator/selector
2 years ago
Jordan Borean 8db9bd7574
Remove deprecated include (#80752)
* Remove deprecated include

* Add tombstone entry for include

* Use string for date

* Use todays date

* Remove uneeded if statement
2 years ago
Matt Clay 2cd1744be3
Use ansible.module_utils.common.text.converters (#80704)
Replace use of old `ansible.module_utils._text` and add a unit test to maintain backwards compatibility.
2 years ago
Matt Martz 42355d181a
Do not double calculate loops and `delegate_to` (#80171) 2 years ago
Matt Clay 72c59cfd98
Avoid trailing spaces in unit test (#80033) 2 years ago
Jordan Borean 1e6b8249e7
Add condition that causes a when to skip a task to output msg (#78918)
* Add condition that causes a when to skip a task

* Fix up tests

* Use false_condition instead of failed_condition

* Remove formatting accidentially added

* Fix sanity
2 years ago
Matt Clay 379058e10f
Update collections.abc imports (#79911)
* Update `collections.abc` imports

- Use `six.moves` for modules and module_utils
- Use `collections.abc` for controller code

This avoids using `ansible.module_utils.common._collections_compat`,
which was added before the vendored `six` was updated to provide these
imports.

* Update _collections_compat to use six.moves

Also update the custom pylint rule to reflect this change.
2 years ago
Matt Clay 25d859f8fb
Clean up unused imports in unit tests (#79893) 2 years ago
Matt Clay c33c8cf086
Remove unused and unreachable unit test code (#79854) 2 years ago
Matt Martz 56d142350d
Add support for importlib.resources (#78915)
* Add support for importlib.resources

* Remove the importlib.resources imports

* return the correct data

* Some code comments, and re-order for consistency

* Disallow traversing packages below an individual collection

* Add a traversable class for namespaces

* Re-use variable

* Utilize itertools.chain.from_iterable

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Simplify logic to check for packages from ansible loaders

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Just a generator expression, instead of a generator

* docstrings

* Add comment about find_spec for our namespaces

* Add some initial unit tests for importlib.resources

* normalize

* Utilize importlib.resources for listing collections

* collections_path is already in config, just use config

* install uses a different default for collections_path

* Remove unused import

* Remove duplicate __truediv__

* Bring back TraversableResources

* Apply some small suggestions from code review

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>

* Remove cross contamination between plugin loader code and CLI code

* Remove unused import

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
2 years ago
Jordan Borean ba6da65a0f
Fix connection/become task loop settings (#78565)
* Fix connection/become task loop settings

* Remove old code
2 years ago
Kate Case bf1ef5a1f3
Replace get_persistent_connection_options in task_executor with get_options (#74446)
Replace get_persistent_connection_options with get_options
Remove special case for network sub_plugin in _set_plugin_options
Try to avoid mock connection pretending to be persistent
Rename variables->options to reflect what they actually are
Gather options for ssh_type_conn on network_cli
Drop reliance on sub_plugin["type"]
2 years ago
Matt Clay 9ee183c4b9
Clean up unit tests. (#78498) 2 years ago
Sloane Hertel 621e782ed0
Add toggle to fix module_defaults with module-as-redirected-action on a per-module basis (#77265)
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults

Add a toggle for action plugins to prefer the resolved module when loading module_defaults

Allow moving away from modules intercepted as actions pattern

Fixes #77059
3 years ago
Matt Clay 3a9a23fb1a
Use unittest.mock instead of mock. (#77883)
This can only be done for controller tests.
3 years ago
Martin Krizek 94eff70030
PlayIterator - remove deprecated states (#77444) 3 years ago
Matt Clay 1a5853d794
Remove obsolete units.compat.mock compat layer. (#77118)
* Remove obsolete units.compat.mock compat layer.
* Update remaining units.compat.mock references.
3 years ago
Brian Coca 9142be2f6c
Allow specifying specific python via shebang (#76677)
modules with python were always normalized to /usr/bin/python,
  while other interpreters could have specific versions.

* now shebang is always constructed by get_shebang and args are preserved
* only update shebang if interpreter changed
* updated test expectation
* added python shebang test
3 years ago
Martin Krizek 5c225dc0f5
Introduce public methods to access PlayIterator._host_states (#74416) 3 years ago
Martin Krizek 395e5e20fa
PlayIterator: use enums for states (#74511) 3 years ago
Matt Clay 7450e87615
Fix up string splitting to prepare for pylint update. (#75479)
* Use rsplit with maxsplit in BaseCacheModule.
* Use maxsplit for accessing first item.
* ansible-test - More efficient splitting.
3 years ago
Sam Doran fa7482c63a
Change interpreter discovery defaults to silently prefer Python 3 (#75049)
Push /usr/bin/python to almost the bottom of the barrel. This makes the strategy to prefer
specific versions of the "mystery meat" version.

* Change INTERPRETER_PYTHON default to 'auto'
    Change description to match change in behavior.
    Change deprecation message to a warning.

* Update docs
* Add porting guide entry
* Update unit tests
* Update integration test
* Allow INTERPRETER_PYTHON_FALLBACK to be configure using a variable
* Prefer platform-python above other Python 2 interpreters
* Add Python 3.10 to the list of interpreters
3 years ago
Brian Coca 415e08c297
Try to get best usable locale (#75033)
specially for when you have parameters in unicode but need
  to scrape responses, C is still the fallback

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
3 years ago
Sam Doran abacf6a108
Use ArgumentSpecValidator in AnsibleModule (#73703)
* Begin using ArgumentSpecValidator in AnsibleModule

* Add check parameters to ArgumentSpecValidator

Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.

* Make errors related to unsupported parameters match existing behavior

Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.

* Set default values after performining checks

* FIx sanity test failure

* Use correct parameters when checking sub options

* Use a dict when iterating over check functions

Referencing by key names makes things a bit more readable IMO.

* Fix bug in comparison for sub options evaluation

* Add options_context to check functions

This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.

* Fix bug in apply_defaults behavior of sub spec validation

* Accept options_conext in get_unsupported_parameters()

If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.

* Build path to the unsupported parameter for error messages.

* Remove unused import

* Update recursive finder test

* Skip if running in check mode

This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.

This is a silght change in behavior, but I believe the correct one.

Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.

* The great purge

Remove all methods related to argument spec validation from AnsibleModule

* Keep _name and kind in the caller and out of the validator

This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.

* Double moustaches work

I wasn't sure if they get stripped or not. Looks like they do. Neat trick.

* Add changelog

* Update unsupported parameter test

The error message changed to include name and kind.

* Remove unused import

* Add better documentation for ArgumentSpecValidator class

* Fix example

* Few more docs fixes

* Mark required and mutually exclusive attributes as private

* Mark validate functions as private

* Reorganize functions in validation.py

* Remove unused imports in basic.py related to argument spec validation

* Create errors is module_utils

We have errors in lib/ansible/errors/ but those cannot be used by modules.

* Update recursive finder test

* Move errors to file rather than __init__.py

* Change ArgumentSpecValidator.validate() interface

Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.

Return the validated parameters if validation is successful rather than True/False.

Update docs and tests.

* Get attribute in loop so that the attribute name can also be used as a parameter

* Shorten line

* Update calling code in AnsibleModule for new validator interface

* Update calling code in validate_argument_spec based in new validation interface

* Base custom exception class off of Exception

* Call the __init__ method of the base Exception class to populate args

* Ensure no_log values are always updated

* Make custom exceptions more hierarchical

This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.

* Just go back to basing off Exception

* Return ValidationResult object on successful validation

Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.

* Make it easier to get error objects from AnsibleValidationResultMultiple

This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.

* Define custom exception for each type of validation failure

These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.

* Update tests based on (yet another) interface change

* Mark several more functions as private

These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.

* Move warnings and deprecations to result object

Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.

* Use subclass for module arg spec validation

The subclass uses global warning and deprecations feature

* Fix up docs

* Remove legal_inputs munging from _handle_aliases()

This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.

Create a private function just for getting legal inputs since that is done in a couple of places.

It may make sense store that on the ValidationResult object.

* Increase test coverage

* Remove unnecessary conditional

ci_complete

* Mark warnings and deprecations as private in the ValidationResult

They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.

* Mark valid_parameter_names as private and populate it during initialization

* Use a global for storing the list of additonal checks to perform

This list is used by the main validate method as well as the sub spec validation.
4 years ago
Matt Davis 4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
4 years ago
Martin Krizek 1b70260d5a
Add implicit role_complete block instead of role._eor (#72208)
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes #69848
4 years ago
Matt Davis 4c0af6c808
fix internal cases of actions calling unqualified module names (#70818)
* fix internal cases of actions calling unqualified module names

* add porting_guide entry
* misc other fixes around action/module resolution broken by redirection

ci_complete

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* address review feedback

* pep8

* unit test fixes

* win fixes

* gather_facts fix module args ignores

* docs sanity

* pep8

* fix timeout test

* fix win name rewrites

Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Davis c616e54a6e
refactor Python module_utils locator (#70610)
* refactor Python module_utils locator

* no longer recursive
* embed special-case module code internally
* share common code between collections/not cases
* fixes #70134
* properly support subpackage redirection
* adds support for FQCN redirect targets used by migration (expands to FQ Python name)
* add tests

* add changelog
4 years ago
Rick Elrod 9cfc76a221
Nuke with_ squashing, deprecated for 2.11 (#70209)
Change:
Removes with_* loop squashing and tests for 2.11

Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Clay c67efe0bd1 Avoid unnecessary comprehensions. 5 years ago
Karl Fogel 225ae65b0f
Make Python path warning say what it means to say (#69669) 5 years ago
Bob Weinand f200487414
Fix filedescriptor out of range in select() when running commands (#65058)
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
5 years ago
Nathaniel Case 3dbc03d58a
Try to load action plugin from the same collection as the module (#66701)
* Try to load network action plugin from the same collection as the module

* Alter tests to match

Just make sure the action plugin is as qualified as the module it is paired with
5 years ago
Sam Doran 3461c682c3
Add mechanism for storing warnings and deprecations outside of AnsibleModule (#58993)
* Move warn() and deprecate() methods out of basic.py
* Use _global_warnings and _global_deprications and create accessor functions
    - This lays the foundation for future functions being moved outside of AnsibleModule
       that need an interface to warnings and deprecations without modifying them.
* Add unit tests for new warn and deprecate functions
5 years ago
Ganesh Nalawade 74e9b1e219
Fix network action plugin load in collection (#65849)
* Fix network action plugin load in collection

Fixes https://github.com/ansible/ansible/issues/65071

*  Load network action plugin that matches the module
   prefix name from list of collections.

* Update changelog

* Fix unit test
5 years ago
Mads Jensen c5d61953b1 Remove empty setUp/tearDown/tearDownClass methods in test classes. 5 years ago