Commit Graph

3403 Commits (fb09e79b2f21573f44427c91e8d2f577fc968559)

Author SHA1 Message Date
Matt Clay fb09e79b2f
[stable-2.19] Fixed string type in tags fieldattribute (#84655) (#85452)
(cherry picked from commit 8207406306)

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
6 months ago
Matt Clay 483b23ce6d
[stable-2.19] Ensure config env/ini values are tagged (#85404) (#85414)
* Ensure config env/ini values are tagged

Config env and ini values now have origin and trust tags applied.

* Remove unused import
(cherry picked from commit 6ff6339191)
6 months ago
Matt Clay 100b20260a
[stable-2.19] Short-circuit legacy network module prefix->action mapping (#85406) (#85413)
* Short-circuit legacy network module prefix->action mapping

* Modified a non-short-circuit compound conditional in a legacy networking path that attempted to resolve an action for any module name containing `_`. The bug was always present, but the typical presentation (an ImportError) was ignored prior to 2.19.
* The legacy networking path should be deprecated and removed in 2.20- a module could still be run under the wrong action if one with a matching prefix is found.

* unit test fix
(cherry picked from commit d6efb7db8a)

Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
6 months ago
Patrick Kingston a633311f97 Test multiple inheritance in _utils get_all_subclasses (#85396)
Should bring code coverage in _utils.py to 100%.
Ensures get_all_subclasses won't enter infinite recursion.

(cherry picked from commit fcffd707c6)
7 months ago
Matt Davis ebae950db2 Encryptedstring redact fixes (#85390)
* misc DTFIX/docstring cleanup

* fix EncryptedString redaction, add tests

Co-authored-by: Matt Clay <matt@mystile.com>

* Fix test failures

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 649c9ec443)
7 months ago
Matt Davis 8c083e4d1d ensure transform_to_native_types converts keys (#85389)
* added basic key visitor support to variable visitor (off by default)
* transform_to_native_types enables new key visit/conversion
* add test

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit db819d2514)
7 months ago
Matt Davis b23aa84a49 Jinja sandbox refinement (#85385)
* DTFIX recategorize

* fix ansible-console generated task dict

* use non-deprecated task shape

* switch Jinja from ImmutableSandboxedEnvironment to SandboxedEnvironment

* Restore ability to call collection mutation methods.
* Restore ability to directly call integer bitwise operator methods.
* Adjust tests.

Co-authored-by: Matt Clay <matt@mystile.com>

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 11f5563895)
7 months ago
Patrick Kingston 68bf0a66cb Add test coverage to ansiballz builder source mapping (#85382)
Should bring code coverage in _builder.py to 100%.

(cherry picked from commit e7c7df7074)
7 months ago
Matt Davis b3dde82ab1 deprecate esoteric/undocumented playbook syntaxes (#85378)
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 547c64f326)
7 months ago
Matt Davis 0ce6b45db2 deprecate invalid variable names in inventory (#85377)
* deprecate invalid variable names in inventory

* was previously a hard error for `Host`
* added missing check to `Group`
* swapped blanket Python keyword prohibition with Jinja singleton and `not` check

Co-authored-by: Matt Clay <matt@mystile.com>

* fix invalid variable name test

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 3c52b14c9e)
7 months ago
Matt Davis 9f7432c5ff display wrap/format arg cleanup (#85352)
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit fa9f286096)
7 months ago
Sloane Hertel 39ed1fc4bc password lookup - re-attempt acquiring lock file regardless of locale (#85318)
* Fix handling FileExistsError, instead of only handling OSError when the human-readable error message is "File exists".

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 8e9f5fb9d5)
7 months ago
Matt Davis 59cdb65926 Misc callback fixes/cleanup (#85344)
* Misc callback fixes/cleanup

* Fix v1 callback method dispatch, fully deprecate v1 methods, add missing tests.
* Clean up callback plugin init/setup code, remove redundancies, improve error messaging.
* Remove unused callback method definitions from base class.

Co-authored-by: Matt Clay <matt@mystile.com>

* switch callback bypass to instance-level from class-level

* preserves any instance-level method magic that implementations were using

* add missing handler dispatch entry

* add tests to ensure all methods are covered

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit eec57ec396)
7 months ago
Maxim Evtush 19f3890275 Fix Typo in Variable Name and Update Test Data for Identifier Validation (#85338)
(cherry picked from commit 778fdcbef1)
7 months ago
Matt Clay 69ac7b1278 Add type annotations to set_module_args (#85329)
(cherry picked from commit 025e9cfae6)
7 months ago
Matt Clay 0cecfd9877 AnsiballZ refactoring and remote debug support (#85289)
(cherry picked from commit 45dd2c0647)
7 months ago
Martin Krizek 33c55b9c25 Fix templating tags on plays and roles (#82314)
Fixes #69903

ci_complete

(cherry picked from commit 532edf4c35)
7 months ago
Matt Davis b95bc19853 fix Marker handling on Jinja macro invocations (#85280)
* always allow Marker args to pass through
* always disable pre-emptive trip-on-retrieval for Macro JinjaCallContext
* add macro-callable template expression result test cases

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 2bed98bd20)
7 months ago
Martin Krizek eb29a662f6 Exceptions clean up (#85176)
* replace usage of `IOError` as it is an alias to `OSError`
* replace usage of `socket.error` as it is an alias to `OSError`
* use subclasses of `OSError` rather than inspecting `errno`s
* utilize `exist_ok` parameter of `os.makedirs` rather than ignoring
  `FileExistsError`

Make the following changes to the exception handling this patch already
updates to be consistent with the new code:
* use `ex` as a name for exception being handled
* use `from ex` when re-raising exception for additional context
* use f-strings and `!r` for quoting
* pass exceptions to the `exception` parameter of `fail_json`
* use `display.error_as_warning` rather than passing stringified
  exception into `display.warning`

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 600c1e67b4)
7 months ago
Matt Davis 0f2bc31753 adjust PluginInfo to use PluginType enum (#85277)
* normalization fixups

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 43c0132caa)
7 months ago
Matt Davis 7b823720ca Prevent template lookup and action from masking `ansible_managed` value (#85075)
* deprecate DEFAULT_MANAGED_STR and prevent masking of ansible_managed var

* adjust public API behavior

* restore backward-compatible behavior on existing public API

(cherry picked from commit 9f0a8075e3)
7 months ago
Matt Davis a943626ce6 Restore 2.18 vault tag YAML dump behavior (#85275)
* Doing conditional redaction/formatting needs other bits that aren't ready for 2.19.

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 2b7204527b)
7 months ago
Matt Clay a57229aa8a Fix YAML loading/dumping and filters (#85266)
- from_yaml/from_yaml_all filters now preserve trust
- YAML dumping can once again handle undecryptable vaulted values
- increased test coverage

Co-authored-by: Matt Davis <nitzmahone@redhat.com>
(cherry picked from commit 1c06c46cc1)
7 months ago
Matt Davis bc624d35b9 Fix incorrect behavior when a Jinja test returns Marker (#85264)
* Avoid Marker trip in plugin wrapper that causes unnecessary early template bailout.
* Remove spurious non-boolean result deprecation warning.
* Add test.

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 6198c7377f)
7 months ago
Matt Davis 3a487532dd fix key conversion on legacy JSON profiles (#85253)
* fix key conversion on legacy JSON profiles

* restore stdlib silent str conversion behavior for int/float/bool/None dict key types
* remove most conversions for non-scalar keys
* add key conversion cases to JSON profile test suite

* typo

* Fix sanity test failures

* Fix _handle_key_str_fallback implementation

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit bbbfbd57e2)
8 months ago
Matt Clay 0576ff3e65 Add missing warning methods and args (#85225)
(cherry picked from commit eafe5fc739)
8 months ago
Matt Davis 85283e7238 Clean up TE error handling, wrap sigalrm handler (#85232)
* Clean up TE error handling, wrap sigalrm handler

* Preserve error detail on AnsibleAction and Connection exceptions.
* Remove multiple layers of unreachable or redundant error handling.
* Wrap manual alarm signal/timeout handling into a context manager, add tests.

Co-authored-by: Matt Clay <matt@mystile.com>

* update error message check in test

* update test timeout message assertions

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit cbcefc53a3)
8 months ago
Brian Coca 311ef75245 async handling changed to use 'true' booleans(#85074)
async_status and async_wrrapper now use booleans instead of 0/1 for 'thruthyness'
gather_facts also updated

Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit d41a3430b7)
8 months ago
Matt Davis 5fd78b07fb Added _TEMPLAR_SANDBOX_MODE config (#85222)
* Added _TEMPLAR_SANDBOX_MODE config

* allows unsafe attribute checks to be disabled in Jinja sandbox

* Update lib/ansible/_internal/_templating/_jinja_bits.py

Co-authored-by: Matt Clay <matt@mystile.com>

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 91453e30af)
8 months ago
Matt Davis 21f478e77b Decouple SSH client verbosity from Ansible display (#85224)
* decouple SSH client verbosity from Ansible display

* remove failing false-coverage unit test

(cherry picked from commit b71d9aa4a5)
8 months ago
Harshvardhan Sharma fa2f1e4750 Improve SUSE distribution detection using VARIANT_ID with fallback (#85152)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit f05b1d1ccf)
8 months ago
Jordan Borean 880b584124 Add support for Windows App Control/WDAC (#84898)
* Add support for Windows App Control/WDAC

Adds preview support for Windows App Control, formerly known as WDAC.
This is a tech preview feature and is designed to test out improvements
needed in future versions of Ansible.

* Use psd1 and parse it through the Ast to avoid any unexpected execution results

* Add tests for various manifest permutations

* Ignore test shebang failure

* Apply suggestions from code review

Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>

* Use more flexible test expectations

* Add type annotations for shell functions

---------

Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
(cherry picked from commit 75f7b2267d)
8 months ago
Matt Clay 638d711a72 ansible-test - Fix option filtering (#85182)
(cherry picked from commit feda0a5c6e)
8 months ago
Matt Clay 01e97d7f80 DTFIX recategorization and error/warning refactor (#85181)
Co-authored-by: Matt Davis <nitzmahone@redhat.com>
(cherry picked from commit 242bb9ebab)
8 months ago
Matt Davis 3a2248a992 recategorize pass over DTFIX-MERGE entries post b4 (#85155)
(cherry picked from commit f5a85493c1)
8 months ago
Matt Davis fddda73042 ensure builtin tests always return bool (#85149)
* added unit test validation and coverage enforcement

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit e5476972da)
8 months ago
Matt Davis 60ddb2954b misc changelog and docstring cleanup (#85148)
* deferred listify_lookup_plugin_terms deprecation

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 7be1359841)
8 months ago
Matt Davis d6a8582da7 fix ensure_type to support vaulted values (#85129)
* restored parity with 2.18

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 9a426fe303)
8 months ago
Matt Davis 9fff6d433d Misc config type coercion fixes (#85119)
* remove dead config comment noise

* update `list` typed config defaults to be lists

* fix tag preservation/propagation in config
* numerous other ensure_type bugfixes
* 100% unit test coverage of ensure_type
* emit warnings on template_default failures
* fix unhandled exception in convert_bool on unhashable inputs

Co-authored-by: Matt Clay <matt@mystile.com>

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit d33bedc48f)
8 months ago
Matt Davis 8c8717a8e4 Switch to stackwalk caller ID (#85095)
* See changelog fragment for most changes.
* Defer early config warnings until display is functioning, eliminating related fallback display logic.
* Added more type annotations and docstrings.
* ansible-test - pylint sanity for deprecations improved.
* Refactored inline legacy resolutions in PluginLoader.

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit ff6998f2b9)
9 months ago
Brian Coca 131175a5a6
ensure predictable permissions on module artifacts (#84948)
and test it!

(cherry picked from commit 9f894b81c2)
9 months ago
Matt Davis f898f9fec6 Implement TaskResult backward compatibility for callbacks (#85039)
* Implement TaskResult backward compatibility for callbacks
* general API cleanup
* misc deprecations

Co-authored-by: Matt Clay <matt@mystile.com>

* fix v2_on_any deprecation exclusion for base

---------

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 03181ac87b)
9 months ago
Brian Coca 89a4900b61
normalize error handler choices (#84998)
use existing to avoid deprecation cycle
normalize test too

(cherry picked from commit 2cbb721f6f)
9 months ago
Lee Garrett 7fc916361e
Fix test_range_templating on 32-bit architectures (#85007)
* Fix test_range_templating on 32-bit architectures

32-bit archtectures like i386, armel, armhf will fail with the error

ansible._internal._templating._errors.AnsibleTemplatePluginRuntimeError: The
filter plugin 'ansible.builtin.random' failed: Python int too large to convert
to C ssize_t

So just pick sys.maxsize (2**31 - 1) so it works on 32 bit machines.

---------

Co-authored-by: Lee Garrett <lgarrett@rocketjump.eu>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 5f6aef95ac)
9 months ago
Matt Davis 82ea3addce
Miscellaneous fixes (#85012)
* Add help_text to play_hosts deprecation

* clean up TaskResult type handling

(cherry picked from commit 1b6b910439)
9 months ago
Matt Clay 80d5f05642
Miscellaneous DT fixes (#84991)
* Use `_UNSET` instead of allowing `ellipsis`

* Fix deprecation warning pre-check

* Deprecation warnings from modules can now be disabled.
* Deprecation warnings from modules get the "can be disabled" notice.

* Include help text in pre-display fatal errors

* Simplify lookup warning/debug messaging

* Fix return type of `timedout` test plugin

* Use `object` for `_UNSET`

* Remove obsolete `convert_data` tests

* Remove unnecessary template from test

* Improve legacy YAML objects backward compat

* Fix templar backward compat for None overrides

(cherry picked from commit 6cc97447aa)
9 months ago
Matt Davis 35750ed321
Templating overhaul, implement Data Tagging (#84621)
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Matt Clay <matt@mystile.com>
9 months ago
Abhijeet Kasurde 5347d4d4fc
distro: detect debian for OS_FAMILY for LMDE 6 (#84941)
Fixes: #84934

Signed-off-by: Abhijeet Kasurde <Akasurde@redhat.com>
9 months ago
Matt Clay 462affa7c4
Clean up interpreter discovery (#84394)
* Clean up interpreter discovery

- Deprecated `auto_legacy` and `auto_legacy_silent`
- Removed obsolete platform fallback config and logic
- Replaced unit tests with integration tests
- Increased test coverage
10 months ago
Jordan Borean 7d0886457a
Windows Exec Runner Update (#84868)
Updates the Windows exec runner in preparation for the WDAC changes.
This new process is designed to improve the way modules are run by
Windows and expose common functionality to run PowerShell code in a
common environment. It also includes futher changes to improve the error
handling to make it easier to see where an error occurred in the running
code.
10 months ago