Commit Graph

12047 Commits (1ecc62ba0609cd75d798e309c6c8dd14958dd01a)

Author SHA1 Message Date
Sloane Hertel d18d4f84ec
_symbolic_mode_to_octal- fix raising ValueError for invalid symbolic modes (#80449)
validate the full user and perm strings instead of just first character

fixes unhelpful unarchive error for some invalid modes
1 year ago
Martin Krizek ed749cf0a0
Remove deprecated FileLock class (#80438) 1 year ago
Martin Krizek 8754d8e1e1
ActionBase: remove deprecated _remote_checksum (#80440) 1 year ago
Matt Clay 4d25e3d54f
Set the minimum setuptools to 45.2.0 (#80649)
Also update the package-data sanity test to use the minimum setuptools version.
1 year ago
Landson Guo ece8da71ea
Add TTY check and argument to disable it (#50603) (#68260)
* Add TTY check and argument to disable it (#50603)

* Fix formatting

* add changelog

* rename flag and updated help description

* add tests for tty check

* replace deprecated uses of assertRaisesRegexp to assertRaisesRegex

* fix yaml syntax

* shorten line 79

* Revert "replace deprecated uses of assertRaisesRegexp to assertRaisesRegex"

This reverts commit cea5fe1655.

* change back to assertRaisesRegexp
1 year ago
Brian Coca b2c0095722
Fix gather parallel (#80597)
* fix meaning of parallel in gather_facts
 * Update docs with note about parallel not always being faster
 * add 'smarter' usage of gahter_timeout for parallel tasks
 * restore async when needed, not always
 * added typing
 * parallelism tests
1 year ago
Martin Krizek 1e8b889b93
Templar: remove deprecated shared_loader_obj param (#80439) 1 year ago
Matt Clay 5e220416a2
ansible-test - Update the default containers (#80635) 1 year ago
Matt Clay 372fc99686
ansible-test - Update pylint to 2.17.3 (#80632) 1 year ago
Martin Krizek a2be8915de
PlayIterator: remove deprecated methods (#80437) 1 year ago
Matt Clay abc58c026b
ansible-test - Use FreeBSD packaged setuptools (#80615)
This will avoid issues during bootstrapping caused by breaking changes in setuptools.
1 year ago
Martin Krizek 989a7d6951
Fix calling v2_playbook_on_notify callback (#80585)
Co-authored-by: Matt Martz <matt@sivel.net>
1 year ago
Maxwell G f4615d69ad
tests: replace a mock import with unittest.mock (#80589) 1 year ago
Martin Krizek 748f534312
Use target of /usr/bin/dnf for dnf version detection (#80550)
Fixes #80376
1 year ago
Matt Clay 7b50571588
ansible-test - Remove deprecated features (#80558) 1 year ago
Matt Clay 1fdeff949c
ansible-test - Update default containers (#80559) 1 year ago
Matt Clay ba47dd9ef4
ansible-test - Update pylint to 2.17.2 (#80555) 1 year ago
Sloane Hertel 694c11d5bd
argspec - fix validating type for required options that are None (#79677)
* Only bypass type validation for null parameters if the default is None. A default is mutually exclusive with required.

* Prevent coercing None to str type. Fail the type check instead.
1 year ago
Sloane Hertel 964e678a7f
ansible-galaxy - fix installing dir sources with a trailing path separator (#79110)
* strip trailing path separator from type=dir sources
1 year ago
Matt Clay 676b731e6f
ansible-test - Replace pytest-forked (#80525)
- Unit tests now report warnings generated during test runs.
- Python 3.12 warnings about `os.fork` usage with threads (due to `pytest-xdist`) are suppressed.
- Added integration tests to verify forked test behavior.
1 year ago
kloud-byun 2e62724a8a
feature: add support for symbolic links when passing "dir" (#80460)
* update the include_vars action plugin to always follow symbolic links when traversing directories
1 year ago
Matt Clay 43c5cbcaef
Fix galaxy CLI unit test assertions (#80504) 1 year ago
Matt Clay 3ec828703f
Fix unit test asserts (#80500) 1 year ago
Matt Clay 74b102b33c
Fix unit tests returning values (#80499) 1 year ago
Matt Clay bd22b849c6
Fix unclosed files in unit tests (#80497) 1 year ago
Martin Krizek 09dd80b4ec
Last handler defined runs, fix for roles (#79558)
Fixes #73643
* clear_notification method and simplify ifs
* Deduplicate code
* Limit number of Templar creations
* Fix sanity
* Preserve handler callbacks order as they were notified
1 year ago
Matt Clay 0371ea08d6
ansible-test - Add support for argcomplete 3 (#80482) 1 year ago
Matt Martz 62211097cb
Expand pylint deprecated plugin to catch ansible.module_utils.common.warnings uses (#80462) 1 year ago
Martin Krizek bc3cc426bf
fetch_url: remove auto disabling decompress when gzip is not available (#80474) 1 year ago
Martin Krizek 9f0a9cf20d
get_action_args_with_defaults: remove deprecated redirected_names arg (#80473) 1 year ago
Matt Martz 2a795e5747
Manage basic.selinux completely, instead of mocking potentially missing module (#80448)
* Manage basic.selinux completely, instead of mocking potentially missing module. Fixes #80436

* pep8
1 year ago
Sean Reifschneider 243aea45ce
Symbolic modes with X or =[ugo] always use original mode (#80132)
* Symbolic modes with X or =[ugo] always use original mode (Fixes #80128)

Here's what's happening, by way of this mode example: u=,u=rX

At the first step in the loop, the "u" bits of are set to 0. On the next
step in the loop, the current stat of the filesystem object is used to
determine X, not the "new_mode" in the previous iteration of the loop. So
while most operations kind of operate left to right, "X" is always going
back to the original file to determine whether to set x bit.

The Linux "chmod" (the only one I've tested) doesn't operate this way. In
it, "X" operates on the current state the loop understands it is in,
based on previous operations (and starting with the file permissions).

This is an issue with "X" and any of the "=[ugo]" settings, because
they are lookups. For example, if a file is 755 and you do "ug=rx,o=u",
file module produces 0557 and chmod produces 0555.

This really becomes a problem when you want to recursively change a
directory of files, and the files are currently 755, but you want to
change the directory to 750 and the files to 640. In chmod you can do
"a=,ug=rX,u+w" (or "a=,u=rwX,g=rX"), and have it apply equally to the
directory and the files. I can't come up with a single way in the ansible
file module to deterministically, recursively, set a directory to 750
and the contents to 640 no matter what the current permissions are,
as the code currently is.

The fix is to pass in "new_mode" to _get_octal_mode_from_symbolic_perms
in lib/ansible/module_utils/basic.py inside _symbolic_mode_to_octal. And
then take "new_mode" as an argument and use it instead of the filesystem
object stat.st_mode value.

* Fixing my new unit test, fixing bug in test comments
1 year ago
Matt Clay f9534fd7b7
ansible-test - Update default containers (#80468) 1 year ago
Matt Clay babdec80cc
Fix the parsing integration tests (#80454)
Most of the bad_parsing tests were no longer running, with several of them no longer being valid.
The invalid tests have been removed and the valid ones rewritten.
1 year ago
Matt Martz 7251938370
ansible.builtin inferred by collection_name == None (#80446) 1 year ago
Felix Fontein 92c694372b
Replace validate-modules's semantic markup parser with antsibull-docs-parser (#80406) 1 year ago
Felix Fontein 96f7fd5539
Extend ansible-doc test with more classic markup. (#80407) 1 year ago
Matt Clay 163f297d7f
Extend bulk issue creator (#80423)
It can now be used to create feature requests, not just deprecation bug reports.
1 year ago
Matt Martz 6d1e355639
Only use the selectors code path when we are prompt matching (#79931) 1 year ago
Sloane Hertel 460abe0cef
Fetch signatures from galaxy after the dependency resolver runs (#80334)
Reduce the number of Galaxy API calls made during dependency resolution by fetching remote signatures afterwards, since these are not used in backtracking.

Reduce the verbosity to `-vvvv` (to match other Galaxy API calls) to see this activity.

Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
1 year ago
Matt Davis 5e131a96c0
bump devel to 2.16.0.dev0 (#80390) 1 year ago
Felix Fontein 2f647e9617
Implement semantic markup support for Ansible documentation in validate-modules. (#80243) 1 year ago
Matt Clay 7fcb9960e6
Use --no-isolation for package-data sanity test (#80377)
The dependencies are already in the sanity test venv. This avoids use of unpinned dependencies and a dependency on a network connection.
1 year ago
tu1h 043a0f3ee8
Fix run_once by instantly tiny post_validate (#78492) (#80051)
Signed-off-by: tu1h <lihai.tu@daocloud.io>
1 year ago
Felix Fontein 0e509ecf25
Support role extension for semantic markup. (#80305) 1 year ago
Martin Krizek a81b787a05
Add new dnf5 module (#80272) 1 year ago
Sloane Hertel 666188892e
fix using templated values for include/import role FA (#80320)
* fix using templated values for include/import role options 'public', 'allow_duplicates', and 'rolespec_validate'

* pass templated values without changing the instance

* Fix templating by setting always_post_validate to True and calling IncludeRole.post_validate() instead

ci_complete

* add changelog
1 year ago
Matt Clay 5d64024f1a
ansible-test - Fix packaging change classification (#80360) 1 year ago
Matt Clay d8adc081d8
ansible-test - Update base and default container (#80355) 1 year ago
Matt Clay 49b5c80c41
ansible-test - Update sanity test requirements (#80354) 1 year ago
Matt Clay 46362bbd27
Update package-data sanity test (#80344)
The test no longer relies on the Makefile.
1 year ago
Matt Clay 052da3c89e
Use `skip/macos` instead of `skip/macos/13.2` (#80343)
The reason for the skip won't be going away with future versions of macOS.
1 year ago
Matt Clay 1d6f019ce6
Remove obsolete integration test aliases (#80342)
* Remove obsolete integration test aliases

* Remove obsolete `skip/osx` aliases

* Use `skip/macos` instead of `skip/osx` in docs
1 year ago
Matt Martz d2f6ea4179
New deb822_repository module (#80018)
Fixes #77073
1 year ago
Felix Fontein 8600a1b927
Bump antsibull-docs version in docs-build sanity test (#80326)
* Bump antsibull-docs to 1.11.0.

* Update all requirements for the docs-build sanity test.

* Run again with Python 3.9.
1 year ago
Felix Fontein 086ae42209
Fix URL processing. (#80295) 1 year ago
Jordan Borean ba4505f5cb
Ansible.Basic - Improve temporary file cleanup process (#80293)
* Ansible.Basic - Improve temporary file cleanup process

* Add comment on struct value used
1 year ago
Matt Clay 8ce102f302
ansible-test - Remove macos/12.0 remote (#80319) 1 year ago
Brian Coca 0fd88717c9
password lookup, handle ident properly when saved (#80251)
* password lookup, handle ident properly when saved

  Currently we format and save ident when present but we didn't account for this when reading the saved file
  Also added some more robust error handling.
1 year ago
Wong Hoi Sing Edison 016b7f71b1
ansible-galaxy - support `resolvelib >= 0.5.3, < 1.1.0` (#80196)
* ansible-galaxy - support `resolvelib >= 0.5.3, < 1.1.0`

<https://pypi.org/project/resolvelib/1.0.1> released on 2023-03-09:

-   <https://github.com/sarugaku/resolvelib/blob/main/CHANGELOG.rst#101-2023-03-09>
-   <https://github.com/sarugaku/resolvelib/releases/tag/1.0.1>

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>

* Trigger CI by pinning resolvelib with latest version

Since resolvelib is pre-installed in our test containers, we should temporarily pin the latest version allowed to force the tests to run with that version. Once the tests have passed that commit can be reverted.

Please make those changes without force pushing, so that we keep the reference to the passing CI run. We can squash the commits when merging the PR so the temporary commits won't be in the final commit merged to the devel branch.

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>

* https://github.com/ansible/ansible/pull/80196#discussion_r1136003637

Also test resolvelib with multiple supported versions.

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>

* Revert "Trigger CI by pinning resolvelib with latest version"

This reverts commit 5518e5dbca.

---------

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
1 year ago
Felix Fontein b398613120
Add support for plugin field in seealso. (#80212) 1 year ago
Matt Clay 054aa92158
Fix url lookup test to use test container (#80284) 1 year ago
Matt Martz ed4b1afc6c
Don't double loop, ensuring that machine formats don't dupe collections onto all paths (#80277)
* Don't double loop, ensuring that machine formats don't dupe collections onto all paths

* Add tests
1 year ago
Matt Martz 42355d181a
Do not double calculate loops and `delegate_to` (#80171) 1 year ago
Felix Fontein fafb23094e
ansible-doc: fix broken seealso links in text output (#80280)
* Fix broken URLs.

* Also remove auto-generated description for modules outside ansible.builtin.
1 year ago
Felix Fontein 4ea50cef23
validate-modules: check_mode attribute and compare with supports_check_mode parameter (#80090)
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
1 year ago
Felix Fontein 5388f6020a
validate-modules: support plugin see-also (#80244) 1 year ago
Matt Clay 6a6824f570
ansible-test - Update CloudStack test container (#80278) 1 year ago
Matt Clay cf8fa5e75c
ansible-test - Update sanity test requirements (#80279) 1 year ago
Felix Fontein a2dc5fcc7d
Implement semantic markup support for Ansible documentation in ansible-doc. (#80242) 1 year ago
Felix Fontein 4ca3a29cd2
Bump antsibull-docs version to 1.10.0 to support semantic markup. (#80201) 1 year ago
Sviatoslav Sydorenko eebfd71a6d
Make PEP 517 mutation tests use pinned old setuptools (#80262)
* Pin setuptools to lowest supported @ PEP 517 test

This allows catching the behavior of builds under old setuptools.

* Stop invoking `setup.py install` in tests

This is not the part we care about since it involves dealing with the
external runtime dependencies rather than building our source
distribution.
1 year ago
Sviatoslav Sydorenko 7097df3eed
📦 Switch sdist build-system to pure setuptools (#80255)
This patch modifies the in-tree build backend to build sdists that swap
out pointers to it in the `pyproject.toml`'s `[build-system]` section.

The effect of this is that the first build from source (for example,
from a Git checkout) uses our PEP 517 in-tree build backend. But the
produced tarball has `build-backend` set to `setuptools.build_meta`
which is the native build backend of `setuptools`. So any following
builds from that sdist will skip using the in-tree build backend,
calling the setuptools' one.
The good news is that if the first build generated the manpages, they
will be included and won't go anywhere even though, a different build
system is in place.

Combined with #80253, this will make sure not to modify the current
source checkout on that first build.

Co-authored-by: Matt Clay <matt@mystile.com>
1 year ago
Matt Clay bad8843124
ansible-test - Update git diff handling (#80202)
This change allows ansible-test to work with newer versions of git on AZP.
1 year ago
Matt Clay f3bc0f18ff
ansible-test - Update base and default container (#80178) 1 year ago
Matt Clay 85b5f31f75
Update mypy to 1.1.1 (#80175) 1 year ago
Sloane Hertel b7a0e0d792
copy - fix check mode with remote_src=True (#78624)
* Don't create dest directory in check mode

uncomment existing test

Fix checking for file attribute changes in check mode and add a test
1 year ago
Matt Clay c564c6e21e
ansible-test - Update pylint requirements (#80172) 1 year ago
Matt Clay 31658031ed
ansible-test - Remove pointless comparison (#80168) 1 year ago
Martin Krizek cfa7acbc19
Raise an error on invalid FA.isa value (#80040)
Avoids bad definitions of playbook classes
1 year ago
Sviatoslav Sydorenko 701f8852ec
Revert "🧪 Switch macOS 13.2 to 12.0 in CI (#80145)" (#80156)
This reverts commit 326e70c02a.
1 year ago
Sloane Hertel 46c08c4467
move test so it runs when the plugin is modified (#80158) 1 year ago
Sloane Hertel 9a03c8f5b3
update paramiko_ssh unit tests load the connection with plugin loader (#80157)
* Load the paramiko_ssh connection plugin properly using plugin loader so it can use config manager when _connect() is called
1 year ago
Brian Coca a1bff416ed
paramiko connection now uses the correct source of data (#79704)
* paramiko connection now uses the correct source of data

* keep orig name

* added missing port to docs

* Update changelogs/fragments/paramiko_config.yml

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>

* test ansible_host templating

* added missing private_key_file

* removed all play_context fallbacks

* Update lib/ansible/plugins/connection/paramiko_ssh.py

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Update lib/ansible/plugins/connection/paramiko_ssh.py

Co-authored-by: Jordan Borean <jborean93@gmail.com>

---------

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
1 year ago
Matt Clay 367d45facd
ansible-test - Fix error with missing pylint home (#80155) 1 year ago
Matt Clay 27287b40c0
ansible-test - Fix pylint error with old home dir (#80151) 1 year ago
Matt Clay 7c62cdeecd
Fix MANIFEST.in includes (#80147) 1 year ago
Sviatoslav Sydorenko 326e70c02a
🧪 Switch macOS 13.2 to 12.0 in CI (#80145)
* 🧪 Switch macOS 13.2 to 12.0 in CI

The former revealed unexpected flakiness while the latter is the
previous value that was used to be stable. This is a temporary revert.

* Skip lookup_url integration test under macOS 12.0
1 year ago
Brian Coca ff3ee9c4bd
ansible-inventory prevent duplicating host entries (#80059)
this happened after implementing limits we introduced a bug that a host would be duplicated if it existed in a group's children
1 year ago
Sviatoslav Sydorenko 8fec2d2a82
Add macOS 13 to CI and ansible-test (#79508)
*  Add macOS 13.2 to `ansible-test`

* 🧪 Replace macOS 12.0 with 13.2 in the CI matrix

* Skip `lookup_url` under macOS 13.2

This is due to https://wefearchange.org/2018/11/forkmacos.rst.html
that manifests itself as follows:

    TASK [lookup_url : Test that retrieving a url works] ***************************
    objc[15394]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
    objc[15394]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in t
    he fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
    ERROR! A worker was found in a dead state

* 📝 Extend ansible-test change note w/ macOS 13.2
1 year ago
Sviatoslav Sydorenko c0e0550bce
🐛 Make integration tests compatible w/ modern Git (#80122)
* 🐛 Make integration tests compatible w/ modern Git

This patch makes use of the `init.defaultBranch` setting to unify
the test across new and old Git versions since one defaults to
`master` and the other uses `main` for the default branch.

Where possible, it uses the `HEAD` committish to avoid having to
normalize the branch name.

The change fixes the following integration tests:

  * `ansible-galaxy`

  * `ansible-galaxy-collection-scm` (recursive collection)

  * `git`

* 🐛Replace `git-symbolic-ref` with a repo template

This custom Git repository template emulates the `init.defaultBranch` setting
on Git versions below 2.28. Ref: https://superuser.com/a/1559582.
Other workarounds mentioned there, like invoking
`git symbolic-ref HEAD refs/heads/main` after each `git init` turned
out to have mysterious side effects that break the tests in surprising ways.

* 🎨 Make Git integration test non-destructive

This patch makes use of the `$HOME` environment variable to trick Git
into using a user-global config generated in the temporary directory.
1 year ago
Sam Doran a56428de11
unarchive - properly handle relative path for `dest` (#75267)
* unarchive - properly handle relative path for dest

* Add integration test

* Return output of underlying commands with increased verbosity

* Revert "Return output of underlying commands with increased verbosity"

This reverts commit a2790c8275cdc5697b65670a0beffdc74b741bf6.

* Warn when a relative destination path was provided
1 year ago
Matt Clay f47bc03599
Relocate the AZP entry point scripts (#80114)
Scripts previously under `test/utils/shippable/` are now under `.azure-pipelines/commands/` instead.
1 year ago
Sloane Hertel b981a9dfcd
add a worker queue to get updates from the main results thread (#79886)
* Create a queue per WorkerProcess to receive intra-task updates
* Update `pause` action to use the worker queue
* Deprecate ConnectionBase()._new_stdin
* Add new `Display`  convenience method `prompt_until` to manage both controller- and worker-sourced prompting without cross-fork stdin sharing, in-worker mechanism to handle request-response over new worker queue.
1 year ago
Matt Clay c2ea56e12e
Relocate and refactor ansible-test diff unit tests (#80113)
This provides the same test coverage as the previous tests, without the dependency on git.
It also includes many more specific test cases with assertions, instead of simply relying on the code to not raise an exception.
1 year ago
Matt Martz 060a27f28f
Do not use hardcoded httpbin.org in uri tests (#80101) 1 year ago
Mark Chappell f64e79be41
sanity tests - ignore pre-release versions for deprecation comparisons (#79185) 1 year ago
Matt Clay a895cae5a1
Relocate validate-modules unit tests (#80087)
This allows the validate-modules unit tests to run as part of the regular unit test suite.
1 year ago
Matt Clay 2f62db842d
ansible-test - Update base and default container (#80086) 1 year ago
Yaakov Selkowitz c028006aaf
Define all processor facts on s390x (#19755) (#79806)
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
1 year ago