* Only bypass type validation for null parameters if the default is None. A default is mutually exclusive with required.
* Prevent coercing None to str type. Fail the type check instead.
- Unit tests now report warnings generated during test runs.
- Python 3.12 warnings about `os.fork` usage with threads (due to `pytest-xdist`) are suppressed.
- Added integration tests to verify forked test behavior.
* Validate task attributes `run_once` and `action` with finalized attrs after individual loop results
* Validate task attribute `ignore_unreachable` using individual loop results
Once there's a way to post validate only certain fields, we can use self._task.post_validate() instead
This replaces the fix introduced in https://github.com/ansible/ansible/pull/80051.
Fixes#73643
* clear_notification method and simplify ifs
* Deduplicate code
* Limit number of Templar creations
* Fix sanity
* Preserve handler callbacks order as they were notified
* Symbolic modes with X or =[ugo] always use original mode (Fixes#80128)
Here's what's happening, by way of this mode example: u=,u=rX
At the first step in the loop, the "u" bits of are set to 0. On the next
step in the loop, the current stat of the filesystem object is used to
determine X, not the "new_mode" in the previous iteration of the loop. So
while most operations kind of operate left to right, "X" is always going
back to the original file to determine whether to set x bit.
The Linux "chmod" (the only one I've tested) doesn't operate this way. In
it, "X" operates on the current state the loop understands it is in,
based on previous operations (and starting with the file permissions).
This is an issue with "X" and any of the "=[ugo]" settings, because
they are lookups. For example, if a file is 755 and you do "ug=rx,o=u",
file module produces 0557 and chmod produces 0555.
This really becomes a problem when you want to recursively change a
directory of files, and the files are currently 755, but you want to
change the directory to 750 and the files to 640. In chmod you can do
"a=,ug=rX,u+w" (or "a=,u=rwX,g=rX"), and have it apply equally to the
directory and the files. I can't come up with a single way in the ansible
file module to deterministically, recursively, set a directory to 750
and the contents to 640 no matter what the current permissions are,
as the code currently is.
The fix is to pass in "new_mode" to _get_octal_mode_from_symbolic_perms
in lib/ansible/module_utils/basic.py inside _symbolic_mode_to_octal. And
then take "new_mode" as an argument and use it instead of the filesystem
object stat.st_mode value.
* Fixing my new unit test, fixing bug in test comments
Reduce the number of Galaxy API calls made during dependency resolution by fetching remote signatures afterwards, since these are not used in backtracking.
Reduce the verbosity to `-vvvv` (to match other Galaxy API calls) to see this activity.
Co-authored-by: Sviatoslav Sydorenko <webknjaz@redhat.com>
* Ensure we default to show all tags when listing tags
'never' was being excluded by default,as it is not part of the 'run tags'
runtime default ('all'). For listing we now add it to the default 'run tags'.
* fix using templated values for include/import role options 'public', 'allow_duplicates', and 'rolespec_validate'
* pass templated values without changing the instance
* Fix templating by setting always_post_validate to True and calling IncludeRole.post_validate() instead
ci_complete
* add changelog
* Improve Ansible.Basic.cs tempdir uniqueness
The current tempdir naming scheme can result in the same name if the
remote worker starts at the same time as another. By using the process
id it should add enough uniqueness to avoid this situation.
* Fix sanity issues
* Fix up compile issue on older hosts
* ansible-galaxy - support `resolvelib >= 0.5.3, < 1.1.0`
<https://pypi.org/project/resolvelib/1.0.1> released on 2023-03-09:
- <https://github.com/sarugaku/resolvelib/blob/main/CHANGELOG.rst#101-2023-03-09>
- <https://github.com/sarugaku/resolvelib/releases/tag/1.0.1>
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* Trigger CI by pinning resolvelib with latest version
Since resolvelib is pre-installed in our test containers, we should temporarily pin the latest version allowed to force the tests to run with that version. Once the tests have passed that commit can be reverted.
Please make those changes without force pushing, so that we keep the reference to the passing CI run. We can squash the commits when merging the PR so the temporary commits won't be in the final commit merged to the devel branch.
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* https://github.com/ansible/ansible/pull/80196#discussion_r1136003637
Also test resolvelib with multiple supported versions.
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* Revert "Trigger CI by pinning resolvelib with latest version"
This reverts commit 5518e5dbca.
---------
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
* clog frag
* Fix retries so that each explicit call to _call_galaxy is retried for the correct number of attempts. Fixes#80174
* Extend retry logic to common URL related connection errors. Fixes#80170
* Extend retries to downloading artifacts
* Extend param docs for change
* Rework the exception handling
* Don't be overly broad, reduce to TimeoutError, and BadStatusLine for now
* _download_file needs to raise AnsibleError.orig_exc
* Remove unused import
* Add IncompleteRead
* Add socket.timeout for py39
* Add 502 to retry codes
* Move http error code checking first
* Use itertools.tee to replay the backoff_iterator instead of using a callable
* Actually set a CLI default of 60s for timeout, to prevent implicit galaxy from using 10s as default from Request.open
* Import typing
* fix type hints
* Use http.HTTPStatus instead of int HTTP error codes where feasible
* Split exception handling
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add missing import
---------
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* ✨ Add macOS 13.2 to `ansible-test`
* 🧪 Replace macOS 12.0 with 13.2 in the CI matrix
* Skip `lookup_url` under macOS 13.2
This is due to https://wefearchange.org/2018/11/forkmacos.rst.html
that manifests itself as follows:
TASK [lookup_url : Test that retrieving a url works] ***************************
objc[15394]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[15394]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in t
he fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
ERROR! A worker was found in a dead state
* 📝 Extend ansible-test change note w/ macOS 13.2
* unarchive - properly handle relative path for dest
* Add integration test
* Return output of underlying commands with increased verbosity
* Revert "Return output of underlying commands with increased verbosity"
This reverts commit a2790c8275cdc5697b65670a0beffdc74b741bf6.
* Warn when a relative destination path was provided
* Create a queue per WorkerProcess to receive intra-task updates
* Update `pause` action to use the worker queue
* Deprecate ConnectionBase()._new_stdin
* Add new `Display` convenience method `prompt_until` to manage both controller- and worker-sourced prompting without cross-fork stdin sharing, in-worker mechanism to handle request-response over new worker queue.
The previous change to the default mode of operation for the plugin is now limited to collections.
This enables easier testing of the plugin in core, so the previously removed tests have been restored.
* Add condition that causes a when to skip a task
* Fix up tests
* Use false_condition instead of failed_condition
* Remove formatting accidentially added
* Fix sanity
* ansible-inventory, add --limit option
* also graph
* optimize the when not providing limit
* added tests and clog
* avoid empty groups, fix tests
* i swear there as an ignore_errors there already!!?!?!?
* Fix stdout test fallback
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* simpler approach using subset against inventory directly
* tyupose
* renamed funciton to what it actuall does
got yaml in line with others about removing empty groups
* have graph ignore limts, also note same for --host
* fixed typo long line
* better test
* cause hosvars are not a thing
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Remove unneeded fallback code for nested defined/undefined tests
The code used to handle defined/undefined tests on objects containing
nested undefined variables. Due to changes in lazy evalution of Jinja2
expressions it is no longer needed, see #56116.
* Change ansible_job_id format
...to be something that does not resemble float or other type which
value could be changed by literal_eval that jinja2_native uses.
Specifically the format of '%d.%d' is converted from str to float and
then back to float which may result in truncating the number resulting
in the job not being found because the job id does not exist.
* Fix detection of available hashlib algorithms
Detection of hashlib algorithms now works on Python 3.x.
The new implementation works on Python 2.7 and later.
Test coverage is provided by both integration and unit tests.
* Add additional details about hashlib in docs
* ansible-test - Fix file permissions for delegation
* Set more restrictive permissions for SSH key
* Check all execute bits, not just owner
* Add a breaking_changes changelog entry
* Improving the documentation on how we generate the default value of the filename parameter
* fix pep8
* removing unnecessary documentation and improving the module's return
* making the RETURN docs
* pep8
* version_added and changelog
* module._diff
* module._diff fix
* add rudimentary tests for new outputs
---------
Co-authored-by: Matt Davis <mrd@redhat.com>
* Update `collections.abc` imports
- Use `six.moves` for modules and module_utils
- Use `collections.abc` for controller code
This avoids using `ansible.module_utils.common._collections_compat`,
which was added before the vendored `six` was updated to provide these
imports.
* Update _collections_compat to use six.moves
Also update the custom pylint rule to reflect this change.
* add null 'manifest' key to metadata for git repo collections containing MANIFEST.json
changelog
* set to Sentinel instead of None
* Test installing a collection in a git repo that contains a MANIFEST.json
* fix test
* Update changelogs/fragments/ansible-galaxy-install-git-src-manifest.yml
* validate-modules - Remove `__future__` limits
Limits on specific `__future__` imports are handled by other sanity tests.
* Add integration test for module/plugin imports.
* Normalize deprecation records.
* Fix alias deprecations in suboptions.
* Report in which option an alias warning happened for suboptions.
* Add deprecation tests for suboptions.
* Also test deprecation in list of dicts.
* Adjust unit tests for toplevel alias deprecation field name change.
* Remove obsolete 'if'
The result dict returned by super.run() will never have a 'skipped' entry.
Without the if, it is more clear that wrap_async is actually defined at the end of the function.
* Remove more dead code.
The result dict will also not contain invocation.
* Remove notes.
Fix ignoring certs when downloading tarballs
Fix ignoring certs when downloading a collection from a specific source that isn't in the configured servers list
* validate-modules: don't fail on invalid YAML
When validate-modules encounters invalid YAML (e.g. in the EXAMPLES
section), it tries to reformat the exception to include the line number
in the Python file instead of the line number of the embedded YAML
document. However, PyYAML doesn't allow modification of the Mark object
(anymore) which leads to a new exception being raised, instead of
reporting the original exception.
As the original exception is not needed in other places anymore, we
don't have to modify it at all and can just compute the right line
number when reporting the error via ansible-test.
Fixes: #75837
* Add test for invalid module doc YAML syntax.
Co-authored-by: Matt Clay <matt@mystile.com>
Also improve the ansible-test-container integration test:
- Add coverage for the no-probe code path.
- Add work-arounds for centos6 containers (to support backporting).
- Avoid systemd debug when the container doesn't use cgroup.
* background threads writing to stdout/stderr can cause children to deadlock if a thread in the parent holds the internal lock on the BufferedWriter wrapper
* prevent writes to std handles during fork by monkeypatching stdout/stderr during display startup to require a mutex lock with fork(); this ensures no background threads can hold the lock during a fork operation
* add integration test that fails reliably on Linux without this fix
Martin Krizek