* Rethread pr/70185 through the dependency resolver
Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath
Don't error while listing collections if a collection's metadata is missing keys required for building a collection.
Give an informative warning if metadata has been badly formatted.
Co-authored-by: Sam Doran <sdoran@redhat.com>
The current language pack fails to install on Ubuntu 22.04.
However, since the langauge only needs to be non-English, changing it preserves the test functionality.
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes#78214
* Write non-ascii via script instead of static file
* Use contexlib.closing to support py27 context manager
* Use executable from task, instead of shebang
* Update encoded bytes to utf16
* Don't set become_flags in become integration test.
Setting the flags in the test overrides the default value in the become plugins, which was not intentional.
* Remove hack for broken implicit tilde support.
It was only broken because the tests were overriding the default become flags for sudo.
* Simplify assertions.
`group` and `setup_cron` integration tests currently install `libuser` and `faketime`
packages respectively, from the non-default edge/testing repositories. It is no
longer necessary to do so for the former so this patch drops the repo option. As
for the latter, this patch includes a note explaining when the same could be done
for it too.
This patch also adds a note on installing `faketime` on Alpine in CI.
Co-Authored-By: Matt Clay <matt@mystile.com>
* Add setup_test_user integration target.
This integration target creates a temporary test user and removes
it when the current integration target finishes. This user is
configured with a password so it can be used with become, even
when the current user is unprivileged.
* Fix slurp integration test.
* Fix unarchive integration test.
* Fix module_utils integration test.
* Move undefined check from concat to finalize
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.
Fixes#78156
ci_complete
* Fix sanity
* ...
* sigh
* [dnf] Fix skip_broken, add test coverage
Change:
- skip_broken was set in config but not actually used in calls to
base.install()
- added a lot of test cases with specialized repo
- got rid of external (docker repo) nobest test cases since the
specialized repo works well for those too
- Slight cleanup and adding comments in dnf module
Test Plan:
- ci_complete
Tickets:
- Fixes#73072
Original-author: Rick Elrod <rick@elrod.me>
* Use a better test for checking results list
ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Relocate update-ignore.py for easier re-use.
* Add script to ease collection testing.
* Skip ignore rewrite if file does not exist.
* Add integration test for the shebang sanity test.
* Fix ansible-test-no-tty integration test.
Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.
* Clean up ansible-test-docker integration test.
Add a new handler class ZipZArchive to use unzip -Z as an alternative to zipinfo
Run 'unzip -Z' in can_handle_archive so we fall back to the next handler if it's not available (failing in is_unarchived is too late)
* Add a test for unzip -Z when zipinfo is not available
* Update test for missing binary altogether by removing /usr/bin from the PATH
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1
Test incompatibilities are removed for resolvelib >= 0.6.0
Test against the latest 0.8.x version and fix requirements
* Fix tests - use a venv for testing the range of resolvelib versions
* Update temporary hardcoded fallback for ansible-test
* Update hardcoded upperbound for sanity tests
* Make error check more flexible
checksum can also accept a checksum only file (no filename beside the checksum).
fixes#54390
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
* Add RHEL 9.0 to CI.
* Restrict network manager inspection to RHEL8
* Skip module tests when astream_name is undefined, undefine it for RHEL9 until 9.1
* Remove redundant test.
Co-authored-by: Matt Martz <matt@sivel.net>
* Add find test to assert patterns with regex
* Add test to assert that nested includes with relative path work
* Add test to assert symmetric_difference supports unhashable types
* ci_complete ci_coverage
* Add test to assert JSON values are converted to Python
* Add test to assert how unexpected module failures behave
* Add test to verify remote_is_local
* ci_complete ci_coverage
* Get real path of local tmp
* ci_complete ci_coverage
* Expanded symmetric_difference tests
* ci_complete ci_coverage
* I don't even know
* ci_complete ci_coverage
* Revert mathstuff changes
* ci_complete ci_coverage
* apt: fix virtual package install version detection
Change 4a62c4e3e4 introduced version
matching in installation.
The problem stems from
if version_installable or version:
pkg_list.append("'%s=%s'" % (name, version_installable or version))
When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.
For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with
$ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...
In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.
The tests are updated to install and remove this package.
Fixes: #76779
* Fix traceback when a supported version of resolvelib is not installed
Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).
* Add tests for unsupported resolvelib versions
* Resolve remaining import sanity test issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults
Add a toggle for action plugins to prefer the resolved module when loading module_defaults
Allow moving away from modules intercepted as actions pattern
Fixes#77059
* Add intentional test coverage for lib/ansible/plugins/inventory/__init__.py
* Add intentional (integration) test coverage for lib/ansible/module_utils/common/network.py
* Add ansible_release info test from https://github.com/ansible/ansible/pull/74673
ci_complete
ci_coverage
Co-authored-by: Rick Elrod <rick@elrod.me>
* ansible-test - Add a Ubuntu 22.04 container.
* ansible-test - Add a Fedora 36 container.
* ansible-test - Update distro containers.
* Fix dnf test on Fedora 36.
* Work around scp test issues.
* Add new templating target, test splitter
* Add test for host:port parsing in parse_address via add_host
* Test already notified listening handler
* Add test for gathering bridge network facts
* Add veth to bridge
* ci_complete ci_coverage
* make the pre-flight sniffing more robust to different failure conditions (was failing on Ubuntu 22.04 VMs)
* remove skip aliases (the test needs to function everywhere to assert that the selinux facts bits behave properly when it's N/A)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
* Expand ansible-doc to tests/filters and fix existing issues
enable filter/test docs if in single file or companion yaml
add docs for several filters/tests plugins
allow .yml companion for docs for other plugins, must be colocated
verify plugins are valid (not modules, cannot)
fix 'per collection' filtering
limit old style deprecation (_ prefix) to builtin/legacy
start move to pathlib for saner path handling
moved some funcitons, kept backwards compat shims with deprecation notice
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Prevent losing unsafe from lookups
This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.
Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar
Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.
The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.
It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.
For more examples see unit and integration tests included in this patch.
Fixes#77535
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Proper error on missing jinja2 override separator
Properly catch the case when no (or a wrong) separator is used in a
jinja2 override, and return a useful error message to the user.
* Support colons in jinja2 override value
By limiting the split to 1, any colons in the value are preserved and
passed on.
* Fall back to implicit namespace.name from the path if the metadata is invalid
* Test listing a collection with null namespace/name/version fields in its galaxy.yml
* Add option --no-fail-on-errors to return errors for ansible-doc --metadata-dump in JSON result instead of failing.
* Adjust changelog fragment.
* Add basic tests.
* Support ignoring of certificates for ansible-galaxy during SCM cloning
* Add integration tests installing a role from an untrusted repository
Test installing the role without --ignore-certs fails
Test installing the role with --ignore-certs is successful
* Add a toggle to control the number of signatures required to verify the authenticity of a collection
* Make the default number of required valid signatures 1
* Add option to make signature verification strict and fail if there are no valid signatures (e.g. "+1")
* Use a regex to validate --required-valid-signature-count
* Add a toggle to limit the gpg status codes that are considered a failure
* Update documentation and changelog
* Add unit and integration tests for the new options
* Fixes#77146
Fix using user-provided signatures when running 'ansible-galaxy collection verify ns.coll --offline'
Add a test for a user-provided signature when running ansible-galaxy collection verify with --offline
Fix displaying overall gpg failure without extra verbosity
Add a test for displaying gpg failure without verbosity
Improve documentation to be more clear that signature verification only currently applies to collections directly sourced from Galaxy servers
* apply owner/group permissions to top folder
* remove unused var unarchive30
* fix permissions for top folders if the tarball include multiple top folders
* added test for top folder ownership
* fix facts delegation loop overwrite
partial revert of change to allow facts to be present in each loop iteration
was not needed in final results as result processing alreayd had the disctiontion
and ended up breaking the assumptions in the calling code.
fixes#76676
* ansible-galaxy collection install|verify:
- Support verifying the origin of the MANIFEST.json when the Galaxy server has provided signatures.
- Allow supplemental signatures to use during verification on the CLI/requirements file.
* ansible-galaxy collection install:
- Support disabling signature verification. This silences the warning provided by ansible-galaxy if the Galaxy server provided signatures it cannot use because no keyring is configured.
- Store Galaxy server metadata alongside installed collections for provenance. This is used by 'ansible-galaxy collection verify --offline'.
* Add unit tests for method that gets signatures from a Galaxy server
* Add integration tests for user-provided signature sources
- Test CLI option combinations
- Test installing collections with valid/invalid signature sources
- Test disabling GPG verification when installing collections
- Test verifying collections with valid/invalid signature sources
* Make signature verification advisory-by-default if signatures are provided by the Galaxy server
- Make the default keyring None
- Warn if the keyring is None but the Galaxy server provided signatures
- Error if the keyring is None but the user supplied signatures
- Error if the keyring is not None but is invalid
* changelog
* add ansible-galaxy user documentation for new options
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Andy Mott <amott@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
* Return rc=0 on success.
Error handling in playbooks generally expects `rc` to be set to 0 when a module has not failed. Playbook authors should not have to check for the existence of `rc` first.
* Use single definition and added changelog
* Fix up tests with new return value
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* add DebianStrategy tests
* ensure hostname can be changed by using become
* use Systemd strat for debian and Base for generic.
* add test to ensure all strategies are available
Signed-off-by: Thomas Sjögren <konstruktoid@users.noreply.github.com>
ssh plugin, use 'correct' information source in all cases
* still fallback to pc
* added inventory to new test
* undef var can still show as parser error on pc
now task_exectuer has a more accurate error handling
* updated tests to conform to new block inheritance
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* minor refactor in other options by pushing common code into functions
* consolidate coll_filter
* more normalizing loader
* dont pass plugin_loader, its global import
* Also dump roles and collections
* adjusted tests to new err msg
* disable namespace filter (unused)
* Make include_role/include_tasks work with any_errors_fatal v2
Fixes#50897
Co-authored-by: Matt Martz <matt@sivel.net>
* Add failed to results in free strategy too
* Fix
* Avoid duplicating results
* ci_complete
Co-authored-by: Matt Martz <matt@sivel.net>
* ansible-test - Improve help for unsupported cwd.
* The `--help` option is now available when an unsupported cwd is in use.
* The `--help` output now shows the same instructions about cwd as would be shown in error messages if the cwd is unsupported.
* Add `--version` support to show the ansible-core version.
* The explanation about cwd usage has been improved to explain more clearly what is required.
Resolves https://github.com/ansible/ansible/issues/64523
Resolves https://github.com/ansible/ansible/issues/67551
* group2 - switch to setup_remote_tmp_dir instead of using output_dir
* output_file needs to be in the tmp dir
* Use comparison that should work with macos
* remove unused dep
* allow inputting 0 for password_expire_{min|max}
0 is meaningful for min days (any time)
0 is technically valid for max_days
* add test for setting both min and max expiry
* [0] return result of execute_command from set_password_expire*
* [1] better return for set_password_expire
* [2] handle returns from set_password_expire*
* only set password expiry if user exists
* collect return-handling code
* combine password min and max into one execution
* handle case where spwd is not present like on macOS and FreeBSD
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Update dnf tests to reflect new behavior.
Previously dnf would report there was nothing to do when trying to install a package from the cache when it was not present.
A recent update to dnf has changed this behavior to match yum, resulting in a failure instead.
* Allow dnf to fail or report no changes.
This adds the ability to specify a package version using >=. This will ensure the package is at the specified version or above.
* If the package is not installed, the latest version will be installed.
* If the package is installed and less than the specified version, it will be upgraded.
* If the package is installed and greater than or equal to the specified version, it will be left alone.
The version selection is handled by Apt itself, so things like the system policy, pinning, etc, are considered.
* Add additional tests for check_mode with shell/command/script
* update check_mode documentation: if a module/action is not skipped in check mode and accurately reflects whether a change is made to the remote support should be 'full'
* Make reporting skipped in check mode mutually exclusive with 'changed: True'
* Add missing documented attributes
* Fix tests to expect skipped=True and changed=True do not occur together
* Fix script check_mode support documentation
* Fix earlier changelog
* document platforms attribute
* Use tasks's check mode since the value from PlayContext does not reflect loop items
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* Ensure the version is valid for directories and collections in git repos before installing
Fix the error message for invalid semantic versions
* Make requested changes
* Add a test case for unhandled ValueError exception
* Add changelog
* Update lib/ansible/galaxy/collection/galaxy_api_proxy.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Reword error message
Include link to learn how to compose a semver version
* Move version validation into the caller, find_matches
* Add tests for more invalid version types
* Remove unused import
Fix raising unexpected error
* Update lib/ansible/galaxy/collection/__init__.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Update lib/ansible/galaxy/dependency_resolution/providers.py
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* fixes#28977
* Include the original error message when pick_handler fails
* Add a test to unarchive a tar file with an invalid extra option
* add a changelog
* Disable the _distutils_hack in newer setuptools. Doesn't fix the underlying issue of the venv finding the _distutils_hack of a setuptools that is not its own.
ci_complete
* re-throw blanket
* check if service is indirect status / add integration test
* Update changelogs/fragments/76453-indirect-systemd-status.yml
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* simplify indirect check / remove integration test restriction on rhel9
* add dummy indirect service for integration tests
* unarchive: fix non-english locales
For GNU Gettext, the LANGUAGE environment variable takes precedence over LANG or LC_ALL. On systems where LANGUAGE was set to a non-english locale, the output of the tar command therefore not understood and the module failed silently ("changed": false, but the archive was not extracted).
* add tests
* changelog
Sloane Hertel