* refactor and remove redundant code in documentation
allow location and building api to be more accessible
fix issues with displaying ansible.legacy and ansible.builtin
ensure we don't x2 process tokens (some modules reference them also) fixes#77764
move to constants vs hardcoded
more informative errors and comments
now have actual filter/test plugins, which expose the filter/test functions
moved filter/test loading/finding logic into jinja2pluginloader, removed dupe implementations
added tests for case in which we unique by basename when listing
Update lib/ansible/utils/plugin_docs.py
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Add a warning for collections that are attempting to be autoloaded to no effect
* Deprecate REQUIRES_WHITELIST and add support for REQUIRES_ENABLED so the docs are accurate
* Fix documentation
* add more vars plugin tests
* Simplify code and add a FIXME for another bug
* fix precedence
* Make setting the class attr at all a warning, even if it's True
* Add fun parsing for _load_name
* include _load_name in messages
This specific integration test gets stuck periodically causing the
Galaxy jobs to be killed on timeout wasting an hour of runtime. The
module that gets stuck waiting on Pulp is an in-test one, called
`setup_collections`. When it works, the task is complete in around 70
seconds but when it doesn't, it just freezes the whole play.
This patch attempts to make it fail faster by putting a reasonable
timeout value of 2 minutes.
On BusyBox systems such as Alpine, chattr on a tmpfs fails with a status of 0 and output only on stderr.
This change updates the test to not assume output on stdout.
* add required file to collection skeleton
This file is required to be able to upload a collection.
It is present in
https://github.com/ansible-collections/collection_template/blob/main/meta/runtime.yml
but that does not get used by default.
Without this, if you use the "ansible-galaxy collection init" command
and you try and publish that collection without adding this file, you
get the error:
"ERROR! Galaxy import process failed: 'requires_ansible' in
meta/runtime.yml is mandatory, but no meta/runtime.yml found (Code:
UNKNOWN)"
Also updates relevant test and adds a changelog fragment
* Relocate venv-pythons.py script.
* Split out unsupported-directory test.
* Split out sanity-import test.
* Split out sanity-validate-modules test.
* Split out units test.
* Split out integration test.
* Split out units-constraints test.
* Split out integration-constraints test.
* Split out coverage test.
* Split out sanity test.
* Split out git test.
* Update test groups.
* Don't actually update the cache in check mode
Add tests for updating the cache in check mode
* Don't mark packages as manually installed in check mode and add a test for it
* changelog
* Update test/integration/targets/apt/tasks/apt.yml
* fix test
* update attributes of files that are symlinks
* update attributes of files that are hard links
* fix default state in documentation
* remove unnecessary suppression
* add to changelog
* `context/target` tests must be in groups 1 - 2.
* `context/controller` tests must be in groups 3 - 5.
This makes it easier to efficiently organize groups and balance test runtimes.
* Rethread pr/70185 through the dependency resolver
Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath
Don't error while listing collections if a collection's metadata is missing keys required for building a collection.
Give an informative warning if metadata has been badly formatted.
Co-authored-by: Sam Doran <sdoran@redhat.com>
The current language pack fails to install on Ubuntu 22.04.
However, since the langauge only needs to be non-English, changing it preserves the test functionality.
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes#78214
* Write non-ascii via script instead of static file
* Use contexlib.closing to support py27 context manager
* Use executable from task, instead of shebang
* Update encoded bytes to utf16
* Don't set become_flags in become integration test.
Setting the flags in the test overrides the default value in the become plugins, which was not intentional.
* Remove hack for broken implicit tilde support.
It was only broken because the tests were overriding the default become flags for sudo.
* Simplify assertions.
`group` and `setup_cron` integration tests currently install `libuser` and `faketime`
packages respectively, from the non-default edge/testing repositories. It is no
longer necessary to do so for the former so this patch drops the repo option. As
for the latter, this patch includes a note explaining when the same could be done
for it too.
This patch also adds a note on installing `faketime` on Alpine in CI.
Co-Authored-By: Matt Clay <matt@mystile.com>
* Add setup_test_user integration target.
This integration target creates a temporary test user and removes
it when the current integration target finishes. This user is
configured with a password so it can be used with become, even
when the current user is unprivileged.
* Fix slurp integration test.
* Fix unarchive integration test.
* Fix module_utils integration test.
* Move undefined check from concat to finalize
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.
Fixes#78156
ci_complete
* Fix sanity
* ...
* sigh
* [dnf] Fix skip_broken, add test coverage
Change:
- skip_broken was set in config but not actually used in calls to
base.install()
- added a lot of test cases with specialized repo
- got rid of external (docker repo) nobest test cases since the
specialized repo works well for those too
- Slight cleanup and adding comments in dnf module
Test Plan:
- ci_complete
Tickets:
- Fixes#73072
Original-author: Rick Elrod <rick@elrod.me>
* Use a better test for checking results list
ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Relocate update-ignore.py for easier re-use.
* Add script to ease collection testing.
* Skip ignore rewrite if file does not exist.
* Add integration test for the shebang sanity test.
* Fix ansible-test-no-tty integration test.
Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.
* Clean up ansible-test-docker integration test.
Add a new handler class ZipZArchive to use unzip -Z as an alternative to zipinfo
Run 'unzip -Z' in can_handle_archive so we fall back to the next handler if it's not available (failing in is_unarchived is too late)
* Add a test for unzip -Z when zipinfo is not available
* Update test for missing binary altogether by removing /usr/bin from the PATH
* ansible-galaxy - support resolvelib versions >= 0.5.3, <= 0.8.1
Test incompatibilities are removed for resolvelib >= 0.6.0
Test against the latest 0.8.x version and fix requirements
* Fix tests - use a venv for testing the range of resolvelib versions
* Update temporary hardcoded fallback for ansible-test
* Update hardcoded upperbound for sanity tests
* Make error check more flexible
checksum can also accept a checksum only file (no filename beside the checksum).
fixes#54390
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
* Add RHEL 9.0 to CI.
* Restrict network manager inspection to RHEL8
* Skip module tests when astream_name is undefined, undefine it for RHEL9 until 9.1
* Remove redundant test.
Co-authored-by: Matt Martz <matt@sivel.net>
* Add find test to assert patterns with regex
* Add test to assert that nested includes with relative path work
* Add test to assert symmetric_difference supports unhashable types
* ci_complete ci_coverage
* Add test to assert JSON values are converted to Python
* Add test to assert how unexpected module failures behave
* Add test to verify remote_is_local
* ci_complete ci_coverage
* Get real path of local tmp
* ci_complete ci_coverage
* Expanded symmetric_difference tests
* ci_complete ci_coverage
* I don't even know
* ci_complete ci_coverage
* Revert mathstuff changes
* ci_complete ci_coverage
* apt: fix virtual package install version detection
Change 4a62c4e3e4 introduced version
matching in installation.
The problem stems from
if version_installable or version:
pkg_list.append("'%s=%s'" % (name, version_installable or version))
When the package is a virtual-package, package_status() is returning
the "version_installable" of the package *satisfying* the
virtual-package; but then this is trying to install the
virtual-package with this version pin.
For example, "yaml-mode" is a virtual package satisifed by
"elpa-yaml-mode" (currently 0.0.14-1) and trying to install it fails
with
$ usr/bin/apt-get -y ... install 'yaml-mode=0.0.14-1'
... failed: E: Version '0.0.14-1' for 'yaml-mode' was not found ...
In the case of a virtual-package with nothing installed to satisfy it,
we should just return blank values to allow apt-get to do it's thing.
The tests are updated to install and remove this package.
Fixes: #76779
* Fix traceback when a supported version of resolvelib is not installed
Try to read the supported version range from the package distribution info and fall back to a hardcoded lowerbound/upperbound (>=0.5.3,<0.6.0).
* Add tests for unsupported resolvelib versions
* Resolve remaining import sanity test issues.
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Martz <matt@sivel.net>
* If there is a platform specific handler, prefer the resolved module over the resolved action when loading module_defaults
Add a toggle for action plugins to prefer the resolved module when loading module_defaults
Allow moving away from modules intercepted as actions pattern
Fixes#77059
* Add intentional test coverage for lib/ansible/plugins/inventory/__init__.py
* Add intentional (integration) test coverage for lib/ansible/module_utils/common/network.py
* Add ansible_release info test from https://github.com/ansible/ansible/pull/74673
ci_complete
ci_coverage
Co-authored-by: Rick Elrod <rick@elrod.me>
* ansible-test - Add a Ubuntu 22.04 container.
* ansible-test - Add a Fedora 36 container.
* ansible-test - Update distro containers.
* Fix dnf test on Fedora 36.
* Work around scp test issues.
* Add new templating target, test splitter
* Add test for host:port parsing in parse_address via add_host
* Test already notified listening handler
* Add test for gathering bridge network facts
* Add veth to bridge
* ci_complete ci_coverage
* make the pre-flight sniffing more robust to different failure conditions (was failing on Ubuntu 22.04 VMs)
* remove skip aliases (the test needs to function everywhere to assert that the selinux facts bits behave properly when it's N/A)
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
* Use a Python subprocess instead of a shell.
* Use InternalError instead of Exception.
* Require capture argument.
* Check for invalid raw_command arguments.
* Removed pointless communicate=True usage.
* Relocate stdout w/o capture check.
* Use threads instead of a subprocess for IO.
* Expand ansible-doc to tests/filters and fix existing issues
enable filter/test docs if in single file or companion yaml
add docs for several filters/tests plugins
allow .yml companion for docs for other plugins, must be colocated
verify plugins are valid (not modules, cannot)
fix 'per collection' filtering
limit old style deprecation (_ prefix) to builtin/legacy
start move to pathlib for saner path handling
moved some funcitons, kept backwards compat shims with deprecation notice
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Prevent losing unsafe from lookups
This patch fixes a bug which under certain conditions results in data
returned from lookups not being marked as unsafe.
Each time Templar.do_template is invoked a new AnsibleContext is
created and stored effectively at two places:
1) as an instance variable in templar_obj.cur_context
2) as a local variable called new_context in do_template method of Templar
Due to custom functionality in Ansible's Context that allows for nested
templating it is possible that during resolving variable's value
template/do_template method is called recursively again, again creating
a new context. At that point the problem manifests itself because as
mentioned in 1) above the context is overwriten on the templar object
which means that any subsequent calls to _lookup will use the new
context to mark it as unsafe which is now different to the local
new_context which is used for testing for unsafe property.
The solution to the problem appears to be to restore the original
context inside do_template and also to eliminate the local variable
new_context to prevent problems in the future.
It appears that we don't have a better way of storing the context other
than as some form of global variable and so this appears to be the
"best" solution possible at this point. Hopefully data tagging will be
the solution here.
For more examples see unit and integration tests included in this patch.
Fixes#77535
* Run code-smell sanity tests in UTF-8 Mode.
* Update subprocess use in sanity test programs.
* Use raw_command instead of run_command with always=True set.
* Add more capture=True usage.
* Don't expose stdin to subprocesses.
* Capture more output. Warn on retry.
* Add more captures.
* Capture coverage cli output.
* Capture windows and network host checks.
* Be explicit about interactive usage.
* Use a shell for non-captured, non-interactive subprocesses.
* Add integration test to assert no TTY.
* Add unit test to assert no TTY.
* Require blocking stdin/stdout/stderr.
* Use subprocess.run in ansible-core sanity tests.
* Remove unused arg.
* Be explicit with subprocess.run check=False.
* Add changelog.
Brian Coca