* Normalize deprecation records.
* Fix alias deprecations in suboptions.
* Report in which option an alias warning happened for suboptions.
* Add deprecation tests for suboptions.
* Also test deprecation in list of dicts.
* Adjust unit tests for toplevel alias deprecation field name change.
(cherry picked from commit 8a7185c224)
* Ensure we are passing ciphers to all url_get calls. Fixes#79717
* Add clog frag
* Fix tests
(cherry picked from commit 2143bcd)
Co-authored-by: Matt Martz <matt@sivel.net>
* validate-modules: don't fail on invalid YAML
When validate-modules encounters invalid YAML (e.g. in the EXAMPLES
section), it tries to reformat the exception to include the line number
in the Python file instead of the line number of the embedded YAML
document. However, PyYAML doesn't allow modification of the Mark object
(anymore) which leads to a new exception being raised, instead of
reporting the original exception.
As the original exception is not needed in other places anymore, we
don't have to modify it at all and can just compute the right line
number when reporting the error via ansible-test.
Fixes: #75837
* Add test for invalid module doc YAML syntax.
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit a7111c4dbb)
Co-authored-by: Evgeni Golov <evgeni@golov.de>
Also improve the ansible-test-container integration test:
- Add coverage for the no-probe code path.
- Add work-arounds for centos6 containers (to support backporting).
- Avoid systemd debug when the container doesn't use cgroup.
(cherry picked from commit 04fc98c794)
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix file touch check mode result (#79360) (#79422)
Fixes#79360
(cherry picked from commit 3936b5c471)
* Rename changelog fragment according to PR number
* Enable the `reboot` integration test in CI
Co-Authored-By: Matt Clay <matt@mystile.com>
(cherry picked from commit bb7ad0f0d8)
* [CI] Move running `reboot` integration test to group 2
PR #79289, follow-up for #78402.
It was breaking the integration-aliases sanity test, but only
became apparent post-merge. This patch fixes that.
(cherry picked from commit 26a4775611)
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* background threads writing to stdout/stderr can cause children to deadlock if a thread in the parent holds the internal lock on the BufferedWriter wrapper
* prevent writes to std handles during fork by monkeypatching stdout/stderr during display startup to require a mutex lock with fork(); this ensures no background threads can hold the lock during a fork operation
* add integration test that fails reliably on Linux without this fix
(cherry picked from commit 1424484be0)
This should prevent color codes from interfering with string matches.
(cherry picked from commit 31f9d60b8d)
Co-authored-by: Matt Clay <matt@mystile.com>
* ansible-galaxy install - fix unnecessary api check when installing a role from git repo (#79090)
* delay server api evaluation until a GalaxyRole needs to make an api call for info, list, and install
(cherry picked from commit cb2e434dd2)
* Fix isinstance check (#79159)
Use GalaxyAPI for isinstance check instead of RoleDistributionServer, since the latter is defined in __main__ sometimes (when running integration tests or ansible-galaxy from source) and importing from ansible.cli.galaxy won't reference the same object.
(cherry picked from commit 89d682464b)
* Log `runme.sh` execution in integration tests (#79263)
* Log `runme.sh` execution in integration tests
This patch adds `set -x` where it's missing in the integration tests.
It also enables `pipefail` in `runme.sh` scripts that use pipes.
* Add a change note for PR #79263
(cherry picked from commit 6674c43edd)
* Delete a bad PR #79263 change note
(cherry picked from commit 9cfb3f73e8)
The latest versions of `git` include a fix for CVE-2022-39253. The fix blocks the file protocol by default.
See: 45c9f05c44/Documentation/RelNotes/2.30.6.txt
(cherry picked from commit 4202acb41b)
Co-authored-by: Matt Clay <matt@mystile.com>
* [stable-2.14] Ensure that we do not squash keywords in validate (#79049)
* Ensure that we do not squash keywords in validate. Fixes#79021
* become_user: nobody should only apply to the test tasks, not the setup_test_user role
* Update how become_user is specified
* Add test to ensure keyword inheritance is working for become
* Add clog frag
* Cache fattributes to prevent re-calculation
* ci_complete
* Remove unnecessary getattr.
(cherry picked from commit 420564c5bc)
Co-authored-by: Matt Martz <matt@sivel.net>
* Remove unneeded merge conflict changes
* omit keyword should reset to context (#78917)
* omit keyword should reset to context
ensure we use context/inheritance when calculating value,
using default only when context is unavailable.
fixes#75692
(cherry picked from commit 9650ddb11c)
* fixes to FA inheritance (#78990)
finalized applies to all field attributes
fix getting parent value
also remove unused/needed extend/prepend signature
moar testing
(cherry picked from commit ff6e4da36a)
* setup role needs it's own info
* Do not crash templating when filter/test name is not a valid Ansible plugin name.
* Store and re-raise KeyError if there was one.
Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
(cherry picked from commit 6d0aeac1e1)
* Fix known_hosts changed status when removing non-existing key (#78748)
* Ensure no change is reported when removing missing key for host with other keys
Co-authored-by: Matt Martz <matt@sivel.net>
* Rename changelog fragment for fix known hosts changed status cherry pick
* Rename changelog according to new PR number
Co-authored-by: Matt Martz <matt@sivel.net>
- Allow disabled, unsupported, unstable and destructive integration test targets to be selected using their respective prefixes.
- Allow unstable tests to run when targeted changes are made and the ``--allow-unstable-changed`` option is specified (resolves https://github.com/ansible/ansible/issues/74213).
(cherry picked from commit d3d7785472)
Co-authored-by: Matt Clay <matt@mystile.com>
* Add --offline option to 'ansible-galaxy collection install' to prevent querying distribution servers
This allows installing/upgrading individual tarfiles to have dependency resolution.
Previously needed to be done manually with --no-deps or else all collections and dependencies needed to be included in the requirements.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* new _fqcn attribute to plugin objects
* unbreak plugins in subdirs
* Fix inadventent changes to _load_name and use existing vars
* add plugin aliases and name property, and replace plugin._load_name where incorrect
* Fix listing plugin names
Fix listing legacy and builtin together
test deprecated plugin documentation
fix doc extensions
remove sometimes inaccurate _load_name handling from plugin.name
* Add tests for REJECT_EXTS and doc extensions
Fix unpredictable collection redirects so non-fqcns in the redirect list are guaranteed to be legacy (instead of determined by the collections keyword)
Move aliases and name properties to _update_object so all plugin types, including doc fragments, can use them
* make legacy plugin names internally consistent
* rename attributes to ansible_name and ansible_aliases
* Recover unix load average into an ansible_loadavg fact
The average load is available out of the box with os.getloadavg().
It adds useful context to the existing ansible_memory_mb and ansible_mounts
facts that provide size and utilization or memory and disk space.
* Add an integration test
* Limit Galaxy API calls during ansible-galaxy collection dependency resolution when possible
Installing a tarfile with a dependency from a Galaxy server (e.g. dependencies: {'ns.coll': '>=1.0.0'}) does not get the available versions of the dependency from the galaxy server if a sufficient version is already installed.
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* refactor and remove redundant code in documentation
allow location and building api to be more accessible
fix issues with displaying ansible.legacy and ansible.builtin
ensure we don't x2 process tokens (some modules reference them also) fixes#77764
move to constants vs hardcoded
more informative errors and comments
now have actual filter/test plugins, which expose the filter/test functions
moved filter/test loading/finding logic into jinja2pluginloader, removed dupe implementations
added tests for case in which we unique by basename when listing
Update lib/ansible/utils/plugin_docs.py
Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
* Add a warning for collections that are attempting to be autoloaded to no effect
* Deprecate REQUIRES_WHITELIST and add support for REQUIRES_ENABLED so the docs are accurate
* Fix documentation
* add more vars plugin tests
* Simplify code and add a FIXME for another bug
* fix precedence
* Make setting the class attr at all a warning, even if it's True
* Add fun parsing for _load_name
* include _load_name in messages
This specific integration test gets stuck periodically causing the
Galaxy jobs to be killed on timeout wasting an hour of runtime. The
module that gets stuck waiting on Pulp is an in-test one, called
`setup_collections`. When it works, the task is complete in around 70
seconds but when it doesn't, it just freezes the whole play.
This patch attempts to make it fail faster by putting a reasonable
timeout value of 2 minutes.
On BusyBox systems such as Alpine, chattr on a tmpfs fails with a status of 0 and output only on stderr.
This change updates the test to not assume output on stdout.
* add required file to collection skeleton
This file is required to be able to upload a collection.
It is present in
https://github.com/ansible-collections/collection_template/blob/main/meta/runtime.yml
but that does not get used by default.
Without this, if you use the "ansible-galaxy collection init" command
and you try and publish that collection without adding this file, you
get the error:
"ERROR! Galaxy import process failed: 'requires_ansible' in
meta/runtime.yml is mandatory, but no meta/runtime.yml found (Code:
UNKNOWN)"
Also updates relevant test and adds a changelog fragment
* Relocate venv-pythons.py script.
* Split out unsupported-directory test.
* Split out sanity-import test.
* Split out sanity-validate-modules test.
* Split out units test.
* Split out integration test.
* Split out units-constraints test.
* Split out integration-constraints test.
* Split out coverage test.
* Split out sanity test.
* Split out git test.
* Update test groups.
* Don't actually update the cache in check mode
Add tests for updating the cache in check mode
* Don't mark packages as manually installed in check mode and add a test for it
* changelog
* Update test/integration/targets/apt/tasks/apt.yml
* fix test
* update attributes of files that are symlinks
* update attributes of files that are hard links
* fix default state in documentation
* remove unnecessary suppression
* add to changelog
* `context/target` tests must be in groups 1 - 2.
* `context/controller` tests must be in groups 3 - 5.
This makes it easier to efficiently organize groups and balance test runtimes.
* Rethread pr/70185 through the dependency resolver
Hang optional metadata toggle on the ConcreteArtifactsManager instead of threading it through whole list codepath
Don't error while listing collections if a collection's metadata is missing keys required for building a collection.
Give an informative warning if metadata has been badly formatted.
Co-authored-by: Sam Doran <sdoran@redhat.com>
The current language pack fails to install on Ubuntu 22.04.
However, since the langauge only needs to be non-English, changing it preserves the test functionality.
* wait_for - Read file and perform comparisons using bytes to avoid decode errors. Fixes#78214
* Write non-ascii via script instead of static file
* Use contexlib.closing to support py27 context manager
* Use executable from task, instead of shebang
* Update encoded bytes to utf16
* Don't set become_flags in become integration test.
Setting the flags in the test overrides the default value in the become plugins, which was not intentional.
* Remove hack for broken implicit tilde support.
It was only broken because the tests were overriding the default become flags for sudo.
* Simplify assertions.
`group` and `setup_cron` integration tests currently install `libuser` and `faketime`
packages respectively, from the non-default edge/testing repositories. It is no
longer necessary to do so for the former so this patch drops the repo option. As
for the latter, this patch includes a note explaining when the same could be done
for it too.
This patch also adds a note on installing `faketime` on Alpine in CI.
Co-Authored-By: Matt Clay <matt@mystile.com>
* Add setup_test_user integration target.
This integration target creates a temporary test user and removes
it when the current integration target finishes. This user is
configured with a password so it can be used with become, even
when the current user is unprivileged.
* Fix slurp integration test.
* Fix unarchive integration test.
* Fix module_utils integration test.
* Move undefined check from concat to finalize
In the classic Jinja2's Environment str() is called on the return value of the
finalize method to potentially trigger the undefined error. That is not
the case in NativeEnvironment where string conversion of the return value is
not desired. We workaround that by checking for Undefined in all of our concat
functions. It seems simpler to do it earlier in the finalize method(s) instead.
As a side-effect it fixes an undefined variable detection in imported templates.
Fixes#78156
ci_complete
* Fix sanity
* ...
* sigh
* [dnf] Fix skip_broken, add test coverage
Change:
- skip_broken was set in config but not actually used in calls to
base.install()
- added a lot of test cases with specialized repo
- got rid of external (docker repo) nobest test cases since the
specialized repo works well for those too
- Slight cleanup and adding comments in dnf module
Test Plan:
- ci_complete
Tickets:
- Fixes#73072
Original-author: Rick Elrod <rick@elrod.me>
* Use a better test for checking results list
ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Relocate update-ignore.py for easier re-use.
* Add script to ease collection testing.
* Skip ignore rewrite if file does not exist.
* Add integration test for the shebang sanity test.
* Fix ansible-test-no-tty integration test.
Previously the test only verified a TTY was not used if a TTY already existed.
This prevented the test from verifying behavior when run in CI.
Now the test creates a PTY before invoking ansible-test.
* Clean up ansible-test-docker integration test.
Felix Fontein