* Disable too-many-ancestors pylint check.
This check was only enabled for ansible-test code.
In some cases this check counts 1 more ancestor in python 3.7+ than under python 3.6, making the check inconsistent and unsuitable for use in ansible-test.
* Disable additional pylint checks:
- import-error
- no-name-in-module
These checks were only enabled for ansible-test code.
These checks provide inconsistent results since they are dependent on available imports, which vary between environments.
It may be practical to enable these checks in the future if changes are made to ensure a consistent test environment for pylint.
* Test galaxy cli against pulp
* linting fix
* Renames and small fixes
* Better handling for resetting pulp
* Clean up some things, add a comment
* I can't spell
* Bump fallaxy, use alternate pulp image
* Only reset pulp when we're are executing against pulp
* Update for updated pulp container
* Update some comments with correct URLs and typos
* Linting fix
* Pin pulp-fedora31 to a digest
* Address review comments for documentation
The current author line wants to match a github author id. But
some people, including the OpenStack project, do not use github,
and additionally do not claim individual ownership but instead
group ownership.
Since there are already a couple of hard-coded examples in the
regex, just add one more. Alternately we could come up with some
mechanism to indicate that the author is purposely not listing
a github id, but that seems a bit heavywight.
* try to load unqualified plugins from whitelist
* necessary for backcompat loading of unqualified collectionized callback plugins redirected from <= 2.9 core
* also added de-duping from actual loaded name
* add tests
* add warning test
* group test script entries by topic
* shorten warning text grep because wrapping is dumb
* fix adhoc callback loading behavior
* collections pass over whitelist wasn't respecting `_run_additional_callbacks`
* adds regression tests for same
* avoid `grep -L` in tests since it breaks the world
The upcoming pyparsing 3 release will require Python 3.5 or later, see:
https://github.com/pypa/packaging/issues/313
Unfortunately pip 8.x and earlier versions do not support python version requirements, which is why this constraint is needed.
* builtin downstream vendoring support
* allows downstream packagers to install packages to `ansible/_vendor` that will automatically be added to head of sys.path during `ansible` package load
* tests
* sort conflicting package names in warning text
* sanity fixes
* skip unnecessary comparison
If we fail to find a member when extracting a directory, try adding a trailing
slash to the member name. In certain cases, the member in the tarfile will
contain a trailing slash but the file name in FILES.json will never contain
the trailing slash.
If unable to find the member, handle the KeyError and print a nicer error.
Also check if a directory exists before creating it since it may have been
extracted from the archive.
Fixes#70009
* Add unit tests
* Use loop for trying to get members
* Validate ansible-base & collection's runtime.yml
Add new test `runtime-metadata`
* Schema validation of file
* Error if a a legacy meta/routing.yml exist in a collection
* removal_date OR removal_version
* Add tombstone validation.
* Allow both ISO 8601 date strings and datetime.date objects (from YAML dates).
* Address review comments.
* Add metadata to test collection.
* Add requirements file.
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Matt Clay <matt@mystile.com>
* Add mccabe complexity testing
* Make mccabe complexity an optional error
* Add mccabe to new sanity pylint requirements
* Add a changelog fragment.
Co-authored-by: Matt Clay <matt@mystile.com>
When using 'remote_src: yes' and 'mode: preserve', the code handling
the file modes has to be handled on the remote node because it's
the one that has access to the source files. This means that the
copy module itself must handle this, rather than the copy action
plugin (which is where all that logic exists). The copy module
handles this when we copy a single file over. But when it is a
directory as the src parameter value, the mode of the files
beneath it are not considered. Subdirectories are copied with
shutil.copytree() which will preserve permissions automatically.
Individual files are copied with shutil.copyfile() which does NOT
preserve permissions. We need to add some calls to shutil.copymode()
to correct that.
Note: This *always* retains individial file permissions. Specifying
a 'mode' other than 'preserve' when giving a source directory for
the 'src' param does not make sense so will be ignored in that case
only.
Fixes#69783
* Add changelog and test
* config: singular ANSIBLE_COLLECTIONS_PATH
Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories. Let's be consistent both internally and with POSIX
tradition.
* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular
* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
* Tag return value docs if they are a dict (and not str/None).
* Try to parse return docs as YAML.
* Properly dump return values in ansible-doc.
* Adjust plugin formatter.
* Add changelog fragment.
* Don't add 'default' for return values.
* Fix plugin_formatter.
* Only try to parse return docs if they are still a string.
* Add tests.
* Warn if RETURN cannot be parsed.
* Adjust tests. Also test for warning.
* if -> elif (otherwise EXAMPLE will be parsed too).
* Always parse return documentation, and fail if it is invalid YAML.
* Polishing.
* Mostly re-enable ansible-doc tests.
Listing from the local collection seems to be somewhat broken. I assume this
is why the test was disabled.
* Lint and make tests work with Python 2.
* Keep FQCNs in plugins (not modules), i.e. restore previous state.
* Support removed_at_date in ansible-doc
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
Changes:
* ansible-doc does not support `removed_at_date` and assumes that
deprecated dict will either have `removed_in` or `version`. This
results in ansible-doc (and hence "sanity --test=ansible-doc")
failing for modules having only `removed_at_date`.
* This patch adds support for `removed_at_date` and also gives it
precedence over `removed_in` or `version`.
* Add tests and changelog
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
Ensure the vcenter provider initialize the `module_defaults` of all
the vmware modules, not just `vmware_guest`.
The VMware CI relies on this for the authentication of the different
VMware modules.
The commit adjust `incidental_vmware_prepare_tests/tasks/init_vcsim.yml`.
The test-suite uses a copy of `vmware_guest` that is not in the
`group/vmware` group. As a result, we need to manually pass the
authentification parameter.
Some code-smell sanity tests for ansible-base use subprocess to invoke ansible commands.
Intercept these commands to make sure the correct script and python version are used.
* ansible-galaxy - Fix role info when role is not installed
Only report the role not found if in offline mode, otherwise query the galaxy API
to get role information.
Fixes#69867
* Improve error message when role is not found in Ansible Galaxy
* Extra whitespace around imported playbook filename are stripped.
* Corrected call for display.deprecated
* Changed warning type & added test
* Added auto verification of raised warning
* More accurate warning message
* Move ansible-test completion code.
* Fix a few type hints.
* Change docker completion based on context.
Collections now use version 2.0 of the default-test-container.
This is an updated version of the earlier 1.x default-test-container with ansible-base and cloud specific requirements removed.
Testing of ansible-base now uses version 1.0 of a new ansible-base-test-container.
This container is similar to the earlier 1.x default-test-container, but with unnecessary duplication of requirements across Python versions removed.
Collections which have tests that depend on requirements no longer present in the default test container should specify them in their test requirements files:
* tests/integration/requirements.txt
* tests/unit/requirements.txt
* Bump test container versions
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Only allow groups which were hardcoded in module_defaults.yml
only load action groups from the collection if module_defaults contains a potential group for the action
* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml
Third party modules should not be using group/ - use the action name instead
* add externalized module_defaults tests
add the missing group and collections
ci_complete
Co-authored-by: Matt Davis <mrd@redhat.com>
* changelog
ci_complete
* Fix import in tests
ci_complete
* Update with requested changes
ci_complete
* don't traceback since we don't validate the contents of module_defaults
ci_complete
Co-authored-by: Matt Davis <mrd@redhat.com>
* Allow to specify collection_name separately for deprecation.
* Use new functionality in Ansible.
* Use new functionality in tests.
* Update tagging/untagging functions.
* Update pylint deprecated sanity test.
* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).
* Improve version validation. Re-add version_added validation.
* Make sure collection names are added to return docs before schema validation.
* Extra checks to avoid crashes on bad data.
* Make C# module utils code work, and update/extend tests.
* Add changelog fragment.
* Stop extracting collection name from potentially tagged versions/dates.
* Simplify C# code.
* Update Windows modules docs.
* Forgot semicolons.
* Split out sanity test requirements.
* Run each --venv test separately.
This provides verification that the requirements for each test are properly specified.
* Use a separate requirements file per sanity test.
* Skip setuptools/cryptography setup for sanity.
* Eliminate pyyaml missing warning.
* Eliminate more pip noise.
* Fix conflicting generate_pip_install commands.
* Add changelog fragment.
* Make AnsibleVaultEncryptedUnicode work more like a string. Fixes#24425
* Remove debugging
* Wrap some things
* Reduce diff
* data should always result in text
* add tests
* Don't just copy and paste, kids
* Add eq and ne back
* Go full UserString copy/paste
* Various version related fixes
* Remove trailing newline
* py2v3
* Add a test that can evaluate whether a variable is vault encrypted
* map was introduces in jinja2 2.7
* moar jinja
* type fix
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Remove duplicate __hash__
* Fix typo
* Add changelog fragment
* ci_complete
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Auto unroll generators produced by jinja filters
* Unroll for native in finalize
* Fix indentation
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add changelog fragment
* ci_complete
* Always unroll regardless of jinja2
* ci_complete
Co-authored-by: Sam Doran <sdoran@redhat.com>
* various deprecation, display, warning, error fixes
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* cleanup, test fixes
* add collection name to deprecated() calls
* clean up redirect entries from uncommitted tests
* fix dep warning/error header text to match previous
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- The `add_host` action now shows an accurate change status.
Test Plan:
- Added a plethora of integration tests.
Tickets:
Fixes#69881
Signed-off-by: Rick Elrod <rick@elrod.me>
* starting metadata sunset
- purged metadata from any requirements
- fix indent in generic handler for yaml content (whey metadata display was off)
- make more resilient against bad formed docs
- removed all metadata from docs template
- remove metadata from schemas
- removed mdata tests and from unrelated tests
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Work around strange behavior of StrictVersion and SemanticVersion constructors that they accept an falsy value.
* Do not accept empty strings as versions.
Change:
- This was removed in 2014 in 122a7021bc.
- The option still exists and is enabled by default and can lead to user
confusion when people aren't expecting packages (or updated
dependencies for it) to get installed and they do.
- Add the option documentation back with a few notes to make it clear what
is happening.
Test Plan:
N/A, no code change, just documentation
Tickets:
- Refs #69497
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix sanity errors
Signed-off-by: Rick Elrod <rick@elrod.me>
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from. Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.
This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.
- hosts: all
strategy: free
gather_facts: false
tasks:
- include_role:
name: random_sleep
- block:
- name: set a fact (1)
include_role:
name: set_a_fact
tasks_from: fact1.yml
- name: set a fact (2)
include_role:
name: set_a_fact
tasks_from: fact2.yml
- name: include didn't run
fail:
msg: >
set_a_fact didn't run
fact1: {{ fact1 | default('not defined')}}
fact2: {{ fact2 | default('not defined') }}"
when: (fact1 is not defined or fact2 is not defined)
Closes#69521
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.
See also https://github.com/ansible/ansible/issues/69516
* do not return the body even if it failed
* add some tests for this and rebase
* import test task
* ignore_errors when fails
Co-authored-by: Jack Zhang <jack.zhang@aspiraconnect.com>
* Enable installing collections from git repositories
* Add tests for installing individual and multiple collections from git repositories
* Test to make sure recursive dependencies with different syntax are deduplicated
* Add documentation
* add a changelog
* Skip Python 2.6
* Only fail if no collections are located in a git repository
Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.
Add a note that the collection(s) in the repo must contain a galaxy.yml
* Add a warning about embedding credentials in SCM URLs
* Update with review suggestions
* suppress sanity compile failure for Python 2.6
Change:
Rather than only using config, have base connection plugins fall back to
play_context.
Test Plan:
- Tested ansible-connection logic against an IOS device
- Tested -k against a VM
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.
* Make tagging/untagging functions more flexible.
* Tag all versions in doc fragments.
* Tag all deprecation versions issued by code.
* Make Display.deprecated() understand tagged versions.
* Extend validation to enforce tagged version numbers.
* Tag versions in tests.
* Lint and fix test.
* Mention collection name in collection loader's deprecation/removal messages.
* Fix error IDs.
* Handle tagged dates in Display.deprecated().
* Also require that removed_at_date and deprecated_aliases.date are tagged.
* Also automatically tag/untag removed_at_date; fix sanity module removal version check.
* Improve error message when invalid version number is used (like '2.14' in collections).
* Allow to deprecate module by date in documentation.
* Make sure deprecation date/version match between module docs and meta/runtime.yml.
* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.
* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.
* Linting.
* Allow to deprecate plugin options by date.
* Add changelog fragment for deprecation by date (also covers #68177).
Gets rid of the unknown field names in the return data.
Allows the plugin return docs to format under the new docs pipeline.
* Expect that package_facts will pass return-syntax-error now.
PR #66461 introduced a regression that resulted in an in correct block in the file
if the block to be inserted did not end with a line separator. Fix this bug and add
tests to cover this scenario.
Fixes#64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.
Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.
Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.
Tickets:
Fixes#34722, fixes#54743, refs #11565.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow to deprecate options and aliases by date instead of only by version.
* Update display.deprecate().
* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.
* Parse date and fail on invalid date.
This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.
* C# code: improve validation and update/extend tests.
* Make sure that deprecate() is not called with both date and version.
* Forgot to remove no longer necessary formatting.
* Adjust order of warnings in C# code.
* Adjust unrelated test.
* Fix grammar (and make that test pass).
* Don't parse date, and adjust message to be same as in #67684.
* Sanity tests: disable date in past test.
* Validate-modules: validate ISO 8601 date format.
* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.
* Use DateTime instead of string for date deprecation.
* Validate that date in deprecated_aliases is actually a DateTime.
* Fix tests.
* Fix rebasing error.
* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.
* Make deprecation date in the past error codes optional.
* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.
* Stop using Python 3.7+ API.
* Make sure errors are actually reported. Re-add 'ansible-' prefix.
* Avoid crashing when 'name' isn't there.
* Linting.
* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Adjust test to latest change.
* Prefer date over version if both end up in Display.deprecated().
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
* Use a custom build container on Shippable.
This supports key generation before git_sync, to avoid issues with pre-migration PRs.
If the node pool is switched to another version, a matching build container should be built and used.
* Don't kill our own custom container.
* [yum] Make package removal confirmation strict
Change:
After removing packages, the yum module does a final check to ensure the
packages are really installed. The check would include packages that
were RPM `Provides:` values of another package.
This means that, for example, if a third-party kernel RPM spec had
`Provides: kernel` in it, removing the stock kernel would be successful
but the check to see if it was really removed would fail and cause
Ansible to report a failure.
Test Plan:
Tested on local CentOS 7 VM with kernel from elrepo which is known to
`Provides: kernel`.
Tickets:
Fixes#69237
Refs #35672
Refs #40723
Signed-off-by: Rick Elrod <rick@elrod.me>
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
* Fix `ansible -K` become_pass regression
Change:
- This fixes a breaking change introduced in
2165f9ac40
Test Plan:
- Local VM for now, with plans to add an integration test for -K going
forward.
Tickets:
Refs #69244
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
as it stands we rerun discovery for the delegated host
unless its saving facts to itself
* fixed test lacking delegate_to mock
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.
Fixes#69457
* Add multipart/form-data functionality
* Fix some linting issues
* Fix error message
* Allow filename to be provided with content
* Add integration test
* Update examples
* General improvements to multipart handling
* Use prepare_multipart for galaxy collection publish
* Properly account for py2 vs py3, ensuring no max header length
* Address test assumptions
* Add unit tests
* Add changelog
* Ensure to use CRLF instead of NL
* Ignore line-endings in fixture
* Consolidate code, add comment
* Bump fallaxy container version
* ci_complete
* ansible-galaxy - remove warning during collection install
If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.
A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.
* Update unit tests
* Unify ansible-galaxy install -r
* Minor nit fixes for docs
* Re-align warnings
* Fix up integration test
* Fix up test where no roles/collections were in file
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.
* Add fixture data and update unit tests
Co-authored-by: Sam Doran <sdoran@redhat.com>
The updated container includes fewer requirements now that the collection migration has completed.
Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
* fix delegation vars usage and reporting
- just pass delegated host vars + task vars to plugins
and avoid poluting with original host vars
- updated tests
* Add a better error for "ansible-galaxy verify" if the MANIFEST.json has been deleted from the installed collection or if the collection hasn't been installed via normal means
* Fix unit tests for the remote collection
If there's something wrong with the local collection's version it will fail before the remote collection is sought
* Add a test for the new error msg
* Prevent the duplicate warning
Mock the new isfile call where needed
* Update lib/ansible/galaxy/collection.py
Co-Authored-By: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Fix filedescriptor out of range in select() when running commands
* Simplify the run_command() code
Now that we're using selectors in run_command(), we can simplify some of
the code.
* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
instead.
Also use a simpler conditional check of whether the selector map is
empty
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:
lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL
The exact error depends on Python 2 or 3, but here is the error on
Python 2:
AttributeError: 'NoneType' object has no attribute 'endswith'
And on Python 3:
TypeError: expected str, bytes or os.PathLike object, not NoneType
This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().
Fixes: #69450
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.
Test Plan:
CI
Tickets:
Fixes#69230
Co-authored-by: Matt Clay <matt@mystile.com>
* validate-modules: deprecated modules in collections
In Collections a module is marked as deprecated via meta/routing.yml
Use this file, rather than the leading `_` as part of the deprecated
test.
* Correct variable
* review comments
* indentation
* Read routing.yml only once
* pep8
* Apply suggestions from code review
Co-authored-by: Matt Clay <matt@mystile.com>
* review: remove duplicated conditional
Co-authored-by: Matt Clay <matt@mystile.com>
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances. There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code. (The most likely case of
this is when it's a global variable and some other code might import
that variable name).
Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement. At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'
Fixes: #69286
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Metadata defaults were not being set if only a few fields were missing.
* ansible-doc with no documentation and no status in metadata should
return empty, just like if there was no documentation and no metadata
at all.
* Fix support for Kali Linux detection
* Add test for Kali Linux detection
* Improve path matching with "in list"
Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Do not pass file mode during recursive copy on symlink files.
The 'file' module cannot deal with mode=preserve. Do not pass that
mode to the module when 'preserve' is used.
* Fix changelog fragment filename
Change:
We were only testing dnf on RHEL previously.
Test on CentOS 8 as well.
Test Plan:
Ran locally in docker.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Extend the logic for custom error handling in the dnf module, so that on
newer DNF (such as DNF that ships with modern Fedora 31 container
images, and ships with RHEL 8.2) we report errors consistently with
older DNF.
Test Plan:
Ran dnf integration tests against an old Fedora 31 container image and a
brand new Fedora 32 container image; tess passed on both.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Make sure collection is a list if a str is given
* Call field validation early on collections
Because we are doing work on modifying the collections value before
it is actually validated, we can validate it ourselves early to make
sure the user supplies either a string or list. Dicts are not valid.
The new validation allows us to simplify the _ensure_default_collection()
function. And since the field is now static, we no longer need to specify
a default for it, which also allows us to simplify the function. Since
the default is now removed, we can also remove the sanity/ignore.txt entry
for collectionsearch.py.
New unit tests are added (and the existing one modified) that allow us to
make sure that we throw a parser error if a user specifies something other
than a string or list for the collections value everywhere it can be specified.
* Revert removing the collection default
The default is actually used, so restore it.
* Fix unit tests in test_helpers.py affected by early collection validation
This patch covers a few changes to get the yum test case working on ppc64le
CentOS. Specifically we needed to enable the EPEL repository on CentOS
as well as ensure some of the architecture-specific tasks use the right
set of binaries during their test.
Currently if virtualenv_command has arguments, then the
search for the binary in the path does not work so the
user has to specify the full path to it.
To allow arguments to be used without having to specify
the path to the binary, we split the module argument into
the command and anything after the first space.
This makes using this module argument more flexible and
user friendly.
Fixes: #52275
Change:
Rather than hardcoding .pyo and .pyc, filter on all BLACKLIST_EXTS in
the non-legacy logic of PluginLoader (_find_fq_plugin). The two harcoded
extensions are part of BLACKLIST_EXTS already and this simply adds the
rest of the blacklisted extensions to the check.
In addition, check .endswith() instead of an exact match of the suffix,
like everywhere else that uses BLACKLIST_EXTS. This allows for
blacklisting, for example, emacs's backup files which can appear after
any extension, leading to things like `foo.py~`.
Test Plan:
Ran `ansible-playbook` against a collection where a `foo.py~` module was
getting executed instead of `foo.py` which also appeared in the same
directory. `foo.py~` is no longer executed.
Tickets:
Fixes#22268
Refs #27235
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow a collection role to call a standalone role by default. Fixes#69101
* tweaked changelog text
* Guard against NoneType
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
* update ActionBase._low_level_execute_command to honor executable
* adding changelog fragment
* renaming changelog fragment to .yml
* noop change to bump shippable
* adding raw_executable integration test
* copying aliases from raw
* removing blank lines
* skipping aix and freebsd
* noop to bump shippable
* moving tests to raw/
* removing become_method: sudo ; it doesn't work on AIX
* removing trailing blank line
* forcing become_method: su to try to get AIX to work
Co-authored-by: Rob Wagner <rob.wagner@sas.com>
Change:
New `cryptography` statically links an openssl that is too new for macOS
10.11, so limit to an older cryptography for now.
Test Plan:
Ran the test with `--remote osx/10.11` and it passed.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Adds some missing coverage for the copy module when `check_mode: True`.
Test Plan:
Ran test with --coverage and looked at the resulting report.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable service integration tests for FreeBSD
Change:
Adds necessary rc file for freebsd, and gets tests passing for it.
Test Plan:
Ran test with `--remote freebsd/12.1` and `--remote freebsd/11.1`. Both
passed.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update test/integration/targets/service/tasks/rc_setup.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
* fix up comment
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Clay <matt@mystile.com>
With https://github.com/pallets/jinja/pull/1190 merged our short-circuit
is no longer valid (has it ever been?) as now data like ' True ' may go
through our ansible_native_concat function as opposed to going through
intermediate call to Jinja2's native_concat before. Now we need to always
send data through literal_eval to ensure native types are returned.
* Testing: Add CentOS Linux On Power platform
* Add arch designation to remotes.
This avoids overloading the provider with the arch.
Also add a changelog entry.
Co-authored-by: Matt Clay <matt@mystile.com>
* ansible-galaxy - fix listing specific role
If the role was not in the first search path, it was reported as not found
* Properly display role description
Default to description to top level description, falling back to the description from within galaxy_info
* Display proper message when a role does not exist
* Add integration tests
* Use context manager
* BSD and macOS ruining all the fun
* Force collections to be static
Templating of collection names does not work at all. Force them to
be static so that a warning is generated for the user.
* Add collectionsearch unit test and fix for reviews
New unit test validates the new _load_collections() code and moves
the new check to the end of the method.
* Change unit test to pytest
* Adjust unit test to use capsys instead of monkeypatch
* Fix pep8 error
* Add changelog fragment
Closes#68704
* subversion module - provide password securely with svn command line option --password-from-stdin when possible, and provide a warning otherwise.
* Update lib/ansible/modules/source_control/subversion.py.
* Add a test.
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Don't trigger full CI run for changes to changelogs/ and docs/ in collections.
* Add changelog fragment.
* Update changelogs/fragments/68550-ansible-test-docs-changelogs.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Update docker.txt to use the OpenSUSE 15.1 container image
Signed-off-by: Rick Elrod <rick@elrod.me>
* handle installing mysql on suse
Signed-off-by: Rick Elrod <rick@elrod.me>
* add changelog fragment
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update changelogs/fragments/ansible-test-opensuse-15.1.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Update tests to use RHEL 7.8.
Keeping support for RHEL 7.6 since collections are still using it.
* Fix tests for RHEL 7.7+ due to extras repo name change.
* fixed fetch traversal from slurp
* ignore slurp result for dest
* fixed naming when source is relative
* fixed bug in local connection plugin
* added tests with fake slurp
* moved existing role tests into runme.sh
* normalized on action excepts
* moved dest transform down to when needed
* added is_subpath check
* fixed bug in local connection
fixes#67793
CVE-2019-3828
* Allow tasks to notify a fqcn handler name
* Add tests. Fixes#68181
* Add changelog fragment
* Add test to ensure handlers are deduped properly with fqcn, role, and just handler names
* Add some docs about new special vars
fial_json() requires a message be given to it to inform the end user of
why the module failed. Prior to this commit, the message had to be a
keyword argument:
module.fail_json(msg='Failed due to error')
Since this is a required parameter, this commit allows the message to be
given as a positional argument instead:
module.fail_json('Failed due to an error')
The last task in a play should now properly report code coverage.
This change should also eliminate empty coverage files, as well as incomplete coverage files resulting from early worker termination.
* Fix test_check_mutually_exclusive exception-checking
Asserting inside of the `with` context of `pytest.raises`
doesn't actually have any effect. So we move the assert
out, using the exception that gets placed into the scope
after we leave the context, and ensure that it actually gets
checked.
This is also what the pytest documentation says to do:
https://docs.pytest.org/en/latest/assert.html#assertions-about-expected-exceptions
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add some tests for check_required_together
Signed-off-by: Rick Elrod <rick@elrod.me>
* use to_native instead of str, for consistency
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add newlines for pep8
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add tests for check_required_arguments
Signed-off-by: Rick Elrod <rick@elrod.me>
* Sort missing keys in error message, since hashes are unsorted and this can be random
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add changelog entry
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow custom inventory plugins and cache plugins
If _load_name is not set correctly the cache plugin can't load the documentation (which is also the arg spec)
Fix the existing inventory plugin in the collections tests
Add integration tests for using a cache plugin in a collection
* Set the attribute on the instance instead of the class
Deprecate importing custom CacheModules directly - they should use the cache_loader
* Add some more test coverage for unarchive
This moves over (and slightly extends) coverage which was found in
incidental_flatpak_remote.
Signed-off-by: Rick Elrod <rick@elrod.me>
* add a group for testing too, user creation does not mean group creation on all platforms
Signed-off-by: Rick Elrod <rick@elrod.me>
* update the test group assert
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix style
Signed-off-by: Rick Elrod <rick@elrod.me>
* block/always
Signed-off-by: Rick Elrod <rick@elrod.me>
* fix vault tmpe file handling
* use local temp dir instead of system temp
* ensure each worker clears dataloader temp files
* added test for dangling temp files
* added notes to data loader
CVE-2020-10685
* Remove some unreachable code in the file module
Remove some cases in file.py which are covered by conditionals a few
lines earlier. Remove the duplicate code which will never be hit.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Restore incidental file coverage from timezone module
Signed-off-by: Rick Elrod <rick@elrod.me>
* Combine two conditionals, add a changelog entry
Signed-off-by: Rick Elrod <rick@elrod.me>
* Make new test syntax consistent, add two stat tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Support pre-releases via new SemanticVersion. Fixes#64905
* Don't treat buildmeta as prerelease
* Don't inherit from str and int
* Add helper method to try and construct a SemanticVersion from a LooseVersion
* Don't count major 0 as pre-release, it's different
* Guard against invalid or no version in LooseVersion
* return a bool
* Add integration tests for pre-release
* Fix up lingering issues with comparisons
* typo fix
* Always allow pre-releases in verify
* Move pre-release filtering into CollectionRequirement, add messaging when a collection only contains pre-releases
* Update changelog
* If explicit requirement allow pre releases
* Enable pre-releases for tar installs, and collections already installed when they are pre-releases
* Drop --pre-release alias, make arg name more clear
* Simplify code into a single line
* Remove build metadata precedence, add some comments, and is_stable helper
* Improve from_loose_version
* Increase test coverage
* linting fix
* Update changelog
* remove azure extras and extras_require support
* Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now.
* Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history.
* Mark azure-requirements as orhpaned.
This keeps the docs around so that existing links from old test runs remain valid.
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix ansible-test coverage analysis option usage.
The `--input-dir` option for `coverage analyze targets generate` was being ignored.
No changelog entry since this feature has not yet been released.
* Move coverage config to fix type annotations.
Declaring the types before referencing them makes sure they're recognized by tools such as PyCharm.
Now empty `*.py` files are ignored during module_utils import analysis for change detection.
This eliminates "No imports found" warnings for files which should have no imports.
* Rename `tests` test to match plugin type.
* Rename `test_infra` test to avoid confusion.
This test target is not a test for test plugins.
* Rename `vars_prompt` test to avoid confusion.
* Update sanity ignores.
* Internally redirect win modules to collection name
* Added comment for how this should be fixed in a subsystem plugin
* add collection plugins for base tests
* Address compat issue for collection loading on py26
* Move import_module shim to utils for compat across the codebase
* Enable collection tests on py2.6
* Update changelog fragment
* Simplify code using sys.moduls
* Move compat to module_utils/compat/importlib
* Add back errantly deleted newline
* Remove hack comment
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
Requirements were incorrectly added to ansible-test in https://github.com/ansible/ansible/pull/61813
These requirements should have been placed into `test/units/requirements.txt` instead.
Now that the relevant content has been migrated out of the repository, the requirements are no longer necessary there either.
No changelog entry for this change since the original changes were not included in any release and also lacked a changelog entry.
* copy plugins.inventory.docker_swarm in to test/support
Signed-off-by: Rick Elrod <rick@elrod.me>
* Copy over functions from module_utils.docker.common and nuke the import
Signed-off-by: Rick Elrod <rick@elrod.me>
* Split test_play_context_make_become_cmd into files
For NWO migration. Split the become module assertions into distinct test
files and functions. For now, this is done naively - there is probably
room to abstract these tests out and remove some of the duplication
later on.
Signed-off-by: Rick Elrod <rick@elrod.me>
* use default_exe variable instead of hardcoding /bin/bash
Signed-off-by: Rick Elrod <rick@elrod.me>
* Move become plugin tests to their proper directory and rename them accordingly
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix up fixtures and imports.
* Remove stray file.
Co-authored-by: Matt Clay <matt@mystile.com>
* Create a dedicated windows-minimal test target.
The windows-minimal target is a copy of the win_ping test, taking the place of that test as the windows minimal test run on multiple python versions.
It includes a private copy of the win_ping module so it will work after migration.
This will keep tests passing during the migration prep process.
* Update sanity ignores.
* Split fallaxy tests into their own group.
This keeps expected pass/fail tests separate during migration prep.
* Move network tests down in matrix.
This will help keep pass/fail tests grouped together during migration prep.
* Group all incidental tests together.
This will keep pass/fail tests separate leading up to migration.
* win_timezone - Allow for _dstoff timezones
* Update win_timezone-Allow-dstoff.yml
* Added doc entry for new format
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Added nxos_lldp_interfaces module
* Linting
* Added RTT, resolved shippable errors
* Added new states
* New states edit
* Updated states
* Updated tests
* Show all interfaces in facts
* Test changes
* Added unit tests
* Linting
* Handled portchannel failing condition
* Renamed auto evpn test
* Made sure that the current module is idempotent with additional tests
* Added tests for route-target import function
* Added tests for route-target export function
* Added tests for route-target both function
* PEP8 syntax fix
* Added route-target import & export function
* Added required 'version_added: "2.8"' in the documentation
* Updated documentation of new route-target options
* Added a test to make sure that in case of `state=absent` on the vrf
level the route-target options are ignored.
* Specified that the route-target options are ignored in case of
`state=absent'.
* Updated the doc to the correct format (using 'C()')
* Changed the VRF Route Target Syntax
Instead of using three different params (route_target_import,
route_target_export, route_target_both) the module uses now only one
param (route_targets) and the direction is specified for each of the
route targets.
Example:
route_targets:
[{rt: '...', direction: '{import|export|both}', state: '...'}]
* Updated Description and Examples to reflect new params
* Updated "version_added"
* pep8 fixes
* If rt['direction'] is not definied, we assume default 'both' and run
the same routine
* Added test with default direction for route-targets
* Documentation fixes
* Split out cache plugin unit tests.
* Rename unit tests to match code under test.
* Relocate unit test code to match code under test.
* Another rename.
* Update sanity ignores.
* ansible-galaxy - optimise some paths and use fake galaxy int tests
* Added init, built, and publish tests
* Test against both mocked Galaxy and AH server
* Finish off writing the install tests
* Fix up broken tests
* Rename test target and add migrated tests
* Use cloud provider for Galaxy implementation
* Added blank static config
* Use correct alias group
* Set release version and fix copy typo
* Remove reset step as it is no longer needed
* Use sane env var names for test container name
pytest-mock 2.0.0, when run locally, gets grumpy when os.path.exists is
messed with and then another method is patched afterwards. Likely
something in the pytest-mock chain uses os.path.exists internally, and
since pytest-mock prohibits context-specific patching, there's not a
good solution. For now, just patch os.path.exists last.
Signed-off-by: Rick Elrod <rick@elrod.me>
* reworked sqs_queue
* Switch default purge_tags behaviour to false.
This matches the behaviour of ec2_tag and ecs_tag.
* Minor lint / review fixups
* Add missing AWS IAM policy for SQS tests
* Move integration tests to using module_defaults: group/aws:...
* add changelog
* Break out the 'compatability' map from our spec definition (gets flagged by the schema validation)
* Tweaks based on review
* add basic examples
* Lint fixups
* Switch out NonExistentQueue logic so it's easier to follow
* Reorder name argument options for consistency
Co-authored-by: Dennis Podkovyrin <dennis.podkovyrin@gmail.com>
Use PostgreSQL 9.5 on FreeBSD 12.0, and PostgreSQL 11 on FreeBSD 12.1 and 11.3 due to the Python packages having a dependency on that version of PostgreSQL and automatically uninstalling PostgreSQL 9.5.
* Use separate PostgreSQL versions for 12.0 and 12.1
* Allow passing through of (almost) all params available on boto methods in aws_api_gateway
* Linting and docs fixes
* Refactored method signature of create_deployment() to use keyword args instead of named args
* Updated version_added flags to 2.10
* Cleanup and improve aws_api__gateway integration test play. Also included new params into test.
* Fixed RETURN docs and some ttests
* Completed RETURN docs and made integration tests match
* Fixed variable names in test and YAML syntax in docs
* Comment out critical sections of integration test
* Fixed update test after figuring out what the error message means. Also updated error message to be more descriptive.
* Fixed test assertion
* Update docs and make tests reflect that endpoint type wont be changed on updates
* Syntax fix
* Add changelog fragment
* Improve aws_api_gateway docs, fix typos.
* Quote doc lines with colon
* AnsibleAWSModule related cleanup - s3_bucket
* Add extra information to s3_bucket timeout failures, it's possible the comparisons are doing something weird...
* Move Bucket Encryption boto support logic into the pre-flight checks
* Use the built in required_by logic
* Rework s3_bucket integration tests
* Add a retry around put_bucket_encryption
s3_client.put_bucket_encryption is occasionally dropped on the floor
by Amazon add some logic to retry s3_client.put_bucket_encryption call
* Catch OperationAborted and retry, it is caused by a conflicting change
still being in progress. (For example an Encryption setting applying)
* Make sure we don't explode if the botocore version's too old
* Review tweaks
* Cleanup tests
* Auto-Retry on ResourceNotFound and RequestInProgress exceptions
* Use AnsibleModule options for required_if logic
* changelog
* Remove (now) duplicate RequestInProgressException catching
* Allow a single retry when attempting to fetch the information about a cert directly after deleting it.
There is a small chance that it goes away while we pull the details.
* add key rotation option
* add changelog fragment
* provide version added as string
* change changelog to minor_changes
* Update changelogs/fragments/67651-aws-kms-key-rotation.yml
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* prevent key upgrade if key rotation was enabled manually. In that case, the key rotation would be disabled, if not mentioned in the playbook
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
Co-authored-by: Mark Chappell <mchappel@redhat.com>
* Initial copy of incidental network tests.
* Update incidental test aliases.
* Add incidental tests to CI.
* Rewrite module references in tests.
This should not be necessary once module redirection is supported.
* Rewrite target references for renamed targets.
* Add support collections for incidental tests.
* Add ignores for test support code.
* Remove echo used for debugging.
Make AWS test plugin to refer to the aws-permissions-for-integration-tests section of the AWS devel docs now that we have them, rather than suggesting to flag down specific users.
This ensures the test passes regardless of the test environment.
Previously, it would pass in some environments but fail locally
in other environments, due to the width being calculated as
different or support for color not being consistently detected.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Support private test plugins in ansible-test.
This feature is exclusively used for testing Ansible itself and is not available to collections.
Content in the newly supported directories will be added in follow-up PRs.
* Add new directory to path classification.
* Support new paths in shebang test.
* Add new directory to manifest.
* fixed#47050
* added changelog fragment
* added quick and basic test
* Revert "added quick and basic test"
This reverts commit 75f4141656.
* added better tests
* now also creating files to copy on the remote
* removed tests for recursive copying which is not supported by remote_src
* sns_topic: (integration tests) Move the tests over to using module defaults
* sns_topic: (integration tests) Add test for behaviour of changed when using delivery_policy
* sns_topic: ensure "changed" behaves properly when managing delivery policies
- a delivery_policy isn't an IAM policy, so compare_policies didn't cope with it
- AWS automatically adds an additional option when you set an HTTP delivery
policy
* Parse the delivery policies so we can test the changes properly
* Update AWS policy to enable management of TargetGroups
* elb_target: (integration tests) migrate to using module_defaults
* elb_target: (integration tests) lookup the AMI by name rather than hard coding AMI IDs
* elb_target_info: (integration tests) finish rename of integration test role
* elb_target: (integration tests) rename various resources to consistently use {{ resource_prefix }}
* elb_target_info: (integration tests) Migrate to using module_defaults
* elb_target_info: (integration tests) Lookup AMI by name rather than hard coding AMI IDs
* Apply suggestions from code review
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
* elb_target: (integration tests) Remove the 'unsupported' alias
* Try bumping up the timeout
* Rules don't permit 'shippable' (resource_prefix uses this when run in shippable)
* Try bumping up more timeouts :/
* Avoid double evaluation of target_health assertion
* Simplify target_type usage a little (rather than constantly performing a lookup)
* mark elb_target tests 'unstable' for now, they're slow
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
* Add warning when running devel. Addresses #67362
* Add config entry to disable devel warning, so it doesn't impact CI
* Update warning about using devel
* cloudfront_distribution: (integration tests) Migrate to using module_defaults
* cloudfront_distribution: (integration tests) Use the ID rather than the alias
Using aliases requires providing a valid SSL certificate, as such we're not longer able to test using an arbitrary hostname
* cloudfront_distribution: (integration tests) Make sure we delete the test s3 bucket when tests fail
* cloudfront_distribution: field_level_encryption_id is now a mandatory field always add it
Setting the field to an empty string has the same effect as the original behaviour.
* Copy & Paste fixup
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
* required_if checks should have three or four parts.
* Validate mutually_exclusive, required_together, required_one_of, required_if and required_by.
* Simplify code.
* Improve messages.
* Add changelog.
* Sanity check.
* Update docs.
* Update ignore.txt.
* Don't continue with tests when terms are not strings.
* Remove ignore.txt entry.
* Make sure validate-modules doesn't choke on things already flagged by schema test.
* Check required_if requirements list for strings.
* Run Ed25519 and Ed448 tests for openssl_csr and openssl_certificate only if key generation succeeded.
* Make openssl_privatekey tests more robust: allow special key generation tests to fail with 'algorithm not supported' on FreeBSD.
These tests rely on the AWS S3 modules, but will not be migrated along with those modules into an AWS collection.
Since these tests will not reside in a collection alongside the modules under test they are being moved to the legacy tests directory.
The legacy tests directory will soon be migrated to a separate repository.
* Add x509_crl module.
* Add integration tests.
* Fix some errors.
* Fix inversion.
* Compare name instead of tpye.
* Fix fail_json() calls.
* Work around rename of serial_number attribute for cryptography 1.4.
* Don't die for non-cert loading errors.
* One more.
* Fix function call.
* Fixed/improved descriptions.
* Don't read issuer from certificate file.
* Allow to ignore timestamps.
* Default value for revocation_date.
* Update tests.
* Mention ignore_timestamps in update docs.
* Support privatekey_content, and require some options only if state is present.
* Allow to pass certificate in directly.
* Add tests.
* Fix required_if.
* Forgot to encode content.
* Forgot to adjust type.
* Allow to return CRL's content directly.
* return_crl_content -> return_content (as in #65400).
* Fix elements.
* Fix messages.
* Use required_one_of and mutually_exclusive instead of doing the checks by hand.
* Fix format.
* Skip tests on AIX.
* Fix typo.
* 'message' parameter is replaced by 'commit_message' in grafana_dashboard
* 'message' parameter is replaced by 'notification_message' in datadog_monitor
This change is required since 'message' as parameter name is used internally by
Ansible core engine.
Fixes: #39295#45362#47132#59617
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix DHCP support in win_dns_client + more
* Fix bugs and test failures, add changelog fragment
* Add idempotency tests for DHCP
* Address review feedback; dedup address-family code
* Remove legacy function
* Remove old reference
* Create ipwcli_dns.py
* add newline at the end
* Update after review and support AAAA
* Update lib/ansible/modules/net_tools/ipwcli_dns.py
Co-Authored-By: Felix Fontein <felix@fontein.de>
* add integration tests and change param user to username
Co-authored-by: Felix Fontein <felix@fontein.de>
* ec2_snapshot and ec2_snapshot_info test suite
* Add a fact to make tests easier to follow.
Test all return values, and add missing RETURN docs to module.
* cloudformation - use mutually_exclusive for on_create_failure and disable_rollback
* cloudformation - remove unit test test_disable_rollback_and_on_failure_defined
* bumped zabbix integration tests to be run against current LTS 4.4
* macros and tags tests for zabbix host + naming changes due to switch to zabbix 4.4
* Add AWSRetry decorator to ec2_vpc_nacl
* Also add a decorator to ec2_vpc_nacl_info to catch things like API rate limit errors.
* add double-removal integration tests to make sure things don't get too slow
* Fixup retry usage for _info
* Simplify changed logic when modifying a NACL
* tweak error message
* Fix for shared snapshot parameter
Fixing this bug:
`Unknown parameter in input: "IsShared", must be one of: DBInstanceIdentifier, DBSnapshotIdentifier, SnapshotType, Filters, MaxRecords, Marker, IncludeShared, IncludePublic, DbiResourceId`
* Updated documentation for shared snapshots
Tags can't get accessed for shared snapshots
* fixed indentation
* added test for shared snapshot
* fixed isPublic parameter to correct IncludePublic parameter
Co-authored-by: Oliver Kastler <oliver@realestate.co.nz>
This avoids confusion with tests named `inventory_*` which do not test inventory plugins.
Tests for inventory scripts are now prefixed with `script_inventory_`.
User can now specify tag and category using dict in vmware_tag_manager
module. This is useful when tag or category name contains colon.
Fixes: #65765
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Start adding ansible-galaxy collection list options
* Working list all collections and list a specific collection
* Nuke debugging cruft
* Use to_text to get a string of the FQCN for sorting
* Improve collection output formatting
- add header
- display collection name and version in separate columns
- width of columns is dynamic based on collection name and version length
* Make role list output match collection output list
- add header
- add columns for role name and version
- make column width dynamic based on name and version length
* remove debug statemnt and extra header
* Revert "Make role list output match collection output list"
This reverts commit a0b3db47bb3b198aafd34c1f1be5b6561af2f928.
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Use validate_collection_path
* Do not warn if a specific collection in found in any search path
* Fix extraneous warning and remove duplicate code
Do not warn when listing a specific collection and it does not exist
in other collection paths.
Restructure the code that loops through collection paths to remove
duplicate conditional code.
* Indicate role path was found
* Use new function name
* WIP Save Point
* Use separate functions for role and collection list
* Wrap error message
There may be a better way to do this besides hard coding a line break, but this
does make the message a lot more readable.
* Add validate_collection_path function (#66441)
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Fix bad syntax
* Correct docstring
* Bikeshed the names
* Properly list a single role
* Simplify _display_warnings()
Only display warnings. Move exception raise back to each caller.
* Move private methods to private functions
They don't need self, so it makes sense to have them as functions
Get rid of _display_warnings() function since it doesn't do anything worthy of
an independent function.
* Add integration tests for ansible-galaxy collection list
* Fix docs sanity test
* Fix bug where ansible_collections dir does not exist
The path may exist, but if there is no ansible_collections dir inside that path,
an exception was raised in find_existing_collections().
Add integration test for this scenario
* Put execute_list() method back
* Add some informational messages for debugging
* Add unit tests
Units tests for the various private methods in support of collection list
* Start adding unit tests for test_execute_list
* Display collection path when listing specific collection
* Add unit tests for listing all collections and specific collection
- Create fixture for creating test objects
- Add function for controlling os.path.isdir results
* Set defaults for minimum collection widths
Ensure that collections with small FQCNs display correctly.
Add unit tests
* Split up unit tests and fix fixtures
Add more fixtures for mocking objects during the specific collection tests
* Change help message for -p in list subcommand
Give accurate description of what it actually does rather than trying to use language shared between sub commands.
* Disable colorized output in unit test
* Add docs for collection list
* Fix integration test on macOS
The temp file path is really long on macOS, so the warning message gets wrapped
across multiple lines. That make seth grep fail. Switch to matching on a smaller
part of the warning.
* Recreate common path options for collections
Improve help about what the '-p' option does and how it works.
* Remove unnecessary elif after continue statements
* Account for duplicate paths in collections_searh_paths
If someone specifies the same path via '-p' that is the COLLECTIONS_PATHS,
do not list the collections twice.
* Docs updates
* [WIP] Add verify subcommand command to 'ansible-galaxy collection'
* Fix pep8 and use consistent display order
* WIP - docs
* Remove some redundancy in verify display messages by using an error queue for each collection
* Share common code and improve output format
* clean up documentation
* typo
* Apply suggestions from code review
Co-Authored-By: Sandra McCann <samccann@redhat.com>
* Move ModifiedContent namedtuple to the global scope
Add a public metadata property
Rename function to _get_json_from_tar_file
* Add some unit tests
* fix using common functions after rebase
* changelog
* Improve efficiency finding specific installed collections
Improve efficiency by only downloading the tar.gz from the galaxy server for comparison after checking that the collection has been installed
Handle multiple collection paths
Fix up tests
* pep8
* reword that for accuracy
* use more common code and verify collection name
* Improve error message and add documentation
* Update unit tests and add coverage for both local and remote collections that are unable to be located
* Only validate collections using the format collection_namespace.collection_name
Fix tests to reflect that
Fix documentation
* Fix a bug when a local collection does not contain a file
* pep8
* Fix formatting
Co-authored-by: Sandra McCann <samccann@redhat.com>
us-east-1e is sometimes picked at random, and has no support for
t3/m5 instance types, which breaks some tests.
Because availability_zones is returned in a consistent (sorted) order,
we should at least get either consistent success or consistent failure.
* win_package - Refactor with msp, appx support
* Added msi test for ALLUSERS
* Added some msix tests, refactored tests
* Added remaining msix tests
* Enable msix sideloading for tests
* Added remaining exe path tests
* Added basic msp tests
* Remove url options now the util no longer has them
* Fix file version check for older Windows hosts
* Remove no_proxy ansible-test setting
* Use same mechanism of become to copy the file with explicit creds
* Added Ansible.Service util and win_service_info
* Fix up util test
* Sigh forgot to update the test and fix sanity
* Try to make tests more robust
* That didn't work, just check the username
* Betraying Queen and country with this doc fix
* More changes for compat
* More OS compatibility
* AnsibleAWSModule related cleanup - redshift
* Apply a backoff on modify_cluster to cope with concurrent operations
* Add AWS 'hacking' policy to allow creation of Redshift ServiceRole
* Adding the retry policies makes the redshift test suite more reliable
* Make validate-modules stop ignore FILE_COMMON_ARGUMENTS.
* Add types to FILE_COMMON_ARGUMENTS and update document fragment to match it.
* Update ignore.txt.
* Add changelog.
Only one integration test target is supported per module. Since there is already a `mysql_replication` integration test, the `mariadb_replication` tests will not execute for the same module.
To avoid issues with tests not running on changes to the `mysql_replication` module and then failing after changes are made and all tests are executed, the test has been marked `unsupported` to prevent it from running in CI.
To re-enable this test for CI it will need to be merged into the `mysql_replication` tests, which will require working around conflicts between the packages required by the two sets of tests.
* Include distro.{os,lsb}_release_info() in distro fixture generation script
* Add distro release information to existing fixtures
* Add some new fixtures for newer distribution versions
* Add fixture for Arch Linux with lsb-release
* Split up lookup integration tests.
* Rename lookup_paths integration test.
This will avoid confusing it for a test of the `paths` lookup plugin, which does not exist.
* Fix lookup_pipe integration test.
The test now verifies it receives the correct output.
Adding a second task also causes code coverage to be properly registered for the lookup plugin.
* Rename ini lookup test to match plugin name.
* Update sanity ignore path.
Modules and plugins can only have one integration test target associated with them.
When there is a conflict between alias(es) and/or the target name, only one target will trigger on changes to the module or plugin.
* Clean up FILE_COMMON_ARGUMENTS.
* postgresql_pg_hba doesn't declare the backup option.
* uri doesn't declare the remote_src option.
* Add documentation.
* maven_artifact seems to use directory_mode, which it doesn't declare.
* Update changelogs/fragments/66389-file-common-arguments.yml
Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst
ci_complete
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
The test fails in the CI with a timeout of vmware_guest_tools_wait. It's
still unclear if this comes from:
- the ESXi environment
- the VM configuration, e.g: the amount of the RAM
- the ISO image itself
Ideally, we should have a light VM with the vmware-tools.
I didn't properly update the commit message via github UI. Revert, to
open a new PR.
This reverts commit 2794142eb3.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This looks to be causing issues for our new ansible.netcommon
collection. Revert for now, until we can properly address.
This reverts commit 53c7f8cbde.
* Refactor coverage file enumeration.
* Relocate sanitize_filename function.
* Support sets when writing JSON files.
* Generalize setting of info_stderr mode.
* Split out coverage path checking.
* Split out collection regex logic.
* Improve sanitize_filename type hints and docs.
* Clean up coverage erase command.
* Fix docs and type hints for initialize_coverage.
* Update type hints on CoverageConfig.
* Split out logic for finding modules.
* Split out arc enumeration.
* Split out powershell coverage enumeration.
* Raise verbosity level of empty coverage warnings.
* Add code coverage target analysis to ansible-test.
* Fix UNC path joining in the powershell shell plugin, add test
* Remove testy bits and a redundant line
* Fix style nits
* Update to use os.ntpath
* Add changelog for #66604
* Try to load network action plugin from the same collection as the module
* Alter tests to match
Just make sure the action plugin is as qualified as the module it is paired with
This change moves all code for the `ansible-test coverage` command into the `coverage` directory.
Each subcommand is split into a separate file.
Only minor spelling changes were made aside from code relocation.
Since https://github.com/ansible/ansible/pull/61006 `vmware_cluster_info`
exposes a new key called `hosts`. The deprecated module
`vmware_cluster_facts` keeps the previous behaviour, and so we must keep
its test-suite unchanged.
This makes it behave in a more idiomatic way
* Fix bug in Darwin facts for free memory
If the vm_stat command is not found, fact gathering would fail with an unhelpful
error message. Handle this gracefully and return a default value for free memory.
* Add unit tests
As per:
https://github.com/pytest-dev/pytest-mock#note-about-usage-as-context-manager
pytest-mock is not meant to be used within a `with` context or as a
decorator. Instead, pytest-mock will automatically unpatch the mocked
methods when each test is complete.
In newer pytest-mock, this use actually throws an exception and causes
the tests to fail.
This hasn't been hit in Ansible's CI yet, because the docker image
that the tests run in uses an older version of pytest-mock. However,
there is no constraint on the upper bound of pytest-mock in
test/lib/ansible_test/_data/requirements/constraints.txt which means
that when running the tests locally, outside of that docker image, the
tests never pass.
This patch removes the `with` context in each such case.
Signed-off-by: Rick Elrod <rick@elrod.me>
PR #66898
This change introduces a new sanity check with code
`parameter-state-invalid-choice` in the `ansible-test sanity`
validator. It enforces modules not to support `list` or `info`
as their `state`.
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-Authored-By: Felix Fontein <felix@fontein.de>
* Fixes#66478
* When no quantity is set, then candlepin server usually uses
default value 1. When more quantities are required, then
candlepin server can automatically choose correct minimal
value.
* Move warn() and deprecate() methods out of basic.py
* Use _global_warnings and _global_deprications and create accessor functions
- This lays the foundation for future functions being moved outside of AnsibleModule
that need an interface to warnings and deprecations without modifying them.
* Add unit tests for new warn and deprecate functions
Add integration test
There are a number of other parameters that result in stack traces as well when this module is used ad-hoc. I'm not sure if we're interested in fixing them all since this module isn't meant to be run ad-hoc.
* Remove redundant use of ec2_argument_spec where we're using AnsibleAWSModule
* Use module.client() instead of the get_aws_connection_info/boto3_conn combo.
* AnsibleAWSModule handles 'HAS_BOTO3'
* Remove unused imports
* Update error message that lambda_policy integration test is looking for when the region's missing
* Revert redshift and s3_bucket
* Added integrations test for ecs_tag
Testing invalid cluster test for the expected message.
Add idempotency test is for adding tests to service and task_definition.
* throttle tests: fix detection of parallel execution
The test wasn't able to detect if too many workers were running.
On my laptop:
- without this change, the 'throttle' target takes ~20 seconds
- with this change, the 'throttle' target takes ~70 seconds
- 1 second isn't long enough to encounter the issue
* Fix throttle test when strategy is 'free' based
'free' strategy allows multiple tasks to be executed in parallel: use
one 'throttledir' per task.
Use 'linear' strategy with a dedicated play for cleanup/setup tasks
* throttle: reset worker idx before queuing a new task
* TestStrategyBase: define task.throttle
otherwise '1' will be used instead of the default value due to the
following expression being equal to '1':
int(templar.template(task_mock.throttle))
Co-authored-by: James Cammarata <jimi@sngx.net>
vmware_tag_info used to return dict of tag information which caused
data loss when there are multiple tags with same name and different category ids.
This fix will add additional fact "tag_info" which will deprecated existing fact
"tag_facts".
The "tag_info" is a list which handles multiple tags with same name.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add a script to update the intersphinx inventory files
* We're about to add intersphinx inventories for separate ansible docs
so we need an easy way to update them. Also, we should be updating
these cache files for other upstreams occassionally as well. With a
script, we can add updating them to a release process.
* Now that we don't know what the version of the cache is, change the
filenames to not contain versions.
* Update the intersphinx cache files with the latest upstream versions
Results of running:
hacking/build-ansible.py update-intersphinx-cache -o docs/docsite -c docs/docsite/rst/conf.py
* Add a comment to the configuration file which says how to structure the intersphinx mapping and why.
* Update docs/docsite/rst/conf.py
Co-Authored-By: Sandra McCann <samccann@redhat.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
When HostVars are part of the data that goes through (de)serialization
when being passed from a worker process to the main process, its
variable manager reference loses some of its attributes due to the
implementation of __getstate__ and __setstate__ (perf utilization).
Since HostVars already has those attributes, use __setstate__ to assign
them.
Fixes#65365
On Python 2, leave all fds open since there is no mechanism to close specific fds with subprocess.Popen() on Python 2
Add unit tests.
Co-authored-by: Matt Martz <matt@sivel.net>
This change contains fixes for argument spec and respective datatypes.
Created separate PR since these changes might alter the behavior of these modules.
Will need shipit from individual module owner(s).
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Fixes#66549
The inefficiency improvement
https://github.com/ansible/ansible/pull/63713 introduced a bug where
`enablerepo` was not being honored if combined with
`disablerepo="*"`. This fixes that issue.
Signed-off-by: Adam Miller <admiller@redhat.com>
This commit address a problem in static mode, if the hostname change,
`host.name` value with change too. And as a result:
results['dns_config_result'][host.name]
will be initialized.
We now record the initial hostname first, and uses it as the key for the
results.
This commit also ensures hostname and search domain change are detected if
`instance.dnsConfig.dhcp` is true.
Finally, if we target a cluster, the `changed` result will depend on the
number of nodes.
By using a `rescue` block, we can potentially hide a really problem in the
block and return a success. This is a bit problematic for a functional
test.
* Add integration tests
* Handle error in _get_diff_data()
* Change to warning rather than error
* Also change failure to warning in assemble action plugin
* Adding documentation and integration test for the new module
* Correcting typo
* removed unused if else block
* changing error messages
* Addressed review comments
* resolving sanity error
* fixed typo in vmware.py
* Added support for datastore cluster
* adding state parameter instead of power on fixed few more review comments
* Documentation update
* Updating argument
* don't mix " and '
* rename functions that perform changes
* add diff, reorganize _present/_absent functions
* update tests to match function reorg
* add result to _exit_hostvars and removal of possibly sensitive information
* shallow copy dict for proper diff, add check_mode
* set check_mode to False for unit tests
* move _parse_meta back to it's original place
* use get_openstack_vars for diff when not in check_mode
* add changelog fragment
For a refresh of the datastore list on the vcenter, this before we
check the present of the newly attached datastores.
A lot of tests depend on the presence of the DS, and this little
change allow us to save =~ 45s everytime.
- remove the unsupported alias
- refactoring
- move the hosts outside of the cluster to avoid any conflict with DRS
- import the `prepare_vmware_tests` role
- does not work with govcsim
While this does properly pass our testing for ansible/ansible devel
branch, it is currently breaking our collection testing for 2.10.
Specifically, this would mean ansible.netcommon would need to directly
import arista.eos or cisco.nxos collections, causing a circular dependency.
This reverts commit e266e5f8b6.
Some adjustments to be able to run the test-suite properly:
- Starts with the ESXi out of the cluster to be able to deploy the
VM on a proper host consistently
- Ensure the resource pool exists
- The resource pool is called `DC0_C0_RP1`, not `Resources`
- Avoid an exception if we try to move a non existing VM.
Put the test in the zuul/vmware/vcenter_2esxi group.
Bind the govcsim service on port 1443, this way we don't need root privilege
to start the container. This allow us to use ansible-test with podman
seamlessly.
The commit also avoid the mapping of port 80. We don't need it.
* Allow to not read content from file.
* Allow to feed content directly into _info modules.
* Allow to feed non-primary content into openssl_certificate, openssl_csr and openssl_publickey.
* Rename changelog.
* Bugfix of 65367: postgresql_query doesn't support non-ASCII characters in SQL files with Python3
* add changelog
* fix
* change changelog fragment, add example
* new connection plugin aws_ssm
Return code may be at the end of the last command output line.
Marking regex.
Ensure command status code is on it's owm line - last 3 lines are not part of command stdout.
* Adding timeout parameter - aws_ssm_timeout
Default 10 second timeout (https://docs.ansible.com/ansible/2.4/intro_configuration.html#timeout) is marginal.
This avoids changing this default and allowing the SSM timeout to be controlled via inventory.
This change wraps commands so commands which may never return do timeout.
* Added integration tests
Added AWS SSM Executor, target and config functions
Fixed more code for integration tests
Improved execution
Added S3 bucket name
Fixed pylint
Reverted lib changes
Reverted few more changes
Improved support for integration test execution
added ansible role for aws_ssm_integration_test setup and teardown and modifiled runme.sh
Reset to 17fa565 commit
inventory file location changed
change inventory file location
deleted meta and handlers folder as it is not required
deleted main.yml inside vars, removed extra space from tasks/main.yml, Added appropriate tags for ec2 and delete test folder as it is not required
deleted main.yml inside vars, removed extra space from tasks/main.yml, Added appropriate tags for ec2 and delete test folder as it is not required
modified task/main.yml
added region variable and fixed pattern for using variable
modified policy for IAM role
moved to first line of scrip set -eux
Updated Session Manager plugin installation
edited custme policy
Included tags for Session Manager plugin installation
Added README.md
Upddated README and added support for ssm-plugin for Amazon-Linux
Added Windows Integration test support
Improved user data for Linux
Added random value generation for the role and policy,delete vars_to_delete.yml
upadte README in vars
fixed typo
update policy
Updated IAM policy file
update playbook
Updated playbook to include ssm-agent userdata
modified jing2 template
modified jing2 template
modified jing2 template and fixed role deletion
fixed role name issue while deleting and task name
Updated playbook to include wait_connection for ec2
Corrected Synatx changes and updated ssm-plugin debian file
Changed region variable to us-east-1
Removed vars file and updated to /tmp dir
fixed typo
Improved setup
Fixed boto3 dependency
Fixed missing tag
Added boto as dependency as well
Improved execution workflow
Trying other way of defining tags
Fixed undefined var
Changed AMI ID to Amazon Linux
Improved Tags
Ok, created different directory for WIndows test execution
Fixed IAM Role Name for Windows
Fixed inventory not found
Improved integration test execution
Fixed Windows Inventory path
Fixed wrong Windows AMI ID
Fixes issue for windows test execution
* Don't attempt to terminate sessions without a session id.
* Added Unit test cases file for AWS SSM Connection plugin
updated test file with close
updated unit test file with start_session
updated test files
* Eliminate AWS CLI dependency for terminal session.
* Removing unused code, cleanup logic.
Reduce mark length - 52^26 should be plenty
Be explicit about subprocess.Popen options
Simplify if/else for mark end
_stdin_readline is not used now
* updated test file
Added exec command and fixed close session unit tests
updated test files
Improved ansible ssm test command
updated file for lint checks
updated for pylint checks
New Unit_testcases for pre-signed URL file
removing additonal spaces and white spaces
remaning error changes
fixed changes
fixed spaces issues
python 2.7 version and whitespaces
python 2.7 version and whitespaces
python 2.7 skip if
space issue with 16:1
Unit test cases for windows and linux
Unit test cases for windows and linux with issues fixed issues
Unit test cases for windows and linux with issues fixed issues1
* Added support for S3 Pre-signed URLs
* Updated documentation and comments
* Documentation and curl dependency removal for controller machine
Fixing lint errors and removing requirements.
* Adding support for Windows remote EC2 instances.
* Added Encoding fixes
* Updating author section and adding obvious requirement for the SSM agent.
* Refactor stdout post processing
Attempt to get real return code on error (test using ansible -m raw -a 'cmd /c exit 99'.
Fixes problem at terminal width (ansible windows -i ./hosts.yml -m setup).
* Refactor back to a single module.
* Fixed fetch file for windows
* ssm usage examples for linux and windows
* Update aws_ssm.py
Service state corrected.
* Strip line continuation when at terminal width - otherwise replace.
Strip ANSI control sequences only for Windows.
Test playbook:
---
- name: test
hosts: windows
gather_facts: False
vars:
small: 'abc'
tasks:
- name:
set_fact:
large: "{{ lookup('password', '/dev/null length=2000 chars=ascii_letters,digits,hexdigits,punctuation') }}"
- name: small fixed
raw: echo '{{ small }}'
register: small_result
changed_when: False
- name: check
assert:
that:
- "(item | length) == (small | length)"
- "item == small"
msg: "'{{ item | length }} must equal '{{ small | length }}' and '{{ item }}' must match '{{ small }}'"
with_items:
- "{{ small_result.stdout_lines[0] }}"
- name: large random
raw: echo '{{ large }}'
register: large_result
changed_when: False
- name: check
assert:
that:
- "(item | length) == (large | length)"
- "item == large"
msg: "'{{ item | length }} must equal '{{ large | length }}' and '{{ item }}' must match '{{ large }}'"
with_items:
- "{{ large_result.stdout_lines[0] }}"
- name: gather facts
setup:
* Correct module parameter names.
* Updated Windows Executable variable.
Updated Windows Executable variable to "ansible_shell_type".
Fixing Examples with raw declaration
Updated the plugin timeout variable.
* Fix to work with dynamic inventory plug-in
* IntegrationTest template updates
* Removing unsupported flag for integration tests.
Fixing unit test.
* Adding shippable group.
* SSM Usage examples with dynamic inventory plugin
* Fixing yamllint errors.
* Fixed Integration tests
* Fixed Integration tests
* Updates for python3.
Removing python3 restriction.
* Remove python3 restriction.
Change block from retry to always
* Fixed Integration tests with Python 3
* Fixed shellcheck
* Fix for Windows which could pick up end mark prematurely
Move debug to _wrap_command and use a single return point
Single-quotes not needed around linux marks
Fix typo in comment
End mark to new command.
* Unit test cases now works on Python2 and Python3
* Skip tests on Python 2.6
* Fix for wait_for_connection module for windows.
* Updated changes as per review comments
* Fixing broken pipe error seen with session-manager-plugin version 1.1.17.0.
Eliminating sleep as this looks to be fixed in session-manager-plugin version 1.1.17.0.
* Adding back delays for Windows with session-manager-plugin 1.1.17.0.
* Updating Windows AMI ID for integration Test
* Upgrading windows ssm agent to the latest
* Adding boilerplate code.
* Windows ami and integration test updates
* Revert "Windows ami and integration test updates"
This reverts commit cd6ca3579b7cda584bd9c065f9c0835bddb23627.
* Updating windows ami for Integration tests
* Integration test suite updates and fixes.
* Updates and fixes
* Eliminate duplicate processing for exit code on failed command.
* Add powershell wrap.
* Refactor windows post_process.
* AMI Lookup, aliases, OSC filter, test suite updates
Co-authored-by: Gaurav Ashtikar <gau1991@gmail.com>
Co-authored-by: Deepak Choudhary <40276333+deepsvc@users.noreply.github.com>
Co-authored-by: Hanumanth <46720371+hanumantharaomvl@users.noreply.github.com>
Co-authored-by: KUMAR MAYANK <mayank@flux7.com>
* (WIP) nxos_l3_interfaces: fix states, add new minor attributes
* sa cleanup
* more regression fixes
* test_8 for add'l code coverage
* Fix regressions to handle mgmt w/o IP
* add 'no system default switchport' to regression setups
* add err msg to terminal_stderr_re so that cli_config will catch L2 failures
* regression test change: /int4/int3/
* Add default rsvd_intf_len for Zuul CI
* Fix replaced-with-no-ipaddr and ip redirect issues
This was previously removed in PR ##60083 but was added again in
PR #61257. This will cause a lot of things to fail if it is uninstalled
during cleanup after the test role, which is why it was originally removed
from the test.
As AnsibleModule._log_invocation is currently implemented, any parameter
with a name that matches PASSWORD_MATCH triggers the no_log warning as a
precaution against parameters that may contain sensitive data, but have not
been marked as sensitive by the module author.
This patch would allow module authors to explicitly mark the aforementioned
parameters as not sensitive thereby bypassing an erroneous warning message,
while still catching parameters which have not been marked at all by the
author.
Adds tests for various no_log states including True, False, and None (as
extracted by AnsibleModule._log_invocation) when applied to an argument with
a name that matches PASSWORD_MATCH.
Fixes: #49465#64656
* Add final commits
* End of night commit
* Add further tests
* What is this doing here?
* Remove file from commit
* Change os check
* Change version check
This avoids installing dnf-plugins-core, which breaks the yum and dnf modules
when uninstalling packages using a wildcard after they have already been removed.
This should resolve issues with the yum integration tests failing after docker tests run.
This causes multiple unneeded http redirects.
Since each http redirect takes a few hundred
milliseconds, handling the redirects starts to
take up a significant amount of wall time.
Fixes: #63281
Ensure we already create the `test_vm3` VM in the `{{ f0 }}` directory.
This to be sure the teardown playbook will be able to clean it up at the
end. This to avoid problem like this one:
```
Multiple virtual machines with same name [test_vm3] found, Folder value is a required parameter to find uniqueness of the virtual machine
```
* Add test for reboot & wait_for_connection
* Add test for ios
* Collection-proof block test
* Add junos test
* Don't try to evaluate cli context unless using the connection
* Prevent infinite recursion
* Use is_sequence, and Mapping throughout, add support for tuples. Fixes#65722
* Address tests
* Remove unused import
* Add changelog
* Add docstring for clarity
* Argh, linting fix
* Not chasing this rabbit
* wrap_var doesn't return a ref to the original item
* no ref tests
* Remove unused import
* #42292 first draft.
* fix some documentation
* use snake dict keynames
* return metric filter variable
* rework documentation
* try to fix documentation. fix condition when defaultValue is 0. update version string
* fix intenion, syntax and undefined
* try solve state documentation
* define type for state in doc
* rework integration test and fix minor typos
* add defaults for integration test
* update documentation, integration tests, and return value
* fix code formatting and documentation
* irgnore errors in ignore block
* Update lib/ansible/modules/cloud/amazon/cloudwatchlogs_log_group_metric_filter.py
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
* add options to metric_transformation
* fix pep-8
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
* Add support for timeout while waiting for state.
* Allow to limit removal wait time.
* Add changelog.
* Forgot version_added.
* Add some check mode tests.
* Use removal_wait_timeout in tests.
* Parse Healthcheck.StartPeriod properly
* Add changelog fragment
* Use proper markup in changelog
Co-Authored-By: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Fix multiple subnet (of same IP version) idempotence for docker_network.
* Add changelog.
* Unit tests no longer make sense, since the part of the code they test has been removed.
* Re-add CIDR validation. Move it to better position (module setup instead of idempotence check).
* Update changelog.
* Only run new tests on VM test images.
* Actually do what is documented. Especially since an empty object is a valid value for aux_addresses.
* properly document ``upgrade`` option as an alias to ``upgrade_all``, and not a separate option
* Update documentation.
* Update examples.
Fixes: #34696
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fixed issue 64479 with lambda_info module
* Added integration tests for lambda_info module
* Moved lambda_info tests into already existing aws_lamda testsuite for easier test setup.
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
Check that all yaml we ship is parsable by the pyyaml c backend. Since
Ansible uses Pyyaml for docs and playbooks, if the yaml files aren't
parsable, they will error out if they were used.
Warn and skip yamllint if libyaml backend is not present
Ignore new errors in examples until someone can fix them
* Move ec2_vpc_net tests to group1, group2 is running much longer
* Allow for VPC CIDRs to be "associat*ing*", things in AWS are "eventually consistent" and occasionally take longer than we would like
elb_network_lb.py: allow UDP and TCP_UDP protocols
- Fixing documentation
- Add support to UDP and TCP_UDP as described on AWS SDK
elb_target_group.py: allow UDP, TLS, TCP_UDP proto
- Fixing documentation
- Making health checks with response codes and paths only in HTTP/HTTPS
- Allow UDP, TLS, TCP_UDP protocols as described on AWS SDK.
others:
- Added changelog fragments
- Integration test
Fixes: #65265
Signed-off-by: Alexandre Mulatinho <alex@mulatinho.net>
* free strategy - include failed hosts that were notified so --force-handlers is used
* trim line length a bit
* Loop over the force handler tests with the strategies linear and free
* rename changelog
* Use the play iterator instead of TQM for accurate failure representation in blocks
* Remove hack in a backwards compatible way for 3rd party plugins
Some teardown tasks are now enabled only if we have ESXi in the configuration
file. On my system and in a vcenter only scenario, `vcenter_folder` goes from
42.7 to 31.2s.
* Fix copy/pasta for ecs_ecr test names
* Add support for lifecycle policies to ecs_ecr
New feature for ecs_ecr to support [ECR Lifecycle Policies][].
Fixes#32003
[ECR Lifecycle Policies]: https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html
* Improve error message for ecs_ecr parsing errors
Replaces the exception and stack trace with a description of what's
actually going wrong from a user perspective.
* Rename delete policy to purge policy
Marks the `delete_policy` parameter as deprecated, to be removed in
Ansible 2.6.
* Add version_added to purge_policy
* Remove changing results based on verbosity
What I really want is --diff support, and changing results based on
verbosity is abnormal.
* Ensure repository name is lowercase
* Fix deprecation cycle to 4 releases
* Use a YAML anchor for credentials
* Remove filters from assertions
* Add minimal permissions needed
* Updating version_added and deprecation cycle
The original PR sat while a few releases happened.
* Bumping version added and deprecation version
We missed the 2.8 release.
* Removing bare except:
This is not allowed and is generally bad practice.
* Fix lint errors
* update ansible release metadata
* Use the new alias deprecation scheme
This was added in the time the PR has been in development, so rework
things to use it.
* Add test coverage
This makes sure that lifecycle_policy is produced when passed in.
*Also a minor suggestion for simplification from PR.
* Restore changes from 62871 lost in rebase
* Add changelog
* Remove version_added for new purge_policy option
Per sanity test fail.
* set supports_check_mode=True
I don't see any reason why this module cannot run in check mode. The following API calls are made, none of which modify configurations
All using boto3.client('cloudformation')
describe_change_set
describe_stacks
describe_stack_events
get_paginator
get_stack_policy
get_template
list_change_sets
list_stack_resources
* duplicate cloudformation_info tasks with checkmode
Duplicated all existing cloudformation_info tests and added check_mode: yes to them
* delete duplicate empty line
* Move new Ansible cli options '--ask-vault-password' and '--vault-pass-file' to the existing calls to add_argument
* Add changelog fragement
* Change order of ansible cli arguments to use --ask-vault-password and --vault-password-file by default
* Update runme.sh in vault integration tests to test new options --ask-vault-password and --vault-pass-file
Rather than silently processing extra_vars filename without @ sign,
CLI now fails with appropriate warning about requirement.
Fixes: #51857
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Run ec2_instance tests in parallel
* Make sure we spin up the iam_instance_role instances in the dedicated VPC
* Rework a little to only build one VPC while still running in parallel
* Start running the tests within shippable
* We only use setup_remote_tmp_dir for the version_fail tests, so only set it up once
* Make sure we clean up if we fail during Environment Setup
* Add a warning about running in parallel
* Minor review tweaks
* Remove unused file
* cloudformation_info updates
Add ability to retrieve stack change sets
Update module to use AnsibleAWSModule
Update backoff to use new decorator style
Updated exceptions to use fail_json_aws
Converted outputs to snake_case where appropriate, for cloudformation_info only to preserve cloudformation_facts functionality.
Unconverted keys are stack_outputs, stack_parameters, stack_policy,
stack_resources, stack_tags and stack_template
* cloudformation_info updates
Add ability to retrieve stack change sets
Update module to use AnsibleAWSModule
Update backoff to use new decorator style
Updated exceptions to use fail_json_aws
Converted outputs to snake_case where appropriate, for cloudformation_info only to preserve cloudformation_facts functionality.
Unconverted keys are stack_outputs, stack_parameters, stack_policy,
stack_resources, stack_tags and stack_template
* Adding integration tests
* Remove unneeded debug statement
* Add myself to Authors
* rename stack-name to match iam policy limits
aws-terminator permission for cloudformation have the following resource
Resource: arn:aws:cloudformation:us-east-1:{{ aws_account_id }}:stack/ansible-test*
updating test stack name to match
* removing .orig file from rebase
* fix stack name, underscore not allowed
* rename integration testsuite to cloudformation
rename per request
* add resource_prefix to resource names
* prefix stack name with ansible-test
IAM policy in aws-terminator requires the stack name to begin with ansible-test
* add single quotes around variable in test
* fix test for display name
mistakenly updated when changing topic_name to use resource_prefix. The test is to check the display name, which is what the stack update changes
* fix value to display name test
* rename main.yaml to main.yml before rebase
* Merge tests with #65643
remove uneeded files
using tests from #65643 with added check to validate changeset is present in info when requested
* fix assert on chageset check
* remove trailing blank line
This allows to group tests by connections, which allow us to disable
local connection testing in collections for the moment.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* Support username command to config users and edit capabilities
* Add disconnected param to disconnect all session for such user
* Edit yaml examples, add types to docs and edit the version
* Add fullname attribute
* Add no_log to password field and mutually_execlusive relations
* Fix pep8 issue
* remove debug line
* support syslog remote module
* change enabled implementation and edit the tests, also support disable override class
* fix the level order and delete types from choices
* support syslog files
* nested doc fix
* required togother
* fix docs type dict
* add file-debug feature same as files logging
* remove duplicated validation and change dict key access method
* add format of upload url and change the way of getting data from dict
* win_cert_stat initial commit with tests
* documentation fix.
first attempt windows server 2008 compatibility
* add formatted dates
removed debug tests
* make choices generic list
* return a list of certificates
use .net x509 store instead of PS cert provider
* fixed tests file
* fix timestamps returning null
* rename to win_certificate_info
* rename tests win_certificate_info
* return certificates as a sorted array
open the store with readonly privileges
* extensions always returned as an array
* Simply sorting of Windows files below other plugin types
Using the sort method with a custom key function uses less memory than creating multiple lists then joining them.
This seemed to be an acceptable use of a lamdba, even though I geneally try to avoid them.
* Fix sorting of plugins inside of collections
Explicitly sort Windows files below others, mimicking what we do in plugin/loader.py
* Add documentation about ansible.builtin and ansible.legacy
Also document to the two different methods used for searching based on the candidate type.
* Add changelog
* Add integration test
* Update comment with expected sort order
* Fix network action plugin load in collection
Fixes https://github.com/ansible/ansible/issues/65071
* Load network action plugin that matches the module
prefix name from list of collections.
* Update changelog
* Fix unit test
* adding the modules to /nxos/storage/ folder
* fix pep8 stuff
* fixed provider specific information from doc
* Fix integration tests
* Added tested against note
* Review comments
* added assertions for the commands sent in the integration tests
interface.port should be a string to be able to use macros in that
value.
This fixes the case when interface.port is a macro (eg.: "{$MACRO}" and
force=false.
Because, until now, setting the interface.port to an integer was the correct way to work with force=false, a type validation has been added
to that parameter.
Previously, if a string was used for interface.port, it was converted
to an integer, the comparison didn't work (if interface not in interfaces)
and the module tried to register the same interface twice, returning an
error.
Zabbix API manual specifies that only 'main, 'type', 'useip' and 'bulk'
are integers.
https://www.zabbix.com/documentation/current/manual/api/reference/hostinterface/object
Tests are changed to use always str in the interface.port.
Two new tests are added.
The first one is to show that now registering a host with force=false and
a macro in interface.port works.
The other one tests that interfaces defined using string for port are
compared correctly when force=false is used. Previously it was
a comparison between int and str, interfaces were seen as different
and an error was thrown because we were trying to create twice a main
interface.
* Try to kindly convert interface port to the string
As suggested, the previous behaviour could break current configurations. This solution accepts integers and strings.
Co-Authored-By: Dusan Matejka <D3DeFi@users.noreply.github.com>
* adding prefix-list to ios_bgp
* adding prefix-list to ios_bgp
* Fix copy-paste bug in test
* Adding neigbor 10.10.20.20 to global neighbor pool
* Added prefix_list_in and _out to documentation
* Rewrite bgp prefix_list test
* Bugfix in yaml data
* More test, assert neighbor 10.10.20.20
* Fix nxos_file_copy option value path validation
* Modify `local_file`, `local_file_directory` and
`remote_file` option type from `str` to `path`
so that the option value is validated in Ansible
for a legitimate path value
* Fix review comments
* Add tags to ecs_task
remove older reference to credentials
* uncomment shell commands to enable/disable account settings
* Fix documentation and pep8 issues
* fix review items for ecs_task tags
use missing_required_lib for tags
change fail_json message to suggested message
switch from task_tags to tags for consisitency
* Add import for missing_required_lib function
* Tidy put-account-setting tasks and add permission
Using `environment` and `command` rather than `shell` avoids the
need for `no_log` and means that people can fix the problem
* update version added for ecs_task tags
* fix tests after removal of ansible_facts from ecs_service_info, add delay when service is still draining
* Add documentation for sanity tests
* nxos_interfaces: RMB state fixes
* shippable fixes
* Add add'l comments per review
* fix long line
* Fix mode/enabled system defaults handling
* fix N3L test skips
* lint
* test updates for titanium images
* doc fix
* Converted ec2_metric_alarm to boto3. Added treat_missing_data option.
* Handle potentially non-existent alarm keys in ec2_metric_alarm module
* Add treat missing data to ec2_metric_alarms wth some tests
Continues the work of #23407
* Clean up ec2_metric_alarm main test playbook
* fix test suite and sanity checks
* more fixes for sanity tests
* fixes to ec2_metric_alarms requested in code review
* import ClientError from botocore, catch generic ClientError
* more fixes from review
drops extra dict in argument spec and set_facts for aws access
* Fix pep8 blank line issue
* switch to fail_json_aws, add idempotency test
* fix under indented continuation
* remove unsupported alias
* Add group to ec2_metric_alarm aliases
* Put alarm prefix before resource prefix to match aws-terminator pr 63
* Add type for treat_missing_data
User can specify associable object types while creating the categories.
VMware API does not allow to modify associable object types after creation of
category.
Fixes: #61220
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
This should cut down about 1hour worth of test time, due to not looping
over the set_fact functions more then 60 times.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* add module aws_step_functions_state_machine_execution
* AWS step functions tests - Use module defaults
* Return all attributes from aws api calls as ansible task output
* aws_sfn - make start and stop execution idempotent and fix check mode
* aws sfn - use build_full_result method of the paginator
* aws sfn - remove changes made to help with local debugging
There is no longer the need to warn on password, it is not supported any
more.
Update nxos_user tests not to purge current SSH user for nxos, otherwise
we loose access for testing.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Fedora 31 tests are failing due to a missing packages error. I am unable to duplicate this issue when running locally, so it is possibly an issue with a mirror that is being used when run from Shippable
Previously you'd get one of the following behaviours:
- A boto3 error
- Nothing would change
- An error that you're not allowed to change the strategy
So of the bahaviour would depend on the random order that AWS returns the list of all Placement Groups
* Use correct var, move cleanup for async
* Add changelog and tests. Fixes#65393. Fixes#65277.
* Kill off all long running async tasks from listen_ports_facts
* Update task to work with older jinja2
Depends-On: ansible/ansible-zuul-jobs#253
We cannot run the following tests without any ESXi host:
- vmware_drs_rule_info
- vmware_drs_rule_facts
- vmware_host_acceptance
- vmware_host_active_directory
- vmware_host_powerstate
* Don't return module error when mysql_connect fails (#64560)
mysql_user expects an Exception when using check_implicit_admin.
* Adds integration tests for mysql_user check_implicit_admin (#64560)
* win_find - refactor to make more performance and use newer style
* win_find - refactor for performance improvements and alignment to find
* More path alignment to find
* Fix yamllint error
* Add test for print() call in module_utils and modules.
* Add changelog fragment.
* Add ignore.txt entries.
* Use blacklist plugin instead of adding a new.
* Update ignore.txt
* win_auto_logon - check, diff and store pass in LSA
* Ensure baseline keys are set for test
* Skip remove item prop on check mode due to win bug
* Start at a cleared baseline to ensure old LSA secrets are cleared
* add new modul
* correct password var
* fix for linting issues
* add return values documentation
* linting fixes
* Get tests working and simplify a bit
Add the `vmware/scenario/vcenter_only` alias to be able to identify the
vmware test roles that can be tested with just on vcenter instance.
Also, add a check to be sure we don't try to attach ESXi hosts if none
has been defined.
* win_description Module
Module to change Windows description and Windows license owner information.
* LiteralPath updated
changed -path to -LiteralPath in the script
* Version and metadata_version
version_added updated to 2.10
Metadata_version set to 1.1
* version updated
version_added changed to '2.10'
* Changes based on feedback
* removed some redundant checks
* Rename win_description.ps1 to win_computer_description.ps1
* Rename win_description.py to win_computer_description.py
* Module name change
* Integration tests added
* added aliases file
* Change compatibility from 2008 to 2008R2
* Update aliases
Matt Clay