Fixes#64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.
Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.
Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.
Tickets:
Fixes#34722, fixes#54743, refs #11565.
Signed-off-by: Rick Elrod <rick@elrod.me>
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
as it stands we rerun discovery for the delegated host
unless its saving facts to itself
* fixed test lacking delegate_to mock
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.
Fixes#69457
* Add multipart/form-data functionality
* Fix some linting issues
* Fix error message
* Allow filename to be provided with content
* Add integration test
* Update examples
* General improvements to multipart handling
* Use prepare_multipart for galaxy collection publish
* Properly account for py2 vs py3, ensuring no max header length
* Address test assumptions
* Add unit tests
* Add changelog
* Ensure to use CRLF instead of NL
* Ignore line-endings in fixture
* Consolidate code, add comment
* Bump fallaxy container version
* ci_complete
* ansible-galaxy - remove warning during collection install
If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.
A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.
* Update unit tests
* Unify ansible-galaxy install -r
* Minor nit fixes for docs
* Re-align warnings
* Fix up integration test
* Fix up test where no roles/collections were in file
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.
* Add fixture data and update unit tests
Co-authored-by: Sam Doran <sdoran@redhat.com>
The updated container includes fewer requirements now that the collection migration has completed.
Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
* Fix filedescriptor out of range in select() when running commands
* Simplify the run_command() code
Now that we're using selectors in run_command(), we can simplify some of
the code.
* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
instead.
Also use a simpler conditional check of whether the selector map is
empty
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
* only show_vars when showing vars
avoid processing function params that can be very expensive
and might not be used at all in called function.
fixes#69357
* Update changelogs/fragments/69357_optimize_inventory_graph_wo_vars.yml
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:
lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL
The exact error depends on Python 2 or 3, but here is the error on
Python 2:
AttributeError: 'NoneType' object has no attribute 'endswith'
And on Python 3:
TypeError: expected str, bytes or os.PathLike object, not NoneType
This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().
Fixes: #69450
* Fix galaxy publish sha256 value format.
The multipart/form content used for the body
of the POST to /api/automation-hub/v3/collections
was missing a newline before the line with the value
of the sha256.
automation-hub/galaxy/django skips the field entirely in
that case and automation-hub code will use None for default
to indicate that no sha256 is provided (an available option).
Fixesansible/galaxy-dev#246
* Add changelog fragment
Co-authored-by: Matt Martz <matt@sivel.net>
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.
Test Plan:
CI
Tickets:
Fixes#69230
Co-authored-by: Matt Clay <matt@mystile.com>
* validate-modules: deprecated modules in collections
In Collections a module is marked as deprecated via meta/routing.yml
Use this file, rather than the leading `_` as part of the deprecated
test.
* Correct variable
* review comments
* indentation
* Read routing.yml only once
* pep8
* Apply suggestions from code review
Co-authored-by: Matt Clay <matt@mystile.com>
* review: remove duplicated conditional
Co-authored-by: Matt Clay <matt@mystile.com>
* use orderdict for yaml dictionaries
* clog
* SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
* allow user to toggle odict off
* removed config, since requored to load config
* remove unused import
* Update changelogs/fragments/yaml_orderd_mappings.yml
Co-authored-by: Matt Clay <matt@mystile.com>
* Update lib/ansible/parsing/yaml/objects.py
Co-authored-by: Matt Clay <matt@mystile.com>
* Update lib/ansible/parsing/yaml/objects.py
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'
Fixes: #69286
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Metadata defaults were not being set if only a few fields were missing.
* ansible-doc with no documentation and no status in metadata should
return empty, just like if there was no documentation and no metadata
at all.
* Address FIXME's in sysctl.py
* Added changelog fragment
* Updated check
* Update conditions
* if not instead of is None
* Restore and delete FIXME comments
* Do not pass file mode during recursive copy on symlink files.
The 'file' module cannot deal with mode=preserve. Do not pass that
mode to the module when 'preserve' is used.
* Fix changelog fragment filename
Add the ability to pass allowerasing to alter the dnf transaction
behavior.
Fixes#24161
This is effectively a port of the original pull request from
poettler-ric who has since abandoned the PR
https://github.com/ansible/ansible/pull/34111
Signed-off-by: Adam Miller <admiller@redhat.com>
Change:
Extend the logic for custom error handling in the dnf module, so that on
newer DNF (such as DNF that ships with modern Fedora 31 container
images, and ships with RHEL 8.2) we report errors consistently with
older DNF.
Test Plan:
Ran dnf integration tests against an old Fedora 31 container image and a
brand new Fedora 32 container image; tess passed on both.
Signed-off-by: Rick Elrod <rick@elrod.me>
With inventory script migrated to their respective collection,
redirect links in documentation to their respective collection
location.
Fixes: #69139
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Make sure collection is a list if a str is given
* Call field validation early on collections
Because we are doing work on modifying the collections value before
it is actually validated, we can validate it ourselves early to make
sure the user supplies either a string or list. Dicts are not valid.
The new validation allows us to simplify the _ensure_default_collection()
function. And since the field is now static, we no longer need to specify
a default for it, which also allows us to simplify the function. Since
the default is now removed, we can also remove the sanity/ignore.txt entry
for collectionsearch.py.
New unit tests are added (and the existing one modified) that allow us to
make sure that we throw a parser error if a user specifies something other
than a string or list for the collections value everywhere it can be specified.
* Revert removing the collection default
The default is actually used, so restore it.
* Fix unit tests in test_helpers.py affected by early collection validation
The call to daemonize() in sysvinit.py was missing the module parameter included in the function definition in service.py.
This pull request simply adds that parameter, as the module is used for error handling in daemonize().
* service_facts: fix for systemd 245
Since systemd 245, `systemctl list-unit-files` comes with a new column
"VENDOR PRESET" [1] and breaks the service_facts module:
$ ansible localhost -m service_facts
localhost | FAILED! => {
"changed": false,
"msg": "Malformed output discovered from systemd list-unit-files: auditd.service disabled disabled "
}
This patch drops the third column to make it work with old and new
systemd. With the new slice operation, IndexError instead of ValueError
is raised if the output contains less than 2 columns.
Test plan: running `ansible-test integration -v service_facts` on
up-to-date Arch Linux
[1] https://github.com/systemd/systemd/pull/14445
* add changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
Openstack dynamic inventory script is moved from community.general to
openstack.cloud, adjust the bot meta accordingly.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Currently if virtualenv_command has arguments, then the
search for the binary in the path does not work so the
user has to specify the full path to it.
To allow arguments to be used without having to specify
the path to the binary, we split the module argument into
the command and anything after the first space.
This makes using this module argument more flexible and
user friendly.
Fixes: #52275
Change:
Rather than hardcoding .pyo and .pyc, filter on all BLACKLIST_EXTS in
the non-legacy logic of PluginLoader (_find_fq_plugin). The two harcoded
extensions are part of BLACKLIST_EXTS already and this simply adds the
rest of the blacklisted extensions to the check.
In addition, check .endswith() instead of an exact match of the suffix,
like everywhere else that uses BLACKLIST_EXTS. This allows for
blacklisting, for example, emacs's backup files which can appear after
any extension, leading to things like `foo.py~`.
Test Plan:
Ran `ansible-playbook` against a collection where a `foo.py~` module was
getting executed instead of `foo.py` which also appeared in the same
directory. `foo.py~` is no longer executed.
Tickets:
Fixes#22268
Refs #27235
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow a collection role to call a standalone role by default. Fixes#69101
* tweaked changelog text
* Guard against NoneType
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
* update ActionBase._low_level_execute_command to honor executable
* adding changelog fragment
* renaming changelog fragment to .yml
* noop change to bump shippable
* adding raw_executable integration test
* copying aliases from raw
* removing blank lines
* skipping aix and freebsd
* noop to bump shippable
* moving tests to raw/
* removing become_method: sudo ; it doesn't work on AIX
* removing trailing blank line
* forcing become_method: su to try to get AIX to work
Co-authored-by: Rob Wagner <rob.wagner@sas.com>
Change:
This corrects an incorrect CVE identifier in the changelog entry for
CVE-2020-1735.
Test Plan:
N/A
Tickets:
Refs #67793, #68720
Signed-off-by: Rick Elrod <rick@elrod.me>
This patch fixes the issue where nested Block copies were created from
incorrect Block object. This resulted in nested Blocks data like ``name``
or ``_uuid`` to contain values from the Block the filter_tagged_tasks
method was called on.
* Testing: Add CentOS Linux On Power platform
* Add arch designation to remotes.
This avoids overloading the provider with the arch.
Also add a changelog entry.
Co-authored-by: Matt Clay <matt@mystile.com>
* ansible-galaxy - fix listing specific role
If the role was not in the first search path, it was reported as not found
* Properly display role description
Default to description to top level description, falling back to the description from within galaxy_info
* Display proper message when a role does not exist
* Add integration tests
* Use context manager
* BSD and macOS ruining all the fun
* also consolidated temp dir name generation, added pid for more 'uniqness'
* generalize error message
* added notes about remote expansion
CVE-2020-1733
fixes#67791
* Force collections to be static
Templating of collection names does not work at all. Force them to
be static so that a warning is generated for the user.
* Add collectionsearch unit test and fix for reviews
New unit test validates the new _load_collections() code and moves
the new check to the end of the method.
* Change unit test to pytest
* Adjust unit test to use capsys instead of monkeypatch
* Fix pep8 error
* Add changelog fragment
Closes#68704
* subversion module - provide password securely with svn command line option --password-from-stdin when possible, and provide a warning otherwise.
* Update lib/ansible/modules/source_control/subversion.py.
* Add a test.
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Don't trigger full CI run for changes to changelogs/ and docs/ in collections.
* Add changelog fragment.
* Update changelogs/fragments/68550-ansible-test-docs-changelogs.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Update docker.txt to use the OpenSUSE 15.1 container image
Signed-off-by: Rick Elrod <rick@elrod.me>
* handle installing mysql on suse
Signed-off-by: Rick Elrod <rick@elrod.me>
* add changelog fragment
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update changelogs/fragments/ansible-test-opensuse-15.1.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Update tests to use RHEL 7.8.
Keeping support for RHEL 7.6 since collections are still using it.
* Fix tests for RHEL 7.7+ due to extras repo name change.
* fixed fetch traversal from slurp
* ignore slurp result for dest
* fixed naming when source is relative
* fixed bug in local connection plugin
* added tests with fake slurp
* moved existing role tests into runme.sh
* normalized on action excepts
* moved dest transform down to when needed
* added is_subpath check
* fixed bug in local connection
fixes#67793
CVE-2019-3828
* Allow tasks to notify a fqcn handler name
* Add tests. Fixes#68181
* Add changelog fragment
* Add test to ensure handlers are deduped properly with fqcn, role, and just handler names
* Add some docs about new special vars
fial_json() requires a message be given to it to inform the end user of
why the module failed. Prior to this commit, the message had to be a
keyword argument:
module.fail_json(msg='Failed due to error')
Since this is a required parameter, this commit allows the message to be
given as a positional argument instead:
module.fail_json('Failed due to an error')
* Always set the discovered interpreter on the delegated host. Fixes#63180
* Make code a little more generic
* Move code into a function
* Implement some changes based on reviews
* Add changelog fragment
* when possible, use filedescriptors from mkstemp to avoid race
* when using path strings, ensure we are always creating the file
CVE-2020-1740
Fixes#67798
Co-authored-by: samdoran
The last task in a play should now properly report code coverage.
This change should also eliminate empty coverage files, as well as incomplete coverage files resulting from early worker termination.
* Fix test_check_mutually_exclusive exception-checking
Asserting inside of the `with` context of `pytest.raises`
doesn't actually have any effect. So we move the assert
out, using the exception that gets placed into the scope
after we leave the context, and ensure that it actually gets
checked.
This is also what the pytest documentation says to do:
https://docs.pytest.org/en/latest/assert.html#assertions-about-expected-exceptions
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add some tests for check_required_together
Signed-off-by: Rick Elrod <rick@elrod.me>
* use to_native instead of str, for consistency
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add newlines for pep8
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add tests for check_required_arguments
Signed-off-by: Rick Elrod <rick@elrod.me>
* Sort missing keys in error message, since hashes are unsorted and this can be random
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add changelog entry
Signed-off-by: Rick Elrod <rick@elrod.me>
* fix vault tmpe file handling
* use local temp dir instead of system temp
* ensure each worker clears dataloader temp files
* added test for dangling temp files
* added notes to data loader
CVE-2020-10685
* Remove some unreachable code in the file module
Remove some cases in file.py which are covered by conditionals a few
lines earlier. Remove the duplicate code which will never be hit.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Restore incidental file coverage from timezone module
Signed-off-by: Rick Elrod <rick@elrod.me>
* Combine two conditionals, add a changelog entry
Signed-off-by: Rick Elrod <rick@elrod.me>
* Make new test syntax consistent, add two stat tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Support pre-releases via new SemanticVersion. Fixes#64905
* Don't treat buildmeta as prerelease
* Don't inherit from str and int
* Add helper method to try and construct a SemanticVersion from a LooseVersion
* Don't count major 0 as pre-release, it's different
* Guard against invalid or no version in LooseVersion
* return a bool
* Add integration tests for pre-release
* Fix up lingering issues with comparisons
* typo fix
* Always allow pre-releases in verify
* Move pre-release filtering into CollectionRequirement, add messaging when a collection only contains pre-releases
* Update changelog
* If explicit requirement allow pre releases
* Enable pre-releases for tar installs, and collections already installed when they are pre-releases
* Drop --pre-release alias, make arg name more clear
* Simplify code into a single line
* Remove build metadata precedence, add some comments, and is_stable helper
* Improve from_loose_version
* Increase test coverage
* linting fix
* Update changelog
* remove azure extras and extras_require support
* Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now.
* Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history.
* Mark azure-requirements as orhpaned.
This keeps the docs around so that existing links from old test runs remain valid.
Co-authored-by: Matt Clay <matt@mystile.com>
Now empty `*.py` files are ignored during module_utils import analysis for change detection.
This eliminates "No imports found" warnings for files which should have no imports.
* Address compat issue for collection loading on py26
* Move import_module shim to utils for compat across the codebase
* Enable collection tests on py2.6
* Update changelog fragment
* Simplify code using sys.moduls
* Move compat to module_utils/compat/importlib
* Add back errantly deleted newline
* Remove hack comment
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* win_timezone - Allow for _dstoff timezones
* Update win_timezone-Allow-dstoff.yml
* Added doc entry for new format
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* ansible-galaxy - optimise some paths and use fake galaxy int tests
* Added init, built, and publish tests
* Test against both mocked Galaxy and AH server
* Finish off writing the install tests
* Fix up broken tests
* Rename test target and add migrated tests
* Use cloud provider for Galaxy implementation
* Added blank static config
* Use correct alias group
* Set release version and fix copy typo
* Remove reset step as it is no longer needed
* Use sane env var names for test container name
* Run modules on windows container
This provides an ability to run Powershell modules on windows container via docker connection. Otherwise, Ansible tries to run python modules on windows containers and fails.
* Removing whitespace in the blank lines
* Adding a changelog fragment
* fix:git module ignores remote_tmp
* WIP: added the changelog fragment and edit comment
* Revert "WIP: added the changelog fragment and edit comment"
This reverts commit 2f739df619.
* WIP: added changelog fragments
* WIP: added changelog fragment
* WIP: fixed spelling in changelog fragment
* reworked sqs_queue
* Switch default purge_tags behaviour to false.
This matches the behaviour of ec2_tag and ecs_tag.
* Minor lint / review fixups
* Add missing AWS IAM policy for SQS tests
* Move integration tests to using module_defaults: group/aws:...
* add changelog
* Break out the 'compatability' map from our spec definition (gets flagged by the schema validation)
* Tweaks based on review
* add basic examples
* Lint fixups
* Switch out NonExistentQueue logic so it's easier to follow
* Reorder name argument options for consistency
Co-authored-by: Dennis Podkovyrin <dennis.podkovyrin@gmail.com>
* Allow passing through of (almost) all params available on boto methods in aws_api_gateway
* Linting and docs fixes
* Refactored method signature of create_deployment() to use keyword args instead of named args
* Updated version_added flags to 2.10
* Cleanup and improve aws_api__gateway integration test play. Also included new params into test.
* Fixed RETURN docs and some ttests
* Completed RETURN docs and made integration tests match
* Fixed variable names in test and YAML syntax in docs
* Comment out critical sections of integration test
* Fixed update test after figuring out what the error message means. Also updated error message to be more descriptive.
* Fixed test assertion
* Update docs and make tests reflect that endpoint type wont be changed on updates
* Syntax fix
* Add changelog fragment
* Improve aws_api_gateway docs, fix typos.
* Quote doc lines with colon
* Cleanup tests
* Auto-Retry on ResourceNotFound and RequestInProgress exceptions
* Use AnsibleModule options for required_if logic
* changelog
* Remove (now) duplicate RequestInProgressException catching
* Allow a single retry when attempting to fetch the information about a cert directly after deleting it.
There is a small chance that it goes away while we pull the details.
* add key rotation option
* add changelog fragment
* provide version added as string
* change changelog to minor_changes
* Update changelogs/fragments/67651-aws-kms-key-rotation.yml
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* prevent key upgrade if key rotation was enabled manually. In that case, the key rotation would be disabled, if not mentioned in the playbook
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
Co-authored-by: Mark Chappell <mchappel@redhat.com>
Handle NoneType error occured due to accessing host system service info
in vmware_host_service_info module.
Fixes: #67615
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* fixed#47050
* added changelog fragment
* added quick and basic test
* Revert "added quick and basic test"
This reverts commit 75f4141656.
* added better tests
* now also creating files to copy on the remote
* removed tests for recursive copying which is not supported by remote_src
* sns_topic: (integration tests) Move the tests over to using module defaults
* sns_topic: (integration tests) Add test for behaviour of changed when using delivery_policy
* sns_topic: ensure "changed" behaves properly when managing delivery policies
- a delivery_policy isn't an IAM policy, so compare_policies didn't cope with it
- AWS automatically adds an additional option when you set an HTTP delivery
policy
* Parse the delivery policies so we can test the changes properly
* Remove dead code key_matches_filter/key_matches_filters
* Fail more cleanly when we don't recognise the 'shape' of KMS policy
* Refactor aws_kms to bring down the complexity
* Minor docs tweaks
* Changelog fragment
* Fixups from review
* cloudfront_distribution: (integration tests) Migrate to using module_defaults
* cloudfront_distribution: (integration tests) Use the ID rather than the alias
Using aliases requires providing a valid SSL certificate, as such we're not longer able to test using an arbitrary hostname
* cloudfront_distribution: (integration tests) Make sure we delete the test s3 bucket when tests fail
* cloudfront_distribution: field_level_encryption_id is now a mandatory field always add it
Setting the field to an empty string has the same effect as the original behaviour.
* Copy & Paste fixup
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
* ec2_tag - Deprecate the use of state=list
* Update lib/ansible/modules/cloud/amazon/ec2_tag.py
* Add changelog and porting_guide entries
Co-authored-by: flowerysong <junk+github@flowerysong.com>
* required_if checks should have three or four parts.
* Validate mutually_exclusive, required_together, required_one_of, required_if and required_by.
* Simplify code.
* Improve messages.
* Add changelog.
* Sanity check.
* Update docs.
* Update ignore.txt.
* Don't continue with tests when terms are not strings.
* Remove ignore.txt entry.
* Make sure validate-modules doesn't choke on things already flagged by schema test.
* Check required_if requirements list for strings.
* added omit_date option for template dump
omit_date=yes removed the date field in the exported template
* Update zabbix_template.py
- ansibot fixes
- better function call handling
* Update zabbix_template_info.py
* Create 67302-zabbix_template_info-add-omit_date-field
* Rename 67302-zabbix_template_info-add-omit_date-field to 67302-zabbix_template_info-add-omit_date-field.yml
* Allow botocore configuration to be configurable for boto3 modules
* Allow modification of the boto user agent
* play nicely with modules that might be modifying config
* changelog
* 'message' parameter is replaced by 'commit_message' in grafana_dashboard
* 'message' parameter is replaced by 'notification_message' in datadog_monitor
This change is required since 'message' as parameter name is used internally by
Ansible core engine.
Fixes: #39295#45362#47132#59617
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* facts virtual: detect KVM when product_name is 'KVM Server'
* added changelog fragment for "facts virtual: detect KVM when product_name is 'KVM Server'"
* Add multi-domain forest Support
cloned extra_args so there is no check for credentials needed.
Fixed Formatting
added missing extra_args to pure state
* minor Fixes
do not clone $extra_member_args again
do not overide $name
better description
* added Changelog
fixed typo in Documentation
* [aws_s3] fix deleting the current objects and the previous versions from a version-enabled bucket
* use existing paginated_list function to keep compatibility with the other places it is called
* changelog
* Add noteworthy change to the porting guide
* Reword that with acozine's suggestion
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Fix DHCP support in win_dns_client + more
* Fix bugs and test failures, add changelog fragment
* Add idempotency tests for DHCP
* Address review feedback; dedup address-family code
* Remove legacy function
* Remove old reference
User can now specify tag and category using dict in vmware_tag_manager
module. This is useful when tag or category name contains colon.
Fixes: #65765
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Start adding ansible-galaxy collection list options
* Working list all collections and list a specific collection
* Nuke debugging cruft
* Use to_text to get a string of the FQCN for sorting
* Improve collection output formatting
- add header
- display collection name and version in separate columns
- width of columns is dynamic based on collection name and version length
* Make role list output match collection output list
- add header
- add columns for role name and version
- make column width dynamic based on name and version length
* remove debug statemnt and extra header
* Revert "Make role list output match collection output list"
This reverts commit a0b3db47bb3b198aafd34c1f1be5b6561af2f928.
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Use validate_collection_path
* Do not warn if a specific collection in found in any search path
* Fix extraneous warning and remove duplicate code
Do not warn when listing a specific collection and it does not exist
in other collection paths.
Restructure the code that loops through collection paths to remove
duplicate conditional code.
* Indicate role path was found
* Use new function name
* WIP Save Point
* Use separate functions for role and collection list
* Wrap error message
There may be a better way to do this besides hard coding a line break, but this
does make the message a lot more readable.
* Add validate_collection_path function (#66441)
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Fix bad syntax
* Correct docstring
* Bikeshed the names
* Properly list a single role
* Simplify _display_warnings()
Only display warnings. Move exception raise back to each caller.
* Move private methods to private functions
They don't need self, so it makes sense to have them as functions
Get rid of _display_warnings() function since it doesn't do anything worthy of
an independent function.
* Add integration tests for ansible-galaxy collection list
* Fix docs sanity test
* Fix bug where ansible_collections dir does not exist
The path may exist, but if there is no ansible_collections dir inside that path,
an exception was raised in find_existing_collections().
Add integration test for this scenario
* Put execute_list() method back
* Add some informational messages for debugging
* Add unit tests
Units tests for the various private methods in support of collection list
* Start adding unit tests for test_execute_list
* Display collection path when listing specific collection
* Add unit tests for listing all collections and specific collection
- Create fixture for creating test objects
- Add function for controlling os.path.isdir results
* Set defaults for minimum collection widths
Ensure that collections with small FQCNs display correctly.
Add unit tests
* Split up unit tests and fix fixtures
Add more fixtures for mocking objects during the specific collection tests
* Change help message for -p in list subcommand
Give accurate description of what it actually does rather than trying to use language shared between sub commands.
* Disable colorized output in unit test
* Add docs for collection list
* Fix integration test on macOS
The temp file path is really long on macOS, so the warning message gets wrapped
across multiple lines. That make seth grep fail. Switch to matching on a smaller
part of the warning.
* Recreate common path options for collections
Improve help about what the '-p' option does and how it works.
* Remove unnecessary elif after continue statements
* Account for duplicate paths in collections_searh_paths
If someone specifies the same path via '-p' that is the COLLECTIONS_PATHS,
do not list the collections twice.
* Docs updates
* [WIP] Add verify subcommand command to 'ansible-galaxy collection'
* Fix pep8 and use consistent display order
* WIP - docs
* Remove some redundancy in verify display messages by using an error queue for each collection
* Share common code and improve output format
* clean up documentation
* typo
* Apply suggestions from code review
Co-Authored-By: Sandra McCann <samccann@redhat.com>
* Move ModifiedContent namedtuple to the global scope
Add a public metadata property
Rename function to _get_json_from_tar_file
* Add some unit tests
* fix using common functions after rebase
* changelog
* Improve efficiency finding specific installed collections
Improve efficiency by only downloading the tar.gz from the galaxy server for comparison after checking that the collection has been installed
Handle multiple collection paths
Fix up tests
* pep8
* reword that for accuracy
* use more common code and verify collection name
* Improve error message and add documentation
* Update unit tests and add coverage for both local and remote collections that are unable to be located
* Only validate collections using the format collection_namespace.collection_name
Fix tests to reflect that
Fix documentation
* Fix a bug when a local collection does not contain a file
* pep8
* Fix formatting
Co-authored-by: Sandra McCann <samccann@redhat.com>
* win_package - Refactor with msp, appx support
* Added msi test for ALLUSERS
* Added some msix tests, refactored tests
* Added remaining msix tests
* Enable msix sideloading for tests
* Added remaining exe path tests
* Added basic msp tests
* Remove url options now the util no longer has them
* Fix file version check for older Windows hosts
* Remove no_proxy ansible-test setting
* Use same mechanism of become to copy the file with explicit creds
* AnsibleAWSModule related cleanup - redshift
* Apply a backoff on modify_cluster to cope with concurrent operations
* Add AWS 'hacking' policy to allow creation of Redshift ServiceRole
* Adding the retry policies makes the redshift test suite more reliable
* Make validate-modules stop ignore FILE_COMMON_ARGUMENTS.
* Add types to FILE_COMMON_ARGUMENTS and update document fragment to match it.
* Update ignore.txt.
* Add changelog.
* Clean up FILE_COMMON_ARGUMENTS.
* postgresql_pg_hba doesn't declare the backup option.
* uri doesn't declare the remote_src option.
* Add documentation.
* maven_artifact seems to use directory_mode, which it doesn't declare.
* Update changelogs/fragments/66389-file-common-arguments.yml
Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst
ci_complete
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
* Refactor coverage file enumeration.
* Relocate sanitize_filename function.
* Support sets when writing JSON files.
* Generalize setting of info_stderr mode.
* Split out coverage path checking.
* Split out collection regex logic.
* Improve sanitize_filename type hints and docs.
* Clean up coverage erase command.
* Fix docs and type hints for initialize_coverage.
* Update type hints on CoverageConfig.
* Split out logic for finding modules.
* Split out arc enumeration.
* Split out powershell coverage enumeration.
* Raise verbosity level of empty coverage warnings.
* Add code coverage target analysis to ansible-test.
The edgeos_config module had a list of commands to filter out to avoid
load failures. This list had a single regular expression which caught
commands that attempted to set pre-encrypted passwords. This behavior is
undesirable for a few reasons.
* It's poorly documented. The documentation makes cryptic mention of a
return value that some commands might be filtered out, but offers no
explanation as to what they are or why.
* It's hard-coded. There's no way for the user to change or disable this
functionality, rendering the commands caught by that expression
completely unusable with the edgeos_config module.
* The obvious workaround is unsafe. The filter catches passwords that
are already encrypted, but is perfectly fine letting the user set
plain-text passwords. EdgeOS will encrypt them upon commit, but this
module encourages unsafe handling of secrets up to that point.
* It's a security vulnerability if the user doesn't know about this
behavior. While the module will warn if commands are filtered, the
user won't know what got filtered out until after the fact, and may
easily miss that warning if they are not vigilant. For something as
sensitive as setting a password, it's not hard to imagine naive use of
this module resulting in incorrect credentials being deployed.
* It provides no discernible benefit. Using the module without filtering
does not result in load failures. If those commands are indeed harmful
for some reason on (old?) versions of EdgeOS, it should be incumbent
upon the user to be scrupulous in what commands they issue, rather
than the module maintaining a blacklist of possible ways the user
might misuse their own system.
* "openssl_certificate - Add option for changing which ACME directory to use with acme-tiny. Set the default ACME directory to Let's Encrypt instead of using acme-tiny's default. (acme-tiny also uses Let's Encrypt at the time being, so no action should neccessary.)"
* "openssl_certificate - Change the required version of acme-tiny to >= 4.0.0"
self._discovered_interpreter_key is None unless a previous iteration
has attempted discovery. In that case, force re-discovery, as the
previous attempt certainly failed.
* Fix UNC path joining in the powershell shell plugin, add test
* Remove testy bits and a redundant line
* Fix style nits
* Update to use os.ntpath
* Add changelog for #66604
* Fix#63077
If the package is already installed the stdout is not as expected by this function. Either remove `--needed` or just noop if we detect pacman returning. We cannot match the stdout string, as that is most likely localized.
```
[root@archBook user]# /usr/bin/pacman --upgrade --noconfirm --noprogressbar --needed /srv/aur/src/i3cat-git/i3cat-git-r38.c6d29dd-1-x86_64.pkg.tar.xz
loading packages...
warning: i3cat-git-r38.c6d29dd-1 is up to date -- skipping
there is nothing to do
```
* Add comment
Add comment
* Add changelog fragment.
Co-authored-by: Felix Fontein <felix@fontein.de>
* aws_netapp_cvs_snapshots - minor required_if fixup (state must be set if state=present)
* ec2 - fix typo in mutually_exclusive definition
* rds_instance: fix typo in mutually_exclusive restore_to_time should be restore_time - currently throws a boto error
This change moves all code for the `ansible-test coverage` command into the `coverage` directory.
Each subcommand is split into a separate file.
Only minor spelling changes were made aside from code relocation.
This makes it behave in a more idiomatic way
* Fix bug in Darwin facts for free memory
If the vm_stat command is not found, fact gathering would fail with an unhelpful
error message. Handle this gracefully and return a default value for free memory.
* Add unit tests
Fixed sphinx theme to navigate "Edit on Github" link to locate correct
plugin, cli source in GitHub repo.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Add 'force_basic_auth' option
By default "force_basic_auth" option is set to False, with this adjustment it is up to the user if it wants to enforce basic authentication.
PR #66898
This change introduces a new sanity check with code
`parameter-state-invalid-choice` in the `ansible-test sanity`
validator. It enforces modules not to support `list` or `info`
as their `state`.
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-Authored-By: Felix Fontein <felix@fontein.de>
* Fixes#66478
* When no quantity is set, then candlepin server usually uses
default value 1. When more quantities are required, then
candlepin server can automatically choose correct minimal
value.
* Move warn() and deprecate() methods out of basic.py
* Use _global_warnings and _global_deprications and create accessor functions
- This lays the foundation for future functions being moved outside of AnsibleModule
that need an interface to warnings and deprecations without modifying them.
* Add unit tests for new warn and deprecate functions
Add integration test
There are a number of other parameters that result in stack traces as well when this module is used ad-hoc. I'm not sure if we're interested in fixing them all since this module isn't meant to be run ad-hoc.
I consistently reach a timeout with the current (10s) timeout. This
commit increases the default value to 30s, this should simplify the life
of our users.
Also, the documentation was incorrect. The value is in second, not minute.
* enable new update rule to delete missing linked templates
New update rule is available from 4.0.16 and 4.4.4 up. Add check for version and enable new update rule.
fixes#66720
* adding fragment file
* Update zabbix_template.py
* Update zabbix_template.py
* make sure current_pass_hash is a string before using it in comparison
* add changelogs/fragments file
* fix changelogs/fragments file: it is actually a bugfix, not a minor change
* throttle tests: fix detection of parallel execution
The test wasn't able to detect if too many workers were running.
On my laptop:
- without this change, the 'throttle' target takes ~20 seconds
- with this change, the 'throttle' target takes ~70 seconds
- 1 second isn't long enough to encounter the issue
* Fix throttle test when strategy is 'free' based
'free' strategy allows multiple tasks to be executed in parallel: use
one 'throttledir' per task.
Use 'linear' strategy with a dedicated play for cleanup/setup tasks
* throttle: reset worker idx before queuing a new task
* TestStrategyBase: define task.throttle
otherwise '1' will be used instead of the default value due to the
following expression being equal to '1':
int(templar.template(task_mock.throttle))
Co-authored-by: James Cammarata <jimi@sngx.net>
* Add new configuration parameter to make this timeout configurable
* Rename keys to be more correct and reformat exception message for whitespace handling
* Rename config away from default prefix. Add vars element and associated changes to support.
* Update 65001-allow_configuring_async_startup_timeout.yml
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
* Fixing errors on empty linked templates
fixes#66417
Line 409 fixes KeyError if existing template has no linked templates - as Zabbix API isn't returning an empty array either.
Line 417 is needed to mark as changed if new/modified template has no linked but existing one has.
Line 442 is needed to actually update even if link_templates was None.
* decode parsed xml root
fixes#66466
Line 533: without .decode() the ET.tostring on python3 returns byte-like object
* Create 66463-zabbix_template-fix-error-linktemplate-and-importdump.yml
* Update changelogs/fragments/66463-zabbix_template-fix-error-linktemplate-and-importdump.yml
Co-Authored-By: Dusan Matejka <D3DeFi@users.noreply.github.com>
* Update changelogs/fragments/66463-zabbix_template-fix-error-linktemplate-and-importdump.yml
Co-Authored-By: Dusan Matejka <D3DeFi@users.noreply.github.com>
Co-authored-by: Dusan Matejka <D3DeFi@users.noreply.github.com>
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Fix bad syntax
* Correct docstring
When HostVars are part of the data that goes through (de)serialization
when being passed from a worker process to the main process, its
variable manager reference loses some of its attributes due to the
implementation of __getstate__ and __setstate__ (perf utilization).
Since HostVars already has those attributes, use __setstate__ to assign
them.
Fixes#65365
Allow the built in commands for checking boot time to be overridden. This is mainly for containers and other odd systems where it would add too much complexity to the plugin code to try and discern the correct command to use.
On Python 2, leave all fds open since there is no mechanism to close specific fds with subprocess.Popen() on Python 2
Add unit tests.
Co-authored-by: Matt Martz <matt@sivel.net>
This change contains fixes for argument spec and respective datatypes.
Created separate PR since these changes might alter the behavior of these modules.
Will need shipit from individual module owner(s).
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Fixes#66549
The inefficiency improvement
https://github.com/ansible/ansible/pull/63713 introduced a bug where
`enablerepo` was not being honored if combined with
`disablerepo="*"`. This fixes that issue.
Signed-off-by: Adam Miller <admiller@redhat.com>
* Add integration tests
* Handle error in _get_diff_data()
* Change to warning rather than error
* Also change failure to warning in assemble action plugin
Redact GitLab Project variables which might include sensetive information
such as password, api_keys and other project related details.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* don't mix " and '
* rename functions that perform changes
* add diff, reorganize _present/_absent functions
* update tests to match function reorg
* add result to _exit_hostvars and removal of possibly sensitive information
* shallow copy dict for proper diff, add check_mode
* set check_mode to False for unit tests
* move _parse_meta back to it's original place
* use get_openstack_vars for diff when not in check_mode
* add changelog fragment
* remove the line that is suppressing the output being shown when running
terraform from ansible
* Address #56934 and #57044
* added changelog for bug fix for missing terraform output
Co-authored-by: Adam <adam.lemanski@gmail.com>
* RoleRequirement - Include stderr in the error message if there's a non-0 return code
* Don't try to concatenate str and bytes
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Deprecate Windows 2008, and 2008 R2
* Remove shippable nodes
* Update windows_faq.rst
Be less specific about 2008/R2 timeframes
* Update setup.ps1
tweak warning text
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
* Update ec2_vol.py
When `name` is specified, it ignores `zone` filter. That is you could not have same tag:Name in different zones.
* Add changelog fragment
* Add collection support with local for supported network platforms
* The legacy behavior of network action plugins using persistent
the framework is to override the connection=local with the actual connection type (network_cli/netconf/httpapi) based
on the value of transport option in provider.
* After the actual connection plugin is identified try to load it from
`ansible.netcommon` collection, if it fails load it from ansible
core. This is done to work with Ansible 2.10 and also maintain backward compatibility for
Ansible 2.9 version to ensure it works with network collections.
* To support this need to pass collection_list value to connection
plugins. As in case of connection=local it loads `persistent`
connection within action plugin which in turn invokes `network_cli`
or `netconf` connection. Similarly `network_cli` connection plugin invokes
`terminal` and `cliconf plugins and `netconf` connection plugin invokes `netconf`
`plugins` based on value on `ansible_network_os`.
* Add deprecation warning for connection=local support and
provider support for applicable platforms
* Fix CI failures
* Allow to not read content from file.
* Allow to feed content directly into _info modules.
* Allow to feed non-primary content into openssl_certificate, openssl_csr and openssl_publickey.
* Rename changelog.
* Bugfix of 65367: postgresql_query doesn't support non-ASCII characters in SQL files with Python3
* add changelog
* fix
* change changelog fragment, add example
As AnsibleModule._log_invocation is currently implemented, any parameter
with a name that matches PASSWORD_MATCH triggers the no_log warning as a
precaution against parameters that may contain sensitive data, but have not
been marked as sensitive by the module author.
This patch would allow module authors to explicitly mark the aforementioned
parameters as not sensitive thereby bypassing an erroneous warning message,
while still catching parameters which have not been marked at all by the
author.
Adds tests for various no_log states including True, False, and None (as
extracted by AnsibleModule._log_invocation) when applied to an argument with
a name that matches PASSWORD_MATCH.
Fixes: #49465#64656
* Modify the parent class for net_put and net_get
module to refer from `ActionModule` class in
network.py action plugin which contains the
common code for network plugins.
* Replace `_handle_template` with `_handle_src_option`
function name which is defined in common class.
* virtual facts: /proc/<pid>/environ is NULL-separated
* non-empty "container" env is a guest container
* This is consistent with `systemd-detect-virt --container`
* fact utils: line_sep option: specify delim for split
* allow using the same function to split null-delimited objects
Since 4.0.0 there is a field proxy_address for limiting proxy requests.
Official Docs: "Proxy address If specified then active proxy requests are only accepted from this list of comma-delimited IP addresses, optionally in CIDR notation, or DNS names of active Zabbix proxy.
This field is only available if an active proxy is selected in the Proxy mode field. Macros are not supported.
This option is supported since Zabbix 4.0.0."
updated documentation/help
fixed test issues
fixed version_added not a string
fix suggestions by D3DeFi
added changelog fragment
applied D3DeFi suggestions
way cleaner and better readable
Updates the user module to support pull request #64733. Neither the
update_password or password_lock field contains sensitive information, so
mark them as such.
* Use is_sequence, and Mapping throughout, add support for tuples. Fixes#65722
* Address tests
* Remove unused import
* Add changelog
* Add docstring for clarity
* Argh, linting fix
* Not chasing this rabbit
* wrap_var doesn't return a ref to the original item
* no ref tests
* Remove unused import
* #65993 - update restart policy (restart policy & restart retries) without restarting the container
* - proper indentation on the continuation-line
- set restart_policy to the correct value independent from the api version
* - move restart_policy definitions into the if block
- add a new variable for the restart_policy configuration value
* add changelog fragment
* typo; minus -> underscore
* rename changelog fragment to contain the correct module name
* rename restart_policy_config_value to just restart_policy and refer to the correct dict values
* Add support for timeout while waiting for state.
* Allow to limit removal wait time.
* Add changelog.
* Forgot version_added.
* Add some check mode tests.
* Use removal_wait_timeout in tests.
* Parse Healthcheck.StartPeriod properly
* Add changelog fragment
* Use proper markup in changelog
Co-Authored-By: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Allow to inspect containers directly.
* Wait for containers to be removed before recreating them.
* Also wait for containers to be removed before creating them.
* Add changelog.
* Fix multiple subnet (of same IP version) idempotence for docker_network.
* Add changelog.
* Unit tests no longer make sense, since the part of the code they test has been removed.
* Re-add CIDR validation. Move it to better position (module setup instead of idempotence check).
* Update changelog.
* Only run new tests on VM test images.
* Actually do what is documented. Especially since an empty object is a valid value for aux_addresses.
In addition to signing update queries also use the TSIG key to sign
lookup queries. By doing that we allow a hidden master to not only to
be looked down network wise, but also TSIG wise.
A bonus benefit of threating update queries and lookup queries more
the same is that will allow for all queries to be refactored into a
shared helper method. Currently we have a bit too much duplicated code
within the module.
Riyad Preukschas