Commit Graph

10324 Commits (e5cc12a64f274e36738fe51d4caa533003ed626b)

Author SHA1 Message Date
Riyad Preukschas e5cc12a64f
blockinfile - Preserve line endings (#66461)
Fixes #64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
5 years ago
Abhijeet Kasurde 564907d8ac
distribution: Refactor test_distribution_version testcases (#69300) 5 years ago
Rick Elrod 304c3e57e8
[ssh] Add new sshpass_prompt option (#68874)
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.

Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.

Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.

Tickets:
Fixes #34722, fixes #54743, refs #11565.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Felix Fontein ea04e0048d
Allow to deprecate options and aliases by date (#68177)
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in #67684.

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
5 years ago
Matt Davis fa81cc6a0c
fix delegated loading when path is not a directory (#69713)
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
5 years ago
Matt Clay 5fc01c0a1d
Use a custom build container on Shippable. (#69660)
* Use a custom build container on Shippable.

This supports key generation before git_sync, to avoid issues with pre-migration PRs.

If the node pool is switched to another version, a matching build container should be built and used.

* Don't kill our own custom container.
5 years ago
Rick Elrod 4aff87770e
[yum] Make package removal confirmation strict (#69592)
* [yum] Make package removal confirmation strict

Change:
After removing packages, the yum module does a final check to ensure the
packages are really installed. The check would include packages that
were RPM `Provides:` values of another package.

This means that, for example, if a third-party kernel RPM spec had
`Provides: kernel` in it, removing the stock kernel would be successful
but the check to see if it was really removed would fail and cause
Ansible to report a failure.

Test Plan:
Tested on local CentOS 7 VM with kernel from elrepo which is known to
`Provides: kernel`.

Tickets:
Fixes #69237
Refs #35672
Refs #40723

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Davis f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
5 years ago
Nilashish Chakraborty dae3ba71a8
Add FQCN mappings for network facts modules (#69601)
* Add FQCN mappings for network facts modules

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add unit tests

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* update entry for ironware

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
5 years ago
Rick Elrod fe9696be52
Fix `ansible -K` become_pass regression (#69629)
* Fix `ansible -K` become_pass regression

Change:
- This fixes a breaking change introduced in
  2165f9ac40

Test Plan:
- Local VM for now, with plans to add an integration test for -K going
  forward.

Tickets:
Refs #69244
5 years ago
Brian Coca de3f7c7739
fix delegated interpreter discovery (#69604)
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
 as it stands we rerun discovery for the delegated host
unless its saving facts to itself
 * fixed test lacking delegate_to mock
5 years ago
James Cammarata a4072ad0e9
Split regular and handler results into their own queues (#69498)
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.

Fixes #69457
5 years ago
Sam Doran cd8920af99
file - return 'state': 'absent' when a file does not exist (#66503)
This was changed in ansible/ansible#51350.
5 years ago
Matt Martz e0f25a2b1f
Strip no log values from module response keys. Fixes #68400 (#69653) 5 years ago
Martin Krizek 1ee70fc272
ansible.utils.vars.isidentifier improvements (#58278)
ci_complete
5 years ago
Brian Coca 9281148b62
correctly merge multiple facts results (#68987)
* correctly merge multiple facts results

  fixes #68532
5 years ago
Sam Doran fe941a4045 Pin Docker version at 19.03.1 5 years ago
Matt Clay 8fe812b212
Remove conflicting plugin from test/support/ dir. (#69628) 5 years ago
Matt Martz b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
5 years ago
Martin Krizek ecd986006e
Properly handle unicode in safe_eval (#68576)
* Properly handle unicode in safe_eval

Fixes #66943

* Update lib/ansible/template/safe_eval.py

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Sam Doran 889da811d7
Pin docker-ce-cli version in tests (#69620)
Installing docker-ce has a dependency of docker-ce-cli. If the version of docker-ci-cli is not specified, it installs the latest version.
5 years ago
Brian Coca 06ecdaa7b1
comment heavy playbook (#68981)
* create examples/play.yml


Co-authored-by: flowerysong <junk+github@flowerysong.com>
5 years ago
Brian Coca df4e83deda
added 'task timeout' feature (#69284)
* added 'task timeout' feature


Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Sam Doran f01de15d25
ansible-galaxy - remove warning during collection install (#69541)
* ansible-galaxy - remove warning during collection install

If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.

A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.

* Update unit tests
5 years ago
Felix Fontein 43acd61901
ansible-test local change detection: use --base-branch if specified (#69508) 5 years ago
Jordan Borean ecea15c508
Unify ansible-galaxy install -r (#67843)
* Unify ansible-galaxy install -r

* Minor nit fixes for docs

* Re-align warnings

* Fix up integration test

* Fix up test where no roles/collections were in file
5 years ago
Matt Clay 01e7915b0a Remove obsolete incidental_azure_rm_webapp test. 5 years ago
Matt Clay c495c92a6e Code cleanup in ansible-test. 5 years ago
Matt Clay 159d79995c Freeze pycodestyle in ansible-test. 5 years ago
Matt Clay e7c2eb519b
Add Shippable request signing to ansible-test. (#69526) 5 years ago
Lukas Pirl 34db57a47f
introduce fact "ansible_processor_nproc": number of usable vcpus (#66569)
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.

* Add fixture data and update unit tests

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Matt Clay 83381cd912 Update the default-test-container in ansible-test.
The updated container includes fewer requirements now that the collection migration has completed.

Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
5 years ago
Matt Clay d0484fbf74 Cleanup for Python 3.9 and pycodestyle compat. 5 years ago
Matt Clay d8e0aadc0d Update ansible-test support for CI providers.
Refactored CI provider code to simplify multiple provider support and addition of new providers.
5 years ago
Matt Clay ca6da5ad63 Remove obsolete incidental test. 5 years ago
Brian Coca 2165f9ac40
fix delegation vars usage (debug still shows inventory_hostname) (#69244)
* fix delegation vars usage and reporting

 - just pass delegated host vars + task vars to plugins
   and avoid poluting with original host vars
 - updated tests
5 years ago
Sloane Hertel 55e29a1464
ansible-galaxy - fall back to galaxy.yml when listing collections (#68925)
Fallback to galaxy.yml for listing collections that don't have a FILES.json or MANIFEST.json

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Sloane Hertel 343ffaa18b
better error for "ansible-galaxy collection verify" if there's no MANIFEST.json (#67498)
* Add a better error for "ansible-galaxy verify" if the MANIFEST.json has been deleted from the installed collection or if the collection hasn't been installed via normal means

* Fix unit tests for the remote collection

If there's something wrong with the local collection's version it will fail before the remote collection is sought

* Add a test for the new error msg

* Prevent the duplicate warning

Mock the new isfile call where needed

* Update lib/ansible/galaxy/collection.py

Co-Authored-By: Martin Krizek <martin.krizek@gmail.com>

Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
5 years ago
Toshio Kuratomi 25c5388fde Update compile skip; all release and docs build scripts require 3.6+ 5 years ago
Bob Weinand f200487414
Fix filedescriptor out of range in select() when running commands (#65058)
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
5 years ago
Brian Coca 87d9b49de2
Fix listing of colleciton plugins with symlinks (#69305)
* Fix listing of colleciton plugins with symlinks
5 years ago
Brian Coca cedfe34619
added unvault lookup plugin (#69087)
* added unvault lookup plugin
5 years ago
Matt Martz 4c4406b2df
Flatten the directory hierarchy of modules (#68966)
* Flatten the directory hierarchy of modules

* Update ignore.txt, flatten units

* Update imports

* Completely flatten the modules directory

* Update sanity ignore

* Fix some sanity test ignores

* Fix relative import

* Fix docs builds without category

* ci_complete

* Clean up docs. ci_complete

* Adjust needs/file alias

* ci_complete

* fix hardcoded ping module paths

Co-authored-by: Matt Davis <mrd@redhat.com>
5 years ago
Felix Fontein 0e15375ffe
Add deprecated removed_in_version and deprecated_aliases version tests (#66920) 5 years ago
Brian Coca 0aa76503dc
avoid fatal tb on bad fqcn callback name (#69440) 5 years ago
Matt Davis 776f4840fc
fix `ansible-test units` to work(ish) under podman (#69462)
* ignore the missing `Networks` key, issue a warning that network disconnection won't function
5 years ago
Mark Goddard aa36b02ede
Fix fileglob plugin with non-existent subdirectory (#69451)
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:

lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL

The exact error depends on Python 2 or 3, but here is the error on
Python 2:

    AttributeError: 'NoneType' object has no attribute 'endswith'

And on Python 3:

    TypeError: expected str, bytes or os.PathLike object, not NoneType

This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().

Fixes: #69450
5 years ago
Martin Krizek 9645304da3
Validate args for includes in handlers too (#57537) 5 years ago
Felix Fontein 5b9418c06c
Fix SemanticVersion comparison (#69395)
* Fix SemanticVersion comparison.

* Complete tests for _Alpha and _Numeric comparators.

* Linting, and add comment.
5 years ago
Rick Elrod de59b17c7f
Add Fedora 32 to CI (#69222)
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.

Test Plan:
CI

Tickets:
Fixes #69230

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago