Commit Graph

453 Commits (801eb73ccbd1c294b5916b121bc8051dc1d39a79)

Author SHA1 Message Date
Miroslav Brabenec bc0caad2ad
Fix parsing of device serial number for RHEL8 (#75876)
* Fix parsing of device serial number for RHEL8

* Add tests and changelog

Co-authored-by: Miroslav Brabenec <miroslav.brabenec@memsource.com>
3 years ago
Matt Martz 79e9dae292
Don't show params when there is an issue with `set_option(s)` (#75805) 3 years ago
Matt Martz 96b221b344
Update vendored copy of distro to 1.6.0 (#75732)
* Update vendored copy of distro to 1.6.0

* copy pasta mistake

* ci_complete
3 years ago
Matt Martz 74f5367673
Ensure `heuristic_log_sanitize` returns correct data if no password found (#75570)
* Ensure heuristic_log_sanitize returns correct data if no password found. See #75542

* Need to allow code to flow through
3 years ago
Brian Coca 3ca50a2200
Clarify get_bin_path error message (#75544) 3 years ago
Matt Clay ce6d8a143c
Code cleanup to prepare for pylint update. (#75475)
* user - Remove unused code.
* Replace deprecated abstractproperty decorator.
* Fix __all__ to be a tuple.
* Use a generator in subelements lookup.
* Use from import in basic.py
* Add changelog fragment.
* Fix selinux unit test.
3 years ago
Matt Clay 04009a77e6
Clean up unit tests to prepare for pylint update. (#75473)
* Fix argument name in mocked function.
* Use from import in unit tests.
* Remove unused imports.
3 years ago
Matt Martz 4dca539a29
Expose `unredirected_headers` to `uri` and `get_url` modules (#75308) 3 years ago
Abhijeet Kasurde e7a3715a90
recursive_diff: handle error when parameters are not dict (#74801)
Co-authored-by: Sam Doran <sdoran@redhat.com>
3 years ago
Brian Coca 415e08c297
Try to get best usable locale (#75033)
specially for when you have parameters in unicode but need
  to scrape responses, C is still the fallback

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Abhijeet Kasurde b0ae3f8a8d
test: Unit tests for validation methods (#75061)
* check_required_one_of()
* check_required_by()
* check_required_if()
* check_missing_parameters()

Fixes: #55994

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doran 9a21e24778
get_distribution - Return distribution for all platforms (#71641)
Since moving to distro, it is possible to return this information for all platforms, not just Linux.
Also return version information for all platfrom not just Linux.
Update unit tests.
Remove some duplicate unit tests though I think there are more to remove.

* Fix docstring formatting

* Minor docstring changes

* Mock distro.id for Solaris service test

* Update comment
4 years ago
Brian Coca 5dd8dc8fd0
minor service_mgr facts fixes (#74894)
* minor service_mgr facts fixes

  handle case in which ps command fails or returns empty
  updated tests since it now does keep trying to detect after ps fails
4 years ago
Johannes Heimansberg b023f34f4a
service_mgr: Detect using symlink if proc/1 and ps fail
* runit and other alternative service managers tend to work via symlink so this covers most of em
* Fixes #74866.
4 years ago
Matt Martz 98138584b7
Don't mutate os.environ in AnsibleModule.run_command, make a copy, and pass to Popen (#74791)
* Don't mutate os.environ in AnsibleModule.run_command, make a copy, and pass to Popen. Fixes #74783

* Simplify code a bit

* More simple

* Address some other potentially non threadsafe operations

* Add if around umask

* Address unit test assumptions

* Add clog frag

* yaml syntax issue
4 years ago
Sloane Hertel ee725846f0
ansible-galaxy - increase page size and add retry decorator for throttling (#74240)
* Get available collection versions with page_size=100 for v2 and limit=100 for v3

* Update unit tests for larger page sizes

* Add a generic retry decorator in module_utils/api.py that accepts an Iterable of delays and a callable to determine if an exception inheriting from Exception should be retried

* Use the new decorator to handle Galaxy API rate limiting

* Add unit tests for new retry decorator

* Preserve the decorated function's metadata with functools.wraps

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
4 years ago
Matt Clay d4413dac08 Fix unit tests on Python 3.10. 4 years ago
Ruairidh MacLeod d101009688
facts - set correct OS family for Rocky Linux (#74530) 4 years ago
Matt Clay a30c55f68a Remove deprecated `common.removed` module_util.
Tests have been updated to reflect its removal as well.
4 years ago
Anatoly Pugachev 04e57d28e3
[facts] add hpux fc info, refactor aix and solaris code (#57406)
* correctly parse device from string
* check for command presence before running them
* check for command presence and return code for solaris and aix as well
* add changelog
4 years ago
Logistic Bot 7099657dd7
Basic: don't call lstat when check_mode (#64279)
While mode is specified in check_mode, don't call lstat.
Since file may not present.

Fixes: #61185
4 years ago
Rick Elrod fcbf5c3185
[module_utils] make to_bits actually return (#74409)
Change:
- Instead of returning the `str` type, return the value that was
  calculated.

Test Plan:
- New unit tests.

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Fernando Correia 7fd989d008
Detect Homebrew on Mac M1 (Apple Silicon) (#74378)
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ

On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
4 years ago
Matt Clay 9ba6cf9a72 Relocate misplaced unit tests. 4 years ago
Matt Martz 17237c1d88
Deprecate Py2.6 support for target nodes (#74165) 4 years ago
Amin Vakil 0c101f3f76
Add epoch_int in date_time facts (#73822)
* Add unit test

* Add changelog
4 years ago
Sam Doran abacf6a108
Use ArgumentSpecValidator in AnsibleModule (#73703)
* Begin using ArgumentSpecValidator in AnsibleModule

* Add check parameters to ArgumentSpecValidator

Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.

* Make errors related to unsupported parameters match existing behavior

Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.

* Set default values after performining checks

* FIx sanity test failure

* Use correct parameters when checking sub options

* Use a dict when iterating over check functions

Referencing by key names makes things a bit more readable IMO.

* Fix bug in comparison for sub options evaluation

* Add options_context to check functions

This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.

* Fix bug in apply_defaults behavior of sub spec validation

* Accept options_conext in get_unsupported_parameters()

If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.

* Build path to the unsupported parameter for error messages.

* Remove unused import

* Update recursive finder test

* Skip if running in check mode

This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.

This is a silght change in behavior, but I believe the correct one.

Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.

* The great purge

Remove all methods related to argument spec validation from AnsibleModule

* Keep _name and kind in the caller and out of the validator

This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.

* Double moustaches work

I wasn't sure if they get stripped or not. Looks like they do. Neat trick.

* Add changelog

* Update unsupported parameter test

The error message changed to include name and kind.

* Remove unused import

* Add better documentation for ArgumentSpecValidator class

* Fix example

* Few more docs fixes

* Mark required and mutually exclusive attributes as private

* Mark validate functions as private

* Reorganize functions in validation.py

* Remove unused imports in basic.py related to argument spec validation

* Create errors is module_utils

We have errors in lib/ansible/errors/ but those cannot be used by modules.

* Update recursive finder test

* Move errors to file rather than __init__.py

* Change ArgumentSpecValidator.validate() interface

Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.

Return the validated parameters if validation is successful rather than True/False.

Update docs and tests.

* Get attribute in loop so that the attribute name can also be used as a parameter

* Shorten line

* Update calling code in AnsibleModule for new validator interface

* Update calling code in validate_argument_spec based in new validation interface

* Base custom exception class off of Exception

* Call the __init__ method of the base Exception class to populate args

* Ensure no_log values are always updated

* Make custom exceptions more hierarchical

This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.

* Just go back to basing off Exception

* Return ValidationResult object on successful validation

Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.

* Make it easier to get error objects from AnsibleValidationResultMultiple

This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.

* Define custom exception for each type of validation failure

These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.

* Update tests based on (yet another) interface change

* Mark several more functions as private

These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.

* Move warnings and deprecations to result object

Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.

* Use subclass for module arg spec validation

The subclass uses global warning and deprecations feature

* Fix up docs

* Remove legal_inputs munging from _handle_aliases()

This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.

Create a private function just for getting legal inputs since that is done in a couple of places.

It may make sense store that on the ValidationResult object.

* Increase test coverage

* Remove unnecessary conditional

ci_complete

* Mark warnings and deprecations as private in the ValidationResult

They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.

* Mark valid_parameter_names as private and populate it during initialization

* Use a global for storing the list of additonal checks to perform

This list is used by the main validate method as well as the sub spec validation.
4 years ago
Abhijeet Kasurde 3811fddede Amazon: Fix distribution facts for older release
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.

Fixes: #73946

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Abhijeet Kasurde fb66b4ffbc
distribution: Add Amazon Linux distribution facts (#73767)
* Update Amazon Linux Distribution facts gathering logic
* Update tests

Fixes: #73742

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doran 2377a0a776
Improve ArgumentSpecValidator unit tests (#73642)
* Add more scenarios to basic valid testing
* Update invalid tests
* Fix test for Python 2
* Condense data
* Add tests for missing required and invalid-elements
* Update aliases tests
* Add invalid scenarios for aliases
* Add tests for _add_error() method
* Fix sanity test failure
4 years ago
Sam Doran b6811dfb61
Add argument spec validator (#73335)
Add argument spec validator class
4 years ago
Matt Davis 4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
4 years ago
Sam Doran 2f5c83dfb1
Add AlmaLinux to the family of Red Hat-like operating systems (#73541)
* Add changelog and fixtures for AlmaLinux support

Co-authored-by: Christoph Schug <com+github@schug.net>
4 years ago
Rick Elrod 9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros (#73445)
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes #73084

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Alexander Sowitzki 218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set (#17029) (#73439)
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
4 years ago
Alexander Sowitzki e6da544310 Let `command` always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
4 years ago
Sam Doran df451636e7
facts - properly report virtualization facts for Linux guests on bhyve (#73204) 4 years ago
Rick Elrod 7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Alexander Sowitzki 6459fbb7bc Make `test_prepare_multipart` succeed on non Debian distros
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
4 years ago
Rick Elrod 20509b6507
[facts] fix version facts on FreeBSD RC/PRERELEASE (#73020)
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
  -RELEASE, -STABLE, or -CURRENT.

Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
  version of FreeBSD.

Tickets:
- Fixes #72331

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Brian Coca 932ba36160
ensure unsafe writes fallback (#70722)
* Ensure we actually fallback to unsafe_writes when set to true

 add integration test
 add fix for get_url not passing the parameter from args
4 years ago
Pilou b464d18fd1
AnsibleModule.set_mode_if_different: handle symlink is in a sticky directory (#45198)
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory

Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
4 years ago
Abhijeet Kasurde a223ea5185
distribution - handle NetBSD OS Family (#70799)
Fixes: #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
4 years ago
Sam Doran e889b1063f
arg_spec - rework _check_arguments() (#72447)
* Move _syslog_facitily to __init__
  No good reason it should not be set for each object

* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
  Change the behavior to return a set, either empty or populated, with unsupported keys.
  Accept legal_inputs as optional which will not required calling handle_aliases before calling
  get_unsupported_parameters().

* Add changelog
* Rework function behavior and documentation
  I realized I missed the original intent of this method when moving it to a function. It
  is meant to compared the parameter keys to legal inputs always, not compare
  parameter keys to argument spec keys, even though the argument spec keys should
  be a subset of legal inputs.

* Add tests
* Fix typo.
* Set internal properties when handling suboptions
4 years ago
Alexei Znamensky 5654de6fce
Return error if cwd directory does not exist (#72390)
* Return warning or error if cwd directory does not exist, in AnsibleModule.run_command()
4 years ago
Jordan Borean caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
4 years ago
Rick Elrod 709484969c
sysctl/openbsd fact fixes (#72070)
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`

Test Plan:
- New unit tests

Tickets:
- Fixes #71968
- Refs #72025
- Refs #72067

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
4 years ago
Christian Loos ea119d3089
fix distribution fact for SLES4SAP (#71559)
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.

This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.

The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
4 years ago
Jorge Vallecillo 94522b7c10
Fix typos (#71970) 4 years ago
Rick Elrod 5cd489af06
Let get_file_attributes() work without `lsattr -v` (#71845)
* Let get_file_attributes() work without `lsattr -v`

Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
  in some cases, it may not.
- The function now takes an optional include_version bool parameter,
  which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
  it returns, we now call it with include_version=False.

Test Plan:
- New unit tests

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago