Prior to this commit, it was impossible to use a module like dnf with a
URL that contains a username with an @ such as an email address
username, because:
dnf:
name: https://foo@example.com:bar@example.com/some.rpm
Would cause netloc parsing to fail. However, the following:
dnf:
name: https://foo%40example.com:bar@example.com/some.rpm
Would also fail because ansible would *not* URL-decode the credentials,
causing the following to be base64 encoded in the Authorization header:
Zm9vJTQwZXhhbXBsZS5jb206YmFyCg==
Which decodes to:
foo%40example.com:foo
Which is *not* the authorized username, and as such, *won't* pass basic
auth.
With this commit, Ansible's url lib behaves like curl, chromium, wget,
etc, and encodes the above to:
Zm9vQGV4YW1wbGUuY29tOmJhcgo=
Which decodes to:
foo@example.com:bar
Which will actually pass the HTTP Basic Auth, and is the same behaviour
that you will find ie. with:
curl -vvI https://foo%40bar:test@example.com 2>&1 |grep Auth | awk '{ print $4 }'
This patch removes an import fallback that was only executed under
Python 2. Now that we don't run tests against that runtime, it
generates an uncovered line. Dropping it will slightly increase the
coverage metric as a side effect.
Adds an option that can have an action plugin tell the module to ignore
options that do not fit its arg spec. This is to enable support for core
running modules that exist outside of the collection that may not be new
enough to support some of the options supplied to it.
* Back out use of communicate, add better comments, add bufsize, and align with subprocess._communicate
* tests
* re-order logic slightly
* more comments
* loopty loop
* yet another comment
* Revert "yet another comment"
This reverts commit 96cd8ada5fa0441b92f2298bdaa6cb40594847d2.
* Revert "loopty loop"
This reverts commit 96ea066f6a7d18902c04a14f18dd79b38e56f5e7.
* ci_complete
* Copy in comment too
* Wording updates
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Back out bufsize
---------
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Remove datetime.datetime.utcfromtimestamp and datetime.datetime.uctnow
from controller code since they are deprecated in Python 3.12.
* Update target side code to use new utcfromtimestamp and utcnow utils in ansible.module_utils.compat.datetime that return aware datetime objects on Python 2.7 and 3.
Co-authored-by: Matt Clay <matt@mystile.com>
* Add test for symbolic to octal when others is omitted
Add case when there should be no permissions for other.
And specific permissions for owner and group.
* Fix permissions test by explicitly setting no permissions for others
* Add additional cases where multiple permissions are specified
* Fix detection of available hashlib algorithms
Detection of hashlib algorithms now works on Python 3.x.
The new implementation works on Python 2.7 and later.
Test coverage is provided by both integration and unit tests.
* Add additional details about hashlib in docs
* Update `collections.abc` imports
- Use `six.moves` for modules and module_utils
- Use `collections.abc` for controller code
This avoids using `ansible.module_utils.common._collections_compat`,
which was added before the vendored `six` was updated to provide these
imports.
* Update _collections_compat to use six.moves
Also update the custom pylint rule to reflect this change.
* Normalize deprecation records.
* Fix alias deprecations in suboptions.
* Report in which option an alias warning happened for suboptions.
* Add deprecation tests for suboptions.
* Also test deprecation in list of dicts.
* Adjust unit tests for toplevel alias deprecation field name change.
After changes:
```
"ansible_locally_reachable_ips": {
"ipv4": [
"127.0.0.0/8",
"127.0.0.1",
"192.168.0.1",
"192.168.1.0/24"
],
"ipv6": [
"::1",
"fe80::2eea:7fff:feca:fe68",
...
]
},
```
192.168.1.0/24 is a local prefix, where any IP address inside this range
is reachable locally (or outside this host if this prefix is announced via
EGP/IGP).
Signed-off-by: Donatas Abraitis <donatas.abraitis@hostinger.com>
* Add OSMC to Debian OS_FAMILY_MAP
- os_family fact of the Debian-based OSMC distribution was not detected correctly
* tweak changelog
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
* Fix distro fact handling for Flatcar
The existence of the file /etc/flatcar/update.conf depends on
bootstrap configuration typically provided by the user. For that
reason this file is unsuitable for determining distro facts for
Flatcar Container Linux.
The distribution_release fact is meaningless in the case of Flatcar
since Flatcar doesn't have named releases. The distribution_version
fact, however, IS meaningful and should contain a number such as
"3139.2.0".
- Use /etc/os-release instead of /etc/flatcar/update.conf.
- Drop the distribution_release fact.
- Set the distribution_version fact.
- Update distro test fixture for Flatcar
- Generate the fixture using gen_distribution_version_testcase.py.
- Override result.distribution and result.os_family manually as the
generator script gives wrong values.
- Use a recent Flatcar version.
Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
* Report OpenSuSE >= 15 as opensuse
Make distro.id() report newer versions of OpenSuSE (at least >=15) also report
as opensuse. They report themselves as opensuse-leap.
* Add a test
- `processor_count` was erroneously set to the number of cores
- `processor_cores` was erroneously set to the number of threads per core
- `processor_vcpus` and `processor_threads_per_core` were not set
- `processor` was a string, while it's supposed to be a list
Before:
```
"ansible_processor": "PowerPC_POWER7",
"ansible_processor_cores": 4,
"ansible_processor_count": 12,
```
After:
```
"ansible_processor": [
"PowerPC_POWER7"
],
"ansible_processor_cores": 12,
"ansible_processor_count": 1,
"ansible_processor_threads_per_core": 4,
"ansible_processor_vcpus": 48,
```
Also add a unit test.
Co-authored-by: Baptiste Jonglez <git@bitsofnetworks.org>
When looking up the `no_log` setting for a parameter that is an alias in
`AnsibleModule._log_invocation()`, the alias value will always be an
empty dictionary since `self.aliases` on the `AnsibleModule` instance is
never updated after initialization. Since the `no_log` setting is on the
canonical parameter not the alias, an incorrect warning is issued if the
parameter matches `PASSWORD_MATCH`.
This PR returns the aliases dictionary as an attribute of the
`ValidationResult` and updates the `aliases` attribute on the
`AnsibleModule` instance.
* user - Remove unused code.
* Replace deprecated abstractproperty decorator.
* Fix __all__ to be a tuple.
* Use a generator in subelements lookup.
* Use from import in basic.py
* Add changelog fragment.
* Fix selinux unit test.
specially for when you have parameters in unicode but need
to scrape responses, C is still the fallback
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Since moving to distro, it is possible to return this information for all platforms, not just Linux.
Also return version information for all platfrom not just Linux.
Update unit tests.
Remove some duplicate unit tests though I think there are more to remove.
* Fix docstring formatting
* Minor docstring changes
* Mock distro.id for Solaris service test
* Update comment
* minor service_mgr facts fixes
handle case in which ps command fails or returns empty
updated tests since it now does keep trying to detect after ps fails
* Don't mutate os.environ in AnsibleModule.run_command, make a copy, and pass to Popen. Fixes#74783
* Simplify code a bit
* More simple
* Address some other potentially non threadsafe operations
* Add if around umask
* Address unit test assumptions
* Add clog frag
* yaml syntax issue
* Get available collection versions with page_size=100 for v2 and limit=100 for v3
* Update unit tests for larger page sizes
* Add a generic retry decorator in module_utils/api.py that accepts an Iterable of delays and a callable to determine if an exception inheriting from Exception should be retried
* Use the new decorator to handle Galaxy API rate limiting
* Add unit tests for new retry decorator
* Preserve the decorated function's metadata with functools.wraps
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* correctly parse device from string
* check for command presence before running them
* check for command presence and return code for solaris and aix as well
* add changelog
Change:
- Instead of returning the `str` type, return the value that was
calculated.
Test Plan:
- New unit tests.
Signed-off-by: Rick Elrod <rick@elrod.me>
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ
On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
* Begin using ArgumentSpecValidator in AnsibleModule
* Add check parameters to ArgumentSpecValidator
Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.
* Make errors related to unsupported parameters match existing behavior
Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.
* Set default values after performining checks
* FIx sanity test failure
* Use correct parameters when checking sub options
* Use a dict when iterating over check functions
Referencing by key names makes things a bit more readable IMO.
* Fix bug in comparison for sub options evaluation
* Add options_context to check functions
This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.
* Fix bug in apply_defaults behavior of sub spec validation
* Accept options_conext in get_unsupported_parameters()
If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.
* Build path to the unsupported parameter for error messages.
* Remove unused import
* Update recursive finder test
* Skip if running in check mode
This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.
This is a silght change in behavior, but I believe the correct one.
Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.
* The great purge
Remove all methods related to argument spec validation from AnsibleModule
* Keep _name and kind in the caller and out of the validator
This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.
* Double moustaches work
I wasn't sure if they get stripped or not. Looks like they do. Neat trick.
* Add changelog
* Update unsupported parameter test
The error message changed to include name and kind.
* Remove unused import
* Add better documentation for ArgumentSpecValidator class
* Fix example
* Few more docs fixes
* Mark required and mutually exclusive attributes as private
* Mark validate functions as private
* Reorganize functions in validation.py
* Remove unused imports in basic.py related to argument spec validation
* Create errors is module_utils
We have errors in lib/ansible/errors/ but those cannot be used by modules.
* Update recursive finder test
* Move errors to file rather than __init__.py
* Change ArgumentSpecValidator.validate() interface
Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.
Return the validated parameters if validation is successful rather than True/False.
Update docs and tests.
* Get attribute in loop so that the attribute name can also be used as a parameter
* Shorten line
* Update calling code in AnsibleModule for new validator interface
* Update calling code in validate_argument_spec based in new validation interface
* Base custom exception class off of Exception
* Call the __init__ method of the base Exception class to populate args
* Ensure no_log values are always updated
* Make custom exceptions more hierarchical
This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.
* Just go back to basing off Exception
* Return ValidationResult object on successful validation
Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.
* Make it easier to get error objects from AnsibleValidationResultMultiple
This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.
* Define custom exception for each type of validation failure
These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.
* Update tests based on (yet another) interface change
* Mark several more functions as private
These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.
* Move warnings and deprecations to result object
Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.
* Use subclass for module arg spec validation
The subclass uses global warning and deprecations feature
* Fix up docs
* Remove legal_inputs munging from _handle_aliases()
This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.
Create a private function just for getting legal inputs since that is done in a couple of places.
It may make sense store that on the ValidationResult object.
* Increase test coverage
* Remove unnecessary conditional
ci_complete
* Mark warnings and deprecations as private in the ValidationResult
They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.
* Mark valid_parameter_names as private and populate it during initialization
* Use a global for storing the list of additonal checks to perform
This list is used by the main validate method as well as the sub spec validation.
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.
Fixes: #73946
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add more scenarios to basic valid testing
* Update invalid tests
* Fix test for Python 2
* Condense data
* Add tests for missing required and invalid-elements
* Update aliases tests
* Add invalid scenarios for aliases
* Add tests for _add_error() method
* Fix sanity test failure
Orion Poplawski