25 Commits (f4b2d4d4bb70d688d437ca79962d8d651b6e731c)

Author	SHA1	Message	Date
Matt Clay	2595b42aca	Prefer unittest.mock over mock. (#77886)	3 years ago
Sam Doran	1b947eaf92	arg_spec - Return aliases in validation result and update aliases (#77576) ... When looking up the `no_log` setting for a parameter that is an alias in `AnsibleModule._log_invocation()`, the alias value will always be an empty dictionary since `self.aliases` on the `AnsibleModule` instance is never updated after initialization. Since the `no_log` setting is on the canonical parameter not the alias, an incorrect warning is issued if the parameter matches `PASSWORD_MATCH`. This PR returns the aliases dictionary as an attribute of the `ValidationResult` and updates the `aliases` attribute on the `AnsibleModule` instance.	3 years ago
Matt Clay	1a5853d794	Remove obsolete units.compat.mock compat layer. (#77118) ... * Remove obsolete units.compat.mock compat layer. * Update remaining units.compat.mock references.	3 years ago
Matt Martz	17237c1d88	Deprecate Py2.6 support for target nodes (#74165)	4 years ago
Sam Doran	abacf6a108	Use ArgumentSpecValidator in AnsibleModule (#73703) ... * Begin using ArgumentSpecValidator in AnsibleModule * Add check parameters to ArgumentSpecValidator Add additional parameters for specifying required and mutually exclusive parameters. Add code to the .validate() method that runs these additional checks. * Make errors related to unsupported parameters match existing behavior Update the punctuation in the message slightly to make it more readable. Add a property to ArgumentSpecValidator to hold valid parameter names. * Set default values after performining checks * FIx sanity test failure * Use correct parameters when checking sub options * Use a dict when iterating over check functions Referencing by key names makes things a bit more readable IMO. * Fix bug in comparison for sub options evaluation * Add options_context to check functions This allows the parent parameter to be added the the error message if a validation error occurs in a sub option. * Fix bug in apply_defaults behavior of sub spec validation * Accept options_conext in get_unsupported_parameters() If options_context is supplied, a tuple of parent key names of unsupported parameter will be created. This allows the full "path" to the unsupported parameter to be reported. * Build path to the unsupported parameter for error messages. * Remove unused import * Update recursive finder test * Skip if running in check mode This was done in the _check_arguments() method. That was moved to a function that has no way of calling fail_json(), so it must be done outside of validation. This is a silght change in behavior, but I believe the correct one. Previously, only unsupported parameters would cause a failure. All other checks would not be executed if the modlue did not support check mode. This would hide validation failures in check mode. * The great purge Remove all methods related to argument spec validation from AnsibleModule * Keep _name and kind in the caller and out of the validator This seems a bit awkward since this means the caller could end up with {name} and {kind} in the error message if they don't run the messages through the .format() method with name and kind parameters. * Double moustaches work I wasn't sure if they get stripped or not. Looks like they do. Neat trick. * Add changelog * Update unsupported parameter test The error message changed to include name and kind. * Remove unused import * Add better documentation for ArgumentSpecValidator class * Fix example * Few more docs fixes * Mark required and mutually exclusive attributes as private * Mark validate functions as private * Reorganize functions in validation.py * Remove unused imports in basic.py related to argument spec validation * Create errors is module_utils We have errors in lib/ansible/errors/ but those cannot be used by modules. * Update recursive finder test * Move errors to file rather than __init__.py * Change ArgumentSpecValidator.validate() interface Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError exceptions for validation failures. Return the validated parameters if validation is successful rather than True/False. Update docs and tests. * Get attribute in loop so that the attribute name can also be used as a parameter * Shorten line * Update calling code in AnsibleModule for new validator interface * Update calling code in validate_argument_spec based in new validation interface * Base custom exception class off of Exception * Call the __init__ method of the base Exception class to populate args * Ensure no_log values are always updated * Make custom exceptions more hierarchical This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to module_utils, or AnsibleError defined in this commit should use the same signature as the original. * Just go back to basing off Exception * Return ValidationResult object on successful validation Create a ValidationResult class. Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful. Update class and method docs. Update unit tests based on interface change. * Make it easier to get error objects from AnsibleValidationResultMultiple This makes the interface cleaner when getting individual error objects contained in a single AnsibleValidationResultMultiple instance. * Define custom exception for each type of validation failure These errors indicate where a validation error occured. Currently they are empty but could contain specific data for each exception type in the future. * Update tests based on (yet another) interface change * Mark several more functions as private These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred public interface. * Move warnings and deprecations to result object Rather than calling deprecate() and warn() directly, store them on the result object so the caller can decide what to do with them. * Use subclass for module arg spec validation The subclass uses global warning and deprecations feature * Fix up docs * Remove legal_inputs munging from _handle_aliases() This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense to do that for an AnsibleModule instance (it should update the parameters before performing validation) and shouldn't be done by the validator. Create a private function just for getting legal inputs since that is done in a couple of places. It may make sense store that on the ValidationResult object. * Increase test coverage * Remove unnecessary conditional ci_complete * Mark warnings and deprecations as private in the ValidationResult They can be made public once we come up with a way to make them more generally useful, probably by creating cusom objects to store the data in more structure way. * Mark valid_parameter_names as private and populate it during initialization * Use a global for storing the list of additonal checks to perform This list is used by the main validate method as well as the sub spec validation.	4 years ago
Abhijeet Kasurde	e439194c8c	basic: Add name of aliases in error message (#69427) ... With this patch, user will be notified with available aliases of arg parameter. Fixes: #58752 Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>	4 years ago
Abhijeet Kasurde	055871cbb8	api: time.clock compatible code (#70650) ... time.clock is removed in Python 3.8. Add time.clock compatible code. Fixes: #70649 Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>	4 years ago
Sam Doran	bc05415109	Only pass kwargs to our string checker not callable checkers (#70151) ... Since only check_type_str() accepts extra param, only pass to our checker and do not pass kwargs to custom checkers. * Add unit tests	4 years ago
Sam Doran	3461c682c3	Add mechanism for storing warnings and deprecations outside of AnsibleModule (#58993) ... * Move warn() and deprecate() methods out of basic.py * Use _global_warnings and _global_deprications and create accessor functions - This lays the foundation for future functions being moved outside of AnsibleModule that need an interface to warnings and deprecations without modifying them. * Add unit tests for new warn and deprecate functions	5 years ago
kaorihinata	3ca4580cb4	Allow no_log=False to silence the no_log warnings for module parameters (#64733) ... As AnsibleModule._log_invocation is currently implemented, any parameter with a name that matches PASSWORD_MATCH triggers the no_log warning as a precaution against parameters that may contain sensitive data, but have not been marked as sensitive by the module author. This patch would allow module authors to explicitly mark the aforementioned parameters as not sensitive thereby bypassing an erroneous warning message, while still catching parameters which have not been marked at all by the author. Adds tests for various no_log states including True, False, and None (as extracted by AnsibleModule._log_invocation) when applied to an argument with a name that matches PASSWORD_MATCH. Fixes: #49465 #64656	5 years ago
Jill R	d49d52eb5f	Add tests for new alias deprecation functionality (#61476) ... * Add tests for the alias deprecation added in #61245	5 years ago
Andrey Klychkov	4e8df9a4b8	unit tests: remove unused imports (#59636)	5 years ago
Felix Fontein	4a574c4d0c	Option parsing: warn if both an option and its alias are specified for a module (#53698) ... * Print warning when both an option and its alias is specified. * Improve output. * Put warnings into self._warnings directly, resp. use self.warn() when handling subspecs. * Add changelog. * Add unit test.	5 years ago
Sam Doran	ff88bd82b5	Move type checking methods out of basic.py and add unit tests (#53687) ... * Move check_type_str() out of basic.py * Move check_type_list() out of basic.py * Move safe_eval() out of basic.py * Move check_type_dict() out of basic.py * Move json importing code to common location * Move check_type_bool() out of basic.py * Move _check_type_int() out of basic.py * Move _check_type_float() out of basic.py * Move _check_type_path() out of basic.py * Move _check_type_raw() out of basic.py * Move _check_type_bytes() out of basic.py * Move _check_type_bits() out of basic.py * Create text.formatters.py Move human_to_bytes, bytes_to_human, and _lenient_lowercase out of basic.py into text.formatters.py Change references in modules to point to function at new location * Move _check_type_jsonarg() out of basic.py * Rename json related functions and put them in common.text.converters Move formatters.py to common.text.formatters.py and update references in modules. * Rework check_type_str() Add allow_conversion option to make the function more self-contained. Move the messaging back to basic.py since those error messages are more relevant to using this function in the context of AnsibleModule and not when using the function in isolation. * Add unit tests for type checking functions * Change _lenient_lowercase to lenient_lowercase per feedback	6 years ago
Sam Doran	43a44e6f35	Move utility functions out of basic.py (#51715) ... Move the following methods to lib/anisble/module_utils/common/validation.py: - _count_terms() - _check_mutually_exclusive() - _check_required_one_of() - _check_required_together() - _check_required_by() - _check_required_arguments() - _check_required_if - fail_on_missing_params() --> create check_missing_parameters()	6 years ago
Felix Fontein	07fcb60d55	Python 2: accept both long and int for type=int (module options) (#53289) ... * Added unit test	6 years ago
Ganesh Nalawade	41e2bd1df5	Add support for elements validation in argspec (#50335) ... * Add support for elements validation in argspec Fixes #48473 * Add support to validate the elements value in argspec when type is `list` * Fix unit test failures * Add unit test for elements validation * Fix CI failures * Fix review comments * Fix unit test and CI failures after rebase	6 years ago
Dag Wieers	cd9471ef17	Introduce new 'required_by' argument_spec option (#28662) ... * Introduce new "required_by' argument_spec option This PR introduces a new **required_by** argument_spec option which allows you to say *"if parameter A is set, parameter B and C are required as well"*. - The difference with **required_if** is that it can only add dependencies if a parameter is set to a specific value, not when it is just defined. - The difference with **required_together** is that it has a commutative property, so: *"Parameter A and B are required together, if one of them has been defined"*. As an example, we need this for the complex options that the xml module provides. One of the issues we often see is that users are not using the correct combination of options, and then are surprised that the module does not perform the requested action(s). This would be solved by adding the correct dependencies, and mutual exclusives. For us this is important to get this shipped together with the new xml module in Ansible v2.4. (This is related to bugfix https://github.com/ansible/ansible/pull/28657) ```python module = AnsibleModule( argument_spec=dict( path=dict(type='path', aliases=['dest', 'file']), xmlstring=dict(type='str'), xpath=dict(type='str'), namespaces=dict(type='dict', default={}), state=dict(type='str', default='present', choices=['absent', 'present'], aliases=['ensure']), value=dict(type='raw'), attribute=dict(type='raw'), add_children=dict(type='list'), set_children=dict(type='list'), count=dict(type='bool', default=False), print_match=dict(type='bool', default=False), pretty_print=dict(type='bool', default=False), content=dict(type='str', choices=['attribute', 'text']), input_type=dict(type='str', default='yaml', choices=['xml', 'yaml']), backup=dict(type='bool', default=False), ), supports_check_mode=True, required_by=dict( add_children=['xpath'], attribute=['value', 'xpath'], content=['xpath'], set_children=['xpath'], value=['xpath'], ), required_if=[ ['count', True, ['xpath']], ['print_match', True, ['xpath']], ], required_one_of=[ ['path', 'xmlstring'], ['add_children', 'content', 'count', 'pretty_print', 'print_match', 'set_children', 'value'], ], mutually_exclusive=[ ['add_children', 'content', 'count', 'print_match','set_children', 'value'], ['path', 'xmlstring'], ], ) ``` * Rebase and fix conflict * Add modules that use required_by functionality * Update required_by schema * Fix rebase issue	6 years ago
Matt Clay	feb5b0b299	Fix unit test issues with pytest >= 4.0.0.	6 years ago
Matt Clay	3033fd96b0	Move unit test compat code out of `lib/ansible/`. (#46996) ... * Move ansible.compat.tests to test/units/compat/. * Fix unit test references to ansible.compat.tests. * Move builtins compat to separate file. * Fix classification of test/units/compat/ dir.	6 years ago
Matt Martz	1663b64e18	Allow subspec defaults to be processed when the parent argument is not supplied (#38967) ... * Allow subspec defaults to be processed when the parent argument is not supplied * Allow this to be configurable via apply_defaults on the parent * Document attributes of arguments in argument_spec * Switch manageiq_connection to use apply_defaults * add choices to api_version in argument_spec	7 years ago
René Moser	2f36b9e5ce	basic: allow one or more when param list having choices (#34537) ... * basic: allow one or more when param list having choices * add unit tests * optimize a bit * re-add get_exception import * a number of existing modules expect to be able to get it from basic.py	7 years ago
Toshio Kuratomi	370a7ace4b	Split basic units (#33510) ... Split the one monolithic test for basic.py into several files * Split test_basic.py along categories. This is preliminary to get a handle on things. Eventually we may want to further split it so each file is only testing a single function. * Cleanup unused imports from splitting test_basic.py * Port atomic_move test to pytest. Working on getting rid of need to maintain procenv * Split a test of symbolic_mode_to_octal to follow unittest best practices Each test should only invoke the function under test once * Port test_argument_spec to pytest. * Fix suboptions failure	7 years ago
Toshio Kuratomi	cd36164239	Porting tests to pytest (#33387) ... * Porting tests to pytest * Achievement Get: No longer need mock/generator.py * Now done via pytest's parametrization * Port safe_eval to pytest * Port text tests to pytest * Port test_set_mode_if_different to pytest * Change conftest AnsibleModule fixtures to be more flexible * Move the AnsibleModules fixtures to module_utils/conftest.py for sharing * Testing the argspec code requires: * injecting both the argspec and the arguments. * Patching the arguments into sys.stdin at a different level * More porting to obsolete mock/procenv.py * Port run_command to pytest * Port known_hosts tests to pytest * Port safe_eval to pytest * Port test_distribution_version.py to pytest * Port test_log to pytest * Port test__log_invocation to pytest * Remove unneeded import of procenv in test_postgresql * Port test_pip to pytest style * As part of this, create a pytest ansiblemodule fixture in modules/conftest.py. This is slightly different than the approach taken in module_utils because here we need to override the AnsibleModule that the modules will inherit from instead of one that we're instantiating ourselves. * Fixup usage of parametrization in test_deprecate_warn * Check that the pip module failed in our test	7 years ago
Matt Davis	3f1ec6b862	add custom module type validation (#27183) ... * Module argument_spec now accepts a callable for the type argument, which is passed through and called with the value when appropriate. On validation/conversion failure, the name of the callable (or its type as a fallback) is used in the error message. * adds basic smoke tests for custom callable validator functionality	7 years ago