* Initial commit for rate limiting
- Detects if error code is 429
- Pauses for random time between .5 and 5 seconds before retrying
- If it fails 10 times, give up and tell user
* Redo structure of request() to support rate limiting
* Hold down timer is now a sliding scale
- 3 * number of retries
- Fails after the 30 second wait
* Whitespace fixes
* Redo implementation using decorators
- Errors aren't tested but code works for regular calls
* Unit tests work for error handling
* Add integration tests for successful retries
* Add condition for 502 errors and retry
* Move _error_report out of the class
* PEP8 fixes
* Add changelog entry
* ansible-galaxy: support multiple servers on install
* Added docs for the server configuration file
* Fix up doc string for requirements file format
* Fix bugs after testing
* Fix kwarg doc and added version
* Fix typo and doc improvement
* Fix base64 encoding and allow --server to override list
* Initial commit for the most basic of unit tests
* Rewrote unit test to actually work
- Uses pytest's fixtures structure, not classes
- Added a test file for importing
* Whitespace fixes
* Draft version of the mock unit test
* Modify code to actually work!
* Add 429 testing
* ansible-test fixes
* Resort lines
* Fix import for 2.x compatibility
On POWER systems, /proc/cpuinfo provides a 'processor' entry as a
counter, and a 'cpu' entry with a description (similar to 'model name'
on x86). Support for POWER in get_cpu_facts was added via the 'cpu'
entry in commit 8746e692c1. Subsequent
support for ARM64 in commit ce4ada93f9
used the 'processor' entry, resulting in double-counting of cores on
POWER systems.
When unit tests were later written for this code in
commit 55306906cf, the erroneous values
were just accepted in the test instead of being diagnosed.
Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
* Clean up comments in integration tests.
Tests reference soon to be outdated paths and implementation details.
* Remove unused test/runner/ reference in test.
This commit allows users to access a vCenter or a ESXi through a
HTTP CONNECT based proxy.
To do so, the users have to set the `proxy_host` and `proxy_port`
variables.
The can also use the `VMWARE_PROXY_HOST` and `VMWARE_PROXY_PORT`
environment variables.
This feature depends on pyvmomi > v6.7.1.2018.12.
Fixes: #42221
Co-Author: Abhijeet Kasurde <akasurde@redhat.com>
Co-Author: Gonéri Le Bouder <goneri@redhat.com>
* Added unit tests for ansibble_utils.py
* Removed unnecessary print statement
* Updated Ip address as review comment
* Updated as per review comments on IP address and test asserts
* Print warning when both an option and its alias is specified.
* Improve output.
* Put warnings into self._warnings directly, resp. use self.warn() when handling subspecs.
* Add changelog.
* Add unit test.
* Get the most detailed version number from distro.version() for CentOS and Debian
* Update tests and fixtures
* Update fixture generation script to gather distro info and work with Python 3
* Update LinuxMint fixtures
* Cleanup fixture formatting
* Improvements based on feedback from abadger:
- use unicode since that is what distro returns
- use frozenset with a tuple
- include link Debian to bug
* Add tests for KubeAPIVersion
* Legibility improvements for KubevirtVM tests
* Create units.utils.kubevirt with common stuff
* Add some VMIRS unit tests
* Update dependence's name to firepower-kickstart
* Check response type before getting attributes
* Add unit test for construct_ansible_facts method
* Update error message
* Adding device_facts module for contribution
* changes added for pylint error
* Updated code to solve ansible-test compile error
* Changes to avoide comile error added
* Review Comments changes update
* Avoided blank line
* pylint error changes
* Removed ansible_facts return in error case
* Updated description
* modules renamed
* changing from ansible_facts to device_info
* avoide pep8 error
* Updated sample output
* version changed to 2.9
* Changed Copyright license to BSD
* Changed 3-clause BSD license to 2-clause BSD
* Added unit test support for ome_device_info
* version change
* removed pylint error in unit test modules
* Avoid Sanity error for unit test modules
* updated version
* Attempt 2 of cert validation fixes
* Remove unused code
* Cleanup the tmp cert using atexit
* Fix linting issues
* Only add SSLValidationHandler when not HAS_SSLCONTEXT
* Catch value errors on non PEM certs
* Only catch NotImplementedError to avoid masking issues
* set self._context even with PyOpenSSLContext for conformity
* Fix error building
* normalize how we interact with the context we create
* Remove unused code
* Address test for py3.7 message difference
* open_url should pass the ca_path through
* Account for new error in url lookup test
* Guard some code behind whether or not we are validating certs
* Make _make_context public
* Move atexit.register up to where the tmp file is created
- Also return url and update docs for other values to indicate they are only returned on success.
- Add integration tests
- Use info variable for common return values
- Use -1 as default status rather than None. This is lines up with with existing code in urls.py
- Add unit tests to ensure status and url are returned on failure
Ensure the module still work with the ESXi where CustomFieldsManager
does not exist.
From: https://www.vmware.com/support/developer/converter-sdk/conv60_apireference/vim.CustomFieldsManager.html
The CustomFieldsManager object is used to add and remove custom fields to
managed entities.
The custom fields values set on managed entities are available through the
customValue property and through the summary objects for VirtualMachine
and HostSystem. They are not available directly through this managed object.
This functionality is only available through VirtualCenter.
Fixes: #56071
* use navigate_value instead navigate_hash
* add async wait method
* update dict compare
* remove unuse methods
* not all modules have timeouts parameter
* navigate_value, the input data may be None
* remove external grep call and parse with python
* use function for repeated code
* use module.get_bin_path() for iscsiutil on HPUX
* some code opt for HPUX
* clean up non-module code, module being defined is a requirement for this code
* import get_bin_path() directly and use without module prefix
* Add integration tests for AIX and HP-UX
* add changelog fragment
* Apply suggestions from code review
Co-Authored-By: mator <matorola@gmail.com>
* Apply suggestions from code review #2
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Remove strict requirement on executable to exist for get_bin_path() as
it will allow facts gathering to continue without an error. Almost all
other files under facts do not have "required=True" (except 2 files,
which should be probably fixed). And check return value for
get_bin_path() , before run attempt.
* add check for AIX lsattr run_command return code
Remove test comparing output to platform.linux_distribution() since we are relying on distro.id() and are not concerned about matching the output of platform.linux_distribution()
* netbox_device: Allow device modification
* Add ability to update and existing device
* Allow check_mode
* Fail when device name is missing
* Fail when cannot resolve ID instead of taking ID 1 by default
* netbox_device: Add diff output
* netbox: Some refactoring
* Add diff output and check_mode to netbox_ip_address
* Deduplicate redundant code into netbox_utils
* netbox_utils: A few unit tests
* Add latest updates from FTD Ansible downstream repository.
- add a better implementation of the upsert operation;
- add API version lookup functionality;
- add filter which remove duplicated references from the list of references;
- fix minor bugs.
* fix issues outlined by ansibot
* fix argument name for _check_enum_method
* Add to_ipv6_subnet function
* Use the correct function for subnet
* Corrected code style and tests
* Corrected testcase assertion
64 bits make 8 octets, or 4 hextets
* Import from correct module directly
* Move check_type_str() out of basic.py
* Move check_type_list() out of basic.py
* Move safe_eval() out of basic.py
* Move check_type_dict() out of basic.py
* Move json importing code to common location
* Move check_type_bool() out of basic.py
* Move _check_type_int() out of basic.py
* Move _check_type_float() out of basic.py
* Move _check_type_path() out of basic.py
* Move _check_type_raw() out of basic.py
* Move _check_type_bytes() out of basic.py
* Move _check_type_bits() out of basic.py
* Create text.formatters.py
Move human_to_bytes, bytes_to_human, and _lenient_lowercase out of basic.py into text.formatters.py
Change references in modules to point to function at new location
* Move _check_type_jsonarg() out of basic.py
* Rename json related functions and put them in common.text.converters
Move formatters.py to common.text.formatters.py and update references in modules.
* Rework check_type_str()
Add allow_conversion option to make the function more self-contained.
Move the messaging back to basic.py since those error messages are more relevant to using this function in the context of AnsibleModule and not when using the function in isolation.
* Add unit tests for type checking functions
* Change _lenient_lowercase to lenient_lowercase per feedback
* Adding support for IP identifiers according to https://tools.ietf.org/html/draft-ietf-acme-ip-05.
* Add changelog.
* Make sure that the authorizations return value is unchanged for CSRs with DNS-only SANs.
* Remove unneeded import.
* type -> identifier_type
* Python 2.6 compatibility.
* Fix unit tests.
* Add IP address normalization.
* Extend tests.
* Move data into fixtures.
* Adjust BOTMETA.
- More info added to module docs.
- Fixed errors and typos in module docs.
- Added parameter types to module docs.
- Some error messages are fixed and/or changed to be more helpful.
- Some code comments changed and added.
- Updated unit tests that test changed error messages.
- Improved module examples.
- Improved docs for custom_params xenserver_guest module parameter
Fixes a bug where parse_distribution_file_ClearLinux() was called on CoreOS (and probably many other distros) and it returned True since it successfully parses the distribution file. Since this file exists on many Linux distributions and they are a very similar format, add an additional check to make sure it is Clear Linux.
Change the order in which distribution files are processed so NA is last. This prevents a match on CoreOS hosts since they also have /etc/os-release and the called matching function for NA is very general and will match CoreOS.
* Add changelog
* Add unit tests
Only add tests for Clear Linux parsing since that was the cause of this issue.
* Rename method and make private
* Use is_iterable, combine transformations
* Remove unused return_values from network modules
* Improve docstrings in new functions
* Add new PASS_VAR
* Add unit tests for list_no_log_values
* Fix unit tests for Python 2.6
Refinements:
- return legal_inputs and update class properties
- remove redundant arguments from method and handle in caller
- add better exception types to method
* Add unit tests for handle_aliases
* uri/win_uri: Make method a free text field
Since various interfaces introduce their own HTTP method (e.g. like
PROPFIND, LIST or TRACE) it's better to leave this up to the user.
* Fix HTTP method check in module_utils urls
* Add integration test for method UNKNOWN
* Clarify the change as requested during review
* added validate_certs paramter to allow for ignoring of SSL certificates
* formatting
* updated NSO unit tests with validate_certs variable
* fixed NSO tests
- xenserver module_util: fixed a bug in gather_vm_params function where
an exception was generated if XenServer product_version_text_short
parameter contained non numeric characters, e.g. "7.1 CU1" on
XenServer version 7.1 with Cummulative Update 1. Code was changed
to use product_version parameter instead which is all numeric.
- xenserver module_util: get_xenserver_version function is changed
to return a list of integers for major, minor and update version
instead of list of strings.
- xenserver module_util: unit tests are updated according to changes.
- xenserver module_util: removed unused imports.
* Add support for elements validation in argspec
Fixes#48473
* Add support to validate the elements value in argspec
when type is `list`
* Fix unit test failures
* Add unit test for elements validation
* Fix CI failures
* Fix review comments
* Fix unit test and CI failures after rebase
* Facts parsing for cmdline can now handle multiple values for a single key.
* Unit tests for cmdline fact parsing
* Review comments
Fixes: #22766
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* First pass at allowing unix socket with urls/uri. See #42341
* Only insert handler as needed
* Fix and add tests
* Add HTTPS functionality for unix sockets
* Additional test fixes
* Create context manager for monkey patching HTTPConnection.connect, de-dupe code, raise better errors
* doc
* Add a few more tests
* Fix __call__
* Remove unused import
* Patch HTTPConnection.connect with the functionality we want, instead of duplicating code and disabling
* Fix var name
* Remove unused var
* Add changelog fragment
* Update uri docs
* Fix rebase indentation issue
* Refactored code
* Added support for Cumulus Linux 2.5.4
* Added support for Cumulus Linux 3.7.3
* Test added
Fixes: #29969
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Introduce new "required_by' argument_spec option
This PR introduces a new **required_by** argument_spec option which allows you to say *"if parameter A is set, parameter B and C are required as well"*.
- The difference with **required_if** is that it can only add dependencies if a parameter is set to a specific value, not when it is just defined.
- The difference with **required_together** is that it has a commutative property, so: *"Parameter A and B are required together, if one of them has been defined"*.
As an example, we need this for the complex options that the xml module provides. One of the issues we often see is that users are not using the correct combination of options, and then are surprised that the module does not perform the requested action(s).
This would be solved by adding the correct dependencies, and mutual exclusives. For us this is important to get this shipped together with the new xml module in Ansible v2.4. (This is related to bugfix https://github.com/ansible/ansible/pull/28657)
```python
module = AnsibleModule(
argument_spec=dict(
path=dict(type='path', aliases=['dest', 'file']),
xmlstring=dict(type='str'),
xpath=dict(type='str'),
namespaces=dict(type='dict', default={}),
state=dict(type='str', default='present', choices=['absent',
'present'], aliases=['ensure']),
value=dict(type='raw'),
attribute=dict(type='raw'),
add_children=dict(type='list'),
set_children=dict(type='list'),
count=dict(type='bool', default=False),
print_match=dict(type='bool', default=False),
pretty_print=dict(type='bool', default=False),
content=dict(type='str', choices=['attribute', 'text']),
input_type=dict(type='str', default='yaml', choices=['xml',
'yaml']),
backup=dict(type='bool', default=False),
),
supports_check_mode=True,
required_by=dict(
add_children=['xpath'],
attribute=['value', 'xpath'],
content=['xpath'],
set_children=['xpath'],
value=['xpath'],
),
required_if=[
['count', True, ['xpath']],
['print_match', True, ['xpath']],
],
required_one_of=[
['path', 'xmlstring'],
['add_children', 'content', 'count', 'pretty_print', 'print_match', 'set_children', 'value'],
],
mutually_exclusive=[
['add_children', 'content', 'count', 'print_match','set_children', 'value'],
['path', 'xmlstring'],
],
)
```
* Rebase and fix conflict
* Add modules that use required_by functionality
* Update required_by schema
* Fix rebase issue
* identity: Add GSSAPI suport for FreeIPA authentication
This enables the usage of GSSAPI for authentication, instead of having
to pass the username and password as part of the playbook run.
If there is GSSAPI support, this makes the password optional, and will
be able to use the KRB5_CLIENT_KTNAME or the KRB5CCNAME environment
variables; which are standard when using kerberos authentication.
Note that this depends on the urllib_gssapi library, and will only
enable this if that library is available.
* identity: Add documentation for GSSAPI authentication for FreeIPA
This documentation describes how to use GSSAPI authentication with the
IPA identity modules.
* identity: Add changelog for GSSAPI support for IPA
This adds the changelog entry for the GSSAPI authentication feature for
the IPA identity module.
* Move docker_ module_utils into subpackage.
* Remove docker_ prefix from module_utils.docker modules.
* Adding jurisdiction for module_utils/docker to $team_docker.
* Making docker* unit tests community supported.
* Linting.
* Python < 2.6 is not supported.
* Refactoring docker-py version comments. Moving them to doc fragments. Cleaning up some indentations.
* facts: solaris: introduce distribution_major version detection for Solaris
Currently, there's no distribution_major in facts module on Solaris OS.
Use "uname -r" output to report major version.
Before the patch we get this on Solaris 11.3 :
$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {}, "changed": false}
and after this patch, output is the following:
$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "11"}, "changed": false}
Tested with Solaris 11.3 and Solaris 10 (both are x86_64 VMs)
Includes patch for test/units.
Fixes#18197
* Try to fix test unit
* should work now...
* fixes for W291 (trailing whitespace) and E265 (block comment)
* mock uname_release for solaris 10 and solaris 11
* facts: solaris: introduce distribution_major version detection for Solaris
Currently, there's no distribution_major in facts module on Solaris OS.
Use "uname -r" output to report major version.
Before the patch we get this on Solaris 11.3 :
$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {}, "changed": false}
and after this patch, output is the following:
$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "11"}, "changed": false}
Tested with Solaris 11.3 and Solaris 10 (both are x86_64 VMs)
Includes patch for test/units.
Fixes#18197
* Try to fix test unit
* should work now...
* fixes for W291 (trailing whitespace) and E265 (block comment)
* mock uname_release for solaris 10 and solaris 11
* typo uname_v -> uname_r
* rebase
* fix pep8 E302: 2 blank lines
* remove int() cast to match test case
* use single function for uname_r and uname_v
* add solaris 11.4 OS to distribution test unit
* fix pep8 sanity - E231 missing whitespace
* distribution_major_version variable strip newline
* mocker test function for mock_get_uname with parameters instead of two different functions
* failed to make one fuction with test unit, revert to use 2 different functions
* try to use single get_uname function
* fix pep8: E703
* parallelize getting mount info
* fixed timeout and made 8 max thread count
- minor cleanup
- avoid empty mount entries
- set timeout on get
- enforce timeout per mount/thread
- make note on failure per mount
- make note on timeout per mount
- ensure proper pool control
- minor fixes
- less vars, simpler code
- move filter 'pre threading'
- remove timeout for all mounts, now per mount
- also use cpu count from multiprocessing lib
- moved 'bind' options out of thread as per comments
- warn on error, more info on failure to get info
* removed info declaration from documentation fragment as this is not implemented
* added optional headers for POST and PUT requests
* updated documentation
* added missing headers field decalaration
* removed info choice from state field
* added tests for the new utm_utils function
* fixed class invocation
* added missing required params
* fixed the pytests
* Move get_all_subclasses out of sys_info as it is unrelated to system
information.
* get_all_subclasses now returns a set() instead of a list.
* Don't port get_platform to sys_info as it is deprecated. Code using
the common API should just use platform.system() directly.
* Rename load_platform_subclass() to get_platform_subclass and do not
instantiate the rturned class.
* Test the compat shims in module_utils/basic.py separately from the new
API in module_utils/common/sys_info.py and module_utils/common/_utils.py
* Revert "allow caller to deal with timeout (#49449)"
This reverts commit 63279823a7.
Flawed on many levels
* Adds poor API to a public function
* Papers over the fact that the public function is doing something bad
by catching exceptions it cannot handle in the first place
* Papers over the real cause of the issue which is a bug in the timeout
decorator
* Doesn't reraise properly
* Catches the wrong exception
Fixes#49824Fixes#49817
* Make the timeout decorator properly raise an exception outside of the function's scope
signal handlers which raise exceptions will never work well because the
exception can be raised anywhere in the called code. This leads to
exception race conditions where the exceptions could end up being
hanlded by unintended pieces of the called code.
The timeout decorator was using just that idiom. It was especially bad
because the decorator syntactically occurs outside of the called code
but because of the signal handler, the exception was being raised inside
of the called code.
This change uses a thread instead of a signal to manage the timeout in
parallel to the execution of the decorated function. Since raising of
the exception happens inside of the decorator, now, instead of inside of
a signal handler, the timeout exception is raised from outside of the
called code as expected which makes reasoning about where exceptions are
to be expected intuitive again.
Fixes#43884
* Add a common case test.
Adding an integration test driven from our unittests. Most of the time
we'll timeout in run_command which is running things in a subprocess.
Create a test for that specific case in case anything funky comes up
between threading and execve.
* Don't use OSError-based TimeoutError as a base class
Unlike most standard exceptions, OSError has a specific parameter list
with specific meanings. Instead follow the example of other stdlib
functions, concurrent.futures and multiprocessing and define a separate
TimeoutException.
* Add comment and docstring to point out that this is not hte Python3 TimeoutError
Kevin Breit