AWS: make jittered_backoff API parameter configurable (#49086)

* AWS: make jittered_backoff API parameter configurable
* Review comments
* minor doc changes

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

lib/ansible/modules/cloud/amazon/GUIDELINES.md

@@ -45,7 +45,7 @@ are included. If you do find an issue, please raise a bug report.
 
 When porting, keep in mind that AnsibleAWSModule also will add the default ec2
 argument spec by default. In pre-port modules, you should see common arguments
-specfied with:
+specified with:
 
 ```
 def main():
@@ -416,6 +416,13 @@ describe_instances(module.client('ec2'), InstanceIds=['i-123456789'])
 The call will be retried the specified number of times, so the calling functions
 don't need to be wrapped in the backoff decorator.
 
+You can also use customization for `retries`, `delay` and `max_delay` parameters used by
+`AWSRetry.jittered_backoff` API using module params. You can take a look into
+[cloudformation](/lib/ansible/modules/cloud/amazon/cloudformation.py) module for example.
+
+To make all Amazon modules uniform, prefix the module param with `backoff_`, so `retries` becomes `backoff_retries`
+ and likewise with `backoff_delay` and `backoff_max_delay`.
+
 ### Returning Values
 
 When you make a call using boto3, you will probably get back some useful information that you

lib/ansible/modules/cloud/amazon/cloudformation.py

@@ -119,6 +119,28 @@ options:
     - Maximum number of CloudFormation events to fetch from a stack when creating or updating it.
     default: 200
     version_added: "2.7"
+  backoff_delay:
+    description:
+    - Number of seconds to wait for the next retry.
+    default: 3
+    version_added: "2.8"
+    type: int
+    required: False
+  backoff_max_delay:
+    description:
+    - Maximum amount of time to wait between retries.
+    default: 30
+    version_added: "2.8"
+    type: int
+    required: False
+  backoff_retries:
+    description:
+    - Number of times to retry operation.
+    - AWS API throttling mechanism fails Cloudformation module so we have to retry a couple of times.
+    default: 10
+    version_added: "2.8"
+    type: int
+    required: False
 
 author: "James S. Martin (@jsmartin)"
 extends_documentation_fragment:
@@ -578,6 +600,9 @@ def main():
         tags=dict(default=None, type='dict'),
         termination_protection=dict(default=None, type='bool'),
         events_limit=dict(default=200, type='int'),
+        backoff_retries=dict(type='int', default=10, required=False),
+        backoff_delay=dict(type='int', default=3, required=False),
+        backoff_max_delay=dict(type='int', default=30, required=False),
     )
     )
 
@@ -649,7 +674,11 @@ def main():
 
     # Wrap the cloudformation client methods that this module uses with
     # automatic backoff / retry for throttling error codes
-    backoff_wrapper = AWSRetry.jittered_backoff(retries=10, delay=3, max_delay=30)
+    backoff_wrapper = AWSRetry.jittered_backoff(
+        retries=module.params.get('backoff_retries'),
+        delay=module.params.get('backoff_delay'),
+        max_delay=module.params.get('backoff_max_delay')
+    )
     cfn.describe_stack_events = backoff_wrapper(cfn.describe_stack_events)
     cfn.create_stack = backoff_wrapper(cfn.create_stack)
     cfn.list_change_sets = backoff_wrapper(cfn.list_change_sets)