module.fail_json(msg="instance_profile_name parameter requires boto version 2.5.0 or higher")
Migrating to boto3
------------------
Prior to Ansible 2.0, modules were written in either boto3 or boto. We are
still porting some modules to boto3. Modules that still require boto should be ported to use boto3 rather than using both libraries (boto and boto3). We would like to remove the boto dependency from all modules.
Porting code to AnsibleAWSModule
---------------------------------
Some old AWS modules use the generic ``AnsibleModule`` as a base rather than the more efficient ``AnsibleAWSModule``. To port an old module to ``AnsibleAWSModule``, change:
..code-block:: python
from ansible.module_utils.basic import AnsibleModule
...
module = AnsibleModule(...)
to:
..code-block:: python
from ansible.module_utils.aws.core import AnsibleAWSModule
...
module = AnsibleAWSModule(...)
Few other changes are required. AnsibleAWSModule
does not inherit methods from AnsibleModule by default, but most useful methods
are included. If you do find an issue, please raise a bug report.
When porting, keep in mind that AnsibleAWSModule also will add the default ec2
argument spec by default. In pre-port modules, you should see common arguments
There are two :ref:`common documentation fragments <module_docs_fragments>`
that should be included into almost all AWS modules:
*``aws`` - contains the common boto connection parameters
*``ec2`` - contains the common region parameter required for many AWS modules
These fragments should be used rather than re-documenting these properties to ensure consistency
and that the more esoteric connection options are documented. For example:
..code-block:: python
DOCUMENTATION = '''
module: my_module
# some lines omitted here
requirements: [ 'botocore', 'boto3' ]
extends_documentation_fragment:
- aws
- ec2
'''
Handling exceptions
===================
You should wrap any boto3 or botocore call in a try block. If an exception is thrown, then there
are a number of possibilities for handling it.
* Catch the general ``ClientError`` or look for a specific error code with
``is_boto3_error_code``.
* Use ``aws_module.fail_json_aws()`` to report the module failure in a standard way
* Retry using AWSRetry
* Use ``fail_json()`` to report the failure without using ``ansible.module_utils.aws.core``
* Do something custom in the case where you know how to handle the exception
For more information on botocore exception handling see the `botocore error documentation <https://botocore.readthedocs.io/en/latest/client_upgrades.html#error-handling>`_.
Using is_boto3_error_code
-------------------------
To use ``ansible.module_utils.aws.core.is_boto3_error_code`` to catch a single
AWS error code, call it in place of ``ClientError`` in your except clauses. In
this case, *only* the ``InvalidGroup.NotFound`` error code will be caught here,
and any other error will be raised for handling elsewhere in the program.
..code-block:: python
try:
info = connection.describe_security_groups(**kwargs)
an existing aliases file such as the `aws_s3 tests aliases file <https://github.com/ansible-collections/amazon.aws/blob/master/tests/integration/targets/aws_s3/aliases>`_.
If your PR has test failures, check carefully to be certain the failure is only due to the missing permissions. If you've ruled out other sources of failure, add a comment with the `ready_for_review`
tag and explain that it's due to missing permissions.
Your pull request cannot be merged until the tests are passing. If your pull request is failing due to missing permissions,
you must collect the minimum IAM permissions required to
run the tests.
There are two ways to figure out which IAM permissions you need for your PR to pass:
* Start with the most permissive IAM policy, run the tests to collect information about which resources your tests actually use, then construct a policy based on that output. This approach only works on modules that use `AnsibleAWSModule`.
* Start with the least permissive IAM policy, run the tests to discover a failure, add permissions for the resource that addresses that failure, then repeat. If your module uses `AnsibleModule` instead of `AnsibleAWSModule`, you must use this approach.
To start with the most permissive IAM policy:
1)`Create an IAM policy <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start>`_ that allows all actions (set ``Action`` and ``Resource`` to ``*```).
2) Run your tests locally with this policy. On AnsibleAWSModule-based modules, the ``debug_botocore_endpoint_logs`` option is automatically set to ``yes``, so you should see a list of AWS ACTIONS after the PLAY RECAP showing all the permissions used. If your tests use a boto/AnsibleModule module, you must start with the least permissive policy (see below).
3) Modify your policy to allow only the actions your tests use. Restrict account, region, and prefix where possible. Wait a few minutes for your policy to update.
4) Run the tests again with a user or role that allows only the new policy.
5) If the tests fail, troubleshoot (see tips below), modify the policy, run the tests again, and repeat the process until the tests pass with a restrictive policy.
6) Open a pull request proposing the minimum required policy to the `CI policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_.
1) Run the integration tests locally with no IAM permissions.
2) Examine the error when the tests reach a failure.
a) If the error message indicates the action used in the request, add the action to your policy.
b) If the error message does not indicate the action used in the request:
- Usually the action is a CamelCase version of the method name - for example, for an ec2 client the method `describe_security_groups` correlates to the action `ec2:DescribeSecurityGroups`.
- Refer to the documentation to identify the action.
c) If the error message indicates the resource ARN used in the request, limit the action to that resource.
d) If the error message does not indicate the resource ARN used:
- Determine if the action can be restricted to a resource by examining the documentation.
- If the action can be restricted, use the documentation to construct the ARN and add it to the policy.
3) Add the action or resource that caused the failure to `an IAM policy <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start>`_. Wait a few minutes for your policy to update.
4) Run the tests again with this policy attached to your user or role.
5) If the tests still fail at the same place with the same error you will need to troubleshoot (see tips below). If the first test passes, repeat steps 2 and 3 for the next error. Repeat the process until the tests pass with a restrictive policy.
6) Open a pull request proposing the minimum required policy to the `CI policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_.
- When you make changes to a policy, wait a few minutes for the policy to update before re-running the tests.
- Use the `policy simulator <https://policysim.aws.amazon.com/>`_ to verify that each action (limited by resource when applicable) in your policy is allowed.
- If you're restricting actions to certain resources, replace resources temporarily with `*`. If the tests pass with wildcard resources, there is a problem with the resource definition in your policy.
- If the initial troubleshooting above doesn't provide any more insight, AWS may be using additional undisclosed resources and actions.
- Examine the AWS FullAccess policy for the service for clues.
- Re-read the AWS documentation, especially the list of `Actions, Resources and Condition Keys <https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_actions-resources-contextkeys.html>`_ for the various AWS services.
- Look at the `cloudonaut <https://iam.cloudonaut.io>`_ documentation as a troubleshooting cross-reference.
- Use a search engine.
- Ask in the Ansible IRC channel #ansible-aws (on freenode IRC).
