Commit Graph

55 Commits (d7845da326e3d0cf74ff813a0e8b84b4137d185d)

Author SHA1 Message Date
Mark Chappell 1f733e2d55 ec2_asg: fix the integration tests (#61212) 5 years ago
Mark Chappell 77e4371460 aws_kms: Update policy on existing keys (when passed) (#60059)
* aws_kms: (integration tests) Use module_defaults to reduce the copy and paste

* aws_kms: (integration tests) make sure policy option functions.

* aws_kms: (integration tests) Move iam_role creation to start of playbook.

iam_roles aren't fully created when iam_role completes, there's a delay on the Amazon side before they're fully recognised.

* aws_kms: Update policy on existing keys (when passed)
5 years ago
Mark Chappell 70777020c4 Fix iam_password_policy integration tests (#60930)
* iam_password_policy: (integration tests) Use module defaults for AWS connection details

* iam_password_policy: (integration tests) Ensure the policy is removed when tests fail

* iam_password_policy: (integration tests) Add regression test for #59102

* iam_password_policy: Only return changed when the policy changes.

* iam_password_policy: PasswordReusePrevention must be omitted to remove/set to 0

* #60930 add changelog

* Update hacking AWS security policy to allow testing of Password Policy Management
5 years ago
Jesse Evers e410dcbfed Add logic to handle multiple actions in an ALB listener rule, Fixes #41861 (#41975)
* added logic to handle multiple actions in an ALB listener rule (#41861)

* fix linting and pep8 issues

* added test for multiple actions using OIDC authentication

* added error messages related to old versions of botocore and multiple actions

* fix action validation error checks (need to check the exception string)

* added logic to make oidc configs idempotent (remove clientsecret for check)

* modified TargetGroupName to TargetGroupArn substitution to account for multiple rule actions

* refactored tests so that it can be run against different versions of botocore

* fix runme.sh to refelct changes to cloud testsuite

* add UseExistingClientSecret to oidc config (AWS api change)

* remove tests for OIDC auth action; add tests for redirect and fixed-response

* add in fixes from markuman and mjmayer

* remove documentation for cognito integration (not sure how to test); added example config for fixed-response and redirect actions

* renamed oidc/multiple action tests; leaving commented due to some AWS API changes

* pep8 fix

* more pep8 fixes

* Restructure elb_application_lb test suite

Move from runme.sh to virtualenv based roles

Update policies to fix tests

Don't log temp dir deletion, so many files in the diff!
5 years ago
Will Thames 60c76be03c rds_instance: add point-in-time instance restore test (#59411) 5 years ago
Aljaž Košir 42073b6331 Add lambda_bucket_event module (#58059) 5 years ago
Will Thames 60fb9fc208 Fix EC2 test suite to work with testing policies (#44387)
* Update testing policies to ensure all required permissions are present
* Tidy up security policies to reduce duplicate permissions
* Make roles static so that they can be present before CI is run,
  meaning that role creation permission is not required by the CI
  itself, only by someone setting up the roles prior to testing
* Move contents to cloudfront policy to network policy to ensure policy
  count (maximum of 10) stays low
* Maintain compute policy below 6144 bytes
5 years ago
Will Thames a6d757e074 Ensure block device instance creation outputs instances
Not waiting outputs results in a format that will never
be matched by the tests

Ensure instances get tidied up

Allow ec2:ReportInstanceStatus

ec2_instance: Improve test cleanup on failing tests

Improve describe/modify attribute error handling

Address feedback on PR
5 years ago
Will Thames a09aa205e1 Fix RDS test suite and minor bugs revealed (#57940)
* Update testing policy to be correct for RDS test suite
* Create read replica in same region to avoid more permissions being
  required
* Ensure modifying DB doesn't try to downgrade engine version
* Add tags to main test suite to limit number of tests run for problem
  solving
5 years ago
Kohei Asano ddf6d096c5 Support the new TLS termination on NLBs (#51327) (#58031) 6 years ago
Will Thames 924352a051 ecs_cluster test suite refactor (#57716)
* Combine testing policies

Because of the maximum of 10 policies per group, need to
consolidate testing policies as best we can.

* Tidy put-account-setting tasks and add permission

Using `environment` and `command` rather than `shell` avoids the
need for `no_log` and means that people can fix the problem

* refactor ecs_cluster test suite

move from runme.sh technique to virtualenv

use ec2_instance rather than ec2 module to
avoid need for boto
6 years ago
Stefan Horning 77ec0549b0 New module for AWS CodeBuild (#47187)
* New AWS module for the CodeBuild service, called aws_codebuild

* Integration test for new aws_codebuild module
6 years ago
Stefan Horning ddfaa83ccf s3_bucket: add encryption capabilities to the module (#55985)
* s3_bucket: add encryption capabilities to the module
6 years ago
mjmayer c8e179fbf1 Aws waf region (#48953)
* Add waiter for AWSRegional

* Add support for WAF Regional

* Add support for regional waf web acl

* Remove set_trace, pep formatting

* Add paginator for regional_waf

* Change name of param for waf_regional

This is more in line with how AWS refers to the service. Additional
 changes made to how client is called. Used ternary to reduce if
 statements

* Change parameter name to waf_regional

* Add support for removal waf regional condition

* Change parameter from cloudfront to waf_regional

* Added state: absent waf rule

* Remove set_trace

* Add integration tests for waf regional

* WIP: adding region parameter to tests

* Add support for waf facts module

* Add region to waf regional integration tests

* Update security policy for waf regional testing

* Add type to documentation for waf_regional param
6 years ago
Bob Boldin b67505d271 AWS: new module ec2_transit_gateway fixes #49376 (#53651)
* AWS: new module ec2_transit_gateway fixes #49376

* Add permissions neeeded for integration tests

* uncomment nolog on creds

* add unsupported to integration test aliases

* remove the shippable/aws/group alias so doesn't conflict with unsupported
6 years ago
Andrea Tartaglia 5c6b16edc3 Fix ec2_instance eventual consistency when wait: false (#51885)
* Do not return 'instances' when wait is false

* Added integration tests for wait: false

* Added changelog fragment

* Fix test suite to work with ec2_instance

* Additional permissions
* Enforce boto3 version
* Fix broken tests
* Improve error messages

* fix linter issues
6 years ago
Tad Merchant b979b26a74 Add launch type to ecs task (#49081)
* adds fargate launch_type to ecs_task module

* White space changes

* fix documentation for running ecs task on fargate

* remove extraneous example from ecs_task

* White space changes

* Adds changelog fragment

* Pluralize minor_changes in changelog fragment

* Add Stop and Start task permissions
6 years ago
Rafael Driutti c68838fb13 AWS Redshift: port module to boto3 and fix parameters check (#37052)
* fix parameters check and port module to boto3

* begin with integration tests

* allow redshift iam policy

* Wait for cluster to be created before moving on to delete it

* Allow sts credentials so this can be run in CI

Don't log credentials

ensure cluster can be removed

* - Replace DIY waiters with boto3 waiters
- test multi node cluster

* catch specific boto3 error codes

* remove wait from test

* add missing alias for shippable

* - Rework modify function.
- Default unavailable parameters to none.
- Add cluster modify test

* Ensure resources are cleaned up if tests fail

* Ensure all botocore ClientError and BotoCoreError exceptions are handled
6 years ago
Will Thames 46fbcf08bc
aws_kms enhancements (#31960)
* Allow creation and deletion of keys (deletion just schedules for
  deletion, recreating an old key is just cancelling its deletion)
* Allow grants to be set, thus enabling encryption contexts to be
  used with keys
* Allow tags to be added and modified
* Add testing for KMS module
* Tidy up aws_kms module to latest standards
6 years ago
Ed Costello b70d5d9aee [AWS] ses rule set module for inbound email processing (#42781)
* Add module ses_rule_set for Amazon SES

* Update behaviours and naming to be consistent with other aws_ses_ modules.

* Add global lock around tests using active rule sets to prevent intermittent test failures.

* Fix deletion of rule sets so that we don't inactivate the active rule set
when force deleting an inactive rule set.
6 years ago
Shuang Wang 0c6513e9b1 add module aws_codecommit to represent AWS CodeCommit (#46161)
* kick off

* done for the day

* beta code and test

* fix a typo

* boto3_conn and boto_exception aren't used in this code, ec2_argument_spec is used but unneeded.

* Returning when find a match avoids doing extra work, especially when pagination is involved

* add new permissions for test

* (output is changed) is preferred over accessing the attribute directly.

* pass the result through camel_dict_to_snake_dict() before returning it.

* AnsibleAWSModule automatically merges the argument_spec.

* deletes the created resources even if a test fails.

* AnsibleAWSModule automatically merges the argument_spec.

* fix typo

* fix pep8

* paginate list_repositories

* specify permissions for test

* cut the unnecessary code.

* add return doc string

* add missed ':'

* fix syntax error: mapping values are not allowed here

* add description for return

* fix syntax error

* rename module name and turn off automated integration test.
6 years ago
Will Thames d2569a3f7d Improve iam_group exception handling (#45599)
* Improve iam_group exception handling

Use AnsibleAWSModule for iam_group and handle BotoCoreErrors
as well as ClientErrors. Use fail_json_aws to improve error messages

* Add minimal iam_group test suite

Update some of the read-only IAM permissions (this is not sufficient
to run the test suite but it gets further than it did until it tries
to add a (non-existent) user)

* Clean up after tests
6 years ago
Ed Costello 96c4efcd95 Add missing s3 permissions for s3 module testing. (#43243) 6 years ago
Will Thames 60e3af42d5 sns_topic boto3 port (#39292)
* Port sns_topic to boto3 and add tests
6 years ago
sdubrul 061877d584 added account_alias in the response of module aws_caller_facts (#42345)
* added account_alias in the response of module aws_caller_facts

* added comment to explain list_account_aliases

* renamed caller_identity to caller_facts as the content is extended

* created changelog

* security-policy needs the iam:ListAccountAliases for this module to work

* test now checks for the added field account_alias

* gracefully handle missing iam:ListAccountAliases permission
6 years ago
flowerysong a08668cf00 Port ec2_tag to boto3 (#39712)
* Add volume manipulation to EC2 integration test policy

* Port ec2_tag to boto3
6 years ago
Calvin Wu 7e42e88cc1 ecs_taskdefinition can absent without containers argument (#41398)
* ecs_taskdefinition can absent without containers argument

* add regression test for absent with arn

* Add PassRole privilege for ecs_cluster to pass
6 years ago
Julien Vey 0f612d1b76 efs_facts: improve performance by reducing the number of api calls (#36520)
* efs_facts: improve performance by reducing the number of api calls

* Remove efs_facts tests from running in CI
6 years ago
Will Thames b235cb8734 aws_eks_cluster: New module for managing AWS EKS (#41183)
* aws_eks: New module for managing AWS EKS

aws_eks module is used for creating and removing EKS clusters.

Includes full test suite and updates to IAM policies to enable it.

* Clean up all security groups

* appease shippable

* Rename aws_eks module to aws_eks_cluster
7 years ago
Will Thames a60fe1946c Remove ECS policies from AWS compute policy
The compute policy was exceeding maximum size and contained
policies that already exist in ecs-policy.

Look up suitable AMIs rather than hardcode

We don't want to maintain multiple image IDs for multiple regions
so use ec2_ami_facts to set a suitable image ID

Improve exception handling
7 years ago
Will Thames 809c7404ab Add two missing VPC permissions (#37896)
Remove VPC permissions from network-policy.json as they mostly duplicate
compute-policy.json permissions - separating the VPC and compute permissions
would likely lead to further confusion.
7 years ago
Madhura-CSI b85970b2b0 New Module: ec2_vpc_vpn_facts (#35983)
* New module: ec2_vpc_vpn_facts

* Add integration tests for ec2_vpc_vpn_facts and the IAM permissions

* Add retry to VPC removal

* Use unique name for VGW

* Always clean up after tests and add retries
7 years ago
Will Thames 29770a297a Fail with nice error message if elb target_type=ip not supported (#38313)
* Add helpful failure message if target_type=ip is not supported

Create test case for target_type=ip not supported

* Update elb_target_group module to latest standards

Use AnsibleAWSModule
Improve exception handling
Improve connection handling
7 years ago
Will Thames 12f2b9506d [aws]Add VPC configuration to ECS modules (#34381)
Enable awsvpc network mode for ECS services and tasks and
their underlying task definitions

Improve test suite to thoroughly test the changes

Use runme.sh technique to run old and new versions of botocore to
ensure that the modules work with older botocore and older network modes
and fail gracefully if awsvpc network mode is used with older botocore
7 years ago
Ed Costello 0d31d1cd24 [cloud]Add aws_ses_identity_policy module for managing SES sending policies (#36623)
* Add aws_ses_identity_policy module for managing SES sending policies

* Add option to AnsibleAWSModule for applying a retry decorator to all calls.

* Add per-callsite opt in to retry behaviours in AnsibleAWSModule

* Update aws_ses_identity_policy module to opt in to retries at all callsites.

* Add test for aws_ses_identity_policy module with inline policy.

* Remove implicit retrys on boto resources since they're not working yet.
7 years ago
Will Thames a1d3cf488d [cloud][test]Add missing IAM policy for cloudfront (#38248)
Cloudfront needs CreateOriginAccessIdentity

Add profile parameter to setup-iam.yml. Could arguably just use
AWS_PROFILE but given that other tasks are using profile, should
be consistent.
7 years ago
Julien Vey 7c07877b1b s3_bucket: add integration tests (#36941)
Also update testing-policies/storage
7 years ago
Julien Vey 51d491f8f0 route53_zone: move to boto3, and enable comment update (#36641) 7 years ago
Will Thames b5a1643e3d Add new aws_waf_condition module (#33110) 7 years ago
Ed Costello d16bc1c3f4 New aws_ses_identity module to manage AWS Simple Email Service Identity (#31140)
* Add aws_ses_identity module

* Update CI alias, add BotoCoreError exception handling.

* Add SES and SNS permissions to hacking/aws_config to run aws_ses_identity integration tests
7 years ago
Marek 5fa29201a7 Port sts_assume_role to boto3 (#32569)
* Ported sts_assume_role to boto3

* Added integration tests
7 years ago
Will Thames 4d58d16793 Add aws_s3 action plugin to find source files as expected (#35028)
People expect to be able to upload files to s3 using standard
locations for files.

Providing an action plugin that effectively rewrites the `src`
key to the result of finding such a file is a great help.

Tests added, and IAM permissions corrected
7 years ago
Will Thames 8d733dbdf0 [cloud] New module cloudfront_distribution (#31284)
* added cloudfont.py, modified cloudfront_facts.py class name and fixed a minor bug

* Improvements to cloudfront_distribution

* Reduce the scope of the cloudfront_distribution module
    * Remove presigning
    * Remove streaming distribution functionality
* Add full test suite for cloudfront distribution
* Meet Ansible AWS guidelines

* Make requested changes

Fix tests

Use built-in waiter

Update copyright
7 years ago
Will Thames 866d7fdce9 [cloud] Create ECS integration test suite (#33757)
Tests for:
* ecs_cluster
* ecs_service
* ecs_service_facts
* ecs_taskdefinition
* ecs_taskdefinition_facts

* Add idempotency testing

Test ecs_cluster, ecs_service and ecs_taskdefinition for trivial
idempotency. Add FIXMEs to the tests because the latter two fail.

Remove unused dependencies
7 years ago
Sloane Hertel f5471b3dcb [cloud] ec2_vpc_net integration tests (#33111)
* Add some integration tests for ec2_vpc_net module

* Add a couple tests for check mode

fix typo

ensure the DHCP option set is cleaned up

* Add permissions to test policy
7 years ago
Will Thames 1ca0c0e7f7 Consolidate IAM policies into fewer, larger policies (#33122)
Due to IAM limits allowing at most 10 policies per group,
need to reduce the number of total policies in use.
7 years ago
Will Thames 0b128e2e29 Add tests for ec2_vpc_route_table (#32036)
* Add tests for ec2_vpc_route_table

Initial tests for ec2_vpc_route_table ready for boto3 porting

* Turn AWS connection information into a YAML block
7 years ago
Daniel Shepherd cfbe9c8aee [cloud] Add IPv6 support for ec2_vpc_subnet module(#30444)
* Add integration test suite for ec2_vpc_subnet

* wrap boto3 connection in try/except

update module documentation and add RETURN docs

add IPv6 support to VPC subnet module

rename ipv6cidr to ipv6_cidr, use required_if for parameter testing, update some failure messages to be more descriptive

DryRun mode was removed from this function a while ago but exception handling was still checking for it, removed

add wait and timeout for subnet creation process

fixup the ipv6 cidr disassociation logic a bit per review

update RETURN values per review

added module parameter check

removed DryRun parameter from boto3 call since it would always be false here

fix subnet wait loop

add a purge_tags parameter, fix the ensure_tags function, update to use compare_aws_tags func

fix tags type error per review

remove **kwargs use in create_subnet function per review

* rebased on #31870, fixed merge conflicts, and updated error messages

* fixes to pass tests

* add test for failure on invalid ipv6 block and update tags test for purge_tags=true function

* fix pylint issue

* fix exception handling error when run with python3

* add ipv6 tests and fix module code

* Add permissions to hacking/aws_config/testing_policies/ec2-policy.json for adding IPv6 cidr blocks to VPC and subnets

* fix type in tests and update assert conditional to check entire returned value

* add AWS_SESSION_TOKEN into environment for aws cli commands to work in CI

* remove key and value options from call to boto3_tag_list_to_ansible_dict

* remove wait loop and use boto3 EC2 waiter

* remove unused register: result vars

* revert az argument default value to original setting default=None
7 years ago
Will Thames 46c4f6311a [cloud] Add retries/backoff to ec2_vpc_subnet module (#31870)
* Allow backoff for describe_subnets

Improve exception handling to latest standards

* Add integration test suite for ec2_vpc_subnet

* Add test for creating subnet without AZ

Fix bug identified by test

Fixes #31905
7 years ago
Sloane Hertel 1dd55acbc2 ec2_group: add rule description support - fixes #29040 (#30273)
* ec2_group: add support for rule descriptions.

* Document rule description feature and add an example using it.

* Fix removing rule descriptions.

* Add integration tests to verify adding/modifying/removing rule descriptions works as expected.

* Add permissions to hacking/aws_config/testing_policies/ec2-policy.json for updating ingress and egress rule descriptions.

* ec2_group: add backwards compatibility with older versions of botocore for rule descriptions.

* Add compatibility with older version of botocore for ec2_group integration tests.

* ec2_group: move HAS_RULE_DESCRIPTION to be checked first.

* Make requested change

* Pass around a variable instead of client

* Make sure has_rule_description defaults to None

* Fail if rule_desc is in any ingress/egress rules and the the botocore version < 1.7.2

* Remove unnecessary variable

* Fix indentation for changed=True when updating rule descriptions.

* minor refactor to remove duplicate code

* add missing parameter

* Fix pep8

* Update test policy.
7 years ago