Commit Graph

580 Commits (f3f4ad93325725b91902984b419502e8b4b97188)

Author SHA1 Message Date
Rick Elrod 3ec18ccb77
Allow announce script to work for base and older (#69768)
Change:
- Generalize the announce script changes from 4dd0f4127

Test Plan:
Harcoded 'hashes' to [] and then:
- Ran with --version 2.9.9 --version 2.10.0.dev1
- Ran with --version 2.9.9
- Ran with --version 2.10.1

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod 4dd0f41270
Update announce script for ansible-base (#69735)
Change:
- s/Ansible/ansible-base/
- Update URLs

Test Plan:
- Hope for the best. (No real way to test this until release day because
  there are no files on releases.a.c for it to hash right now. A staging
  releng environment would fix this.)

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Toshio Kuratomi efff35a7cd Remove unused imports in the porting_guide command plugin 5 years ago
Toshio Kuratomi 26704b915b Turn pathlib paths into strs
Some APIs do not take a pathlib.  They need to have a string
representation of a path.  Transform the default path to a str
so those APIs will work with the default value.
5 years ago
Toshio Kuratomi 957ad8e769 Fix the command plugin to use the ABCMeta metaclass
The abstract* decorators don't have any effect unless the class has an
ABCMeta metaclass.
5 years ago
Toshio Kuratomi a3c400d0ce Remove outdated docs and unneeded import from change_detection build file 5 years ago
Rick Elrod 96c56b119d Add a script for adding backport references
Change:
- Add a place for adding backport-related scripts in the future
- Add a script for adding backport references

Test Plan:
- Used it for this latest batch of PR reference-adding.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Evgeni Golov cdad594b16
plugin_formatter: sys.exit does not take a file argument (#68016)
Cleanup of leftover from bcdfdc0cc3.

sys.exit does not take any named argument.
5 years ago
Matt Martz a513a62536
Remove cgroup_perf_recap_graph.py, since that callback no longer exists here (#68779) 5 years ago
Matt Clay 7323d5dd0d Fix references to old egg-info directory. 5 years ago
Rick Elrod bd9e31fcb0
Add a custom policy for hackers using ansible-test (#68535)
Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Jill Rouleau 0bf7a08eb6 Remove hacking/aws_config
AWS hacking config has been moved to the AWS CI terminator repo, and is no
longer required in base.
https://github.com/mattclay/aws-terminator/pull/91
5 years ago
Jordan Borean 36fc5840b7
hacking - Add script to easily rebalance Shippable groups (#67976)
* hacking - Add script to easily rebalance Shippable groups

* Fix py26 compile issue

* Add option to automatically change test target aliases
5 years ago
Matt Clay d049888a92 Update Shippable hacking scripts default branch. 5 years ago
Matt Clay 707eea3afa Add new options to `hacking/shippable/incidental.py` (#68384)
* Add `--plugin-path` option to `incidental.py`.

* Report on plugins with no test target.

* Add `--verbose` option to script.
5 years ago
Rick Elrod 649f657f91 Fix coverage script to handle ongoing runs (#68380)
* Fix get_recent_coverage_runs.py to handle ongoing runs

Signed-off-by: Rick Elrod <rick@elrod.me>

* Color code test status to make it easier to see at a glance

Signed-off-by: Rick Elrod <rick@elrod.me>

* fix lint

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay 82516a424e Fix hacking/shippable/incidental.py source search. 5 years ago
Rick Elrod d87d96077f Add script for getting URLs of coverage runs
Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay f18f480a3c Add hacking/shippable/incidental.py script. (#68182)
* Add hacking/shippable/incidental.py script.

* Add README.md.
5 years ago
Rick Elrod 787089cba2 First attempt at making the shippable downloader recurse to pull parent results of reruns (#68154)
* First attempt at making the shippable downloader recurse to pull parent results of reruns

Signed-off-by: Rick Elrod <rick@elrod.me>

* Update message for previous run download.

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Mark Chappell da30e6d2e1
sqs_queue - Move to boto3 and add support for various extra features (#66795)
* reworked sqs_queue

* Switch default purge_tags behaviour to false.

This matches the behaviour of ec2_tag and ecs_tag.

* Minor lint / review fixups

* Add missing AWS IAM policy for SQS tests

* Move integration tests to using module_defaults: group/aws:...

* add changelog

* Break out the 'compatability' map from our spec definition (gets flagged by the schema validation)

* Tweaks based on review

* add basic examples

* Lint fixups

* Switch out NonExistentQueue logic so it's easier to follow

* Reorder name argument options for consistency

Co-authored-by: Dennis Podkovyrin <dennis.podkovyrin@gmail.com>
5 years ago
mmoyle cfe96b2092
add module cloudformation_exports (#67349)
* add module cloudformation_exports

* add RETURN, add aliases group, clean up yaml

* update return value. uncomment security_token. remove cloudformation shortcut

* fix typo

* try to delete test stack

* rename stack

* add cleanup and assert. try to set stack name with variable

* create s3 bucket instead

* set bucket name

* add tests, remove unsed key and import, add iam role, add to module_defaults

* import exceptions, fix assert syntax

* fix assert

* Update test/integration/targets/cloudformation_exports/tasks/main.yml

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* fix export name

* renamed module

Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
5 years ago
Mark Chappell 9c6495d4d4
elb_target / elb_target_info : Integration test fixups (#61256)
* Update AWS policy to enable management of TargetGroups

* elb_target: (integration tests) migrate to using module_defaults

* elb_target: (integration tests) lookup the AMI by name rather than hard coding AMI IDs

* elb_target_info: (integration tests) finish rename of integration test role

* elb_target: (integration tests) rename various resources to consistently use {{ resource_prefix }}

* elb_target_info: (integration tests) Migrate to using module_defaults

* elb_target_info: (integration tests) Lookup AMI by name rather than hard coding AMI IDs

* Apply suggestions from code review

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* elb_target: (integration tests) Remove the 'unsupported' alias

* Try bumping up the timeout

* Rules don't permit 'shippable' (resource_prefix uses this when run in shippable)

* Try bumping up more timeouts :/

* Avoid double evaluation of target_health assertion

* Simplify target_type usage a little (rather than constantly performing a lookup)

* mark elb_target tests 'unstable' for now, they're slow

Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
5 years ago
Matt Clay 19f6fc68d4
Relocate Shippable tools. (#67556)
* Move Shippable tools to hacking directory.

These limits the `test/utils/shippable/` directory to scripts required for CI.

* Fix `test/utils/shippable/` file classification.

* Update package-data sanity test.
5 years ago
Mark Chappell b3db41e6d8
Fix hacking policy (#67579) 5 years ago
Mark Chappell 9edcda7ef7
Refactor iam_role to bring down the complexity score (#66027)
* Simplify BotoCore- / Client- Error try/except loops where we don't need different behaviour

* Refactor IAM Role manipulation to reduce complexity scores

* Missing permissions

* Add retry decorator by default

* compare_attached_role_policies is dead code, remove it
5 years ago
Mark Chappell 8d574c3770
AnsibleAWSModule related cleanup - redshift (#66779)
* AnsibleAWSModule related cleanup - redshift

* Apply a backoff on modify_cluster to cope with concurrent operations

* Add AWS 'hacking' policy to allow creation of Redshift ServiceRole

* Adding the retry policies makes the redshift test suite more reliable
5 years ago
Rick Elrod 236773bfea
Improve distribution unit tests (#67300)
* Include distro.{os,lsb}_release_info() in distro fixture generation script
* Add distro release information to existing fixtures
* Add some new fixtures for newer distribution versions
* Add fixture for Arch Linux with lsb-release
5 years ago
Dick Visser 5b93a14a0f
Add anchor to each parameter row (#66895)
* Add anchor to each paramater row

* Update docs/templates/plugin.rst.j2

Co-Authored-By: Felix Fontein <felix@fontein.de>

* Insert full keys into plugin docs.

* Added visible links.

Co-authored-by: Felix Fontein <felix@fontein.de>
5 years ago
Prasad Katti b16525b841 [aws] integration tests for ec2_vol (#66193) 5 years ago
Toshio Kuratomi 0da13469d3 Add a script to update the intersphinx inventory (#66646)
* Add a script to update the intersphinx inventory files

* We're about to add intersphinx inventories for separate ansible docs
  so we need an easy way to update them.  Also, we should be updating
  these cache files for other upstreams occassionally as well.  With a
  script, we can add updating them to a release process.
* Now that we don't know what the version of the cache is, change the
  filenames to not contain versions.

* Update the intersphinx cache files with the latest upstream versions

Results of running:

hacking/build-ansible.py update-intersphinx-cache -o docs/docsite -c docs/docsite/rst/conf.py

* Add a comment to the configuration file which says how to structure the intersphinx mapping and why.

* Update docs/docsite/rst/conf.py

Co-Authored-By: Sandra McCann <samccann@redhat.com>

Co-authored-by: Sandra McCann <samccann@redhat.com>
5 years ago
Clint Byrum 284f26303c Add support for ECR Lifecycle Policies to ecs_ecr (#48997)
* Fix copy/pasta for ecs_ecr test names

* Add support for lifecycle policies to ecs_ecr

New feature for ecs_ecr to support [ECR Lifecycle Policies][].

Fixes #32003

 [ECR Lifecycle Policies]: https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html

* Improve error message for ecs_ecr parsing errors

Replaces the exception and stack trace with a description of what's
actually going wrong from a user perspective.

* Rename delete policy to purge policy

Marks the `delete_policy` parameter as deprecated, to be removed in
Ansible 2.6.

* Add version_added to purge_policy

* Remove changing results based on verbosity

What I really want is --diff support, and changing results based on
verbosity is abnormal.

* Ensure repository name is lowercase

* Fix deprecation cycle to 4 releases

* Use a YAML anchor for credentials

* Remove filters from assertions

* Add minimal permissions needed

* Updating version_added and deprecation cycle

The original PR sat while a few releases happened.

* Bumping version added and deprecation version

We missed the 2.8 release.

* Removing bare except:

This is not allowed and is generally bad practice.

* Fix lint errors

* update ansible release metadata

* Use the new alias deprecation scheme

This was added in the time the PR has been in development, so rework
things to use it.

* Add test coverage

This makes sure that lifecycle_policy is produced when passed in.

*Also a minor suggestion for simplification from PR.

* Restore changes from 62871 lost in rebase

* Add changelog

* Remove version_added for new purge_policy option

Per sanity test fail.
5 years ago
Prasad Katti b8729b2544 cloudformation integration tests (#65643) 5 years ago
Prasad Katti 056b035c98 add module aws_step_functions_state_machine_execution (#64431)
* add module aws_step_functions_state_machine_execution

* AWS step functions tests - Use module defaults

* Return all attributes from aws api calls as ansible task output

* aws_sfn - make start and stop execution idempotent and fix check mode

* aws sfn - use build_full_result method of the paginator

* aws sfn - remove changes made to help with local debugging
5 years ago
Prasad Katti 37ce55fd79 lightsail - Use AnsibleAWSModule (#65275)
* lightsail - Use AnsibleAWSModule

- Use AnsibleAWSModule
- Refactor the logic for wait into a separate function (Fixes #63869)
- Handle exceptions in find_instance_info and add a fail_if_not_found parameter
- Add a new state `rebooted` as an alias for `restarted`. AWS calls the action Reboot.
- Add required_if clause for when state is present

* lightsail - Use the default keypair if one is not provided

* lightsail - add a required_if for when state=present

* Update short description for lightsail module
5 years ago
Prasad Katti 95bd92da04 Add integration tests for aws lightsail (#63770)
* Add integration tests for aws lightsail

* lightsail - use module_defaults instead of aws_connection_info

* lightsail tests - assert instance state on create

* Fix yaml syntax error

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* [lightsail] create keypair as part of the testsuite

* Fix lightsail actions in compute-policy

* Add ability to delete keypair in lightsail_keypair
5 years ago
Mark Chappell a815fdf8bb Update Route53 IAM policy so the Route53 tests run (#64886) 5 years ago
Mark Chappell 551b17b8a2 ec2_vpc_net_info: integration tests (#62649)
* ec2_vpc_net: (integration tests) migrate to using module_defaults

* ec2_vpc_net: (integration tests) use a private subnet for the tests

* ec2_vpc_net_info: Add integration tests

* ec2_vpc_net_info: add cidr_block_association_set to documentation

* Update AWS hacking test policy to allow VPC CIDR disassociation

* Update test/integration/targets/ec2_vpc_net/tasks/main.yml

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* Store vpc2 ID to make it clearer which VPC we're changing

* Be more consistent with our quoting

* Explicitly test that the VPC IDs haven't changed
5 years ago
Mark Chappell b5f484dcc3 ec2_eip Don't throw an exception when re-releasing an EIP (idempotency) (#62332)
* ec2_eip: (integration tests) move to using module_defaults

* ec2_eip: (integration tests) expand integration tests

Also clean up a little
- Delete EIPs when we finish testing them (reduce the chance of hitting limits)
- Rejig deletion so that it works when runs fail
- Add tests for ec2_eip_info

* ec2_eip: Minor doc tweaks

* ec2_eip: Don't throw an exception when we try to disassociate an already disassociated EIP

* ec2_eip: Add missing IAM policy (manage IGWs)

* ec2_eip: (integration tests) Use the VPC as a crude lock to avoid running parallel tests

We test that untagged EIPs come and go as we expect, if multiple tests are
running in parallel this confuses things

* Fix ec2_eip association
5 years ago
Matthew Davis 4ee9f40e62 Add aws_acm module (#60552)
* convert aws_acm_facts to AnsibleAWSModule

* factor aws_acm_facts into module_utils

* add more filtering options for aws_acm_info

* add aws_acm module and tests

* uncomment aws_acm test

* fix linting for aws_acm

* fix __future__ linting for aws_acm

* fix linting for aws_acm

* fix linting for aws_acm

* fix linting for aws_acm

* fix linting for aws_acm

* fix aws_acm_info arg type

* remove test for old module name aws_acm_facts

* simplify AWS ACM client creation

* fix indent typo in aws_acm test

* catch BotoCoreError in aws_acm

* fix indent typo in aws_acm test

* tighten AWS ACM test policy resource

* move aws acm int test to venv

* remove errant file

* fix AWS ACM int test perms

* undo copyright addition to wrong file

* fix invalid log message in aws_acm

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* rephrase aws_acm_info doc from facts to information

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* rename aws_facts var to aws_info

* remove case insensitivity for aws_acm pem compare

* add no_log for aws_acm credential setting

* add per-test prefix to aws_acm test resource names

* make aws_acm use crypto module_util

* clarify copyright for aws_acm

* make aws_acm int test clearer

* add explicit crypto dependency to aws_acm

* change requests for aws_acm pr

* fix wrong copyright owner aws_acm test

* fix wrong copyright owner aws_acm test

* rewrite aws_acm cert chain compare with regex, no dependency

* fix linting for aws_acm unit test

* fix linting for aws_acm unit test

* fix linting and duplicate ignore

* fix failed cert chain split in aws_acm, add more tests

* remove errant file

* more linting fixes for aws_acm

* fix sanity ignore

* rewrite cert compare in aws_acm to use base64 decode

* improve regex for pem cert chain split in aws_acm

* undo changes to crypto module util for aws_acm

* increment ansible version for new aws_acm module

* convert aws_acm return(x) to return x

* increment version added for aws_acm_info new features

* fix linting

* fix bugs with AWS ACM

* fix bad rebase

* disable AWS ACM integration test, due to AWS account limit issue

* remove aws acm integration test from shippable group
5 years ago
Sandra McCann c63ef6d911 update too old version to 2.4 (#64167) 5 years ago
Mark Chappell ef7d060a3f AWS module_utils: Clear out Sanity Test issues (#63991) 5 years ago
Matt Martz 553767ef50
Reword the ticket stub for collections (#63917) 5 years ago
Mark Chappell 4d72b69035 rds_subnet_group : Sanity Check fixes (docs) and Integration tests (#63214)
* rds_subnet_group: Fixup sanity test issues

* rds_subnet_group: Add integration tests

* rds_subnet_group: Add testing policy
5 years ago
Sam Doran 4de516d35f Do not set ANSIBLE_LIBRARY in env-setup.fish (#63688)
Setting this env var was removed from env-setup back in 2014. Not sure why it is in this setup script and it interferes with the defult search path
5 years ago
Mark Chappell 8d0737edf0 Integration tests for s3_logging (#63257)
* s3_logging: (integration tests) updated AWS policy

* s3_logging: fix sanity test issues

* s3_logging: Integration tests

* Add pauses to cope with evenual consistency

* Mark s3_logging tests as 'unsupported' for now due to testing instability
5 years ago
Mark Chappell dbc9444572 ec2_vpc_nacl and ec2_vpc_nacl_info migrate to AnsibleAWSModule and add tests (#63163)
* Move EC2 networking objects into network-policy.json

* ec2_vpc_nacl: Add integration tests

* ec2_vpc_nacl: Migrate tests to use module_defaults

* ec2_vpc_nacl: (integration tests) Add missing AWS permissions

* ec2_vpc_nacl: (integration tests) Update tests for ipv6 support

* ec2_vpc_nacl: Migrate to AnsibleAWSModule

* Fix sanity tests for ec2_vpc_nacl and ec2_vpc_nacl_info

* ec2_vpc_nacl_info: Migrate to AnsibleAWSModule

* ec2_vpc_nacl_info: (integration tests) Rename from ec2_vpc_nacl_facts to ec2_vpc_nacl_info and add a test using a filter (by tag)

* Pick availability zones dynamically

Rather than assuming that AZa and AZb always exist (they don't), query to find out which AZs we have available first

* Test that the NACLs we get back are actually the *saml* NACL rather than duplicates/delete remove

* Cleanup IPv6 tests a little.

Note: IPv6 support for ec2_vpc_nacl not complete yet.

This provides the initial framework, and should ensure things don't start exploding when support is added.

* Removing subnets by name from a NACL *is* now supported

* Fix ec2_vpc_nacl return documentation
5 years ago
Mark Chappell 0239f70648 cloudtrail: Initial integration tests (#61919) 5 years ago
Mark Chappell 40660e7f6e iam_role : support managing max session duration and deleting the instance profile it creates (#62014)
* iam_role: Add support for managing MaxSessionDuration

* iam_role: Add support for deleting the IAM Instance Profiles we created

* iam_role: migrate all boto failures to fail_json_aws for consistency

* iam_role: test validity of path so we can throw a more understandable error

* iam_role: (integration tests) Split iam_role integration tests from sts_assume_role tests

- Make the iam_role tests more comprehensive
- Add tests for iam_role_info

* iam_role: (integration tests) Make some of our pauses optional

If the tests appear to be flakey we may need to enable standard_pauses
5 years ago
Felix Fontein 4393940a41 Docs formatting: recursively preprocess suboptions (#61900)
* Simplify code, move option massaging code into own function.
* Process suboptions recursively.
5 years ago
Toshio Kuratomi b437a19f6d
create-deprecated-issues script can now add to a specified project (#61901)
* create-deprecated-issues script can now add to a specified project

* Migrate the create deprecated issues script into a subcommand of build-ansible

* Remove deprecated-issue script from ignore list
5 years ago
Mark Chappell e0ebc8c9b4 Fixup aws_secret integration tests (#61241)
* aws_secret: (integration tests) Move tests to using module_defaults

* Update hacking aws security policy to enable management of secrets

* aws_secret: (integration tests) Fixup integration tests
- Update tests to use resource_prefix as a prefix rather than a suffix
- Pause after role creation to cope with AWS being slow (and returning before the role it ready)
5 years ago
Tom De Keyser 6f74fca238 New module for AWS Step Functions state machines (#59116)
* add new module: aws_stepfunctions_state_machine

* add integration tests for new module: aws_stepfunctions_state_machine

* fix sanity checks

* use files/ folder instead for integration test

* rename role name in integration test

* attempt further permissions

* iam states prefix

* iam integration test prefix

* add iam policy for running step functions state machine actions

* slightly increase iam permission scope

* rename integration test folder to proper name

* move main() method to end of file

* move contents of integration-policy.json for state machines to compute-policy.json

* make check_mode return proper changed value + add check_mode integration tests

* rename module to aws_step_functions_state_machine

* fix missed rename in integration test variable

* add purge_tags option

* bump to version 2.10
5 years ago
Matt Martz 4e2c70c13e
Include version_added in report.py (#61857) 5 years ago
Matt Martz 02c7f2f518
Add collections ticket stub (#62001)
* Add collections ticket stub

* typo

Co-Authored-By: John R Barker <john@johnrbarker.com>
5 years ago
Mark Chappell b8650c0a50 aws_asg: Fix idempotency when using tags and metrics (#61284)
* Update AWS hacking policy to enable ASG Tagging management

* aws_asg: Add tests for ASG Tagging (including idempotency)

* aws_asg: ignore sort order when comparing tags on the ASG (fix idempotency)

* ec2_asg: (integration tests) test for idempotency when managing metrics collection

* ec2_asg: sort list of enabled metrics to ensure clean comparisons.
5 years ago
Mark Chappell 832e03d932 Fixup iam_group integration tests and return value documentation (#61243)
* iam_group: (integration tests) migrate tests to module_defaults

* iam_group: (integration tests) migrate to using temporary user and group with {{ resource_prefix }}

* iam_group: (integration tests) fix test, checking the return values

* iam_group: (integration tests) Add some more tests around the behaviour of 'changed'

* iam_group: (docs) Update documentation of iam_group return value

* Update AWS testing policies to enable group/user management
5 years ago
Mark Chappell adfaefb732 ec2_launch_template: Fix integration tests (#61260)
* ec2_launch_template: (integration tests) make sure security_token is optional

* ec2_launch_template: (integration tests) add dependencies at the top level so they're pulled into the docker containers

* Update Hacking Compute Policies for Launch Templates
5 years ago
Mark Chappell 35359959de aws_codecommit: Fix integration tests and Add support for updating the description (#61263)
* Update DevOps AWS policy
- Fix typos in permission names
- While AWS claims you can use 'arn:aws:codecommit:*' it errors unless you use '*'

* aws_codecommit: (integration tests) Migrate to module_defaults

* aws_codecommit: (integration tests) Fix integration tests

* aws_codecommit: (integration tests) Add tests for updating the description

* aws_codecommit: Add support for updating the description and rename "comment" option to "description"
5 years ago
Toshio Kuratomi 8f6dbb732a
[WIP] Cleanups and version bumping for 2.10 (#61572)
* Cleanups and version bumping for 2.10

* Fix changelog url now that stable has been branched

* Fix the lenth of the porting guide title now that the version is two digits
5 years ago
Matt Clay f510d59943
Support relative imports in AnsiballZ. (#61196) 5 years ago
Mark Chappell 1f733e2d55 ec2_asg: fix the integration tests (#61212) 5 years ago
Mark Chappell 77e4371460 aws_kms: Update policy on existing keys (when passed) (#60059)
* aws_kms: (integration tests) Use module_defaults to reduce the copy and paste

* aws_kms: (integration tests) make sure policy option functions.

* aws_kms: (integration tests) Move iam_role creation to start of playbook.

iam_roles aren't fully created when iam_role completes, there's a delay on the Amazon side before they're fully recognised.

* aws_kms: Update policy on existing keys (when passed)
5 years ago
Mark Chappell 70777020c4 Fix iam_password_policy integration tests (#60930)
* iam_password_policy: (integration tests) Use module defaults for AWS connection details

* iam_password_policy: (integration tests) Ensure the policy is removed when tests fail

* iam_password_policy: (integration tests) Add regression test for #59102

* iam_password_policy: Only return changed when the policy changes.

* iam_password_policy: PasswordReusePrevention must be omitted to remove/set to 0

* #60930 add changelog

* Update hacking AWS security policy to allow testing of Password Policy Management
5 years ago
Toshio Kuratomi d9b3af523b Galaxy meta docs table (#60171)
* Use an rst table instead of a raw html table

* Rst is easier to read so we want to use it wherever possible
* Fix the jinja2 filters which create links so that they do not include
  extraneous whitespace in the URL

* Normalize description data before sending them to the templates
5 years ago
Matt Clay d651bda123
Relocate ansible-test code. (#60147)
* Initial move of `test/runner/` content.

`test/runner/lib/` -> `test/lib/ansible_test/_internal/`
`test/runner/`     -> `test/lib/ansible_test/_internal/data/`

* Initial move of `test/sanity/` content.

`test/sanity/` -> `test/lib/ansible_test/_internal/data/sanity/` (except `test/sanity/ignore.txt`)

* Initial move of `test/units/pytest/` content.

`test/units/pytest/` -> `test/lib/ansible_test/_internal/data/pytest/`

* Follow-up move of `test/runner/unit/` content.

`test/lib/ansible_test/_internal/data/unit/` -> `test/lib/ansible_test/tests/unit/`

* Initial move of `ansible.cfg` content.

`test/units/ansible.cfg` -> `test/lib/ansible_test/_internal/data/units/ansible.cfg`
`test/env/ansible.cfg` -> `test/lib/ansible_test/_internal/data/env/ansible.cfg`

* Follow-up move of `data` directory.

`test/lib/ansible_test/_internal/data/` -> `test/lib/ansible_test/_data/`

* Update import statements.

* Add missing __init__.py for unit tests.

* Fix path references and miscellaneous issues.
5 years ago
Jesse Evers e410dcbfed Add logic to handle multiple actions in an ALB listener rule, Fixes #41861 (#41975)
* added logic to handle multiple actions in an ALB listener rule (#41861)

* fix linting and pep8 issues

* added test for multiple actions using OIDC authentication

* added error messages related to old versions of botocore and multiple actions

* fix action validation error checks (need to check the exception string)

* added logic to make oidc configs idempotent (remove clientsecret for check)

* modified TargetGroupName to TargetGroupArn substitution to account for multiple rule actions

* refactored tests so that it can be run against different versions of botocore

* fix runme.sh to refelct changes to cloud testsuite

* add UseExistingClientSecret to oidc config (AWS api change)

* remove tests for OIDC auth action; add tests for redirect and fixed-response

* add in fixes from markuman and mjmayer

* remove documentation for cognito integration (not sure how to test); added example config for fixed-response and redirect actions

* renamed oidc/multiple action tests; leaving commented due to some AWS API changes

* pep8 fix

* more pep8 fixes

* Restructure elb_application_lb test suite

Move from runme.sh to virtualenv based roles

Update policies to fix tests

Don't log temp dir deletion, so many files in the diff!
5 years ago
Toshio Kuratomi 0e9f002073 Fix build-ansible.py release-announcement
* When the release scripts were changed, this file to implement release
  announcements was left out

* Add announce.py to the compile skip lists as only the release engineers
  run this during package build.
5 years ago
Will Thames 60c76be03c rds_instance: add point-in-time instance restore test (#59411) 5 years ago
Jordan Borean 65049620ee
Generate galaxy.yml based on single source of truth (#59170)
* Generate galaxy.yml based on single source of truth

* Fix up tests and align file names

* Minor Makefile tweak

* Remove link in galaxy.yml file and make it a template file

* Moved collections docs to dev_guide

* change Makefile clean path

* Added readme to example meta file

* review fixes

* Use newer style for doc generation script

* Fix mistake in dev_guide index

* removed uneeded file, fixed links and added preview banner

* Moved banner for sanity test
5 years ago
Aljaž Košir 42073b6331 Add lambda_bucket_event module (#58059) 5 years ago
Toshio Kuratomi 5392d8697d
Docs scripts cleanup (#59169)
* Fix dump_keywords to require attribute docs

* Change the documentation commands to make definition files positional

Since the definition files are mandatory, make them positional
parameters instead of options.
5 years ago
Toshio Kuratomi 019d078a5a
Move common build code from _build_helpers (#55986)
We have some common code used by several docs scripts.  Migrate that
into the build-only shared code repository.

* Move lib/ansible/utils/_build_helpers.py to the directory for common
  build code
* Migrate docs/bin/dump_config.py to a build-ansible subcommand
* Migrate dump_keywords to the build-ansible framework
  * Make the script more maintainable by using functions and good
    variable names
  * Port to Python3 idioms
  * Fix bug so that private attributes will be undocumented
* Move generate_man to a build-ansible subcommand
* Port plugin_formatter to a build-ansible subcommand
* Rework command_plugins so that docs scripts can target Python-3.4+ and
  releng-only subcommands can use more recent versions of Python.
  The architecture is now that command_plugins/* need to be importable
  on Python-3.4.  The init_parsers() method needs to run on Python-3.4.
  But the main() method can utilize features of more recent Python as
  long as it fits within those parameters.
* Update docs build requirements

Port the plugin_formatter to build-ansible framework
5 years ago
Toshio Kuratomi 33d2728879 Rename python files in hacking/ directory to have .py suffix
ansible-test only passes files which have the .py suffix for sanity
tests on python files.  This change will allow sanity tests to run on
the Python files in hacking/

* Rename test-module to test-module.py
* Symlink test-module for backwards compat since end users may be using
  test-module
* Fix test-module sanity errors that are now triggered
* Rename ansible_profile to ansible-profile.py
* Rename build-ansible
5 years ago
Will Thames 60fb9fc208 Fix EC2 test suite to work with testing policies (#44387)
* Update testing policies to ensure all required permissions are present
* Tidy up security policies to reduce duplicate permissions
* Make roles static so that they can be present before CI is run,
  meaning that role creation permission is not required by the CI
  itself, only by someone setting up the roles prior to testing
* Move contents to cloudfront policy to network policy to ensure policy
  count (maximum of 10) stays low
* Maintain compute policy below 6144 bytes
5 years ago
Will Thames a6d757e074 Ensure block device instance creation outputs instances
Not waiting outputs results in a format that will never
be matched by the tests

Ensure instances get tidied up

Allow ec2:ReportInstanceStatus

ec2_instance: Improve test cleanup on failing tests

Improve describe/modify attribute error handling

Address feedback on PR
5 years ago
Sam Doran ab6a9ef130
Get minor version number for CentOS and Debian (#57814)
* Get the most detailed version number from distro.version() for CentOS and Debian
* Update tests and fixtures
* Update fixture generation script to gather distro info and work with Python 3
* Update LinuxMint fixtures
* Cleanup fixture formatting
* Improvements based on feedback from abadger:
    - use unicode since that is what distro returns
    - use frozenset with a tuple
    - include link Debian to bug
5 years ago
Elijah Lynn 19c6448459 Fix "no matches for wildcard" error in env-setup.fish script. (#58345)
See https://github.com/ansible/ansible/pull/58345#discussion_r297288979
5 years ago
Will Thames eda5dd826f Add rds_snapshot module (#39994)
* new module uses modern ansible AWS standards
* adds additional tests for snapshots
* Update return_skeleton_generator for python3 - should
  set type to `str`, not `string`.
5 years ago
Will Thames a09aa205e1 Fix RDS test suite and minor bugs revealed (#57940)
* Update testing policy to be correct for RDS test suite
* Create read replica in same region to avoid more permissions being
  required
* Ensure modifying DB doesn't try to downgrade engine version
* Add tags to main test suite to limit number of tests run for problem
  solving
5 years ago
Kohei Asano ddf6d096c5 Support the new TLS termination on NLBs (#51327) (#58031) 5 years ago
Will Thames 924352a051 ecs_cluster test suite refactor (#57716)
* Combine testing policies

Because of the maximum of 10 policies per group, need to
consolidate testing policies as best we can.

* Tidy put-account-setting tasks and add permission

Using `environment` and `command` rather than `shell` avoids the
need for `no_log` and means that people can fix the problem

* refactor ecs_cluster test suite

move from runme.sh technique to virtualenv

use ec2_instance rather than ec2 module to
avoid need for boto
5 years ago
Stefan Horning 77ec0549b0 New module for AWS CodeBuild (#47187)
* New AWS module for the CodeBuild service, called aws_codebuild

* Integration test for new aws_codebuild module
5 years ago
Ed Costello 2013d4abc4 Update setup-iam playbook to use aws_caller_info rather than deprecated (#57675)
aws_caller_facts
5 years ago
Stefan Horning ddfaa83ccf s3_bucket: add encryption capabilities to the module (#55985)
* s3_bucket: add encryption capabilities to the module
6 years ago
mjmayer c8e179fbf1 Aws waf region (#48953)
* Add waiter for AWSRegional

* Add support for WAF Regional

* Add support for regional waf web acl

* Remove set_trace, pep formatting

* Add paginator for regional_waf

* Change name of param for waf_regional

This is more in line with how AWS refers to the service. Additional
 changes made to how client is called. Used ternary to reduce if
 statements

* Change parameter name to waf_regional

* Add support for removal waf regional condition

* Change parameter from cloudfront to waf_regional

* Added state: absent waf rule

* Remove set_trace

* Add integration tests for waf regional

* WIP: adding region parameter to tests

* Add support for waf facts module

* Add region to waf regional integration tests

* Update security policy for waf regional testing

* Add type to documentation for waf_regional param
6 years ago
Toshio Kuratomi 3161a91d7e
Implement a framework for having common code for release scripts (#55893)
* Implement a framework for having common code for release scripts

* Release scripts will go through hacking/build-ansible.  build-ansible is
  a pluggable script which will set a directory that has common code for
  non-enduser scripts.  It will then invoke the plugin which implements
  that subcommand.  Uses straight.plugin for loading each sub-command.

* We're going to add tools which are needed to test ansible (the changelog
  generation, for instance) so we need to include the pieces relevant to
  that in the tarball.

* Add straight.plugin to the sanity test requirements for the same
  reason

* Skip compile test just for build-ansible plugins which won't be run as
  part of sanity tests.
6 years ago
Matt Martz 65dcb4242a
Fix deprecated issue creator (#55327) 6 years ago
Toshio Kuratomi e5a31e81b6 Add a script to generate a new porting guide
* porting-guide.py is allowed to be Python-3.6+ as it will only be run
  by release engineers
6 years ago
Karolis Tamutis 54384e7a12 Make test-module use default value for interpreter (#54053)
* Make test-module use default value for interpreter

* Changing from static interpreter path to sys.executable as per #54053

* A little ntegration test for #54053
6 years ago
Toshio Kuratomi 8dd46d6f95
Fix punctuation around issue reporting link (#54295)
Figure out how to format the release announcement so a link isn't
directly followed by a period which would hinder cutand paste but uses
proper grammar and punctuation.
6 years ago
Toshio Kuratomi 5fb416ae34 Add a script to generate twitter and mailing list announcements
Announcements taken from https://github.com/ansible/community/wiki/RelEng:-ReleaseProcess
and then cleaned up:

* Update issue reporting blurb from feedback from acozine and gundalow
* Add a subject and to line for email output
* Ignore long line tests on the jinja templates (as jinja doesn't give
  enough control to get rid of newlines when text wrapping)
* Skip shebang and compile tests for older pythons since this is a
  release engineer-only script.  (ok'd by mattclay)
6 years ago
Sloane Hertel 7da565b3ae
parse botocore.endpoint logs into a list of AWS actions (#49312)
* Add an option to parse botocore.endpoint logs for the AWS actions performed during a task

Add a callback to consolidate all AWS actions used by modules

Added some documentation to the AWS guidelines

* Enable aws_resource_actions callback only for AWS tests

* Add script to help generate policies

* Set debug_botocore_endpoint_logs via environment variable for all AWS integration tests

Ensure AWS tests inherit environment

(also remove AWS CLI in aws_rds inventory tests and use the module)
6 years ago
Bob Boldin b67505d271 AWS: new module ec2_transit_gateway fixes #49376 (#53651)
* AWS: new module ec2_transit_gateway fixes #49376

* Add permissions neeeded for integration tests

* uncomment nolog on creds

* add unsupported to integration test aliases

* remove the shippable/aws/group alias so doesn't conflict with unsupported
6 years ago
Andrea Tartaglia 5c6b16edc3 Fix ec2_instance eventual consistency when wait: false (#51885)
* Do not return 'instances' when wait is false

* Added integration tests for wait: false

* Added changelog fragment

* Fix test suite to work with ec2_instance

* Additional permissions
* Enforce boto3 version
* Fix broken tests
* Improve error messages

* fix linter issues
6 years ago
Tad Merchant b979b26a74 Add launch type to ecs task (#49081)
* adds fargate launch_type to ecs_task module

* White space changes

* fix documentation for running ecs task on fargate

* remove extraneous example from ecs_task

* White space changes

* Adds changelog fragment

* Pluralize minor_changes in changelog fragment

* Add Stop and Start task permissions
6 years ago
Rafael Driutti c68838fb13 AWS Redshift: port module to boto3 and fix parameters check (#37052)
* fix parameters check and port module to boto3

* begin with integration tests

* allow redshift iam policy

* Wait for cluster to be created before moving on to delete it

* Allow sts credentials so this can be run in CI

Don't log credentials

ensure cluster can be removed

* - Replace DIY waiters with boto3 waiters
- test multi node cluster

* catch specific boto3 error codes

* remove wait from test

* add missing alias for shippable

* - Rework modify function.
- Default unavailable parameters to none.
- Add cluster modify test

* Ensure resources are cleaned up if tests fail

* Ensure all botocore ClientError and BotoCoreError exceptions are handled
6 years ago
Will Thames 46fbcf08bc
aws_kms enhancements (#31960)
* Allow creation and deletion of keys (deletion just schedules for
  deletion, recreating an old key is just cancelling its deletion)
* Allow grants to be set, thus enabling encryption contexts to be
  used with keys
* Allow tags to be added and modified
* Add testing for KMS module
* Tidy up aws_kms module to latest standards
6 years ago
Toshio Kuratomi dfd8b659c0 Have update-bundled check for updates to all bundled code
Add a test for whether we have bundled code inside of ansible that needs to be updated
6 years ago
Matt Martz 7a89d373ac Perf graphing (#46346)
* csv of memory usage

* Fix var

* Configurable output file

* Add cpu profiling

* Valdiate the existence of cgroup files

* Add guard to prevent exception when trying to reset max memory value

* to_bytes/to_text and docs updates

* Add support for CPU results

* Just track the max, don't log all results, and then calculate max

* Restore cgroup_memory_recap, and move new functionality into cgroup_perf_recap

* Add pid count tracking, restructure to support more profilers

* Add cli tool for graphing cgroup_perf_recap data

* csv_output_dir is a path

* Correct CALLBACK_NAME

* Include uuid in csv data

* fix linting errors

* Bump version_added

* Create helper funciton to create dict from list of keys, with callable default

* Updated notes to include pids

* Print a newline after each section

* Plugin improvements

* Add option to supporess recap display
* Add default for output directory
* Add option to dictate whether or not to write files
* Add JSON-seq output option

* s/uuid/task_uuid

* Use bytes for paths

* Increase polling interval length for pids/memory

* Reduce instance attrs, change how we invoke profilers

* Shorten some line lengths

* Remove more instance attrs

* Fix some typos

* document directory creation, and catch exceptions

* Enable per task file outputs, and filename customization

* s/per_task_file/file_per_task/g
6 years ago