You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ansible/docs/docsite/rst/dev_guide/testing_integration.rst

229 lines
7.9 KiB
ReStructuredText

:orphan:
.. _testing_integration:
*****************
Integration tests
*****************
.. contents:: Topics
The Ansible integration Test system.
Tests for playbooks, by playbooks.
Some tests may require credentials. Credentials may be specified with `credentials.yml`.
Some tests may require root.
Quick Start
===========
It is highly recommended that you install and activate the ``argcomplete`` python package.
It provides tab completion in ``bash`` for the ``ansible-test`` test runner.
Configuration
=============
ansible-test command
--------------------
The example below assumes ``bin/`` is in your ``$PATH``. An easy way to achieve that
is to initialize your environment with the ``env-setup`` command::
source hacking/env-setup
ansible-test --help
You can also call ``ansible-test`` with the full path::
bin/ansible-test --help
integration_config.yml
----------------------
Making your own version of ``integration_config.yml`` can allow for setting some
tunable parameters to help run the tests better in your environment. Some
tests (e.g. cloud) will only run when access credentials are provided. For more
information about supported credentials, refer to the various ``cloud-config-*.template``
files in the ``test/integration/`` directory.
Prerequisites
=============
Some tests assume things like hg, svn, and git are installed, and in path. Some tests
(such as those for Amazon Web Services) need separate definitions, which will be covered
later in this document.
(Complete list pending)
Non-destructive Tests
=====================
These tests will modify files in subdirectories, but will not do things that install or remove packages or things
outside of those test subdirectories. They will also not reconfigure or bounce system services.
.. note:: Running integration tests within Docker
To protect your system from any potential changes caused by integration tests, and to ensure a sensible set of dependencies are available we recommend that you always run integration tests with the ``--docker`` option, for example ``--docker centos8``. See the `list of supported docker images <https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_data/completion/docker.txt>`_ for options (the ``default`` image is used for sanity and unit tests, as well as for platform independent integration tests such as those for cloud modules).
.. note:: Avoiding pulling new Docker images
Use the ``--docker-no-pull`` option to avoid pulling the latest container image. This is required when using custom local images that are not available for download.
Run as follows for all POSIX platform tests executed by our CI system::
ansible-test integration --docker fedora29 -v shippable/
You can target a specific tests as well, such as for individual modules::
ansible-test integration -v ping
Use the following command to list all the available targets::
ansible-test integration --list-targets
.. note:: Bash users
If you use ``bash`` with ``argcomplete``, obtain a full list by doing: ``ansible-test integration <tab><tab>``
Destructive Tests
=================
These tests are allowed to install and remove some trivial packages. You will likely want to devote these
to a virtual environment, such as Docker. They won't reformat your filesystem::
ansible-test integration --docker fedora29 -v destructive/
Windows Tests
=============
These tests exercise the ``winrm`` connection plugin and Windows modules. You'll
need to define an inventory with a remote Windows 2008 or 2012 Server to use
for testing, and enable PowerShell Remoting to continue.
Running these tests may result in changes to your Windows host, so don't run
them against a production/critical Windows environment.
Enable PowerShell Remoting (run on the Windows host via Remote Desktop)::
Enable-PSRemoting -Force
Define Windows inventory::
cp inventory.winrm.template inventory.winrm
${EDITOR:-vi} inventory.winrm
Run the Windows tests executed by our CI system::
ansible-test windows-integration -v shippable/
Tests in Docker containers
==========================
If you have a Linux system with Docker installed, running integration tests using the same Docker containers used by
the Ansible continuous integration (CI) system is recommended.
.. note:: Docker on non-Linux
Using Docker Engine to run Docker on a non-Linux host (such as macOS) is not recommended.
Some tests may fail, depending on the image used for testing.
Using the ``--docker-privileged`` option when running ``integration`` (not ``network-integration`` or ``windows-integration``) may resolve the issue.
Running Integration Tests
-------------------------
To run all CI integration test targets for POSIX platforms in a Ubuntu 16.04 container::
ansible-test integration --docker ubuntu1604 -v shippable/
You can also run specific tests or select a different Linux distribution.
For example, to run tests for the ``ping`` module on a Ubuntu 14.04 container::
ansible-test integration -v ping --docker ubuntu1404
Container Images
----------------
Python 2
````````
Most container images are for testing with Python 2:
- centos6
- centos7
- fedora28
- opensuse15py2
- ubuntu1404
- ubuntu1604
Python 3
````````
To test with Python 3 use the following images:
- fedora29
- opensuse15
- ubuntu1604py3
- ubuntu1804
Legacy Cloud Tests
==================
Some of the cloud tests run as normal integration tests, and others run as legacy tests; see the
:ref:`testing_integration_legacy` page for more information.
Other configuration for Cloud Tests
===================================
In order to run some tests, you must provide access credentials in a file named
``cloud-config-aws.yml`` or ``cloud-config-cs.ini`` in the test/integration
directory. Corresponding .template files are available for for syntax help. The newer AWS
tests now use the file test/integration/cloud-config-aws.yml
IAM policies for AWS
====================
Ansible needs fairly wide ranging powers to run the tests in an AWS account. This rights can be provided to a dedicated user. These need to be configured before running the test.
testing-policies
----------------
[backport][2.10]Docs backportapalooza3 (#70695) * Suggest ansible ad-hoc command while developing module (#70308) If a local module has no documentation, the doc command will fail without any hints of what is wrong. Add another way to confirm the presence of a local module. * Update docs/docsite/rst/dev_guide/developing_locally.rst Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit 82e5d03bdb6ebff0d8548a5d78296a6c96c8ea32) * Update AWS Integration test docmentation (#70454) (cherry picked from commit e1ba7dc52a0a9d8c81e92323a4d15c11171dba5c) * Update hacking/shippable docs. The `--all` option downloads more than is needed for analyzing code coverage. (cherry picked from commit fb7740ae3b5b660bd7f4a845293aab1612b24b9b) * Document that subversion module requires subversion (#70537) (cherry picked from commit 64c2cb273f91ea54275db85b5cee1d3e7ad117da) * update documentation link to python 3 (#70509) update the docs link for strftime on the filters page to point to the python3 docs (cherry picked from commit f7db428375bc39ce3e71c5a595dcd53bfe0d7425) * Update windows_winrm.rst (#70306) gcc also needed to be installed alongside python-devel, krb5-devel, krb5-libs, and krb5-workstation. (cherry picked from commit f4ea43c4a0c0bc718136de6c7e33a7ae111c1fad) * updated requirements file for docs build (#70609) (cherry picked from commit 38ccfb4a3e33fcaec54d82900d67e20226374f65) * Clarify that index_var is 0 indexed (#70548) A little further down the page is another index, ansible_loop.index, which shares a similar description but is 1 indexed. Its zero indexed twin has a 0 suffix. ``ansible_loop.index`` The current iteration of the loop. (1 indexed) ``ansible_loop.index0`` The current iteration of the loop. (0 indexed) To remove ambiguity around the usage of index_var, explicitly mention that this variable is 0 indexed. (cherry picked from commit c410311f55b3f00831b8d3de06fbd3084f705a3d) * docs: update module development docs (#70594) Update module development docs for flattened modules directory. Fixes: #70261 (at least partially) Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit 20209c508f13b018b8f44f77749001979aa5f048) * Doc: fix examples of changelog entries. (#70551) (cherry picked from commit edcd1a1a7003dd918f312919ce3bc043d7d4e8c9) * Document tags are not supported with task meta. (#70590) fixes #70338 (cherry picked from commit 40591d5fbbe9878427fc5b1b46ec820f69feba1a) * docs: update date format in removed_at_date (#70597) removed_at_date requires YYYY-MM-DD format. Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit 375c6b4ae4b809eace0ef6783e70349d04d5dc6a) * partial update of community docs to reflect collections transition (#70488) (cherry picked from commit f1f782fc371e239a3f18aa3551506e8a4326b72e) Co-authored-by: Michael Ritsema <michaelritsema@users.noreply.github.com> Co-authored-by: Mark Chappell <mchappel@redhat.com> Co-authored-by: Matt Clay <matt@mystile.com> Co-authored-by: Alan Rominger <arominge@redhat.com> Co-authored-by: FloMiau <37121807+FloMiau@users.noreply.github.com> Co-authored-by: mahadelmi <mahadelmi@cmail.carleton.ca> Co-authored-by: Sayee <57951841+sayee-jadhav@users.noreply.github.com> Co-authored-by: Karl Goetz <goetzk@users.noreply.github.com> Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com> Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru> Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com> Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
4 years ago
The GitHub repository `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator/>`_
contains two sets of policies used for all existing AWS module integratoin tests.
The `hacking/aws_config/setup_iam.yml` playbook can be used to setup two groups:
- `ansible-integration-ci` will have the policies applied necessary to run any
integration tests not marked as `unsupported` and are designed to mirror those
used by Ansible's CI.
- `ansible-integration-unsupported` will have the additional policies applied
necessary to run the integraion tests marked as `unsupported` including tests
for managing IAM roles, users and groups.
Once the groups have been created, you'll need to create a user and make the user a member of these
groups. The policies are designed to minimize the rights of that user. Please note that while this policy does limit
the user to one region, this does not fully restrict the user (primarily due to the limitations of the Amazon ARN
notation). The user will still have wide privileges for viewing account definitions, and will also able to manage
some resources that are not related to testing (for example, AWS lambdas with different names). Tests should not
be run in a primary production account in any case.
Other Definitions required
--------------------------
Apart from installing the policy and giving it to the user identity running the tests, a
lambda role `ansible_integration_tests` has to be created which has lambda basic execution
privileges.
Network Tests
=============
For guidance on writing network test see :ref:`testing_resource_modules`.
Where to find out more
======================
If you'd like to know more about the plans for improving testing Ansible, join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.