Network and security devices separate configuration into sections (such as interfaces, VLANs, and so on) that apply to a network or security service. Ansible resource modules take advantage of this to allow users to configure subsections or resources within the device configuration. Resource modules provide a consistent experience across different network and security devices. For example, a network resource module may only update the configuration for a specific portion of the network interfaces, VLANs, ACLs, and so on for a network device. The resource module:
#. Fetches a piece of the configuration (fact gathering), for example, the interfaces configuration.
#. Converts the returned configuration into key-value pairs.
#. Places those key-value pairs into an internal agnostic structured data format.
Now that the configuration data is normalized, the user can update and modify the data and then use the resource module to send the configuration data back to the device. This results in a full round-trip configuration update without the need for manual parsing, data manipulation, and data model management.
The resource module has two top-level keys - ``config`` and ``state``:
*``config`` defines the resource configuration data model as key-value pairs. The type of the ``config`` option can be ``dict`` or ``list of dict`` based on the resource managed. That is, if the device has a single global configuration, it should be a ``dict`` (for example, a global LLDP configuration). If the device has multiple instances of configuration, it should be of type ``list`` with each element in the list of type ``dict`` (for example, interfaces configuration).
*``state`` defines the action the resource module takes on the end device.
The ``state`` for a new resource module should support the following values (as applicable for the devices that support them):
merged
Ansible merges the on-device configuration with the provided configuration in the task.
replaced
Ansible replaces the on-device configuration subsection with the provided configuration subsection in the task.
overridden
Ansible overrides the on-device configuration for the resource with the provided configuration in the task. Use caution with this state as you could remove your access to the device (for example, by overriding the management interface configuration).
deleted
Ansible deletes the on-device configuration subsection and restores any default settings.
gathered
Ansible displays the resource details gathered from the network device and accessed with the ``gathered`` key in the result.
rendered
Ansible renders the provided configuration in the task in the device-native format (for example, Cisco IOS CLI). Ansible returns this rendered configuration in the ``rendered`` key in the result. Note this state does not communicate with the network device and can be used offline.
parsed
Ansible parses the configuration from the ``running_configuration`` option into Ansible structured data in the ``parsed`` key in the result. Note this does not gather the configuration from the network device so this state can be used offline.
Modules in Ansible-maintained collections must support these state values. If you develop a module with only "present" and "absent" for state, you may submit it to a community collection.
..note::
The states ``rendered``, ``gathered``, and ``parsed`` do not perform any change on the device.
..seealso::
`Deep Dive on VLANs Resource Modules for Network Automation <https://www.ansible.com/blog/deep-dive-on-vlans-resource-modules-for-network-automation>`_
Walkthrough of how state values are implemented for VLANs.
Developing network and security resource modules
=================================================
The Ansible Engineering team ensures the module design and code pattern within Ansible-maintained collections is uniform across resources and across platforms to give a vendor-agnostic feel and deliver good quality code. We recommend you use the `resource module builder <https://github.com/ansible-network/resource_module_builder>`_ to develop a resource module.
The highlevel process for developing a resource module is:
#. Create and share a resource model design in the `resource module models repository <https://github.com/ansible-network/resource_module_models>`_ as a PR for review.
#. Download the latest version of the `resource module builder <https://github.com/ansible-network/resource_module_builder>`_.
#. Run the ``resource module builder`` to create a collection scaffold from your approved resource model.
#. Write the code to implement your resource module.
#. Develop integration and unit tests to verify your resource module.
#. Create a PR to the appropriate collection that you want to add your new resource module to. See :ref:`contributing_maintained_collections` for details on determining the correct collection for your module.
Understanding the model and resource module builder
The resource module builder is an Ansible Playbook that helps developers scaffold and maintain an Ansible resource module. It uses a model as the single source of truth for the module. This model is a ``yaml`` file that is used for the module DOCUMENTATION section and the argument spec.
You must create a model for your new resource. The model is the single source of truth for both the argspec and docstring, keeping them in sync. Once your model is approved, you can use the resource module builder to generate three items based on the model:
For any subsequent changes to the functionality, update the model first and use the resource module builder to update the module argspec and docstring.
- The state of the configuration after module completion.
type: str
choices:
- merged
- replaced
- overridden
- deleted
default: merged
EXAMPLES:
- deleted_example_01.txt
- merged_example_01.txt
- overridden_example_01.txt
- replaced_example_01.txt
Notice that you should include examples for each of the states that the resource supports. The resource module builder also includes these in the sample model.
Share this model as a PR for review at `resource module models repository <https://github.com/ansible-network/resource_module_models>`_. You can also see more model examples at that location.
* Entry in ``module_utils/<ansible_network_os>/facts/facts.py`` for ``get_facts`` API to keep ``<ansible_network_os>_facts`` module and facts gathered for the resource module in sync for every subset.
* Entry of Resource subset in FACTS_RESOURCE_SUBSETS list in ``module_utils/<ansible_network_os>/facts/facts.py`` to make facts collection work.
* Implement ``execute_module`` API that loads the configuration to device and generates the result with ``changed``, ``commands``, ``before`` and ``after`` keys.
* Call ``get_facts`` API that returns the ``<resource>`` configuration facts or return the difference if the device has onbox diff support.
* Compare facts gathered and given key-values if diff is not supported.
* Generate final configuration.
Utils
*``module_utils/<ansible_network_os>/utils``.
* Utilities for the ``<ansible_network_os>`` platform.
You should run ``ansible-test sanity`` and ``tox -elinters`` from the collection root directory before pushing your PR to an Ansible-maintained collection. The CI runs both and will fail if these tests fail. See :ref:`developing_testing` for details on ``ansible-test sanity``.
To install the necessary packages:
#. Ensure you have a valid Ansible development environment configured. See :ref:`environment_setup` for details.
#. Run ``pip install -r requirements.txt`` from the collection root directory.
Running ``tox -elinters``:
* Reads :file:`tox.ini` from the collection root directory and installs required dependencies (such as ``black`` and ``flake8``).
* Runs these with preconfigured options (such as line-length and ignores.)
* Runs ``black`` in check mode to show which files will be formatted without actually formatting them.
The tests rely on a role generated by the resource module builder. After changes to the resource module builder, the role should be regenerated and the tests modified and run as needed. To generate the role after changes:
High-level integration test requirements for new resource modules are as follows:
#. Write a test case for every state.
#. Write additional test cases to test the behavior of the module when an empty ``config.yaml`` is given.
#. Add a round trip test case. This involves a ``merge`` operation, followed by ``gather_facts``, a ``merge`` update with additional configuration, and then reverting back to the base configuration using the previously gathered facts with the ``state`` set to ``overridden``.
#. Wherever applicable, assertions should check after and before ``dicts`` against a hard coded Source of Truth.
.._using_zuul_resource_modules:
We use Zuul as the CI to run the integration test.
* To view the report, click :guilabel:`Details` on the CI comment in the PR
* To view a failure report, click :guilabel:`ansible/check` and select the failed test.
* To view logs while the test is running, check for your PR number in the `Zull status board <https://dashboard.zuul.ansible.com/t/ansible/status>`_.
* To fix static test failure locally, run the :command:`tox -e black`**inside the root folder of collection**.
To view The Ansible run logs and debug test failures:
#. Click the failed job to get the summary, and click :guilabel:`Logs` for the log.
#. Click :guilabel:`console` and scroll down to find the failed test.
#. Click :guilabel:`>` next to the failed test for complete details.
Integration test structure
...........................
Each test case should generally follow this pattern:
* setup —> test —> assert —> test again (for idempotency) —> assert —> tear down (if needed) -> done. This keeps test playbooks from becoming monolithic and difficult to troubleshoot.
* Include a name for each task that is not an assertion. You can add names to assertions as well, but it is easier to identify the broken task within a failed test if you add a name for each task.
* Files containing test cases must end in ``.yaml``
Implementation
..............
For platforms that support ``connection: local``*and*``connection: network_cli`` use the following guidance:
* Name the :file:`targets/` directories after the module name.
* The :file:`main.yaml` file should just reference the transport.
The following example walks through the integration tests for the ``vyos.vyos.vyos_l3_interfaces`` module in the `vyos.vyos <https://github.com/ansible-collections/vyos.vyos/tree/master/tests/integration>`_ collection:
Your tests should detect resources (such as interfaces) at runtime rather than hard-coding them into the test. This allows the test to run on a variety of systems.
See the complete test example of this at https://github.com/ansible-collections/cisco.nxos/blob/master/tests/integration/targets/prepare_nxos_tests/tasks/main.yml.
Running network integration tests
..................................
Ansible uses Zuul to run an integration test suite on every PR, including new tests introduced by that PR. To find and fix problems in network modules, run the network integration test locally before you submit a PR.
First, create an inventory file that points to your test machines. The inventory group should match the platform name (for example, ``eos``, ``ios``):
This example will run against all ``vyos`` modules. Note that ``vyos_.*`` is a regex match, not a bash wildcard - include the `.` if you modify this example.
See `test/integration/targets/nxos_bgp/tasks/main.yaml <https://github.com/ansible-collections/cisco.nxos/blob/master/tests/integration/targets/nxos_bgp/tasks/main.yaml>`_ for how this is implemented in the tests.
For more options:
..code-block:: bash
ansible-test network-integration --help
If you need additional help or feedback, reach out in ``#ansible-network`` on Freenode.
Unit test requirements
-----------------------
High-level unit test requirements that new resource modules should follow:
#. Write test cases for all the states with all possible combinations of config values.
#. Write test cases to test the error conditions ( negative scenarios).
#. Check the value of ``changed`` and ``commands`` keys in every test case.
We run all unit test cases on our Zuul test suite, on the latest python version supported by our CI setup.
Use the :ref:`same procedure <using_zuul_resource_modules>` as the integration tests to view Zuul unit tests reports and logs.
See :ref:`unit module testing <testing_units_modules>` for general unit test details.
.. end of cut n .. parsed-literal::
Example: Unit testing Ansible network resource modules
come from external libraries, you can use ``fixtures`` to read in pre-generated data. The text files for this pre-generated data live in ``test/units/modules/network/PLATFORM/fixtures/``. See for example the `eos_l2_interfaces.cfg file <https://github.com/ansible-collections/arista.eos/blob/master/tests/unit/modules/network/eos/fixtures/eos_l2_interfaces_config.cfg>`_.
See the unit test file `test_eos_l2_interfaces <https://github.com/ansible-collections/arista.eos/blob/master/tests/unit/modules/network/eos/test_eos_l2_interfaces.py>`_