archive
/
ansible
mirror of https://github.com/ansible/ansible.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Replace 'simple module development' topic (#26267)

Browse Source 
* WIP: Initial bulk move of quick start content into rst docs. Needs rewrite.

* Swapping out old quick start for new material (thanks @tstringer!). Still work in progress.

* Fixed formatting; removed extraneous file.

* Fixed some formatting errors.

* Once more, with feeling.

* Fixed formatting error
pull/26746/merge
scottb 7 years ago committed by GitHub
parent 245a4d30b9
commit 4f01a323a0
2 changed files with 1814 additions and 262 deletions
Show Stats Download Patch File Download Diff File

498
docs/docsite/rst/dev_guide/developing_modules_general.rst
Unescape Escape View File

@ -1,333 +1,313 @@
.. _module_dev_tutorial_sample: .. _module_dev_tutorial_sample:
Building A Simple Module Ansible Module Development Walkthrough
```````````````````````` ======================================
Let's build a very basic module to get and set the system time. For starters, let's build
a module that just outputs the current time.
We are going to use Python here but any language is possible. Only File I/O and outputting to standard In this section, we will walk through developing, testing, and debugging an Ansible module.
out are required. So, bash, C++, clojure, Python, Ruby, whatever you want
is fine.
Now Python Ansible modules contain some extremely powerful shortcuts (that all the core modules use) What's covered in this section:
but first we are going to build a module the very hard way. The reason we do this is because modules
written in any language OTHER than Python are going to have to do exactly this. We'll show the easy
way later.
So, here's an example. You would never really need to build a module to set the system time, - `Environment setup <#environment-setup>`__
the 'command' module could already be used to do this. - `New module development <#new-module-development>`__
- `Local/direct module testing <#localdirect-module-testing>`__
- `Playbook module testing <#playbook-module-testing>`__
- `Debugging (local) <#debugging-local>`__
- `Debugging (remote) <#debugging-remote>`__
- `Unit testing <#unit-testing>`__
- Integration testing (coming soon)
- `Communication and development
support <#communication-and-development-support>`__
- `Credit <#credit>`__
Reading the modules that come with Ansible (linked above) is a great way to learn how to write
modules. Keep in mind, though, that some modules in Ansible's source tree are internalisms,
so look at :ref:`service` or :ref:`yum`, and don't stare too close into things like ``async_wrapper`` or
you'll turn to stone. Nobody ever executes ``async_wrapper`` directly.
Ok, let's get going with an example. We're going to use Python. For starters, save this as a file named :file:`timetest.py` Environment setup
=================
.. code-block:: python 1. Clone the Ansible repository:
``$ git clone https://github.com/ansible/ansible.git``
2. Change directory into the repository root dir: ``$ cd ansible``
3. Create a virtual environment: ``$ python3 -m venv venv`` (or for
Python 2 ``$ virtualenv venv``. Note, this requires you to install
the virtualenv package: ``$ pip install virtualenv``)
4. Activate the virtual environment: ``$ . venv/bin/activate``
5. Install development requirements:
``$ pip install -r requirements.txt``
6. Run the environment setup script for each new dev shell process:
``$ . hacking/env-setup``
#!/usr/bin/python .. note:: After the initial setup above, every time you are ready to start
developing Ansible you should be able to just run the following from the
import datetime root of the Ansible repo:
import json ``$ . venv/bin/activate && . hacking/env-setup``
date = str(datetime.datetime.now())
print(json.dumps({
"time" : date
}))
.. _module_testing:
Testing Your Module
```````````````````
There's a useful test script in the source checkout for Ansible:
.. code-block:: shell-session
git clone git://github.com/ansible/ansible.git
source ansible/hacking/env-setup
For instructions on setting up Ansible from source, please see
:doc:`../intro_installation`.
Let's run the script you just wrote with that:
.. code-block:: shell-session
ansible/hacking/test-module -m ./timetest.py
You should see output that looks something like this:
.. code-block:: json
{"time": "2012-03-14 22:13:48.539183"}
If you did not, you might have a typo in your module, so recheck it and try again.
.. _reading_input: New module development
======================
Reading Input If you are creating a new module that doesn't exist, you would start
````````````` working on a whole new file. Here is an example:
Let's modify the module to allow setting the current time. We'll do this by seeing
if a key value pair in the form `time=<string>` is passed into the module.
Ansible internally saves arguments to an arguments file. So we must read the file - Navigate to the directory that you want to develop your new module
and parse it. The arguments file is just a string, so any form of arguments are legal. in. E.g. ``$ cd lib/ansible/modules/cloud/azure/``
Here we'll do some basic parsing to treat the input as key=value. - Create your new module file: ``$ touch my_new_test_module.py``
- Paste this simple into the new module file: (explanation in comments)::
The example usage we are trying to achieve to set the time is::
time time="March 14 22:10"
If no time parameter is set, we'll just leave the time as is and return the current time.
.. note::
This is obviously an unrealistic idea for a module. You'd most likely just
use the command module. However, it makes for a decent tutorial.
Let's look at the code. Read the comments as we'll explain as we go. Note that this
is highly verbose because it's intended as an educational example. You can write modules
a lot shorter than this:
.. code-block:: python
#!/usr/bin/python #!/usr/bin/python
# import some python modules that we'll use. These are all ANSIBLE_METADATA = {
# available in Python's core 'metadata_version': '1.0',
'status': ['preview'],
import datetime 'supported_by': 'curated'
import sys }
import json
import os
import shlex
# read the argument string from the arguments file
args_file = sys.argv[1]
args_data = file(args_file).read()
# For this module, we're going to do key=value style arguments.
# Modules can choose to receive json instead by adding the string:
# WANT_JSON
# Somewhere in the file.
# Modules can also take free-form arguments instead of key-value or json
# but this is not recommended.
arguments = shlex.split(args_data)
for arg in arguments:
# ignore any arguments without an equals in it
if "=" in arg:
(key, value) = arg.split("=")
# if setting the time, the key 'time'
# will contain the value we want to set the time to
if key == "time":
# now we'll affect the change. Many modules DOCUMENTATION = '''
# will strive to be idempotent, generally ---
# by not performing any actions if the current module: my_sample_module
# state is the same as the desired state.
# See 'service' or 'yum' in the main git tree short_description: This is my sample module
# for an illustrative example.
version_added: "2.4"
description:
- "This is my longer description explaining my sample module"
options:
name:
description:
- This is the message to send to the sample module
required: true
new:
description:
- Control to demo if the result of this module is changed or not
required: false
extends_documentation_fragment
- azure
author:
- Your Name (@yourhandle)
'''
EXAMPLES = '''
# Pass in a message
- name: Test with a message
my_new_test_module:
name: hello world
# pass in a message and have changed true
- name: Test with a message and changed output
my_new_test_module:
name: hello world
new: true
# fail the module
- name: Test failure of the module
my_new_test_module:
name: fail me
'''
RETURN = '''
original_message:
description: The original name param that was passed in
type: str
message:
description: The output message that the sample module generates
'''
rc = os.system("date -s \"%s\"" % value) from ansible.module_utils.basic import AnsibleModule
# always handle all possible errors def run_module():
# # define the available arguments/parameters that a user can pass to
# when returning a failure, include 'failed' # the module
# in the return data, and explain the failure module_args = dict(
# in 'msg'. Both of these conventions are name=dict(type='str', required=True),
# required however additional keys and values new=dict(type='bool', required=False, default=False)
# can be added. )
if rc != 0: # seed the result dict in the object
print(json.dumps({ # we primarily care about changed and state
"failed" : True, # change is if this module effectively modified the target
"msg" : "failed setting the time" # state will include any data that you want your module to pass back
})) # for consumption, for example, in a subsequent task
sys.exit(1) result = dict(
changed=False,
original_message='',
message=''
)
# when things do not fail, we do not # the AnsibleModule object will be our abstraction working with Ansible
# have any restrictions on what kinds of # this includes instantiation, a couple of common attr would be the
# data are returned, but it's always a # args/params passed to the execution, as well as if the module
# good idea to include whether or not # supports check mode
# a change was made, as that will allow module = AnsibleModule(
# notifiers to be used in playbooks. argument_spec=module_args,
supports_check_mode=True
)
date = str(datetime.datetime.now()) # if the user is working with this module in only check mode we do not
print(json.dumps({ # want to make any changes to the environment, just return the current
"time" : date, # state with no modifications
"changed" : True if module.check_mode:
})) return result
sys.exit(0)
# if no parameters are sent, the module may or # manipulate or modify the state as needed (this is going to be the
# may not error out, this one will just # part where your module will do what it needs to do)
# return the time result['original_message'] = module.params['name']
result['message'] = 'goodbye'
date = str(datetime.datetime.now()) # use whatever logic you need to determine whether or not this module
print(json.dumps({ # made any modifications to your target
"time" : date if module.params['new']:
})) result['changed'] = True
Let's test that module:: # during the execution of the module, if there is an exception or a
# conditional state that effectively causes a failure, run
# AnsibleModule.fail_json() to pass in the message and the result
if module.params['name'] == 'fail me':
module.fail_json(msg='You requested this to fail', **result)
ansible/hacking/test-module -m ./timetest.py -a "time=\"March 14 12:23\"" # in the event of a successful module execution, you will want to
# simple AnsibleModule.exit_json(), passing the key/value results
module.exit_json(**result)
This should return something like:: def main():
run_module()
{"changed": true, "time": "2012-03-14 12:23:00.000307"} if __name__ == '__main__':
main()
.. _binary_module_reading_input: Local/direct module testing
===========================
Binary Modules Input You may want to test the module on the local machine without targeting a
++++++++++++++++++++ remote host. This is a great way to quickly and easily debug a module
that can run locally.
Support for binary modules was added in Ansible 2.2. When Ansible detects a binary module, it will proceed to - Create an arguments file in ``/tmp/args.json`` with the following
supply the argument input as a file on ``argv[1]`` that is formatted as JSON. The JSON contents of that file content: (explanation below)
would resemble something similar to the following payload for a module accepting the same arguments as the
``ping`` module:
.. code-block:: json .. code:: json
{ {
"data": "pong", "ANSIBLE_MODULE_ARGS": {
"_ansible_verbosity": 4, "name": "hello",
"_ansible_diff": false, "new": true
"_ansible_debug": false, }
"_ansible_check_mode": false,
"_ansible_no_log": false
} }
.. _module_provided_facts: - If you are using a virtual environment (highly recommended for
development) activate it: ``$ . venv/bin/activate``
- Setup the environment for development: ``$ . hacking/env-setup``
- Run your test module locally and directly:
``$ python ./my_new_test_module.py /tmp/args.json``
Module Provided 'Facts' This should be working output that resembles something like the
```````````````````````` following:
The :ref:`setup` module that ships with Ansible provides many variables about a system that can be used in playbooks ::
and templates. However, it's possible to also add your own facts without modifying the system module. To do
this, just have the module return a `ansible_facts` key, like so, along with other return data:
.. code-block:: json {"changed": true, "state": {"original_message": "hello", "new_message": "goodbye"}, "invocation": {"module_args": {"name": "hello", "new": true}}}
{ The arguments file is just a basic json config file that you can
"changed" : true, use to pass the module your parameters to run the module it
"rc" : 5,
"ansible_facts" : {
"leptons" : 5000,
"colors" : {
"red" : "FF0000",
"white" : "FFFFFF"
}
}
}
These 'facts' will be available to all statements called after that module (but not before) in the playbook. Playbook module testing
A good idea might be to make a module called 'site_facts' and always call it at the top of each playbook, though =======================
we're always open to improving the selection of core facts in Ansible as well.
Returning a new fact from a python module could be done like:: If you want to test your new module, you can now consume it with an
Ansible playbook.
module.exit_json(msg=message, ansible_facts=dict(leptons=5000, colors=my_colors)) - Create a playbook in any directory: ``$ touch testmod.yml``
- Add the following to the new playbook file \`\`\`yaml ---
- name: test my new module connection: local hosts: localhost
.. _common_module_boilerplate: tasks: - name: run the new module my\_new\_test\_module: name: 'hello'
new: true register: testout
Common Module Boilerplate ::
`````````````````````````
As mentioned, if you are writing a module in Python, there are some very powerful shortcuts you can use. - name: dump test output
Modules are still transferred as one file, but an arguments file is no longer needed, so these are not debug:
only shorter in terms of code, they are actually FASTER in terms of execution time. msg: '{{ testout }}'
Rather than mention these here, the best way to learn is to read some of the `source of the modules <https://github.com/ansible/ansible/tree/devel/lib/ansible/modules>`_ that come with Ansible. ::
The 'group' and 'user' modules are reasonably non-trivial and showcase what this looks like. - Run the playbook and analyze the output: `$ ansible-playbook ./testmod.yml`
Key parts include always importing the boilerplate code from Debugging (local)
:mod:`ansible.module_utils.basic` like this: =================
.. code-block:: python If you want to break into a module and step through with the debugger, locally running the module you can do:
from ansible.module_utils.basic import AnsibleModule - Set a breakpoint in the module: `import pdb; pdb.set_trace()`
if __name__ == '__main__': - Run the module on the local machine: `$ python -m pdb ./my_new_test_module.py ./args.json`
main()
.. note::
Prior to Ansible-2.1.0, importing only what you used from
:mod:`ansible.module_utils.basic` did not work. You needed to use
a wildcard import like this:
.. code-block:: python Debugging (remote)
==================
from ansible.module_utils.basic import * In the event you want to debug a module that is running on a remote target (i.e. not localhost), one way to do this is the following:
And instantiating the module class like: - On your controller machine (running Ansible) set `ANSIBLE_KEEP_REMOTE_FILES=1` (this tells Ansible to retain the modules it sends to the remote machine instead of removing them)
- Run your playbook targetting the remote machine and specify ``-vvvv`` (the verbose output will show you many things, including the remote location that Ansible uses for the modules)
.. code-block:: python - Take note of the remote path Ansible used on the remote host
- SSH into the remote target after the completion of the playbook
def main(): - Navigate to the directory (most likely it is going to be your ansible remote user defined or implied from the playbook: ``~/.ansible/tmp/ansible-tmp-...``)
module = AnsibleModule( - Here you should see the module that you executed from your Ansible controller, but this is the zipped file that Ansible sent to the remote host. You can run this by specifying ``python my_test_module.py`` (not necessary)
argument_spec = dict( - To debug, though, we will want to extra this zip out to the original module format: ``python my_test_module.py explode`` (Ansible will expand the module into ``./debug-dir``)
state = dict(default='present', choices=['present', 'absent']), - Navigate to ``./debug-dir`` (notice that unzipping has caused the generation of ``ansible_module_my_test_module.py``)
name = dict(required=True), - Modify or set a breakpoint in the unzipped module
enabled = dict(required=True, type='bool'), - Ensure that the unzipped module is executable: ``$ chmod 755 ansible_module_my_test_module.py``
something = dict(aliases=['whatever']) - Run the unzipped module directly passing the args file: ``$ ./ansible_module_my_test_module.py args`` (args is the file that contains the params that were originally passed. Good for repro and debugging)
)
)
The :class:`AnsibleModule` provides lots of common code for handling returns, parses your arguments Unit testing
for you, and allows you to check inputs. ============
Successful returns are made like this: Unit tests for modules will be appropriately located in ``./test/units/modules``. You must first setup your testing environment. In this example, we're using Python 3.5.
.. code-block:: python - Install the requirements (outside of your virtual environment): ``$ pip3 install -r ./test/runner/requirements/units.txt``
- To run all tests do the following: ``$ ansible-test units --python 3.5`` (you must run ``. hacking/env-setup`` prior to this)
module.exit_json(changed=True, something_else=12345) .. note:: Ansible uses pytest for unit testing.
And failures are just as simple (where `msg` is a required parameter to explain the error): To run pytest against a single test module, you can do the following (provide the path to the test module appropriately):
.. code-block:: python ``$ pytest -r a --cov=. --cov-report=html --fulltrace --color yes
test/units/modules/.../test/my_new_test_module.py``
module.fail_json(msg="Something fatal happened") Going Further
=============
There are also other useful functions in the module class, such as :func:`module.sha1(path)`. See If you are starting new development or fixing a bug, create a new branch:
:file:`lib/ansible/module_utils/basic.py` in the source checkout for implementation details.
Again, modules developed this way are best tested with the :file:`hacking/test-module` script in the git ``$ git checkout -b my-new-branch``.
source checkout. Because of the magic involved, this is really the only way the scripts
can function outside of Ansible.
If submitting a module to Ansible's core code, which we encourage, use of If you are planning on contributing
:class:`AnsibleModule` is required. back to the main Ansible repository, fork the Ansible repository into
your own GitHub account and developing against the new non-devel branch
in your fork. When you believe you have a good working code change,
submit a pull request to the Ansible repository.
.. _developing_for_check_mode: If you want to submit a new module to the upstream Ansible repo, be sure
to run through sanity checks first. For example:
Supporting Check Mode ``$ ansible-test sanity -v --docker --python 2.7 MODULE_NAME``
`````````````````````
.. versionadded:: 1.1
Modules may optionally support `check mode <http://docs.ansible.com/ansible/playbooks_checkmode.html>`_. If the user runs Ansible in check mode, a module should try to predict and report whether changes will occur but not actually make any changes (modules that do not support check mode will also take no action, but just will not report what changes they might have made). Note that this example requires docker to be installed and running. If you'd rather not use a
container for this, you can choose to use ``--tox`` instead of ``--docker``.
For your module to support check mode, you must pass ``supports_check_mode=True`` when instantiating the AnsibleModule object. The AnsibleModule.check_mode attribute will evaluate to True when check mode is enabled. For example:
.. code-block:: python Communication and development support
=====================================
module = AnsibleModule( Join the IRC channel ``#ansible-devel`` on freenode for discussions
argument_spec = dict(...), surrounding Ansible development.
supports_check_mode=True
)
if module.check_mode: For questions and discussions pertaining to using the Ansible product,
# Check if any changes would be made but don't actually make those changes use the ``#ansible`` channel.
module.exit_json(changed=check_if_system_state_would_be_changed())
Remember that, as module developer, you are responsible for ensuring that no Credit
system state is altered when the user enables check mode. ======
If your module does not support check mode, when the user runs Ansible in check Thank you to Thomas Stringer (`@tstring <https://github.com/tstringer>`_) for contributing source
mode, your module will simply be skipped. material for this topic.

1572
docs/docsite/rst/dev_guide/developing_modules_general_OLD.rst
View File

File diff suppressed because it is too large Load Diff
Write Preview
Loading…
Cancel
Save