mirror of https://github.com/ansible/ansible.git
Continued refurb of dev content
Scott Butler
8 years ago
parent
05aed6e52e
commit
427ce69645
@ -1,19 +0,0 @@
|
|||||||
Developer Information
|
|
||||||
`````````````````````
|
|
||||||
|
|
||||||
Learn how to build modules of your own in any language, and also how to extend Ansible through several kinds of plugins. Explore Ansible's Python API and write Python plugins to integrate with other solutions in your environment.
|
|
||||||
|
|
||||||
.. toctree::
|
|
||||||
:maxdepth: 2
|
|
||||||
|
|
||||||
developing_api
|
|
||||||
developing_inventory
|
|
||||||
developing_modules
|
|
||||||
developing_module_utilities
|
|
||||||
developing_plugins
|
|
||||||
developing_core
|
|
||||||
developing_test_pr
|
|
||||||
developing_releases
|
|
||||||
|
|
||||||
Developers will also likely be interested in the fully-discoverable in :doc:`tower`. It's great for embedding Ansible in all manner of applications.
|
|
||||||
|
|
||||||
@ -1,194 +0,0 @@
|
|||||||
Python API
|
|
||||||
==========
|
|
||||||
|
|
||||||
.. contents:: Topics
|
|
||||||
|
|
||||||
Please note that while we make this API available it is not intended for direct consumption, it is here
|
|
||||||
for the support of the Ansible command line tools. We try not to make breaking changes but we reserve the
|
|
||||||
right to do so at any time if it makes sense for the Ansible toolset.
|
|
||||||
|
|
||||||
|
|
||||||
The following documentation is provided for those that still want to use the API directly, but be mindful this is not something the Ansible team supports.
|
|
||||||
|
|
||||||
There are several interesting ways to use Ansible from an API perspective. You can use
|
|
||||||
the Ansible python API to control nodes, you can extend Ansible to respond to various python events, you can
|
|
||||||
write various plugins, and you can plug in inventory data from external data sources. This document
|
|
||||||
covers the execution and Playbook API at a basic level.
|
|
||||||
|
|
||||||
If you are looking to use Ansible programmatically from something other than Python, trigger events asynchronously,
|
|
||||||
or have access control and logging demands, take a look at :doc:`tower`
|
|
||||||
as it has a very nice REST API that provides all of these things at a higher level.
|
|
||||||
|
|
||||||
Ansible is written in its own API so you have a considerable amount of power across the board.
|
|
||||||
This chapter discusses the Python API.
|
|
||||||
|
|
||||||
.. _python_api:
|
|
||||||
|
|
||||||
The Python API is very powerful, and is how the all the ansible CLI tools are implemented.
|
|
||||||
In version 2.0 the core ansible got rewritten and the API was mostly rewritten.
|
|
||||||
|
|
||||||
.. note:: Ansible relies on forking processes, as such the API is not thread safe.
|
|
||||||
|
|
||||||
.. _python_api_20:
|
|
||||||
|
|
||||||
Python API 2.0
|
|
||||||
--------------
|
|
||||||
|
|
||||||
In 2.0 things get a bit more complicated to start, but you end up with much more discrete and readable classes::
|
|
||||||
|
|
||||||
|
|
||||||
#!/usr/bin/env python
|
|
||||||
|
|
||||||
import json
|
|
||||||
from collections import namedtuple
|
|
||||||
from ansible.parsing.dataloader import DataLoader
|
|
||||||
from ansible.vars import VariableManager
|
|
||||||
from ansible.inventory import Inventory
|
|
||||||
from ansible.playbook.play import Play
|
|
||||||
from ansible.executor.task_queue_manager import TaskQueueManager
|
|
||||||
from ansible.plugins.callback import CallbackBase
|
|
||||||
|
|
||||||
class ResultCallback(CallbackBase):
|
|
||||||
"""A sample callback plugin used for performing an action as results come in
|
|
||||||
|
|
||||||
If you want to collect all results into a single object for processing at
|
|
||||||
the end of the execution, look into utilizing the ``json`` callback plugin
|
|
||||||
or writing your own custom callback plugin
|
|
||||||
"""
|
|
||||||
def v2_runner_on_ok(self, result, **kwargs):
|
|
||||||
"""Print a json representation of the result
|
|
||||||
|
|
||||||
This method could store the result in an instance attribute for retrieval later
|
|
||||||
"""
|
|
||||||
host = result._host
|
|
||||||
print json.dumps({host.name: result._result}, indent=4)
|
|
||||||
|
|
||||||
Options = namedtuple('Options', ['connection', 'module_path', 'forks', 'become', 'become_method', 'become_user', 'check'])
|
|
||||||
# initialize needed objects
|
|
||||||
variable_manager = VariableManager()
|
|
||||||
loader = DataLoader()
|
|
||||||
options = Options(connection='local', module_path='/path/to/mymodules', forks=100, become=None, become_method=None, become_user=None, check=False)
|
|
||||||
passwords = dict(vault_pass='secret')
|
|
||||||
|
|
||||||
# Instantiate our ResultCallback for handling results as they come in
|
|
||||||
results_callback = ResultCallback()
|
|
||||||
|
|
||||||
# create inventory and pass to var manager
|
|
||||||
inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list='localhost')
|
|
||||||
variable_manager.set_inventory(inventory)
|
|
||||||
|
|
||||||
# create play with tasks
|
|
||||||
play_source = dict(
|
|
||||||
name = "Ansible Play",
|
|
||||||
hosts = 'localhost',
|
|
||||||
gather_facts = 'no',
|
|
||||||
tasks = [
|
|
||||||
dict(action=dict(module='shell', args='ls'), register='shell_out'),
|
|
||||||
dict(action=dict(module='debug', args=dict(msg='{{shell_out.stdout}}')))
|
|
||||||
]
|
|
||||||
)
|
|
||||||
play = Play().load(play_source, variable_manager=variable_manager, loader=loader)
|
|
||||||
|
|
||||||
# actually run it
|
|
||||||
tqm = None
|
|
||||||
try:
|
|
||||||
tqm = TaskQueueManager(
|
|
||||||
inventory=inventory,
|
|
||||||
variable_manager=variable_manager,
|
|
||||||
loader=loader,
|
|
||||||
options=options,
|
|
||||||
passwords=passwords,
|
|
||||||
stdout_callback=results_callback, # Use our custom callback instead of the ``default`` callback plugin
|
|
||||||
)
|
|
||||||
result = tqm.run(play)
|
|
||||||
finally:
|
|
||||||
if tqm is not None:
|
|
||||||
tqm.cleanup()
|
|
||||||
|
|
||||||
|
|
||||||
.. _python_api_old:
|
|
||||||
|
|
||||||
Python API pre 2.0
|
|
||||||
------------------
|
|
||||||
|
|
||||||
It's pretty simple::
|
|
||||||
|
|
||||||
import ansible.runner
|
|
||||||
|
|
||||||
runner = ansible.runner.Runner(
|
|
||||||
module_name='ping',
|
|
||||||
module_args='',
|
|
||||||
pattern='web*',
|
|
||||||
forks=10
|
|
||||||
)
|
|
||||||
datastructure = runner.run()
|
|
||||||
|
|
||||||
The run method returns results per host, grouped by whether they
|
|
||||||
could be contacted or not. Return types are module specific, as
|
|
||||||
expressed in the :doc:`modules` documentation.::
|
|
||||||
|
|
||||||
{
|
|
||||||
"dark" : {
|
|
||||||
"web1.example.com" : "failure message"
|
|
||||||
},
|
|
||||||
"contacted" : {
|
|
||||||
"web2.example.com" : 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
A module can return any type of JSON data it wants, so Ansible can
|
|
||||||
be used as a framework to rapidly build powerful applications and scripts.
|
|
||||||
|
|
||||||
.. _detailed_api_old_example:
|
|
||||||
|
|
||||||
Detailed API Example
|
|
||||||
````````````````````
|
|
||||||
|
|
||||||
The following script prints out the uptime information for all hosts::
|
|
||||||
|
|
||||||
#!/usr/bin/python
|
|
||||||
|
|
||||||
import ansible.runner
|
|
||||||
import sys
|
|
||||||
|
|
||||||
# construct the ansible runner and execute on all hosts
|
|
||||||
results = ansible.runner.Runner(
|
|
||||||
pattern='*', forks=10,
|
|
||||||
module_name='command', module_args='/usr/bin/uptime',
|
|
||||||
).run()
|
|
||||||
|
|
||||||
if results is None:
|
|
||||||
print "No hosts found"
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
print "UP ***********"
|
|
||||||
for (hostname, result) in results['contacted'].items():
|
|
||||||
if not 'failed' in result:
|
|
||||||
print "%s >>> %s" % (hostname, result['stdout'])
|
|
||||||
|
|
||||||
print "FAILED *******"
|
|
||||||
for (hostname, result) in results['contacted'].items():
|
|
||||||
if 'failed' in result:
|
|
||||||
print "%s >>> %s" % (hostname, result['msg'])
|
|
||||||
|
|
||||||
print "DOWN *********"
|
|
||||||
for (hostname, result) in results['dark'].items():
|
|
||||||
print "%s >>> %s" % (hostname, result)
|
|
||||||
|
|
||||||
Advanced programmers may also wish to read the source to ansible itself,
|
|
||||||
for it uses the API (with all available options) to implement the ``ansible``
|
|
||||||
command line tools (``lib/ansible/cli/``).
|
|
||||||
|
|
||||||
.. seealso::
|
|
||||||
|
|
||||||
:doc:`developing_inventory`
|
|
||||||
Developing dynamic inventory integrations
|
|
||||||
:doc:`developing_modules`
|
|
||||||
How to develop modules
|
|
||||||
:doc:`developing_plugins`
|
|
||||||
How to develop plugins
|
|
||||||
`Development Mailing List <http://groups.google.com/group/ansible-devel>`_
|
|
||||||
Mailing list for development topics
|
|
||||||
`irc.freenode.net <http://irc.freenode.net>`_
|
|
||||||
#ansible IRC chat channel
|
|
||||||
|
|
||||||
@ -1,24 +0,0 @@
|
|||||||
Developing the Ansible Core Engine
|
|
||||||
==================================
|
|
||||||
|
|
||||||
Although many of the pieces of the Ansible Core Engine are plugins that can be
|
|
||||||
swapped out via playbook directives or configuration, there are still pieces
|
|
||||||
of the Engine that are not modular. The documents here give insight into how
|
|
||||||
those pieces work together.
|
|
||||||
|
|
||||||
.. toctree::
|
|
||||||
:maxdepth: 1
|
|
||||||
|
|
||||||
developing_program_flow_modules
|
|
||||||
|
|
||||||
.. seealso::
|
|
||||||
|
|
||||||
:doc:`developing_api`
|
|
||||||
Learn about the Python API for task execution
|
|
||||||
:doc:`developing_plugins`
|
|
||||||
Learn about developing plugins
|
|
||||||
`Mailing List <http://groups.google.com/group/ansible-devel>`_
|
|
||||||
The development mailing list
|
|
||||||
`irc.freenode.net <http://irc.freenode.net>`_
|
|
||||||
#ansible-devel IRC chat channel
|
|
||||||
|
|
||||||
@ -1,99 +0,0 @@
|
|||||||
Developing Dynamic Inventory Sources
|
|
||||||
====================================
|
|
||||||
|
|
||||||
.. contents:: Topics
|
|
||||||
:local:
|
|
||||||
|
|
||||||
As described in :doc:`intro_dynamic_inventory`, ansible can pull inventory information from dynamic sources, including cloud sources.
|
|
||||||
|
|
||||||
How do we write a new one?
|
|
||||||
|
|
||||||
Simple! We just create a script or program that can print JSON in the right format when fed the proper arguments.
|
|
||||||
You can do this in any language.
|
|
||||||
|
|
||||||
.. _inventory_script_conventions:
|
|
||||||
|
|
||||||
Script Conventions
|
|
||||||
``````````````````
|
|
||||||
|
|
||||||
When the external node script is called with the single argument ``--list``, the script must output a JSON encoded hash/dictionary of all the groups to be managed to stdout. Each group's value should be either a hash/dictionary containing a list of each host/IP, potential child groups, and potential group variables, or simply a list of host/IP addresses, like so::
|
|
||||||
|
|
||||||
{
|
|
||||||
"databases" : {
|
|
||||||
"hosts" : [ "host1.example.com", "host2.example.com" ],
|
|
||||||
"vars" : {
|
|
||||||
"a" : true
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"webservers" : [ "host2.example.com", "host3.example.com" ],
|
|
||||||
"atlanta" : {
|
|
||||||
"hosts" : [ "host1.example.com", "host4.example.com", "host5.example.com" ],
|
|
||||||
"vars" : {
|
|
||||||
"b" : false
|
|
||||||
},
|
|
||||||
"children": [ "marietta", "5points" ]
|
|
||||||
},
|
|
||||||
"marietta" : [ "host6.example.com" ],
|
|
||||||
"5points" : [ "host7.example.com" ]
|
|
||||||
}
|
|
||||||
|
|
||||||
.. versionadded:: 1.0
|
|
||||||
|
|
||||||
Before version 1.0, each group could only have a list of hostnames/IP addresses, like the webservers, marietta, and 5points groups above.
|
|
||||||
|
|
||||||
When called with the arguments ``--host <hostname>`` (where <hostname> is a host from above), the script must print either an empty JSON
|
|
||||||
hash/dictionary, or a hash/dictionary of variables to make available to templates and playbooks. Printing variables is optional,
|
|
||||||
if the script does not wish to do this, printing an empty hash/dictionary is the way to go::
|
|
||||||
|
|
||||||
{
|
|
||||||
"favcolor" : "red",
|
|
||||||
"ntpserver" : "wolf.example.com",
|
|
||||||
"monitoring" : "pack.example.com"
|
|
||||||
}
|
|
||||||
|
|
||||||
.. _inventory_script_tuning:
|
|
||||||
|
|
||||||
Tuning the External Inventory Script
|
|
||||||
````````````````````````````````````
|
|
||||||
|
|
||||||
.. versionadded:: 1.3
|
|
||||||
|
|
||||||
The stock inventory script system detailed above works for all versions of Ansible, but calling
|
|
||||||
``--host`` for every host can be rather expensive, especially if it involves expensive API calls to
|
|
||||||
a remote subsystem. In Ansible
|
|
||||||
1.3 or later, if the inventory script returns a top level element called "_meta", it is possible
|
|
||||||
to return all of the host variables in one inventory script call. When this meta element contains
|
|
||||||
a value for "hostvars", the inventory script will not be invoked with ``--host`` for each host. This
|
|
||||||
results in a significant performance increase for large numbers of hosts, and also makes client
|
|
||||||
side caching easier to implement for the inventory script.
|
|
||||||
|
|
||||||
The data to be added to the top level JSON dictionary looks like this::
|
|
||||||
|
|
||||||
{
|
|
||||||
|
|
||||||
# results of inventory script as above go here
|
|
||||||
# ...
|
|
||||||
|
|
||||||
"_meta" : {
|
|
||||||
"hostvars" : {
|
|
||||||
"moocow.example.com" : { "asdf" : 1234 },
|
|
||||||
"llama.example.com" : { "asdf" : 5678 },
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
.. seealso::
|
|
||||||
|
|
||||||
:doc:`developing_api`
|
|
||||||
Python API to Playbooks and Ad Hoc Task Execution
|
|
||||||
:doc:`developing_modules`
|
|
||||||
How to develop modules
|
|
||||||
:doc:`developing_plugins`
|
|
||||||
How to develop plugins
|
|
||||||
`Ansible Tower <http://ansible.com/ansible-tower>`_
|
|
||||||
REST API endpoint and GUI for Ansible, syncs with dynamic inventory
|
|
||||||
`Development Mailing List <http://groups.google.com/group/ansible-devel>`_
|
|
||||||
Mailing list for development topics
|
|
||||||
`irc.freenode.net <http://irc.freenode.net>`_
|
|
||||||
#ansible IRC chat channel
|
|
||||||
@ -1,264 +0,0 @@
|
|||||||
===========================
|
|
||||||
Porting Modules to Python 3
|
|
||||||
===========================
|
|
||||||
|
|
||||||
Ansible modules are not the usual Python-3 porting exercise. There are two
|
|
||||||
factors that make it harder to port them than most code:
|
|
||||||
|
|
||||||
1. Many modules need to run on Python-2.4 in addition to Python-3.
|
|
||||||
2. A lot of mocking has to go into unittesting a Python-3 module. So it's
|
|
||||||
harder to test that your porting has fixed everything or to make sure that
|
|
||||||
later commits haven't regressed.
|
|
||||||
|
|
||||||
Which version of Python-3.x and which version of Python-2.x are our minimums?
|
|
||||||
=============================================================================
|
|
||||||
|
|
||||||
The short answer is Python-3.5 and Python-2.4 but please read on for more
|
|
||||||
information.
|
|
||||||
|
|
||||||
For Python-3 we are currently using Python-3.5 as a minimum on both the
|
|
||||||
controller and the managed nodes. This was chosen as it's the version of
|
|
||||||
Python3 in Ubuntu-16.04, the first long-term support (LTS) distribution to
|
|
||||||
ship with Python3 and not Python2. Much of our code would still work with
|
|
||||||
Python-3.4 but there are always bugfixes and new features in any new upstream
|
|
||||||
release. Taking advantage of this relatively new version allows us not to
|
|
||||||
worry about workarounds for problems and missing features in that older
|
|
||||||
version.
|
|
||||||
|
|
||||||
For Python-2, the default is for the controller to run on Python-2.6 and
|
|
||||||
modules to run on Python-2.4. This allows users with older distributions that
|
|
||||||
are stuck on Python-2.4 to manage their machines. Modules are allowed to drop
|
|
||||||
support for Python-2.4 when one of their dependent libraries require a higher
|
|
||||||
version of python. This is not an invitation to add unnecessary dependent
|
|
||||||
libraries in order to force your module to be usable only with a newer version
|
|
||||||
of Python. Instead it is an acknowledgment that some libraries (for instance,
|
|
||||||
boto3 and docker-py) will only function with newer Python.
|
|
||||||
|
|
||||||
.. note:: When will we drop support for Python-2.4?
|
|
||||||
|
|
||||||
The only long term supported distro that we know of with Python-2.4 is
|
|
||||||
RHEL5 (and its rebuilds like CentOS5) which is supported until April of
|
|
||||||
2017. Whatever major release we make in or after April of 2017 (probably
|
|
||||||
2.4.0) will no longer have support for Python-2.4 on the managed machines.
|
|
||||||
Previous major release series's that we support (2.3.x) will continue to
|
|
||||||
support Python-2.4 on the managed nodes.
|
|
||||||
|
|
||||||
We know of no long term supported distributions with Python-2.5 so the new
|
|
||||||
minimum Python-2 version will be Python-2.6. This will let us take
|
|
||||||
advantage of the forwards-compat features of Python-2.6 so porting and
|
|
||||||
maintainance of Python-2/Python-3 code will be easier after that.
|
|
||||||
|
|
||||||
|
|
||||||
Supporting only Python-2 or only Python-3
|
|
||||||
=========================================
|
|
||||||
|
|
||||||
Sometimes a module's dependent libraries only run on Python-2 or only run on
|
|
||||||
Python-3. We do not yet have a strategy for these modules but we'll need to
|
|
||||||
come up with one. I see three possibilities:
|
|
||||||
|
|
||||||
1. We treat these libraries like any other libraries that may not be installed
|
|
||||||
on the system. When we import them we check if the import was successful.
|
|
||||||
If so, then we continue. If not we return an error about the library being
|
|
||||||
missing. Users will have to find out that the library is unavailable on
|
|
||||||
their version of Python either by searching for the library on their own or
|
|
||||||
reading the requirements section in :command:`ansible-doc`.
|
|
||||||
|
|
||||||
2. The shebang line is the only metadata that Ansible extracts from a module
|
|
||||||
so we may end up using that to specify what we mean. Something like
|
|
||||||
``#!/usr/bin/python`` means the module will run on both Python-2 and
|
|
||||||
Python-3, ``#!/usr/bin/python2`` means the module will only run on
|
|
||||||
Python-2, and ``#!/usr/bin/python3`` means the module will only run on
|
|
||||||
Python-3. Ansible's code will need to be modified to accommodate this.
|
|
||||||
For :command:`python2`, if ``ansible_python2_interpreter`` is not set, it
|
|
||||||
will have to fallback to `` ansible_python_interpreter`` and if that's not
|
|
||||||
set, fallback to ``/usr/bin/python``. For :command:`python3`, Ansible
|
|
||||||
will have to first try ``ansible_python3_interpreter`` and then fallback to
|
|
||||||
``/usr/bin/python3`` as normal.
|
|
||||||
|
|
||||||
3. We add a way for Ansible to retrieve metadata about modules. The metadata
|
|
||||||
will include the version of Python that is required.
|
|
||||||
|
|
||||||
Methods 2 and 3 will both require that we modify modules or otherwise add this
|
|
||||||
additional information somewhere. 2 needs only a little code changes in
|
|
||||||
executor/module_common.py to parse. 3 will require a lot of work. This is
|
|
||||||
probably not worthwhile if this is the only change but could be worthwhile if
|
|
||||||
there's other things as well. 1 requires that we port all modules to work
|
|
||||||
with python3 syntax but only the code path to get to the library import being
|
|
||||||
attempted and then a fail_json() being called because the libraries are
|
|
||||||
unavailable needs to actually work.
|
|
||||||
|
|
||||||
.. note:: Metadata proposal in progress
|
|
||||||
|
|
||||||
A metadata specification is being created to address module
|
|
||||||
maintainership. In the future we will likely extend this to record that a module
|
|
||||||
works with Python2 and 3, Python2 only, or Python3 only.
|
|
||||||
|
|
||||||
Tips, tricks, and idioms to adopt
|
|
||||||
=================================
|
|
||||||
|
|
||||||
Exceptions
|
|
||||||
----------
|
|
||||||
|
|
||||||
In code which already needs Python-2.6+ (For instance, because a library it
|
|
||||||
depends on only runs on Python >= 2.6) it is okay to port directly to the new
|
|
||||||
exception catching syntax::
|
|
||||||
|
|
||||||
try:
|
|
||||||
a = 2/0
|
|
||||||
except ValueError as e:
|
|
||||||
module.fail_json(msg="Tried to divide by zero!")
|
|
||||||
|
|
||||||
For modules which also run on Python-2.4, we have to use an uglier
|
|
||||||
construction to make this work under both Python-2.4 and Python-3::
|
|
||||||
|
|
||||||
from ansible.module_utils.pycompat24 import get_exception
|
|
||||||
[...]
|
|
||||||
|
|
||||||
try:
|
|
||||||
a = 2/0
|
|
||||||
except ValueError:
|
|
||||||
e = get_exception()
|
|
||||||
module.fail_json(msg="Tried to divide by zero!")
|
|
||||||
|
|
||||||
Octal numbers
|
|
||||||
-------------
|
|
||||||
|
|
||||||
In Python-2.4, octal literals are specified as ``0755``. In Python-3, that is
|
|
||||||
invalid and octals must be specified as ``0o755``. To bridge this gap,
|
|
||||||
modules should create their octals like this::
|
|
||||||
|
|
||||||
# Can't use 0755 on Python-3 and can't use 0o755 on Python-2.4
|
|
||||||
EXECUTABLE_PERMS = int('0755', 8)
|
|
||||||
|
|
||||||
Outputting octal numbers may also need to be changed. In python2 we often did
|
|
||||||
this to return file permissions::
|
|
||||||
|
|
||||||
mode = int('0775', 8)
|
|
||||||
result['mode'] = oct(mode)
|
|
||||||
|
|
||||||
This would give the user ``result['mode'] == '0755'`` in their playbook. In
|
|
||||||
python3, :func:`oct` returns the format with the lowercase ``o`` in it like:
|
|
||||||
``result['mode'] == '0o755'``. If a user had a conditional in their playbook
|
|
||||||
or was using the mode in a template the new format might break things. We
|
|
||||||
need to return the old form of mode for backwards compatibility. You can do
|
|
||||||
it like this::
|
|
||||||
|
|
||||||
mode = int('0775', 8)
|
|
||||||
result['mode'] = '0%03o' % mode
|
|
||||||
|
|
||||||
You should use this wherever backwards compatibility is a concern or you are
|
|
||||||
dealing with file permissions. (With file permissions a user may be feeding
|
|
||||||
the mode into another program or to another module which doesn't understand
|
|
||||||
the python syntax for octal numbers. ``[zero][digit][digit][digit]`` is
|
|
||||||
understood by most everything and therefore the right way to express octals in
|
|
||||||
these circumstances.
|
|
||||||
|
|
||||||
Bundled six
|
|
||||||
-----------
|
|
||||||
|
|
||||||
The third-party python-six library exists to help projects create code that
|
|
||||||
runs on both Python-2 and Python-3. Ansible includes version 1.4.1 in
|
|
||||||
module_utils so that other modules can use it without requiring that it is
|
|
||||||
installed on the remote system. To make use of it, import it like this::
|
|
||||||
|
|
||||||
from ansible.module_utils import six
|
|
||||||
|
|
||||||
.. note:: Why version 1.4.1?
|
|
||||||
|
|
||||||
six-1.4.1 is the last version of python-six to support Python-2.4. As
|
|
||||||
long as Ansible modules need to run on Python-2.4 we won't be able to
|
|
||||||
update the bundled copy of six.
|
|
||||||
|
|
||||||
Compile Test
|
|
||||||
------------
|
|
||||||
|
|
||||||
We have Shippable compiling all modules with various versions of Python to check
|
|
||||||
that the modules conform to the syntax at those versions. When you've
|
|
||||||
ported a module so that its syntax works with Python-3, we need to remove it from
|
|
||||||
the blacklist so that the module is included in the syntax check.
|
|
||||||
|
|
||||||
The file `test/utils/shippable/sanity-skip-python3.txt` contains the list of
|
|
||||||
modules which should not be tested (because we know that they are older modules which
|
|
||||||
have not yet been ported to pass the Python-3 syntax checks. To get another
|
|
||||||
old module to compile with Python-3, remove the entry for it from the list.
|
|
||||||
The goal is to have the LIST be empty.
|
|
||||||
|
|
||||||
String Model
|
|
||||||
------------
|
|
||||||
|
|
||||||
One of the big differences between Python2 and Python3 is the string model.
|
|
||||||
In Python2, most APIs take byte strings (the Python2 ``str`` type). Using the
|
|
||||||
text type (in Python2, this is the ``unicode`` type) often leads to tracebacks
|
|
||||||
because the strings need to be converted to bytes and Python fails to do that
|
|
||||||
correctly. In Python3, the situation is somewhat reversed. Most APIs take
|
|
||||||
text strings (this is **Python3's** ``str`` type). When you have byte strings
|
|
||||||
(the Python3 ``bytes`` type) you sometimes get errors when attempting to
|
|
||||||
combine those with text strings. Note, however, that under the hood, Python
|
|
||||||
still has to convert text to bytes to interface operating system libraries and
|
|
||||||
system calls. This means that you can still get tracebacks when passing
|
|
||||||
text to APIs which call those OS level facilities.
|
|
||||||
|
|
||||||
For module_utils, code we've decided to make the environment work with "native
|
|
||||||
strings". This means that on Python2, things should work if you use the byte
|
|
||||||
string type. In Python3, code should work if you give it text strings. The
|
|
||||||
reason for this is so that third party modules written for Python2 don't start
|
|
||||||
issuing UnicodeError exceptions once we've ported module_utils to work under
|
|
||||||
Python3. We'll need to gather experience to see if this is going to work out
|
|
||||||
well for modules as well or if we should give the module_utils API explicit
|
|
||||||
switches so that modules can choose to operate with text type all of the time.
|
|
||||||
|
|
||||||
Helpers
|
|
||||||
~~~~~~~
|
|
||||||
|
|
||||||
For converting between bytes, text, and native strings we have three helper
|
|
||||||
functions. These are :func:`ansible.module_utils._text.to_bytes`,
|
|
||||||
:func:`ansible.module_utils._text.to_native`, and
|
|
||||||
:func:`ansible.module_utils._text.to_text`. These are similar to using
|
|
||||||
``bytes.decode()`` and ``unicode.encode()`` with a few differences.
|
|
||||||
|
|
||||||
* By default they try very hard not to traceback.
|
|
||||||
* The default encoding is "utf-8"
|
|
||||||
* There are two error strategies that don't correspond one-to-one with
|
|
||||||
a python codec error handler. These are ``surrogate_or_strict`` and
|
|
||||||
``surrogate_or_replace``. ``surrogate_or_strict`` will use the ``surrogateescape``
|
|
||||||
error handler if available (mostly on python3) or strict if not. It is most
|
|
||||||
appropriate to use when dealing with something that needs to round trip its
|
|
||||||
value like file paths database keys, etc. Without ``surrogateescape`` the best
|
|
||||||
thing these values can do is generate a traceback that our code can catch
|
|
||||||
and decide how to show an error message. ``surrogate_or_replace`` is for
|
|
||||||
when a value is going to be displayed to the user. If the
|
|
||||||
``surrogateescape`` error handler is not present, it will replace
|
|
||||||
undecodable byte sequences with a replacement character.
|
|
||||||
|
|
||||||
================================
|
|
||||||
Porting Core Ansible to Python 3
|
|
||||||
================================
|
|
||||||
|
|
||||||
The Ansible code which runs controller-side is easier to port to Python3 in
|
|
||||||
one important way: We do not have to support Python-2.4 on the controller.
|
|
||||||
We only have to support Python-2.6 and above. However, this doesn't eliminate
|
|
||||||
the work that has to be done. The controller is a much more complicated piece
|
|
||||||
of code than any individual module. Making it Python2 and Python3 compatible
|
|
||||||
is a much more complex task.
|
|
||||||
|
|
||||||
String Model
|
|
||||||
------------
|
|
||||||
|
|
||||||
By and large, the controller uses the standard best practice of storing
|
|
||||||
everything internally as text type and converting to and from bytes at the
|
|
||||||
borders. In many places we hardcode these byte values as utf-8. Thus yaml
|
|
||||||
and inventory files are encoded in utf-8. Filenames are also utf-8. This may
|
|
||||||
not be the right answer forever but it is sufficient for now. If there's
|
|
||||||
demand from users to handle encodings other than utf-8 after the code works on
|
|
||||||
Python3 we can look into what strategy to take for supporting other encodings.
|
|
||||||
|
|
||||||
In some cases, storing values as a byte string is not necessarily a choice
|
|
||||||
without drawbacks. For instance, filenames and environment variables on POSIX
|
|
||||||
systems are a sequence of bytes. By using text to represent filenames we
|
|
||||||
prevent filenames that are undecodable in utf-8 and filenames that are not
|
|
||||||
text at all from working. We made the choice to represent these as text for
|
|
||||||
now due to code paths that handle filenames not being able to handle bytes
|
|
||||||
end-to-end. PyYAML on Python3 and jinja2 on both Python2 and Python3, for
|
|
||||||
instance, are meant to work with text. Any decision to allow filenames to be
|
|
||||||
byte values will have to address how we deal with those pieces of the code as
|
|
||||||
well.
|
|
||||||
@ -1,209 +0,0 @@
|
|||||||
Developing Plugins
|
|
||||||
==================
|
|
||||||
|
|
||||||
.. contents:: Topics
|
|
||||||
|
|
||||||
Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own.
|
|
||||||
|
|
||||||
The following types of plugins are available:
|
|
||||||
|
|
||||||
- *Callback* plugins enable you to hook into Ansible events for display or logging purposes.
|
|
||||||
- *Connection* plugins define how to communicate with inventory hosts.
|
|
||||||
- *Lookup* plugins are used to pull data from an external source.
|
|
||||||
- *Vars* plugins inject additional variable data into Ansible runs that did not come from an inventory, playbook, or the command line.
|
|
||||||
|
|
||||||
This section describes the various types of plugins and how to implement them.
|
|
||||||
|
|
||||||
|
|
||||||
.. _developing_callbacks:
|
|
||||||
|
|
||||||
Callback Plugins
|
|
||||||
----------------
|
|
||||||
|
|
||||||
Callback plugins enable adding new behaviors to Ansible when responding to events.
|
|
||||||
|
|
||||||
.. _callback_examples:
|
|
||||||
|
|
||||||
Example Callback Plugins
|
|
||||||
++++++++++++++++++++++++
|
|
||||||
|
|
||||||
Ansible comes with a number of callback plugins that you can look at for examples. These can be found in `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_.
|
|
||||||
|
|
||||||
The `log_plays
|
|
||||||
<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/log_plays.py>`_
|
|
||||||
callback is an example of how to intercept playbook events to a log
|
|
||||||
file, and the `mail
|
|
||||||
<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/mail.py>`_
|
|
||||||
callback sends email when playbooks complete.
|
|
||||||
|
|
||||||
The `osx_say
|
|
||||||
<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/osx_say.py>`_
|
|
||||||
callback provided is particularly entertaining -- it will respond with
|
|
||||||
computer synthesized speech on OS X in relation to playbook events,
|
|
||||||
and is guaranteed to entertain and/or annoy coworkers.
|
|
||||||
|
|
||||||
.. _configuring_callbacks:
|
|
||||||
|
|
||||||
Configuring Callback Plugins
|
|
||||||
++++++++++++++++++++++++++++
|
|
||||||
|
|
||||||
To activate a callback, drop it in a callback directory as configured in `ansible.cfg`.
|
|
||||||
|
|
||||||
Plugins are loaded in alphanumeric order; for example, a plugin implemented in a file named `1_first.py` would run before a plugin file named `2_second.py`.
|
|
||||||
|
|
||||||
Callbacks need to be whitelisted in your `ansible.cfg` file in order to function. For example::
|
|
||||||
|
|
||||||
#callback_whitelist = timer, mail, mycallbackplugin
|
|
||||||
|
|
||||||
|
|
||||||
Writing to stdout
|
|
||||||
`````````````````
|
|
||||||
|
|
||||||
If your callback plugin needs to write to stdout, you should define CALLBACK_TYPE = stdout in the subclass, and then the stdout plugin needs to be configured in `ansible.cfg` to override the default. For example::
|
|
||||||
|
|
||||||
#stdout_callback = mycallbackplugin
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. _callback_development:
|
|
||||||
|
|
||||||
Developing Callback Plugins
|
|
||||||
+++++++++++++++++++++++++++
|
|
||||||
|
|
||||||
Callback plugins are created by creating a new class with the Base(Callbacks) class as the parent::
|
|
||||||
|
|
||||||
from ansible.plugins.callback import CallbackBase
|
|
||||||
from ansible import constants as C
|
|
||||||
|
|
||||||
class CallbackModule(CallbackBase):
|
|
||||||
|
|
||||||
From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`. For a complete list of methods that you can override, please see ``__init__.py`` in the `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_ directory.
|
|
||||||
|
|
||||||
|
|
||||||
The following example shows how Ansible's timer plugin is implemented::
|
|
||||||
|
|
||||||
# Make coding more python3-ish
|
|
||||||
from __future__ import (absolute_import, division, print_function)
|
|
||||||
__metaclass__ = type
|
|
||||||
|
|
||||||
from datetime import datetime
|
|
||||||
|
|
||||||
from ansible.plugins.callback import CallbackBase
|
|
||||||
|
|
||||||
|
|
||||||
class CallbackModule(CallbackBase):
|
|
||||||
"""
|
|
||||||
This callback module tells you how long your plays ran for.
|
|
||||||
"""
|
|
||||||
CALLBACK_VERSION = 2.0
|
|
||||||
CALLBACK_TYPE = 'aggregate'
|
|
||||||
CALLBACK_NAME = 'timer'
|
|
||||||
CALLBACK_NEEDS_WHITELIST = True
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
|
|
||||||
super(CallbackModule, self).__init__()
|
|
||||||
|
|
||||||
self.start_time = datetime.now()
|
|
||||||
|
|
||||||
def days_hours_minutes_seconds(self, runtime):
|
|
||||||
minutes = (runtime.seconds // 60) % 60
|
|
||||||
r_seconds = runtime.seconds - (minutes * 60)
|
|
||||||
return runtime.days, runtime.seconds // 3600, minutes, r_seconds
|
|
||||||
|
|
||||||
def playbook_on_stats(self, stats):
|
|
||||||
self.v2_playbook_on_stats(stats)
|
|
||||||
|
|
||||||
def v2_playbook_on_stats(self, stats):
|
|
||||||
end_time = datetime.now()
|
|
||||||
runtime = end_time - self.start_time
|
|
||||||
self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds" % (self.days_hours_minutes_seconds(runtime)))
|
|
||||||
|
|
||||||
Note that the CALLBACK_VERSION and CALLBACK_NAME definitons are required.
|
|
||||||
|
|
||||||
.. _developing_connection_type_plugins:
|
|
||||||
|
|
||||||
Connection Type Plugins
|
|
||||||
-----------------------
|
|
||||||
|
|
||||||
By default, ansible ships with a 'paramiko' SSH, native ssh (just called 'ssh'), 'local' connection type, and there are also some minor players like 'chroot' and 'jail'. All of these can be used
|
|
||||||
in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines. The basics of these connection types
|
|
||||||
are covered in the :doc:`intro_getting_started` section. Should you want to extend Ansible to support other transports (SNMP? Message bus?
|
|
||||||
Carrier Pigeon?) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins
|
|
||||||
directory. The value of 'smart' for a connection allows selection of paramiko or openssh based on system capabilities, and chooses
|
|
||||||
'ssh' if OpenSSH supports ControlPersist, in Ansible 1.2.1 and later. Previous versions did not support 'smart'.
|
|
||||||
|
|
||||||
More documentation on writing connection plugins is pending, though you can jump into `lib/ansible/plugins/connection <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/connection>`_ and figure things out pretty easily.
|
|
||||||
|
|
||||||
.. _developing_lookup_plugins:
|
|
||||||
|
|
||||||
Lookup Plugins
|
|
||||||
--------------
|
|
||||||
|
|
||||||
Language constructs like "with_fileglob" and "with_items" are implemented via lookup plugins. Just like other plugin types, you can write your own.
|
|
||||||
|
|
||||||
More documentation on writing lookup plugins is pending, though you can jump into `lib/ansible/plugins/lookup <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/lookup>`_ and figure
|
|
||||||
things out pretty easily.
|
|
||||||
|
|
||||||
.. _developing_vars_plugins:
|
|
||||||
|
|
||||||
Vars Plugins
|
|
||||||
------------
|
|
||||||
|
|
||||||
Playbook constructs like 'host_vars' and 'group_vars' work via 'vars' plugins. They inject additional variable
|
|
||||||
data into ansible runs that did not come from an inventory, playbook, or command line. Note that variables
|
|
||||||
can also be returned from inventory, so in most cases, you won't need to write or understand vars_plugins.
|
|
||||||
|
|
||||||
More documentation on writing vars plugins is pending, though you can jump into `lib/ansible/inventory/vars_plugins <https://github.com/ansible/ansible/tree/devel/lib/ansible/inventory/vars_plugins>`_ and figure
|
|
||||||
things out pretty easily.
|
|
||||||
|
|
||||||
If you find yourself wanting to write a vars_plugin, it's more likely you should write an inventory script instead.
|
|
||||||
|
|
||||||
.. _developing_filter_plugins:
|
|
||||||
|
|
||||||
Filter Plugins
|
|
||||||
--------------
|
|
||||||
|
|
||||||
If you want more Jinja2 filters available in a Jinja2 template (filters like to_yaml and to_json are provided by default), they can be extended by writing a filter plugin. Most of the time, when someone comes up with an idea for a new filter they would like to make available in a playbook, we'll just include them in 'core.py' instead.
|
|
||||||
|
|
||||||
Jump into `lib/ansible/plugins/filter <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/filter>`_ for details.
|
|
||||||
|
|
||||||
.. _distributing_plugins:
|
|
||||||
|
|
||||||
Distributing Plugins
|
|
||||||
--------------------
|
|
||||||
|
|
||||||
Plugins are loaded from both Python's site_packages (those that ship with ansible) and a configured plugins directory, which defaults
|
|
||||||
to /usr/share/ansible/plugins, in a subfolder for each plugin type::
|
|
||||||
|
|
||||||
* action
|
|
||||||
* lookup
|
|
||||||
* callback
|
|
||||||
* connection
|
|
||||||
* filter
|
|
||||||
* strategy
|
|
||||||
* cache
|
|
||||||
* test
|
|
||||||
* shell
|
|
||||||
|
|
||||||
To change this path, edit the ansible configuration file.
|
|
||||||
|
|
||||||
In addition, plugins can be shipped in a subdirectory relative to a top-level playbook, in folders named the same as indicated above.
|
|
||||||
|
|
||||||
They can also be shipped as part of a role, in a subdirectory named as indicated above. The plugin will be availiable as soon as the role
|
|
||||||
is called.
|
|
||||||
|
|
||||||
.. seealso::
|
|
||||||
|
|
||||||
:doc:`modules`
|
|
||||||
List of built-in modules
|
|
||||||
:doc:`developing_api`
|
|
||||||
Learn about the Python API for task execution
|
|
||||||
:doc:`developing_inventory`
|
|
||||||
Learn about how to develop dynamic inventory sources
|
|
||||||
:doc:`developing_modules`
|
|
||||||
Learn about how to write Ansible modules
|
|
||||||
`Mailing List <http://groups.google.com/group/ansible-devel>`_
|
|
||||||
The development mailing list
|
|
||||||
`irc.freenode.net <http://irc.freenode.net>`_
|
|
||||||
#ansible IRC chat channel
|
|
||||||
@ -1,523 +0,0 @@
|
|||||||
.. _flow_modules:
|
|
||||||
|
|
||||||
=======
|
|
||||||
Modules
|
|
||||||
=======
|
|
||||||
|
|
||||||
This in-depth dive helps you understand Ansible's program flow to execute
|
|
||||||
modules. It is written for people working on the portions of the Core Ansible
|
|
||||||
Engine that execute a module. Those writing Ansible Modules may also find this
|
|
||||||
in-depth dive to be of interest, but individuals simply using Ansible Modules
|
|
||||||
will not likely find this to be helpful.
|
|
||||||
|
|
||||||
.. _flow_types_of_modules:
|
|
||||||
|
|
||||||
Types of Modules
|
|
||||||
================
|
|
||||||
|
|
||||||
Ansible supports several different types of modules in its code base. Some of
|
|
||||||
these are for backwards compatibility and others are to enable flexibility.
|
|
||||||
|
|
||||||
.. _flow_action_plugins:
|
|
||||||
|
|
||||||
Action Plugins
|
|
||||||
--------------
|
|
||||||
|
|
||||||
Action Plugins look like modules to end users who are writing :term:`playbooks` but
|
|
||||||
they're distinct entities for the purposes of this document. Action Plugins
|
|
||||||
always execute on the controller and are sometimes able to do all work there
|
|
||||||
(for instance, the ``debug`` Action Plugin which prints some text for the user to
|
|
||||||
see or the ``assert`` Action Plugin which can test whether several values in
|
|
||||||
a playbook satisfy certain criteria.)
|
|
||||||
|
|
||||||
More often, Action Plugins set up some values on the controller, then invoke an
|
|
||||||
actual module on the managed node that does something with these values. An
|
|
||||||
easy to understand version of this is the :ref:`template Action Plugin
|
|
||||||
<template>`. The :ref:`template Action Plugin <template>` takes values from
|
|
||||||
the user to construct a file in a temporary location on the controller using
|
|
||||||
variables from the playbook environment. It then transfers the temporary file
|
|
||||||
to a temporary file on the remote system. After that, it invokes the
|
|
||||||
:ref:`copy module <copy>` which operates on the remote system to move the file
|
|
||||||
into its final location, sets file permissions, and so on.
|
|
||||||
|
|
||||||
.. _flow_new_style_modules:
|
|
||||||
|
|
||||||
New-style Modules
|
|
||||||
-----------------
|
|
||||||
|
|
||||||
All of the modules that ship with Ansible fall into this category.
|
|
||||||
|
|
||||||
New-style modules have the arguments to the module embedded inside of them in
|
|
||||||
some manner. Non-new-style modules must copy a separate file over to the
|
|
||||||
managed node, which is less efficient as it requires two over-the-wire
|
|
||||||
connections instead of only one.
|
|
||||||
|
|
||||||
.. _flow_python_modules:
|
|
||||||
|
|
||||||
Python
|
|
||||||
^^^^^^
|
|
||||||
|
|
||||||
New-style Python modules use the :ref:`Ansiballz` framework for constructing
|
|
||||||
modules. All official modules (shipped with Ansible) use either this or the
|
|
||||||
:ref:`powershell module framework <flow_powershell_modules>`.
|
|
||||||
|
|
||||||
These modules use imports from :code:`ansible.module_utils` in order to pull in
|
|
||||||
boilerplate module code, such as argument parsing, formatting of return
|
|
||||||
values as :term:`JSON`, and various file operations.
|
|
||||||
|
|
||||||
.. note:: In Ansible, up to version 2.0.x, the official Python modules used the
|
|
||||||
:ref:`module_replacer` framework. For module authors, :ref:`Ansiballz` is
|
|
||||||
largely a superset of :ref:`module_replacer` functionality, so you usually
|
|
||||||
do not need to know about one versus the other.
|
|
||||||
|
|
||||||
.. _flow_powershell_modules:
|
|
||||||
|
|
||||||
Powershell
|
|
||||||
^^^^^^^^^^
|
|
||||||
|
|
||||||
New-style powershell modules use the :ref:`module_replacer` framework for
|
|
||||||
constructing modules. These modules get a library of powershell code embedded
|
|
||||||
in them before being sent to the managed node.
|
|
||||||
|
|
||||||
.. _flow_jsonargs_modules:
|
|
||||||
|
|
||||||
JSONARGS
|
|
||||||
^^^^^^^^
|
|
||||||
|
|
||||||
Scripts can arrange for an argument string to be placed within them by placing
|
|
||||||
the string ``<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>`` somewhere inside of the
|
|
||||||
file. The module typically sets a variable to that value like this::
|
|
||||||
|
|
||||||
json_arguments = """<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>"""
|
|
||||||
|
|
||||||
Which is expanded as::
|
|
||||||
|
|
||||||
json_arguments = """{"param1": "test's quotes", "param2": "\"To be or not to be\" - Hamlet"}"""
|
|
||||||
|
|
||||||
.. note:: Ansible outputs a :term:`JSON` string with bare quotes. Double quotes are
|
|
||||||
used to quote string values, double quotes inside of string values are
|
|
||||||
backslash escaped, and single quotes may appear unescaped inside of
|
|
||||||
a string value. To use JSONARGS, your scripting language must have a way
|
|
||||||
to handle this type of string. The example uses Python's triple quoted
|
|
||||||
strings to do this. Other scripting languages may have a similar quote
|
|
||||||
character that won't be confused by any quotes in the JSON or it may
|
|
||||||
allow you to define your own start-of-quote and end-of-quote characters.
|
|
||||||
If the language doesn't give you any of these then you'll need to write
|
|
||||||
a :ref:`non-native JSON module <flow_want_json_modules>` or
|
|
||||||
:ref:`Old-style module <flow_old_style_modules>` instead.
|
|
||||||
|
|
||||||
The module typically parses the contents of ``json_arguments`` using a JSON
|
|
||||||
library and then use them as native variables throughout the rest of its code.
|
|
||||||
|
|
||||||
.. _flow_want_json_modules:
|
|
||||||
|
|
||||||
Non-native want JSON modules
|
|
||||||
----------------------------
|
|
||||||
|
|
||||||
If a module has the string ``WANT_JSON`` in it anywhere, Ansible treats
|
|
||||||
it as a non-native module that accepts a filename as its only command line
|
|
||||||
parameter. The filename is for a temporary file containing a :term:`JSON`
|
|
||||||
string containing the module's parameters. The module needs to open the file,
|
|
||||||
read and parse the parameters, operate on the data, and print its return data
|
|
||||||
as a JSON encoded dictionary to stdout before exiting.
|
|
||||||
|
|
||||||
These types of modules are self-contained entities. As of Ansible 2.1, Ansible
|
|
||||||
only modifies them to change a shebang line if present.
|
|
||||||
|
|
||||||
.. seealso:: Examples of Non-native modules written in ruby are in the `Ansible
|
|
||||||
for Rubyists <https://github.com/ansible/ansible-for-rubyists>`_ repository.
|
|
||||||
|
|
||||||
.. _flow_binary_modules:
|
|
||||||
|
|
||||||
Binary Modules
|
|
||||||
--------------
|
|
||||||
|
|
||||||
From Ansible 2.2 onwards, modules may also be small binary programs. Ansible
|
|
||||||
doesn't perform any magic to make these portable to different systems so they
|
|
||||||
may be specific to the system on which they were compiled or require other
|
|
||||||
binary runtime dependencies. Despite these drawbacks, a site may sometimes
|
|
||||||
have no choice but to compile a custom module against a specific binary
|
|
||||||
library if that's the only way they have to get access to certain resources.
|
|
||||||
|
|
||||||
Binary modules take their arguments and will return data to Ansible in the same
|
|
||||||
way as :ref:`want JSON modules <flow_want_json_modules>`.
|
|
||||||
|
|
||||||
.. seealso:: One example of a `binary module
|
|
||||||
<https://github.com/ansible/ansible/blob/devel/test/integration/library/helloworld.go>`_
|
|
||||||
written in go.
|
|
||||||
|
|
||||||
.. _flow_old_style_modules:
|
|
||||||
|
|
||||||
Old-style Modules
|
|
||||||
-----------------
|
|
||||||
|
|
||||||
Old-style modules are similar to
|
|
||||||
:ref:`want JSON modules <flow_want_json_modules>`, except that the file that
|
|
||||||
they take contains ``key=value`` pairs for their parameters instead of
|
|
||||||
:term:`JSON`.
|
|
||||||
|
|
||||||
Ansible decides that a module is old-style when it doesn't have any of the
|
|
||||||
markers that would show that it is one of the other types.
|
|
||||||
|
|
||||||
.. _flow_how_modules_are_executed:
|
|
||||||
|
|
||||||
How modules are executed
|
|
||||||
========================
|
|
||||||
|
|
||||||
When a user uses :program:`ansible` or :program:`ansible-playbook`, they
|
|
||||||
specify a task to execute. The task is usually the name of a module along
|
|
||||||
with several parameters to be passed to the module. Ansible takes these
|
|
||||||
values and processes them in various ways before they are finally executed on
|
|
||||||
the remote machine.
|
|
||||||
|
|
||||||
.. _flow_executor_task_executor:
|
|
||||||
|
|
||||||
executor/task_executor
|
|
||||||
----------------------
|
|
||||||
|
|
||||||
The TaskExecutor receives the module name and parameters that were parsed from
|
|
||||||
the :term:`playbook <playbooks>` (or from the command line in the case of
|
|
||||||
:command:`/usr/bin/ansible`). It uses the name to decide whether it's looking
|
|
||||||
at a module or an :ref:`Action Plugin <flow_action_plugins>`. If it's
|
|
||||||
a module, it loads the :ref:`Normal Action Plugin <flow_normal_action_plugin>`
|
|
||||||
and passes the name, variables, and other information about the task and play
|
|
||||||
to that Action Plugin for further processing.
|
|
||||||
|
|
||||||
.. _flow_normal_action_plugin:
|
|
||||||
|
|
||||||
Normal Action Plugin
|
|
||||||
--------------------
|
|
||||||
|
|
||||||
The ``normal`` Action Plugin executes the module on the remote host. It is
|
|
||||||
the primary coordinator of much of the work to actually execute the module on
|
|
||||||
the managed machine.
|
|
||||||
|
|
||||||
* It takes care of creating a connection to the managed machine by
|
|
||||||
instantiating a ``Connection`` class according to the inventory
|
|
||||||
configuration for that host.
|
|
||||||
* It adds any internal Ansible variables to the module's parameters (for
|
|
||||||
instance, the ones that pass along ``no_log`` to the module).
|
|
||||||
* It takes care of creating any temporary files on the remote machine and
|
|
||||||
cleans up afterwards.
|
|
||||||
* It does the actual work of pushing the module and module parameters to the
|
|
||||||
remote host, although the :ref:`module_common <flow_executor_module_common>`
|
|
||||||
code described in the next section does the work of deciding which format
|
|
||||||
those will take.
|
|
||||||
* It handles any special cases regarding modules (for instance, various
|
|
||||||
complications around Windows modules that must have the same names as Python
|
|
||||||
modules, so that internal calling of modules from other Action Plugins work.)
|
|
||||||
|
|
||||||
Much of this functionality comes from the :class:`BaseAction` class,
|
|
||||||
which lives in :file:`plugins/action/__init__.py`. It makes use of
|
|
||||||
``Connection`` and ``Shell`` objects to do its work.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
When :term:`tasks <tasks>` are run with the ``async:`` parameter, Ansible
|
|
||||||
uses the ``async`` Action Plugin instead of the ``normal`` Action Plugin
|
|
||||||
to invoke it. That program flow is currently not documented. Read the
|
|
||||||
source for information on how that works.
|
|
||||||
|
|
||||||
.. _flow_executor_module_common:
|
|
||||||
|
|
||||||
executor/module_common.py
|
|
||||||
-------------------------
|
|
||||||
|
|
||||||
Code in :file:`executor/module_common.py` takes care of assembling the module
|
|
||||||
to be shipped to the managed node. The module is first read in, then examined
|
|
||||||
to determine its type. :ref:`PowerShell <flow_powershell_modules>` and
|
|
||||||
:ref:`JSON-args modules <flow_jsonargs_modules>` are passed through
|
|
||||||
:ref:`Module Replacer <module_replacer>`. New-style
|
|
||||||
:ref:`Python modules <flow_python_modules>` are assembled by :ref:`Ansiballz`.
|
|
||||||
:ref:`Non-native-want-JSON <flow_want_json_modules>`,
|
|
||||||
:ref:`Binary modules <flow_binary_modules>`, and
|
|
||||||
:ref:`Old-Style modules <flow_old_style_modules>` aren't touched by either of
|
|
||||||
these and pass through unchanged. After the assembling step, one final
|
|
||||||
modification is made to all modules that have a shebang line. Ansible checks
|
|
||||||
whether the interpreter in the shebang line has a specific path configured via
|
|
||||||
an ``ansible_$X_interpreter`` inventory variable. If it does, Ansible
|
|
||||||
substitutes that path for the interpreter path given in the module. After
|
|
||||||
this, Ansible returns the complete module data and the module type to the
|
|
||||||
:ref:`Normal Action <flow_normal_action_plugin>` which continues execution of
|
|
||||||
the module.
|
|
||||||
|
|
||||||
Next we'll go into some details of the two assembler frameworks.
|
|
||||||
|
|
||||||
.. _module_replacer:
|
|
||||||
|
|
||||||
Module Replacer
|
|
||||||
^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
The Module Replacer framework is the original framework implementing new-style
|
|
||||||
modules. It is essentially a preprocessor (like the C Preprocessor for those
|
|
||||||
familiar with that programming language). It does straight substitutions of
|
|
||||||
specific substring patterns in the module file. There are two types of
|
|
||||||
substitutions:
|
|
||||||
|
|
||||||
* Replacements that only happen in the module file. These are public
|
|
||||||
replacement strings that modules can utilize to get helpful boilerplate or
|
|
||||||
access to arguments.
|
|
||||||
|
|
||||||
- :code:`from ansible.module_utils.MOD_LIB_NAME import *` is replaced with the
|
|
||||||
contents of the :file:`ansible/module_utils/MOD_LIB_NAME.py` These should
|
|
||||||
only be used with :ref:`new-style Python modules <flow_python_modules>`.
|
|
||||||
- :code:`#<<INCLUDE_ANSIBLE_MODULE_COMMON>>` is equivalent to
|
|
||||||
:code:`from ansible.module_utils.basic import *` and should also only apply
|
|
||||||
to new-style Python modules.
|
|
||||||
- :code:`# POWERSHELL_COMMON` substitutes the contents of
|
|
||||||
:file:`ansible/module_utils/powershell.ps1`. It should only be used with
|
|
||||||
:ref:`new-style Powershell modules <flow_powershell_modules>`.
|
|
||||||
|
|
||||||
* Replacements that are used by ``ansible.module_utils`` code. These are internal
|
|
||||||
replacement patterns. They may be used internally, in the above public
|
|
||||||
replacements, but shouldn't be used directly by modules.
|
|
||||||
|
|
||||||
- :code:`"<<ANSIBLE_VERSION>>"` is substituted with the Ansible version. In
|
|
||||||
:ref:`new-style Python modules <flow_python_modules>` under the
|
|
||||||
:ref:`Ansiballz` frameworkthe proper way is to instead instantiate an
|
|
||||||
:class:`AnsibleModule` and then access the version from
|
|
||||||
:attr:``AnsibleModule.ansible_version``.
|
|
||||||
- :code:`"<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>"` is substituted with
|
|
||||||
a string which is the Python ``repr`` of the :term:`JSON` encoded module
|
|
||||||
parameters. Using ``repr`` on the JSON string makes it safe to embed in
|
|
||||||
a Python file. In new-style Python modules under the Ansiballz framework
|
|
||||||
this is better accessed by instantiating an :class:`AnsibleModule` and
|
|
||||||
then using :attr:`AnsibleModule.params`.
|
|
||||||
- :code:`<<SELINUX_SPECIAL_FILESYSTEMS>>` substitutes a string which is
|
|
||||||
a comma separated list of file systems which have a file system dependent
|
|
||||||
security context in SELinux. In new-style Python modules, if you really
|
|
||||||
need this you should instantiate an :class:`AnsibleModule` and then use
|
|
||||||
:attr:`AnsibleModule._selinux_special_fs`. The variable has also changed
|
|
||||||
from a comma separated string of file system names to an actual python
|
|
||||||
list of filesystem names.
|
|
||||||
- :code:`<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>` substitutes the module
|
|
||||||
parameters as a JSON string. Care must be taken to properly quote the
|
|
||||||
string as JSON data may contain quotes. This pattern is not substituted
|
|
||||||
in new-style Python modules as they can get the module parameters another
|
|
||||||
way.
|
|
||||||
- The string :code:`syslog.LOG_USER` is replaced wherever it occurs with the
|
|
||||||
``syslog_facility`` which was named in :file:`ansible.cfg` or any
|
|
||||||
``ansible_syslog_facility`` inventory variable that applies to this host. In
|
|
||||||
new-style Python modules this has changed slightly. If you really need to
|
|
||||||
access it, you should instantiate an :class:`AnsibleModule` and then use
|
|
||||||
:attr:`AnsibleModule._syslog_facility` to access it. It is no longer the
|
|
||||||
actual syslog facility and is now the name of the syslog facility. See
|
|
||||||
the :ref:`documentation on internal arguments <flow_internal_arguments>`
|
|
||||||
for details.
|
|
||||||
|
|
||||||
.. _Ansiballz:
|
|
||||||
|
|
||||||
Ansiballz
|
|
||||||
^^^^^^^^^
|
|
||||||
|
|
||||||
Ansible 2.1 switched from the :ref:`module_replacer` framework to the
|
|
||||||
Ansiballz framework for assembling modules. The Ansiballz framework differs
|
|
||||||
from module replacer in that it uses real Python imports of things in
|
|
||||||
:file:`ansible/module_utils` instead of merely preprocessing the module. It
|
|
||||||
does this by constructing a zipfile -- which includes the module file, files
|
|
||||||
in :file:`ansible/module_utils` that are imported by the module, and some
|
|
||||||
boilerplate to pass in the module's parameters. The zipfile is then Base64
|
|
||||||
encoded and wrapped in a small Python script which decodes the Base64 encoding
|
|
||||||
and places the zipfile into a temp directory on the managed node. It then
|
|
||||||
extracts just the ansible module script from the zip file and places that in
|
|
||||||
the temporary directory as well. Then it sets the PYTHONPATH to find python
|
|
||||||
modules inside of the zip file and invokes :command:`python` on the extracted
|
|
||||||
ansible module.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Ansible wraps the zipfile in the Python script for two reasons:
|
|
||||||
|
|
||||||
* for compatibility with Python-2.4 and Python-2.6 which have less
|
|
||||||
featureful versions of Python's ``-m`` command line switch.
|
|
||||||
* so that pipelining will function properly. Pipelining needs to pipe the
|
|
||||||
Python module into the Python interpreter on the remote node. Python
|
|
||||||
understands scripts on stdin but does not understand zip files.
|
|
||||||
|
|
||||||
In Ansiballz, any imports of Python modules from the
|
|
||||||
:py:mod:`ansible.module_utils` package trigger inclusion of that Python file
|
|
||||||
into the zipfile. Instances of :code:`#<<INCLUDE_ANSIBLE_MODULE_COMMON>>` in
|
|
||||||
the module are turned into :code:`from ansible.module_utils.basic import *`
|
|
||||||
and :file:`ansible/module-utils/basic.py` is then included in the zipfile.
|
|
||||||
Files that are included from :file:`module_utils` are themselves scanned for
|
|
||||||
imports of other Python modules from :file:`module_utils` to be included in
|
|
||||||
the zipfile as well.
|
|
||||||
|
|
||||||
.. warning::
|
|
||||||
At present, the Ansiballz Framework cannot determine whether an import
|
|
||||||
should be included if it is a relative import. Always use an absolute
|
|
||||||
import that has :py:mod:`ansible.module_utils` in it to allow Ansiballz to
|
|
||||||
determine that the file should be included.
|
|
||||||
|
|
||||||
.. _flow_passing_module_args:
|
|
||||||
|
|
||||||
Passing args
|
|
||||||
~~~~~~~~~~~~
|
|
||||||
|
|
||||||
In :ref:`module_replacer`, module arguments are turned into a JSON-ified
|
|
||||||
string and substituted into the combined module file. In :ref:`Ansiballz`,
|
|
||||||
the JSON-ified string is passed into the module via stdin. When
|
|
||||||
a :class:`ansible.module_utils.basic.AnsibleModule` is instantiated,
|
|
||||||
it parses this string and places the args into
|
|
||||||
:attr:`AnsibleModule.params` where it can be accessed by the module's
|
|
||||||
other code.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Internally, the :class:`AnsibleModule` uses the helper function,
|
|
||||||
:py:func:`ansible.module_utils.basic._load_params`, to load the parameters
|
|
||||||
from stdin and save them into an internal global variable. Very dynamic
|
|
||||||
custom modules which need to parse the parameters prior to instantiating
|
|
||||||
an ``AnsibleModule`` may use ``_load_params`` to retrieve the
|
|
||||||
parameters. Be aware that ``_load_params`` is an internal function and
|
|
||||||
may change in breaking ways if necessary to support changes in the code.
|
|
||||||
However, we'll do our best not to break it gratuitously, which is not
|
|
||||||
something that can be said for either the way parameters are passed or
|
|
||||||
the internal global variable.
|
|
||||||
|
|
||||||
.. _flow_internal_arguments:
|
|
||||||
|
|
||||||
Internal arguments
|
|
||||||
^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Both :ref:`module_replacer` and :ref:`Ansiballz` send additional arguments to
|
|
||||||
the module beyond those which the user specified in the playbook. These
|
|
||||||
additional arguments are internal parameters that help implement global
|
|
||||||
Ansible features. Modules often do not need to know about these explicitly as
|
|
||||||
the features are implemented in :py:mod:`ansible.module_utils.basic` but certain
|
|
||||||
features need support from the module so it's good to know about them.
|
|
||||||
|
|
||||||
_ansible_no_log
|
|
||||||
~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This is a boolean. If it's True then the playbook specified ``no_log`` (in
|
|
||||||
a task's parameters or as a play parameter). This automatically affects calls
|
|
||||||
to :py:meth:`AnsibleModule.log`. If a module implements its own logging then
|
|
||||||
it needs to check this value. The best way to look at this is for the module
|
|
||||||
to instantiate an :class:`AnsibleModule` and then check the value of
|
|
||||||
:attr:`AnsibleModule.no_log`.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
``no_log`` specified in a module's argument_spec are handled by a different mechanism.
|
|
||||||
|
|
||||||
_ansible_debug
|
|
||||||
~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This is a boolean that turns on more verbose logging. If a module uses
|
|
||||||
:py:meth:`AnsibleModule.debug` rather than :py:meth:`AnsibleModule.log` then
|
|
||||||
the messages are only logged if this is True. This also turns on logging of
|
|
||||||
external commands that the module executes. This can be changed via
|
|
||||||
the``debug`` setting in :file:`ansible.cfg` or the environment variable
|
|
||||||
:envvar:`ANSIBLE_DEBUG`. If, for some reason, a module must access this, it
|
|
||||||
should do so by instantiating an :class:`AnsibleModule` and accessing
|
|
||||||
:attr:`AnsibleModule._debug`.
|
|
||||||
|
|
||||||
_ansible_diff
|
|
||||||
~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This boolean is turned on via the ``--diff`` command line option. If a module
|
|
||||||
supports it, it will tell the module to show a unified diff of changes to be
|
|
||||||
made to templated files. The proper way for a module to access this is by
|
|
||||||
instantiating an :class:`AnsibleModule` and accessing
|
|
||||||
:attr:`AnsibleModule._diff`.
|
|
||||||
|
|
||||||
_ansible_verbosity
|
|
||||||
~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This value could be used for finer grained control over logging. However, it
|
|
||||||
is currently unused.
|
|
||||||
|
|
||||||
_ansible_selinux_special_fs
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This is a list of names of filesystems which should have a special selinux
|
|
||||||
context. They are used by the :class:`AnsibleModule` methods which operate on
|
|
||||||
files (changing attributes, moving, and copying). The list of names is set
|
|
||||||
via a comma separated string of filesystem names from :file:`ansible.cfg`::
|
|
||||||
|
|
||||||
# ansible.cfg
|
|
||||||
[selinux]
|
|
||||||
special_context_filesystems=nfs,vboxsf,fuse,ramfs
|
|
||||||
|
|
||||||
If a module cannot use the builtin ``AnsibleModule`` methods to manipulate
|
|
||||||
files and needs to know about these special context filesystems, it should
|
|
||||||
instantiate an ``AnsibleModule`` and then examine the list in
|
|
||||||
:attr:`AnsibleModule._selinux_special_fs`.
|
|
||||||
|
|
||||||
This replaces :attr:`ansible.module_utils.basic.SELINUX_SPECIAL_FS` from
|
|
||||||
:ref:`module_replacer`. In module replacer it was a comma separated string of
|
|
||||||
filesystem names. Under Ansiballz it's an actual list.
|
|
||||||
|
|
||||||
.. versionadded:: 2.1
|
|
||||||
|
|
||||||
_ansible_syslog_facility
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This parameter controls which syslog facility ansible module logs to. It may
|
|
||||||
be set by changing the ``syslog_facility`` value in :file:`ansible.cfg`. Most
|
|
||||||
modules should just use :meth:`AnsibleModule.log` which will then make use of
|
|
||||||
this. If a module has to use this on its own, it should instantiate an
|
|
||||||
:class:`AnsibleModule` and then retrieve the name of the syslog facility from
|
|
||||||
:attr:`AnsibleModule._syslog_facility`. The code will look slightly different
|
|
||||||
than it did under :ref:`module_replacer` due to how hacky the old way was::
|
|
||||||
|
|
||||||
# Old way
|
|
||||||
import syslog
|
|
||||||
syslog.openlog(NAME, 0, syslog.LOG_USER)
|
|
||||||
|
|
||||||
# New way
|
|
||||||
import syslog
|
|
||||||
facility_name = module._syslog_facility
|
|
||||||
facility = getattr(syslog, facility_name, syslog.LOG_USER)
|
|
||||||
syslog.openlog(NAME, 0, facility)
|
|
||||||
|
|
||||||
.. versionadded:: 2.1
|
|
||||||
|
|
||||||
_ansible_version
|
|
||||||
~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
This parameter passes the version of ansible that runs the module. To access
|
|
||||||
it, a module should instantiate an :class:`AnsibleModule` and then retrieve it
|
|
||||||
from :attr:`AnsibleModule.ansible_version`. This replaces
|
|
||||||
:attr:`ansible.module_utils.basic.ANSIBLE_VERSION` from
|
|
||||||
:ref:`module_replacer`.
|
|
||||||
|
|
||||||
.. versionadded:: 2.1
|
|
||||||
|
|
||||||
.. _flow_special_considerations:
|
|
||||||
|
|
||||||
Special Considerations
|
|
||||||
----------------------
|
|
||||||
|
|
||||||
.. _flow_pipelining:
|
|
||||||
|
|
||||||
Pipelining
|
|
||||||
^^^^^^^^^^
|
|
||||||
|
|
||||||
Ansible can transfer a module to a remote machine in one of two ways:
|
|
||||||
|
|
||||||
* it can write out the module to a temporary file on the remote host and then
|
|
||||||
use a second connection to the remote host to execute it with the
|
|
||||||
interpreter that the module needs
|
|
||||||
* or it can use what's known as pipelining to execute the module by piping it
|
|
||||||
into the remote interpreter's stdin.
|
|
||||||
|
|
||||||
Pipelining only works with modules written in Python at this time because
|
|
||||||
Ansible only knows that Python supports this mode of operation. Supporting
|
|
||||||
pipelining means that whatever format the module payload takes before being
|
|
||||||
sent over the wire must be executable by Python via stdin.
|
|
||||||
|
|
||||||
.. _flow_args_over_stdin:
|
|
||||||
|
|
||||||
Why pass args over stdin?
|
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
||||||
|
|
||||||
Passing arguments via stdin was chosen for the following reasons:
|
|
||||||
|
|
||||||
* When combined with :ref:`pipelining`, this keeps the module's arguments from
|
|
||||||
temporarily being saved onto disk on the remote machine. This makes it
|
|
||||||
harder (but not impossible) for a malicious user on the remote machine to
|
|
||||||
steal any sensitive information that may be present in the arguments.
|
|
||||||
* Command line arguments would be insecure as most systems allow unprivileged
|
|
||||||
users to read the full commandline of a process.
|
|
||||||
* Environment variables are usually more secure than the commandline but some
|
|
||||||
systems limit the total size of the environment. This could lead to
|
|
||||||
truncation of the parameters if we hit that limit.
|
|
||||||
|
|
||||||
@ -1,48 +0,0 @@
|
|||||||
Releases
|
|
||||||
========
|
|
||||||
|
|
||||||
.. contents:: Topics
|
|
||||||
:local:
|
|
||||||
|
|
||||||
.. _schedule:
|
|
||||||
|
|
||||||
Release Schedule
|
|
||||||
````````````````
|
|
||||||
Ansible is on a 'flexible' 4 month release schedule, sometimes this can be extended if there is a major change that requires a longer cycle (i.e. 2.0 core rewrite).
|
|
||||||
Currently modules get released at the same time as the main Ansible repo, even though they are separated into ansible-modules-core and ansible-modules-extras.
|
|
||||||
|
|
||||||
The major features and bugs fixed in a release should be reflected in the CHANGELOG.md, minor ones will be in the commit history (FIXME: add git exmaple to list).
|
|
||||||
When a fix/feature gets added to the `devel` branch it will be part of the next release, some bugfixes can be backported to previous releases and might be part of a minor point release if it is deemed necessary.
|
|
||||||
|
|
||||||
Sometimes an RC can be extended by a few days if a bugfix makes a change that can have far reaching consequences, so users have enough time to find any new issues that may stem from this.
|
|
||||||
|
|
||||||
.. _methods:
|
|
||||||
|
|
||||||
Release methods
|
|
||||||
````````````````
|
|
||||||
|
|
||||||
Ansible normally goes through a 'release candidate', issuing an RC1 for a release, if no major bugs are discovered in it after 5 business days we'll get a final release.
|
|
||||||
Otherwise fixes will be applied and an RC2 will be provided for testing and if no bugs after 2 days, the final release will be made, iterating this last step and incrementing the candidate number as we find major bugs.
|
|
||||||
|
|
||||||
|
|
||||||
.. _freezing:
|
|
||||||
|
|
||||||
Release feature freeze
|
|
||||||
``````````````````````
|
|
||||||
|
|
||||||
During the release candidate process, the focus will be on bugfixes that affect the RC, new features will be delayed while we try to produce a final version. Some bugfixes that are minor or don't affect the RC will also be postponed until after the release is finalized.
|
|
||||||
|
|
||||||
.. seealso::
|
|
||||||
|
|
||||||
:doc:`developing_api`
|
|
||||||
Python API to Playbooks and Ad Hoc Task Execution
|
|
||||||
:doc:`developing_modules`
|
|
||||||
How to develop modules
|
|
||||||
:doc:`developing_plugins`
|
|
||||||
How to develop plugins
|
|
||||||
`Ansible Tower <http://ansible.com/ansible-tower>`_
|
|
||||||
REST API endpoint and GUI for Ansible, syncs with dynamic inventory
|
|
||||||
`Development Mailing List <http://groups.google.com/group/ansible-devel>`_
|
|
||||||
Mailing list for development topics
|
|
||||||
`irc.freenode.net <http://irc.freenode.net>`_
|
|
||||||
#ansible IRC chat channel
|
|
||||||
@ -1,193 +0,0 @@
|
|||||||
Helping Testing PRs
|
|
||||||
```````````````````
|
|
||||||
|
|
||||||
If you're a developer, one of the most valuable things you can do is look at the github issues list and help fix bugs. We almost always prioritize bug fixing over
|
|
||||||
feature development, so clearing bugs out of the way is one of the best things you can do.
|
|
||||||
|
|
||||||
Even if you're not a developer, helping test pull requests for bug fixes and features is still immensely valuable.
|
|
||||||
|
|
||||||
This goes for testing new features as well as testing bugfixes.
|
|
||||||
|
|
||||||
In many cases, code should add tests that prove it works but that's not ALWAYS possible and tests are not always comprehensive, especially when a user doesn't have access
|
|
||||||
to a wide variety of platforms, or that is using an API or web service.
|
|
||||||
|
|
||||||
In these cases, live testing against real equipment can be more valuable than automation that runs against simulated interfaces.
|
|
||||||
In any case, things should always be tested manually the first time too.
|
|
||||||
|
|
||||||
Thankfully helping test ansible is pretty straightforward, assuming you are already used to how ansible works.
|
|
||||||
|
|
||||||
Get Started with A Source Checkout
|
|
||||||
++++++++++++++++++++++++++++++++++
|
|
||||||
|
|
||||||
You can do this by checking out ansible, making a test branch off the main one, merging a GitHub issue, testing,
|
|
||||||
and then commenting on that particular issue on GitHub. Here's how:
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Testing source code from GitHub pull requests sent to us does have some inherent risk, as the source code
|
|
||||||
sent may have mistakes or malicious code that could have a negative impact on your system. We recommend
|
|
||||||
doing all testing on a virtual machine, whether a cloud instance, or locally. Some users like Vagrant
|
|
||||||
or Docker for this, but they are optional. It is also useful to have virtual machines of different Linux or
|
|
||||||
other flavors, since some features (apt vs. yum, for example) are specific to those OS versions.
|
|
||||||
|
|
||||||
First, you will need to configure your testing environment with the necessary tools required to run our test
|
|
||||||
suites. You will need at least::
|
|
||||||
|
|
||||||
git
|
|
||||||
python-nosetests (sometimes named python-nose)
|
|
||||||
python-passlib
|
|
||||||
python-mock
|
|
||||||
|
|
||||||
If you want to run the full integration test suite you'll also need the following packages installed::
|
|
||||||
|
|
||||||
svn
|
|
||||||
hg
|
|
||||||
python-pip
|
|
||||||
gem
|
|
||||||
|
|
||||||
Second, if you haven't already, clone the Ansible source code from GitHub::
|
|
||||||
|
|
||||||
git clone https://github.com/ansible/ansible.git --recursive
|
|
||||||
cd ansible/
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
If you have previously forked the repository on GitHub, you could also clone it from there.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
If updating your repo for testing something module related, use "git rebase origin/devel" and then "git submodule update" to fetch
|
|
||||||
the latest development versions of modules. Skipping the "git submodule update" step will result in versions that will be stale.
|
|
||||||
|
|
||||||
Activating The Source Checkout
|
|
||||||
++++++++++++++++++++++++++++++
|
|
||||||
|
|
||||||
The Ansible source includes a script that allows you to use Ansible directly from source without requiring a
|
|
||||||
full installation, that is frequently used by developers on Ansible.
|
|
||||||
|
|
||||||
Simply source it (to use the Linux/Unix terminology) to begin using it immediately::
|
|
||||||
|
|
||||||
source ./hacking/env-setup
|
|
||||||
|
|
||||||
This script modifies the PYTHONPATH enviromnent variables (along with a few other things), which will be temporarily
|
|
||||||
set as long as your shell session is open.
|
|
||||||
|
|
||||||
If you'd like your testing environment to always use the latest source, you could call the command from startup scripts (for example,
|
|
||||||
`.bash_profile`).
|
|
||||||
|
|
||||||
Finding A Pull Request and Checking It Out On A Branch
|
|
||||||
++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
|
||||||
|
|
||||||
Next, find the pull request you'd like to test and make note of the line at the top which describes the source
|
|
||||||
and destination repositories. It will look something like this::
|
|
||||||
|
|
||||||
Someuser wants to merge 1 commit into ansible:devel from someuser:feature_branch_name
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
It is important that the PR request target be ansible:devel, as we do not accept pull requests into any other branch. Dot releases are cherry-picked manually by ansible staff.
|
|
||||||
|
|
||||||
The username and branch at the end are the important parts, which will be turned into git commands as follows::
|
|
||||||
|
|
||||||
git checkout -b testing_PRXXXX devel
|
|
||||||
git pull https://github.com/someuser/ansible.git feature_branch_name
|
|
||||||
|
|
||||||
The first command creates and switches to a new branch named testing_PRXXXX, where the XXXX is the actual issue number associated with the pull request (for example, 1234). This branch is based on the devel branch. The second command pulls the new code from the users feature branch into the newly created branch.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
If the GitHub user interface shows that the pull request will not merge cleanly, we do not recommend proceeding if you are not somewhat familiar with git and coding, as you will have to resolve a merge conflict. This is the responsibility of the original pull request contributor.
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Some users do not create feature branches, which can cause problems when they have multiple, un-related commits in their version of `devel`. If the source looks like `someuser:devel`, make sure there is only one commit listed on the pull request.
|
|
||||||
|
|
||||||
Finding a Pull Request for Ansible Modules
|
|
||||||
++++++++++++++++++++++++++++++++++++++++++
|
|
||||||
Ansible modules are in separate repositories, which are managed as Git submodules. Here's a step by step process for checking out a PR for an Ansible extras module, for instance:
|
|
||||||
|
|
||||||
1. git clone https://github.com/ansible/ansible.git
|
|
||||||
2. cd ansible
|
|
||||||
3. git submodule init
|
|
||||||
4. git submodule update --recursive [ fetches the submodules ]
|
|
||||||
5. cd lib/ansible/modules/extras
|
|
||||||
6. git fetch origin pull/1234/head:pr/1234 [ fetches the specific PR ]
|
|
||||||
7. git checkout pr/1234 [ do your testing here ]
|
|
||||||
8. cd /path/to/ansible/clone
|
|
||||||
9. git submodule update --recursive
|
|
||||||
|
|
||||||
For Those About To Test, We Salute You
|
|
||||||
++++++++++++++++++++++++++++++++++++++
|
|
||||||
|
|
||||||
At this point, you should be ready to begin testing!
|
|
||||||
|
|
||||||
If the PR is a bug-fix pull request, the first things to do are to run the suite of unit and integration tests, to ensure
|
|
||||||
the pull request does not break current functionality::
|
|
||||||
|
|
||||||
# Unit Tests
|
|
||||||
make tests
|
|
||||||
|
|
||||||
# Integration Tests
|
|
||||||
cd test/integration
|
|
||||||
make
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Ansible does provide integration tests for cloud-based modules as well, however we do not recommend using them for some users
|
|
||||||
due to the associated costs from the cloud providers. As such, typically it's better to run specific parts of the integration battery
|
|
||||||
and skip these tests.
|
|
||||||
|
|
||||||
Integration tests aren't the end all beat all - in many cases what is fixed might not *HAVE* a test, so determining if it works means
|
|
||||||
checking the functionality of the system and making sure it does what it said it would do.
|
|
||||||
|
|
||||||
Pull requests for bug-fixes should reference the bug issue number they are fixing.
|
|
||||||
|
|
||||||
We encourage users to provide playbook examples for bugs that show how to reproduce the error, and these playbooks should be used to verify the bugfix does resolve
|
|
||||||
the issue if available. You may wish to also do your own review to poke the corners of the change.
|
|
||||||
|
|
||||||
Since some reproducers can be quite involved, you might wish to create a testing directory with the issue # as a sub-
|
|
||||||
directory to keep things organized::
|
|
||||||
|
|
||||||
mkdir -p testing/XXXX # where XXXX is again the issue # for the original issue or PR
|
|
||||||
cd testing/XXXX
|
|
||||||
<create files or git clone example playbook repo>
|
|
||||||
|
|
||||||
While it should go without saying, be sure to read any playbooks before you run them. VMs help with running untrusted content greatly,
|
|
||||||
though a playbook could still do something to your computing resources that you'd rather not like.
|
|
||||||
|
|
||||||
Once the files are in place, you can run the provided playbook (if there is one) to test the functionality::
|
|
||||||
|
|
||||||
ansible-playbook -vvv playbook_name.yml
|
|
||||||
|
|
||||||
If there's no playbook, you may have to copy and paste playbook snippets or run an ad-hoc command that was pasted in.
|
|
||||||
|
|
||||||
Our issue template also included sections for "Expected Output" and "Actual Output", which should be used to gauge the output
|
|
||||||
from the provided examples.
|
|
||||||
|
|
||||||
If the pull request resolves the issue, please leave a comment on the pull request, showing the following information:
|
|
||||||
|
|
||||||
* "Works for me!"
|
|
||||||
* The output from `ansible --version`.
|
|
||||||
|
|
||||||
In some cases, you may wish to share playbook output from the test run as well.
|
|
||||||
|
|
||||||
Example!::
|
|
||||||
|
|
||||||
Works for me! Tested on `Ansible 1.7.1`. I verified this on CentOS 6.5 and also Ubuntu 14.04.
|
|
||||||
|
|
||||||
If the PR does not resolve the issue, or if you see any failures from the unit/integration tests, just include that output instead::
|
|
||||||
|
|
||||||
This doesn't work for me.
|
|
||||||
|
|
||||||
When I ran this my toaster started making loud noises!
|
|
||||||
|
|
||||||
Output from the toaster looked like this:
|
|
||||||
|
|
||||||
```
|
|
||||||
BLARG
|
|
||||||
StrackTrace
|
|
||||||
RRRARRGGG
|
|
||||||
```
|
|
||||||
|
|
||||||
When you are done testing a feature branch, you can remove it with the following command::
|
|
||||||
|
|
||||||
git branch -D someuser-feature_branch_name
|
|
||||||
|
|
||||||
We understand some users may be inexperienced with git, or other aspects of the above procedure, so feel free to stop by ansible-devel
|
|
||||||
list for questions and we'd be happy to help answer them.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Loading…
Reference in New Issue