diff --git a/docsite/rst/dev_guide/developing_plugins.rst b/docsite/rst/dev_guide/developing_plugins.rst index 696ba0af7b9..12baaf50e19 100644 --- a/docsite/rst/dev_guide/developing_plugins.rst +++ b/docsite/rst/dev_guide/developing_plugins.rst @@ -53,7 +53,17 @@ Plugins are loaded in alphanumeric order; for example, a plugin implemented in a Callbacks need to be whitelisted in your `ansible.cfg` file in order to function. For example:: - #callback_whitelist = timer, mail, myplugin + #callback_whitelist = timer, mail, mycallbackplugin + + +Writing to stdout +````````````````` + +If your callback plugin needs to write to stdout, you should define CALLBACK_TYPE = stdout in the subclass, and then the stdout plugin needs to be configured in `ansible.cfg` to override the default. For example:: + + #stdout_callback = mycallbackplugin + + .. _callback_development: @@ -109,12 +119,7 @@ The following example shows how Ansible's timer plugin is implemented:: runtime = end_time - self.start_time self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds" % (self.days_hours_minutes_seconds(runtime))) -Note that the CALLBACK_VERSION and CALLBACK_NAME definitons are required. If your callback plugin needs to write to stdout, you should define CALLBACK_TYPE = stdout in the subclass, and then the stdout plugin needs to be configured in `ansible.cfg` to override the default. For example:: - - #stdout_callback = mycallbackplugin - - - +Note that the CALLBACK_VERSION and CALLBACK_NAME definitons are required. .. _developing_connection_type_plugins: @@ -135,10 +140,62 @@ More documentation on writing connection plugins is pending, though you can jump Lookup Plugins -------------- -Language constructs like "with_fileglob" and "with_items" are implemented via lookup plugins. Just like other plugin types, you can write your own. +Lookup plugins are used to pull in data from external data stores. Lookup plugins can be used within playbooks for both looping - playbook language constructs like "with_fileglob" and "with_items" are implemented via lookup plugins - and to return values into a variable or parameter. -More documentation on writing lookup plugins is pending, though you can jump into `lib/ansible/plugins/lookup `_ and figure -things out pretty easily. +Here's a simple lookup plugin implementation - this lookup returns the contents of a text file as a variable:: + + from ansible.errors import AnsibleError, AnsibleParserError + from ansible.plugins.lookup import LookupBase + + try: + from __main__ import display + except ImportError: + from ansible.utils.display import Display + display = Display() + + + class LookupModule(LookupBase): + + def run(self, terms, variables=None, **kwargs): + + ret = [] + + for term in terms: + display.debug("File lookup term: %s" % term) + + # Find the file in the expected search path + lookupfile = self.find_file_in_search_path(variables, 'files', term) + display.vvvv(u"File lookup using %s as file" % lookupfile) + try: + if lookupfile: + contents, show_data = self._loader._get_file_contents(lookupfile) + ret.append(contents.rstrip()) + else: + raise AnsibleParserError() + except AnsibleParserError: + raise AnsibleError("could not locate file in lookup: %s" % term) + + return ret + +An example of how this lookup is called:: + + --- + - hosts: all + vars: + contents: "{{ lookup('file', '/etc/foo.txt') }}" + + tasks: + + - debug: msg="the value of foo.txt is {{ contents }}" + +Errors encountered during execution should be returned by raising AnsibleError() with a message describing the error. Any strings returned by your lookup plugin implementation that could ever contain non-ASCII characters must be converted into Python's unicode type becasue the strings will be run through jinja2. To do this, you can use:: + + from ansible.module_utils._text import to_text + result_string = to_text(result_string) + +For more example lookup plugins, check out the source code for the lookup plugins that are included with Ansible here: `lib/ansible/plugins/lookup `_. + +For usage examples of lookup plugins, see `Using Lookups `_. .. _developing_vars_plugins: diff --git a/docsite/rst/dev_guide/overview_architecture.rst b/docsite/rst/dev_guide/overview_architecture.rst index a08eec0dd79..97899405344 100644 --- a/docsite/rst/dev_guide/overview_architecture.rst +++ b/docsite/rst/dev_guide/overview_architecture.rst @@ -15,6 +15,11 @@ Ansible works by connecting to your nodes and pushing out small programs, called Your library of modules can reside on any machine, and there are no servers, daemons, or databases required. Typically you'll work with your favorite terminal program, a text editor, and probably a version control system to keep track of changes to your content. +Plugins +------- + +Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own. + Inventory ```````````````````` diff --git a/docsite/rst/developing.rst b/docsite/rst/developing.rst deleted file mode 100644 index 845eedeabb9..00000000000 --- a/docsite/rst/developing.rst +++ /dev/null @@ -1,19 +0,0 @@ -Developer Information -````````````````````` - -Learn how to build modules of your own in any language, and also how to extend Ansible through several kinds of plugins. Explore Ansible's Python API and write Python plugins to integrate with other solutions in your environment. - -.. toctree:: - :maxdepth: 2 - - developing_api - developing_inventory - developing_modules - developing_module_utilities - developing_plugins - developing_core - developing_test_pr - developing_releases - -Developers will also likely be interested in the fully-discoverable in :doc:`tower`. It's great for embedding Ansible in all manner of applications. - diff --git a/docsite/rst/developing_api.rst b/docsite/rst/developing_api.rst deleted file mode 100644 index 21fe082ff24..00000000000 --- a/docsite/rst/developing_api.rst +++ /dev/null @@ -1,194 +0,0 @@ -Python API -========== - -.. contents:: Topics - -Please note that while we make this API available it is not intended for direct consumption, it is here -for the support of the Ansible command line tools. We try not to make breaking changes but we reserve the -right to do so at any time if it makes sense for the Ansible toolset. - - -The following documentation is provided for those that still want to use the API directly, but be mindful this is not something the Ansible team supports. - -There are several interesting ways to use Ansible from an API perspective. You can use -the Ansible python API to control nodes, you can extend Ansible to respond to various python events, you can -write various plugins, and you can plug in inventory data from external data sources. This document -covers the execution and Playbook API at a basic level. - -If you are looking to use Ansible programmatically from something other than Python, trigger events asynchronously, -or have access control and logging demands, take a look at :doc:`tower` -as it has a very nice REST API that provides all of these things at a higher level. - -Ansible is written in its own API so you have a considerable amount of power across the board. -This chapter discusses the Python API. - -.. _python_api: - -The Python API is very powerful, and is how the all the ansible CLI tools are implemented. -In version 2.0 the core ansible got rewritten and the API was mostly rewritten. - -.. note:: Ansible relies on forking processes, as such the API is not thread safe. - -.. _python_api_20: - -Python API 2.0 --------------- - -In 2.0 things get a bit more complicated to start, but you end up with much more discrete and readable classes:: - - - #!/usr/bin/env python - - import json - from collections import namedtuple - from ansible.parsing.dataloader import DataLoader - from ansible.vars import VariableManager - from ansible.inventory import Inventory - from ansible.playbook.play import Play - from ansible.executor.task_queue_manager import TaskQueueManager - from ansible.plugins.callback import CallbackBase - - class ResultCallback(CallbackBase): - """A sample callback plugin used for performing an action as results come in - - If you want to collect all results into a single object for processing at - the end of the execution, look into utilizing the ``json`` callback plugin - or writing your own custom callback plugin - """ - def v2_runner_on_ok(self, result, **kwargs): - """Print a json representation of the result - - This method could store the result in an instance attribute for retrieval later - """ - host = result._host - print json.dumps({host.name: result._result}, indent=4) - - Options = namedtuple('Options', ['connection', 'module_path', 'forks', 'become', 'become_method', 'become_user', 'check']) - # initialize needed objects - variable_manager = VariableManager() - loader = DataLoader() - options = Options(connection='local', module_path='/path/to/mymodules', forks=100, become=None, become_method=None, become_user=None, check=False) - passwords = dict(vault_pass='secret') - - # Instantiate our ResultCallback for handling results as they come in - results_callback = ResultCallback() - - # create inventory and pass to var manager - inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list='localhost') - variable_manager.set_inventory(inventory) - - # create play with tasks - play_source = dict( - name = "Ansible Play", - hosts = 'localhost', - gather_facts = 'no', - tasks = [ - dict(action=dict(module='shell', args='ls'), register='shell_out'), - dict(action=dict(module='debug', args=dict(msg='{{shell_out.stdout}}'))) - ] - ) - play = Play().load(play_source, variable_manager=variable_manager, loader=loader) - - # actually run it - tqm = None - try: - tqm = TaskQueueManager( - inventory=inventory, - variable_manager=variable_manager, - loader=loader, - options=options, - passwords=passwords, - stdout_callback=results_callback, # Use our custom callback instead of the ``default`` callback plugin - ) - result = tqm.run(play) - finally: - if tqm is not None: - tqm.cleanup() - - -.. _python_api_old: - -Python API pre 2.0 ------------------- - -It's pretty simple:: - - import ansible.runner - - runner = ansible.runner.Runner( - module_name='ping', - module_args='', - pattern='web*', - forks=10 - ) - datastructure = runner.run() - -The run method returns results per host, grouped by whether they -could be contacted or not. Return types are module specific, as -expressed in the :doc:`modules` documentation.:: - - { - "dark" : { - "web1.example.com" : "failure message" - }, - "contacted" : { - "web2.example.com" : 1 - } - } - -A module can return any type of JSON data it wants, so Ansible can -be used as a framework to rapidly build powerful applications and scripts. - -.. _detailed_api_old_example: - -Detailed API Example -```````````````````` - -The following script prints out the uptime information for all hosts:: - - #!/usr/bin/python - - import ansible.runner - import sys - - # construct the ansible runner and execute on all hosts - results = ansible.runner.Runner( - pattern='*', forks=10, - module_name='command', module_args='/usr/bin/uptime', - ).run() - - if results is None: - print "No hosts found" - sys.exit(1) - - print "UP ***********" - for (hostname, result) in results['contacted'].items(): - if not 'failed' in result: - print "%s >>> %s" % (hostname, result['stdout']) - - print "FAILED *******" - for (hostname, result) in results['contacted'].items(): - if 'failed' in result: - print "%s >>> %s" % (hostname, result['msg']) - - print "DOWN *********" - for (hostname, result) in results['dark'].items(): - print "%s >>> %s" % (hostname, result) - -Advanced programmers may also wish to read the source to ansible itself, -for it uses the API (with all available options) to implement the ``ansible`` -command line tools (``lib/ansible/cli/``). - -.. seealso:: - - :doc:`developing_inventory` - Developing dynamic inventory integrations - :doc:`developing_modules` - How to develop modules - :doc:`developing_plugins` - How to develop plugins - `Development Mailing List `_ - Mailing list for development topics - `irc.freenode.net `_ - #ansible IRC chat channel - diff --git a/docsite/rst/developing_core.rst b/docsite/rst/developing_core.rst deleted file mode 100644 index c0067d74f86..00000000000 --- a/docsite/rst/developing_core.rst +++ /dev/null @@ -1,24 +0,0 @@ -Developing the Ansible Core Engine -================================== - -Although many of the pieces of the Ansible Core Engine are plugins that can be -swapped out via playbook directives or configuration, there are still pieces -of the Engine that are not modular. The documents here give insight into how -those pieces work together. - -.. toctree:: - :maxdepth: 1 - - developing_program_flow_modules - -.. seealso:: - - :doc:`developing_api` - Learn about the Python API for task execution - :doc:`developing_plugins` - Learn about developing plugins - `Mailing List `_ - The development mailing list - `irc.freenode.net `_ - #ansible-devel IRC chat channel - diff --git a/docsite/rst/developing_inventory.rst b/docsite/rst/developing_inventory.rst deleted file mode 100644 index 797f436edf4..00000000000 --- a/docsite/rst/developing_inventory.rst +++ /dev/null @@ -1,99 +0,0 @@ -Developing Dynamic Inventory Sources -==================================== - -.. contents:: Topics - :local: - -As described in :doc:`intro_dynamic_inventory`, ansible can pull inventory information from dynamic sources, including cloud sources. - -How do we write a new one? - -Simple! We just create a script or program that can print JSON in the right format when fed the proper arguments. -You can do this in any language. - -.. _inventory_script_conventions: - -Script Conventions -`````````````````` - -When the external node script is called with the single argument ``--list``, the script must output a JSON encoded hash/dictionary of all the groups to be managed to stdout. Each group's value should be either a hash/dictionary containing a list of each host/IP, potential child groups, and potential group variables, or simply a list of host/IP addresses, like so:: - - { - "databases" : { - "hosts" : [ "host1.example.com", "host2.example.com" ], - "vars" : { - "a" : true - } - }, - "webservers" : [ "host2.example.com", "host3.example.com" ], - "atlanta" : { - "hosts" : [ "host1.example.com", "host4.example.com", "host5.example.com" ], - "vars" : { - "b" : false - }, - "children": [ "marietta", "5points" ] - }, - "marietta" : [ "host6.example.com" ], - "5points" : [ "host7.example.com" ] - } - -.. versionadded:: 1.0 - -Before version 1.0, each group could only have a list of hostnames/IP addresses, like the webservers, marietta, and 5points groups above. - -When called with the arguments ``--host `` (where is a host from above), the script must print either an empty JSON -hash/dictionary, or a hash/dictionary of variables to make available to templates and playbooks. Printing variables is optional, -if the script does not wish to do this, printing an empty hash/dictionary is the way to go:: - - { - "favcolor" : "red", - "ntpserver" : "wolf.example.com", - "monitoring" : "pack.example.com" - } - -.. _inventory_script_tuning: - -Tuning the External Inventory Script -```````````````````````````````````` - -.. versionadded:: 1.3 - -The stock inventory script system detailed above works for all versions of Ansible, but calling -``--host`` for every host can be rather expensive, especially if it involves expensive API calls to -a remote subsystem. In Ansible -1.3 or later, if the inventory script returns a top level element called "_meta", it is possible -to return all of the host variables in one inventory script call. When this meta element contains -a value for "hostvars", the inventory script will not be invoked with ``--host`` for each host. This -results in a significant performance increase for large numbers of hosts, and also makes client -side caching easier to implement for the inventory script. - -The data to be added to the top level JSON dictionary looks like this:: - - { - - # results of inventory script as above go here - # ... - - "_meta" : { - "hostvars" : { - "moocow.example.com" : { "asdf" : 1234 }, - "llama.example.com" : { "asdf" : 5678 }, - } - } - - } - -.. seealso:: - - :doc:`developing_api` - Python API to Playbooks and Ad Hoc Task Execution - :doc:`developing_modules` - How to develop modules - :doc:`developing_plugins` - How to develop plugins - `Ansible Tower `_ - REST API endpoint and GUI for Ansible, syncs with dynamic inventory - `Development Mailing List `_ - Mailing list for development topics - `irc.freenode.net `_ - #ansible IRC chat channel diff --git a/docsite/rst/developing_modules_python3.rst b/docsite/rst/developing_modules_python3.rst deleted file mode 100644 index 84d84910d62..00000000000 --- a/docsite/rst/developing_modules_python3.rst +++ /dev/null @@ -1,264 +0,0 @@ -=========================== -Porting Modules to Python 3 -=========================== - -Ansible modules are not the usual Python-3 porting exercise. There are two -factors that make it harder to port them than most code: - -1. Many modules need to run on Python-2.4 in addition to Python-3. -2. A lot of mocking has to go into unittesting a Python-3 module. So it's - harder to test that your porting has fixed everything or to make sure that - later commits haven't regressed. - -Which version of Python-3.x and which version of Python-2.x are our minimums? -============================================================================= - -The short answer is Python-3.5 and Python-2.4 but please read on for more -information. - -For Python-3 we are currently using Python-3.5 as a minimum on both the -controller and the managed nodes. This was chosen as it's the version of -Python3 in Ubuntu-16.04, the first long-term support (LTS) distribution to -ship with Python3 and not Python2. Much of our code would still work with -Python-3.4 but there are always bugfixes and new features in any new upstream -release. Taking advantage of this relatively new version allows us not to -worry about workarounds for problems and missing features in that older -version. - -For Python-2, the default is for the controller to run on Python-2.6 and -modules to run on Python-2.4. This allows users with older distributions that -are stuck on Python-2.4 to manage their machines. Modules are allowed to drop -support for Python-2.4 when one of their dependent libraries require a higher -version of python. This is not an invitation to add unnecessary dependent -libraries in order to force your module to be usable only with a newer version -of Python. Instead it is an acknowledgment that some libraries (for instance, -boto3 and docker-py) will only function with newer Python. - -.. note:: When will we drop support for Python-2.4? - - The only long term supported distro that we know of with Python-2.4 is - RHEL5 (and its rebuilds like CentOS5) which is supported until April of - 2017. Whatever major release we make in or after April of 2017 (probably - 2.4.0) will no longer have support for Python-2.4 on the managed machines. - Previous major release series's that we support (2.3.x) will continue to - support Python-2.4 on the managed nodes. - - We know of no long term supported distributions with Python-2.5 so the new - minimum Python-2 version will be Python-2.6. This will let us take - advantage of the forwards-compat features of Python-2.6 so porting and - maintainance of Python-2/Python-3 code will be easier after that. - - -Supporting only Python-2 or only Python-3 -========================================= - -Sometimes a module's dependent libraries only run on Python-2 or only run on -Python-3. We do not yet have a strategy for these modules but we'll need to -come up with one. I see three possibilities: - -1. We treat these libraries like any other libraries that may not be installed - on the system. When we import them we check if the import was successful. - If so, then we continue. If not we return an error about the library being - missing. Users will have to find out that the library is unavailable on - their version of Python either by searching for the library on their own or - reading the requirements section in :command:`ansible-doc`. - -2. The shebang line is the only metadata that Ansible extracts from a module - so we may end up using that to specify what we mean. Something like - ``#!/usr/bin/python`` means the module will run on both Python-2 and - Python-3, ``#!/usr/bin/python2`` means the module will only run on - Python-2, and ``#!/usr/bin/python3`` means the module will only run on - Python-3. Ansible's code will need to be modified to accommodate this. - For :command:`python2`, if ``ansible_python2_interpreter`` is not set, it - will have to fallback to `` ansible_python_interpreter`` and if that's not - set, fallback to ``/usr/bin/python``. For :command:`python3`, Ansible - will have to first try ``ansible_python3_interpreter`` and then fallback to - ``/usr/bin/python3`` as normal. - -3. We add a way for Ansible to retrieve metadata about modules. The metadata - will include the version of Python that is required. - -Methods 2 and 3 will both require that we modify modules or otherwise add this -additional information somewhere. 2 needs only a little code changes in -executor/module_common.py to parse. 3 will require a lot of work. This is -probably not worthwhile if this is the only change but could be worthwhile if -there's other things as well. 1 requires that we port all modules to work -with python3 syntax but only the code path to get to the library import being -attempted and then a fail_json() being called because the libraries are -unavailable needs to actually work. - -.. note:: Metadata proposal in progress - - A metadata specification is being created to address module - maintainership. In the future we will likely extend this to record that a module - works with Python2 and 3, Python2 only, or Python3 only. - -Tips, tricks, and idioms to adopt -================================= - -Exceptions ----------- - -In code which already needs Python-2.6+ (For instance, because a library it -depends on only runs on Python >= 2.6) it is okay to port directly to the new -exception catching syntax:: - - try: - a = 2/0 - except ValueError as e: - module.fail_json(msg="Tried to divide by zero!") - -For modules which also run on Python-2.4, we have to use an uglier -construction to make this work under both Python-2.4 and Python-3:: - - from ansible.module_utils.pycompat24 import get_exception - [...] - - try: - a = 2/0 - except ValueError: - e = get_exception() - module.fail_json(msg="Tried to divide by zero!") - -Octal numbers -------------- - -In Python-2.4, octal literals are specified as ``0755``. In Python-3, that is -invalid and octals must be specified as ``0o755``. To bridge this gap, -modules should create their octals like this:: - - # Can't use 0755 on Python-3 and can't use 0o755 on Python-2.4 - EXECUTABLE_PERMS = int('0755', 8) - -Outputting octal numbers may also need to be changed. In python2 we often did -this to return file permissions:: - - mode = int('0775', 8) - result['mode'] = oct(mode) - -This would give the user ``result['mode'] == '0755'`` in their playbook. In -python3, :func:`oct` returns the format with the lowercase ``o`` in it like: -``result['mode'] == '0o755'``. If a user had a conditional in their playbook -or was using the mode in a template the new format might break things. We -need to return the old form of mode for backwards compatibility. You can do -it like this:: - - mode = int('0775', 8) - result['mode'] = '0%03o' % mode - -You should use this wherever backwards compatibility is a concern or you are -dealing with file permissions. (With file permissions a user may be feeding -the mode into another program or to another module which doesn't understand -the python syntax for octal numbers. ``[zero][digit][digit][digit]`` is -understood by most everything and therefore the right way to express octals in -these circumstances. - -Bundled six ------------ - -The third-party python-six library exists to help projects create code that -runs on both Python-2 and Python-3. Ansible includes version 1.4.1 in -module_utils so that other modules can use it without requiring that it is -installed on the remote system. To make use of it, import it like this:: - - from ansible.module_utils import six - -.. note:: Why version 1.4.1? - - six-1.4.1 is the last version of python-six to support Python-2.4. As - long as Ansible modules need to run on Python-2.4 we won't be able to - update the bundled copy of six. - -Compile Test ------------- - -We have Shippable compiling all modules with various versions of Python to check -that the modules conform to the syntax at those versions. When you've -ported a module so that its syntax works with Python-3, we need to remove it from -the blacklist so that the module is included in the syntax check. - -The file `test/utils/shippable/sanity-skip-python3.txt` contains the list of -modules which should not be tested (because we know that they are older modules which -have not yet been ported to pass the Python-3 syntax checks. To get another -old module to compile with Python-3, remove the entry for it from the list. -The goal is to have the LIST be empty. - -String Model ------------- - -One of the big differences between Python2 and Python3 is the string model. -In Python2, most APIs take byte strings (the Python2 ``str`` type). Using the -text type (in Python2, this is the ``unicode`` type) often leads to tracebacks -because the strings need to be converted to bytes and Python fails to do that -correctly. In Python3, the situation is somewhat reversed. Most APIs take -text strings (this is **Python3's** ``str`` type). When you have byte strings -(the Python3 ``bytes`` type) you sometimes get errors when attempting to -combine those with text strings. Note, however, that under the hood, Python -still has to convert text to bytes to interface operating system libraries and -system calls. This means that you can still get tracebacks when passing -text to APIs which call those OS level facilities. - -For module_utils, code we've decided to make the environment work with "native -strings". This means that on Python2, things should work if you use the byte -string type. In Python3, code should work if you give it text strings. The -reason for this is so that third party modules written for Python2 don't start -issuing UnicodeError exceptions once we've ported module_utils to work under -Python3. We'll need to gather experience to see if this is going to work out -well for modules as well or if we should give the module_utils API explicit -switches so that modules can choose to operate with text type all of the time. - -Helpers -~~~~~~~ - -For converting between bytes, text, and native strings we have three helper -functions. These are :func:`ansible.module_utils._text.to_bytes`, -:func:`ansible.module_utils._text.to_native`, and -:func:`ansible.module_utils._text.to_text`. These are similar to using -``bytes.decode()`` and ``unicode.encode()`` with a few differences. - -* By default they try very hard not to traceback. -* The default encoding is "utf-8" -* There are two error strategies that don't correspond one-to-one with - a python codec error handler. These are ``surrogate_or_strict`` and - ``surrogate_or_replace``. ``surrogate_or_strict`` will use the ``surrogateescape`` - error handler if available (mostly on python3) or strict if not. It is most - appropriate to use when dealing with something that needs to round trip its - value like file paths database keys, etc. Without ``surrogateescape`` the best - thing these values can do is generate a traceback that our code can catch - and decide how to show an error message. ``surrogate_or_replace`` is for - when a value is going to be displayed to the user. If the - ``surrogateescape`` error handler is not present, it will replace - undecodable byte sequences with a replacement character. - -================================ -Porting Core Ansible to Python 3 -================================ - -The Ansible code which runs controller-side is easier to port to Python3 in -one important way: We do not have to support Python-2.4 on the controller. -We only have to support Python-2.6 and above. However, this doesn't eliminate -the work that has to be done. The controller is a much more complicated piece -of code than any individual module. Making it Python2 and Python3 compatible -is a much more complex task. - -String Model ------------- - -By and large, the controller uses the standard best practice of storing -everything internally as text type and converting to and from bytes at the -borders. In many places we hardcode these byte values as utf-8. Thus yaml -and inventory files are encoded in utf-8. Filenames are also utf-8. This may -not be the right answer forever but it is sufficient for now. If there's -demand from users to handle encodings other than utf-8 after the code works on -Python3 we can look into what strategy to take for supporting other encodings. - -In some cases, storing values as a byte string is not necessarily a choice -without drawbacks. For instance, filenames and environment variables on POSIX -systems are a sequence of bytes. By using text to represent filenames we -prevent filenames that are undecodable in utf-8 and filenames that are not -text at all from working. We made the choice to represent these as text for -now due to code paths that handle filenames not being able to handle bytes -end-to-end. PyYAML on Python3 and jinja2 on both Python2 and Python3, for -instance, are meant to work with text. Any decision to allow filenames to be -byte values will have to address how we deal with those pieces of the code as -well. diff --git a/docsite/rst/developing_plugins.rst b/docsite/rst/developing_plugins.rst deleted file mode 100644 index 2ba4b226dc2..00000000000 --- a/docsite/rst/developing_plugins.rst +++ /dev/null @@ -1,209 +0,0 @@ -Developing Plugins -================== - -.. contents:: Topics - -Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own. - -The following types of plugins are available: - -- *Callback* plugins enable you to hook into Ansible events for display or logging purposes. -- *Connection* plugins define how to communicate with inventory hosts. -- *Lookup* plugins are used to pull data from an external source. -- *Vars* plugins inject additional variable data into Ansible runs that did not come from an inventory, playbook, or the command line. - -This section describes the various types of plugins and how to implement them. - - -.. _developing_callbacks: - -Callback Plugins ----------------- - -Callback plugins enable adding new behaviors to Ansible when responding to events. - -.. _callback_examples: - -Example Callback Plugins -++++++++++++++++++++++++ - -Ansible comes with a number of callback plugins that you can look at for examples. These can be found in `lib/ansible/plugins/callback `_. - -The `log_plays -`_ -callback is an example of how to intercept playbook events to a log -file, and the `mail -`_ -callback sends email when playbooks complete. - -The `osx_say -`_ -callback provided is particularly entertaining -- it will respond with -computer synthesized speech on OS X in relation to playbook events, -and is guaranteed to entertain and/or annoy coworkers. - -.. _configuring_callbacks: - -Configuring Callback Plugins -++++++++++++++++++++++++++++ - -To activate a callback, drop it in a callback directory as configured in `ansible.cfg`. - -Plugins are loaded in alphanumeric order; for example, a plugin implemented in a file named `1_first.py` would run before a plugin file named `2_second.py`. - -Callbacks need to be whitelisted in your `ansible.cfg` file in order to function. For example:: - - #callback_whitelist = timer, mail, mycallbackplugin - - -Writing to stdout -````````````````` - -If your callback plugin needs to write to stdout, you should define CALLBACK_TYPE = stdout in the subclass, and then the stdout plugin needs to be configured in `ansible.cfg` to override the default. For example:: - - #stdout_callback = mycallbackplugin - - - -.. _callback_development: - -Developing Callback Plugins -+++++++++++++++++++++++++++ - -Callback plugins are created by creating a new class with the Base(Callbacks) class as the parent:: - - from ansible.plugins.callback import CallbackBase - from ansible import constants as C - - class CallbackModule(CallbackBase): - -From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`. For a complete list of methods that you can override, please see ``__init__.py`` in the `lib/ansible/plugins/callback `_ directory. - - -The following example shows how Ansible's timer plugin is implemented:: - - # Make coding more python3-ish - from __future__ import (absolute_import, division, print_function) - __metaclass__ = type - - from datetime import datetime - - from ansible.plugins.callback import CallbackBase - - - class CallbackModule(CallbackBase): - """ - This callback module tells you how long your plays ran for. - """ - CALLBACK_VERSION = 2.0 - CALLBACK_TYPE = 'aggregate' - CALLBACK_NAME = 'timer' - CALLBACK_NEEDS_WHITELIST = True - - def __init__(self): - - super(CallbackModule, self).__init__() - - self.start_time = datetime.now() - - def days_hours_minutes_seconds(self, runtime): - minutes = (runtime.seconds // 60) % 60 - r_seconds = runtime.seconds - (minutes * 60) - return runtime.days, runtime.seconds // 3600, minutes, r_seconds - - def playbook_on_stats(self, stats): - self.v2_playbook_on_stats(stats) - - def v2_playbook_on_stats(self, stats): - end_time = datetime.now() - runtime = end_time - self.start_time - self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds" % (self.days_hours_minutes_seconds(runtime))) - -Note that the CALLBACK_VERSION and CALLBACK_NAME definitons are required. - -.. _developing_connection_type_plugins: - -Connection Type Plugins ------------------------ - -By default, ansible ships with a 'paramiko' SSH, native ssh (just called 'ssh'), 'local' connection type, and there are also some minor players like 'chroot' and 'jail'. All of these can be used -in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines. The basics of these connection types -are covered in the :doc:`intro_getting_started` section. Should you want to extend Ansible to support other transports (SNMP? Message bus? -Carrier Pigeon?) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins -directory. The value of 'smart' for a connection allows selection of paramiko or openssh based on system capabilities, and chooses -'ssh' if OpenSSH supports ControlPersist, in Ansible 1.2.1 and later. Previous versions did not support 'smart'. - -More documentation on writing connection plugins is pending, though you can jump into `lib/ansible/plugins/connection `_ and figure things out pretty easily. - -.. _developing_lookup_plugins: - -Lookup Plugins --------------- - -Language constructs like "with_fileglob" and "with_items" are implemented via lookup plugins. Just like other plugin types, you can write your own. - -More documentation on writing lookup plugins is pending, though you can jump into `lib/ansible/plugins/lookup `_ and figure -things out pretty easily. - -.. _developing_vars_plugins: - -Vars Plugins ------------- - -Playbook constructs like 'host_vars' and 'group_vars' work via 'vars' plugins. They inject additional variable -data into ansible runs that did not come from an inventory, playbook, or command line. Note that variables -can also be returned from inventory, so in most cases, you won't need to write or understand vars_plugins. - -More documentation on writing vars plugins is pending, though you can jump into `lib/ansible/inventory/vars_plugins `_ and figure -things out pretty easily. - -If you find yourself wanting to write a vars_plugin, it's more likely you should write an inventory script instead. - -.. _developing_filter_plugins: - -Filter Plugins --------------- - -If you want more Jinja2 filters available in a Jinja2 template (filters like to_yaml and to_json are provided by default), they can be extended by writing a filter plugin. Most of the time, when someone comes up with an idea for a new filter they would like to make available in a playbook, we'll just include them in 'core.py' instead. - -Jump into `lib/ansible/plugins/filter `_ for details. - -.. _distributing_plugins: - -Distributing Plugins --------------------- - -Plugins are loaded from both Python's site_packages (those that ship with ansible) and a configured plugins directory, which defaults -to /usr/share/ansible/plugins, in a subfolder for each plugin type:: - - * action - * lookup - * callback - * connection - * filter - * strategy - * cache - * test - * shell - -To change this path, edit the ansible configuration file. - -In addition, plugins can be shipped in a subdirectory relative to a top-level playbook, in folders named the same as indicated above. - -They can also be shipped as part of a role, in a subdirectory named as indicated above. The plugin will be availiable as soon as the role -is called. - -.. seealso:: - - :doc:`modules` - List of built-in modules - :doc:`developing_api` - Learn about the Python API for task execution - :doc:`developing_inventory` - Learn about how to develop dynamic inventory sources - :doc:`developing_modules` - Learn about how to write Ansible modules - `Mailing List `_ - The development mailing list - `irc.freenode.net `_ - #ansible IRC chat channel diff --git a/docsite/rst/developing_program_flow_modules.rst b/docsite/rst/developing_program_flow_modules.rst deleted file mode 100644 index 66bb875895c..00000000000 --- a/docsite/rst/developing_program_flow_modules.rst +++ /dev/null @@ -1,523 +0,0 @@ -.. _flow_modules: - -======= -Modules -======= - -This in-depth dive helps you understand Ansible's program flow to execute -modules. It is written for people working on the portions of the Core Ansible -Engine that execute a module. Those writing Ansible Modules may also find this -in-depth dive to be of interest, but individuals simply using Ansible Modules -will not likely find this to be helpful. - -.. _flow_types_of_modules: - -Types of Modules -================ - -Ansible supports several different types of modules in its code base. Some of -these are for backwards compatibility and others are to enable flexibility. - -.. _flow_action_plugins: - -Action Plugins --------------- - -Action Plugins look like modules to end users who are writing :term:`playbooks` but -they're distinct entities for the purposes of this document. Action Plugins -always execute on the controller and are sometimes able to do all work there -(for instance, the ``debug`` Action Plugin which prints some text for the user to -see or the ``assert`` Action Plugin which can test whether several values in -a playbook satisfy certain criteria.) - -More often, Action Plugins set up some values on the controller, then invoke an -actual module on the managed node that does something with these values. An -easy to understand version of this is the :ref:`template Action Plugin -