From 7df31aaca1cc0c936f23bc458facca4d89771ba2 Mon Sep 17 00:00:00 2001 From: John R Barker Date: Sat, 7 Jan 2017 19:38:52 +0000 Subject: [PATCH] Fix all RST errors for docs/docsite/rst (#20005) * developing_modules.rst is now in dev_guide, sync changes and delete the old version * Cleaner RST & formatted code * Tidyup roadmaps * Link to repomerge * Pull in abadger's fixes From https://github.com/ansible/ansible/compare/docs-code-block-fixes?expand=1 * Clean docsite/rst (apart from ROADMAP --- docs/docsite/conf.py | 2 +- docs/docsite/rst/dev_guide/developing.rst | 18 - docs/docsite/rst/dev_guide/developing_api.rst | 4 +- .../rst/dev_guide/developing_inventory.rst | 2 +- .../rst/dev_guide/developing_modules.rst | 197 +++-- .../rst/dev_guide/developing_plugins.rst | 43 +- .../developing_program_flow_modules.rst | 10 +- .../rst/dev_guide/developing_test_pr.rst | 47 +- docs/docsite/rst/dev_guide/index.rst | 9 +- docs/docsite/rst/dev_guide/repomerge.rst | 2 + docs/docsite/rst/developing_modules.rst | 827 ------------------ docs/docsite/rst/faq.rst | 61 +- docs/docsite/rst/guide_cloudstack.rst | 4 +- docs/docsite/rst/guide_rolling_upgrade.rst | 8 +- docs/docsite/rst/intro_bsd.rst | 2 +- docs/docsite/rst/intro_configuration.rst | 14 +- docs/docsite/rst/intro_patterns.rst | 4 +- docs/docsite/rst/intro_windows.rst | 4 +- docs/docsite/rst/modules.rst | 4 +- docs/docsite/rst/modules_extra.rst | 2 +- docs/docsite/rst/modules_intro.rst | 4 +- docs/docsite/rst/playbooks.rst | 1 + .../docsite/rst/playbooks_advanced_syntax.rst | 4 +- docs/docsite/rst/playbooks_best_practices.rst | 2 +- docs/docsite/rst/playbooks_conditionals.rst | 2 +- docs/docsite/rst/playbooks_debugger.rst | 6 +- docs/docsite/rst/playbooks_filters.rst | 53 +- docs/docsite/rst/playbooks_filters_ipaddr.rst | 8 +- docs/docsite/rst/playbooks_intro.rst | 2 +- docs/docsite/rst/playbooks_lookups.rst | 32 +- docs/docsite/rst/playbooks_loops.rst | 10 +- docs/docsite/rst/playbooks_roles.rst | 4 +- docs/docsite/rst/playbooks_variables.rst | 79 +- docs/docsite/rst/porting_guide_2.0.rst | 11 +- docs/docsite/rst/roadmap/ROADMAP_2_1.rst | 10 +- docs/docsite/rst/roadmap/ROADMAP_2_2.rst | 52 +- docs/docsite/rst/roadmap/ROADMAP_2_3.rst | 6 +- docs/docsite/rst/roadmap/ROADMAP_2_4.rst | 10 +- 38 files changed, 464 insertions(+), 1096 deletions(-) delete mode 100644 docs/docsite/rst/dev_guide/developing.rst delete mode 100644 docs/docsite/rst/developing_modules.rst diff --git a/docs/docsite/conf.py b/docs/docsite/conf.py index d6000187013..34166b833e3 100644 --- a/docs/docsite/conf.py +++ b/docs/docsite/conf.py @@ -100,7 +100,7 @@ exclude_patterns = ['modules'] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' -highlight_language = 'yaml' +highlight_language = 'YAML+Jinja' #Substitutions, variables, entities, & shortcuts for text which do not need to link to anything. #For titles which should be a link, use the intersphinx anchors set at the index, chapter, and section levels, such as qi_start_: diff --git a/docs/docsite/rst/dev_guide/developing.rst b/docs/docsite/rst/dev_guide/developing.rst deleted file mode 100644 index 8b423d58d0f..00000000000 --- a/docs/docsite/rst/dev_guide/developing.rst +++ /dev/null @@ -1,18 +0,0 @@ -Developer Information -````````````````````` - -Learn how to build modules of your own in any language, and also how to extend Ansible through several kinds of plugins. Explore Ansible's Python API and write Python plugins to integrate with other solutions in your environment. - -.. toctree:: - :maxdepth: 1 - - developing_api - developing_inventory - developing_modules - developing_plugins - developing_core - developing_test_pr - developing_releases - -Developers will also likely be interested in the fully-discoverable in :doc:`tower`. It's great for embedding Ansible in all manner of applications. - diff --git a/docs/docsite/rst/dev_guide/developing_api.rst b/docs/docsite/rst/dev_guide/developing_api.rst index 21fe082ff24..bb2cbc43426 100644 --- a/docs/docsite/rst/dev_guide/developing_api.rst +++ b/docs/docsite/rst/dev_guide/developing_api.rst @@ -16,7 +16,7 @@ write various plugins, and you can plug in inventory data from external data sou covers the execution and Playbook API at a basic level. If you are looking to use Ansible programmatically from something other than Python, trigger events asynchronously, -or have access control and logging demands, take a look at :doc:`tower` +or have access control and logging demands, take a look at :doc:`../tower` as it has a very nice REST API that provides all of these things at a higher level. Ansible is written in its own API so you have a considerable amount of power across the board. @@ -125,7 +125,7 @@ It's pretty simple:: The run method returns results per host, grouped by whether they could be contacted or not. Return types are module specific, as -expressed in the :doc:`modules` documentation.:: +expressed in the :doc:`../modules` documentation.:: { "dark" : { diff --git a/docs/docsite/rst/dev_guide/developing_inventory.rst b/docs/docsite/rst/dev_guide/developing_inventory.rst index 5775e041e1d..b6f96e2aab3 100644 --- a/docs/docsite/rst/dev_guide/developing_inventory.rst +++ b/docs/docsite/rst/dev_guide/developing_inventory.rst @@ -4,7 +4,7 @@ Developing Dynamic Inventory Sources .. contents:: Topics :local: -As described in :doc:`intro_dynamic_inventory`, ansible can pull inventory information from dynamic sources, including cloud sources. +As described in :doc:`../intro_dynamic_inventory`, Ansible can pull inventory information from dynamic sources, including cloud sources. How do we write a new one? diff --git a/docs/docsite/rst/dev_guide/developing_modules.rst b/docs/docsite/rst/dev_guide/developing_modules.rst index 78f9ad5f881..37bf84165ff 100644 --- a/docs/docsite/rst/dev_guide/developing_modules.rst +++ b/docs/docsite/rst/dev_guide/developing_modules.rst @@ -9,7 +9,7 @@ return information to ansible by printing a JSON string to stdout before exiting. They take arguments in in one of several ways which we'll go into as we work through this tutorial. -See :doc:`modules` for a list of various ones developed in core. +See :doc:`../modules` for a list of existing modules. Modules can be written in any language and are found in the path specified by :envvar:`ANSIBLE_LIBRARY` or the ``--module-path`` command line option. @@ -51,7 +51,9 @@ modules. Keep in mind, though, that some modules in Ansible's source tree are so look at :ref:`service` or :ref:`yum`, and don't stare too close into things like ``async_wrapper`` or you'll turn to stone. Nobody ever executes ``async_wrapper`` directly. -Ok, let's get going with an example. We'll use Python. For starters, save this as a file named :file:`timetest.py`:: +Ok, let's get going with an example. We'll use Python. For starters, save this as a file named :file:`timetest.py` + +.. code-block:: python #!/usr/bin/python @@ -68,21 +70,27 @@ Ok, let's get going with an example. We'll use Python. For starters, save this Testing Modules ```````````````` -There's a useful test script in the source checkout for Ansible:: +There's a useful test script in the source checkout for Ansible: + +.. code-block:: shell-session git clone git://github.com/ansible/ansible.git --recursive source ansible/hacking/env-setup For instructions on setting up Ansible from source, please see -:doc:`intro_installation`. +:doc:`../intro_installation`. + +Let's run the script you just wrote with that: -Let's run the script you just wrote with that:: +.. code-block:: shell-session ansible/hacking/test-module -m ./timetest.py -You should see output that looks something like this:: +You should see output that looks something like this: - {'time': '2012-03-14 22:13:48.539183'} +.. code-block:: json + + {"time": "2012-03-14 22:13:48.539183"} If you did not, you might have a typo in your module, so recheck it and try again. @@ -111,7 +119,9 @@ If no time parameter is set, we'll just leave the time as is and return the curr Let's look at the code. Read the comments as we'll explain as we go. Note that this is highly verbose because it's intended as an educational example. You can write modules -a lot shorter than this:: +a lot shorter than this: + +.. code-block:: python #!/usr/bin/python @@ -211,7 +221,9 @@ Binary Modules Input Support for binary modules was added in Ansible 2.2. When Ansible detects a binary module, it will proceed to supply the argument input as a file on ``argv[1]`` that is formatted as JSON. The JSON contents of that file would resemble something similar to the following payload for a module accepting the same arguments as the -``ping`` module:: +``ping`` module: + +.. code-block:: json { "data": "pong", @@ -229,10 +241,12 @@ Module Provided 'Facts' The :ref:`setup` module that ships with Ansible provides many variables about a system that can be used in playbooks and templates. However, it's possible to also add your own facts without modifying the system module. To do -this, just have the module return a `ansible_facts` key, like so, along with other return data:: +this, just have the module return a `ansible_facts` key, like so, along with other return data: + +.. code-block:: json { - "changed" : True, + "changed" : true, "rc" : 5, "ansible_facts" : { "leptons" : 5000, @@ -265,7 +279,9 @@ Rather than mention these here, the best way to learn is to read some of the `so The 'group' and 'user' modules are reasonably non-trivial and showcase what this looks like. Key parts include always importing the boilerplate code from -:mod:`ansible.module_utils.basic` like this:: +:mod:`ansible.module_utils.basic` like this: + +.. code-block:: python from ansible.module_utils.basic import AnsibleModule if __name__ == '__main__': @@ -274,11 +290,15 @@ Key parts include always importing the boilerplate code from .. note:: Prior to Ansible-2.1.0, importing only what you used from :mod:`ansible.module_utils.basic` did not work. You needed to use - a wildcard import like this:: + a wildcard import like this: + +.. code-block:: python from ansible.module_utils.basic import * -And instantiating the module class like:: +And instantiating the module class like: + +.. code-block:: python def main(): module = AnsibleModule( @@ -293,11 +313,15 @@ And instantiating the module class like:: The :class:`AnsibleModule` provides lots of common code for handling returns, parses your arguments for you, and allows you to check inputs. -Successful returns are made like this:: +Successful returns are made like this: + +.. code-block:: python module.exit_json(changed=True, something_else=12345) -And failures are just as simple (where `msg` is a required parameter to explain the error):: +And failures are just as simple (where `msg` is a required parameter to explain the error): + +.. code-block:: python module.fail_json(msg="Something fatal happened") @@ -322,7 +346,9 @@ mode, the module should try to predict whether changes will occur. For your module to support check mode, you must pass ``supports_check_mode=True`` when instantiating the AnsibleModule object. The AnsibleModule.check_mode attribute -will evaluate to True when check mode is enabled. For example:: +will evaluate to True when check mode is enabled. For example: + +.. code-block:: python module = AnsibleModule( argument_spec = dict(...), @@ -344,7 +370,9 @@ mode, your module will simply be skipped. Common Pitfalls ``````````````` -You should also never do this in a module:: +You should also NEVER do this in a module: + +.. code-block:: python print "some status message" @@ -358,6 +386,12 @@ how the command module is implemented. If a module returns stderr or otherwise fails to produce valid JSON, the actual output will still be shown in Ansible, but the command will not succeed. +Don't write to files directly; use a temporary file and then use the `atomic_move` function from `ansibile.module_utils.basic` to move the updated temporary file into place. This prevents data corruption and ensures that the correct context for the file is kept. + +Avoid creating a module that does the work of other modules; this leads to code duplication and divergence, and makes things less uniform, unpredictable and harder to maintain. Modules should be the building blocks. Instead of creating a module that does the work of other modules, use Plays and Roles instead. + +Avoid creating 'caches'. Ansible is designed without a central server or authority, so you cannot guarantee it will not run with different permissions, options or locations. If you need a central authority, have it on top of Ansible (for example, using bastion/cm/ci server or tower); do not try to build it into modules. + Always use the hacking/test-module script when developing modules and it will warn you about these kind of things. @@ -407,7 +441,9 @@ Example See an example documentation string in the checkout under `examples/DOCUMENTATION.yml `_. -Include it in your module file like this:: +Include it in your module file like this: + +.. code-block:: python #!/usr/bin/python # Copyright header.... @@ -419,8 +455,6 @@ Include it in your module file like this:: # ... snip ... ''' -If an argument takes both C(True)/C(False) and C(Yes)/C(No), the documentation should use C(True) and C(False). - The ``description``, and ``notes`` fields support formatting with some special macros. @@ -451,17 +485,17 @@ the ``copy`` module:: description: destination file/path returned: success type: string - sample: "/path/to/file.txt" + sample: /path/to/file.txt src: description: source file used for the copy on the target machine returned: changed type: string - sample: "/home/httpd/.ansible/tmp/ansible-tmp-1423796390.97-147729857856000/source" + sample: /home/httpd/.ansible/tmp/ansible-tmp-1423796390.97-147729857856000/source md5sum: description: md5 checksum of the file after running copy returned: when supported type: string - sample: "2a5aeecc61dc98c4d780b14b330e3282" + sample: 2a5aeecc61dc98c4d780b14b330e3282 ... ''' @@ -507,7 +541,9 @@ some helper methods to do just that. If you are using Ansible with the :envvar:`ANSIBLE_KEEP_REMOTE_FILES` environment variables to keep the remote module file, here's a sample of how -your debugging session will start:: +your debugging session will start: + +.. code-block:: sh $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible localhost -m ping -a 'data=debugging_session' -vvv <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: badger @@ -515,13 +551,13 @@ your debugging session will start:: <127.0.0.1> PUT /var/tmp/tmpjdbJ1w TO /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping <127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping' localhost | SUCCESS => { - "changed": false, + "changed": false, "invocation": { "module_args": { "data": "debugging_session" - }, + }, "module_name": "ping" - }, + }, "ping": "debugging_session" } @@ -533,7 +569,9 @@ That way it prints the file name of the temporary module file for you to see. If you want to examine the wrapper file you can. It will show a small python script with a large, base64 encoded string. The string contains the module that is going to be executed. Run the wrapper's explode command to turn the -string into some python files that you can work with:: +string into some python files that you can work with: + +.. code-block:: shell-session $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping explode Module expanded into: @@ -571,7 +609,9 @@ When you look into the debug_dir you'll see a directory structure like this:: the module code you have written. Once you edit the code or arguments in the exploded tree you need some way to -run it. There's a separate wrapper subcommand for this:: +run it. There's a separate wrapper subcommand for this: + +.. code-block:: shell-session $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping execute {"invocation": {"module_args": {"data": "debugging_session"}}, "changed": false, "ping": "debugging_session"} @@ -639,16 +679,18 @@ The following checklist items are important guidelines for people who want to c * Modules must be written to support Python 2.4. If this is not possible, required minimum python version and rationale should be explained in the requirements section in ``DOCUMENTATION``. This minimum requirement will be advanced to Python-2.6 in Ansible-2.4. * Modules must be written to use proper Python-3 syntax. At some point in the future we'll come up with rules for running on Python-3 but we're not there yet. See :doc:`developing_modules_python3` for help on how to do this. * Modules must have a metadata section. For the vast majority of new modules, - the metadata should look exactly like this:: + the metadata should look exactly like this: + +.. code-block:: python ANSIBLE_METADATA = {'status': ['preview'], 'supported_by': 'community', 'version': '1.0'} - The complete module metadata specification is here: https://github.com/ansible/proposals/issues/30 +The complete module metadata specification is here: https://github.com/ansible/proposals/issues/30 + * Documentation: Make sure it exists * Module documentation should briefly and accurately define what each module and option does, and how it works with others in the underlying system. Documentation should be written for broad audience--readable both by experts and non-experts. This documentation is not meant to teach a total novice, but it also should not be reserved for the Illuminati (hard balance). - * If an argument takes both C(True)/C(False) and C(Yes)/C(No), the documentation should use C(True) and C(False). * Descriptions should always start with a capital letter and end with a full stop. Consistency always helps. * The `required` setting is only required when true, otherwise it is assumed to be false. * If `required` is false/missing, `default` may be specified (assumed 'null' if missing). Ensure that the default parameter in docs matches default parameter in code. @@ -682,7 +724,7 @@ The following checklist items are important guidelines for people who want to c * Fail predictably--if we must fail, do it in a way that is the most expected. Either mimic the underlying tool or the general way the system works. * Modules should not do the job of other modules, that is what roles are for. Less magic is more. * Don't reinvent the wheel. Part of the problem is that code sharing is not that easy nor documented, we also need to expand our base functions to provide common patterns (retry, throttling, etc). - * Support check mode. This is not required for all modules, as it won't make sense for certain ones, but please attempt to include this when applicable). For more information, refer to :ref:`check_mode_drift` and :ref:`check_mode_dry`. + * Support check mode. This is not required for all modules, as it won't make sense for certain ones, but please attempt to include this when applicable). For more information, refer to :ref:`check_mode_drift` and :ref:`check_mode_dry`. * Exceptions: The module must handle them. (exceptions are bugs) * Give out useful messages on what you were doing and you can add the exception message to that. * Avoid catchall exceptions, they are not very useful unless the underlying API gives very good error messages pertaining the attempted action. @@ -696,7 +738,9 @@ The following checklist items are important guidelines for people who want to c - ec2 * The module must not use sys.exit() --> use fail_json() from the module object. -* Import custom packages in try/except and handled with fail_json() in main() e.g.:: +* Import custom packages in try/except and handled with fail_json() in main() e.g. + +.. code-block:: python try: import foo @@ -710,7 +754,9 @@ The following checklist items are important guidelines for people who want to c * Do not use wildcards for importing other python modules (ex: ``from ansible.module_utils.basic import *``). This used to be required for code imported from ``ansible.module_utils`` but, from Ansible-2.1 onwards, it's just an outdated and bad practice. * The module must have a `main` function that wraps the normal execution. * Call your :func:`main` from a conditional so that it would be possible to - import them into unittests in the future example:: + import them into unittests in the future example + +.. code-block:: python if __name__ == '__main__': main() @@ -727,53 +773,80 @@ The following checklist items are important guidelines for people who want to c fields of a dictionary and return the dictionary. * When fetching URLs, please use either fetch_url or open_url from ansible.module_utils.urls rather than urllib2; urllib2 does not natively verify TLS certificates and so is insecure for https. - +* facts modules must return facts in the ansible_facts field of the result + dictionary. :ref:`module_provided_facts` +* modules that are purely about fact gathering need to implement check_mode. + they should not cause any changes anyway so it should be as simple as adding + check_mode=True when instantiating AnsibleModule. (The reason is that + playbooks which conditionalize based on fact information will only + conditionalize correctly in check_mode if the facts are returned in + check_mode). +* Basic auth: module_utils.api has some helpers for doing basic auth with + module_utils.urls.fetch_url(). If you use those you may find you also want + to fallback on environment variables for default values. If you do that, + be sure to use non-generic environment variables (like + :envvar:`API__USERNAME`). Using generic environment variables + like :envvar:`API_USERNAME` would conflict between modules. Windows modules checklist ````````````````````````` -* Favour native powershell and .net ways of doing things over calls to COM libraries or calls to native executables which may or may not be present in all versions of windows +* Favour native powershell and .net ways of doing things over calls to COM libraries or calls to native executables which may or may not be present in all versions of Windows * modules are in powershell (.ps1 files) but the docs reside in same name python file (.py) * look at ansible/lib/ansible/module_utils/powershell.ps1 for common code, avoid duplication * Ansible uses strictmode version 2.0 so be sure to test with that enabled -* start with:: - #!powershell +All powershell modules must start: - then:: - +.. code-block:: powershell + + #!powershell - then:: + # WANT_JSON # POWERSHELL_COMMON - - then, to parse all arguments into a variable modules generally use:: + +To parse all arguments into a variable modules generally use: + +.. code-block:: powershell $params = Parse-Args $args -* Arguments: - * Try and use state present and state absent like other modules - * You need to check that all your mandatory args are present. You can do this using the builtin Get-AnsibleParam function. - * Required arguments:: +Arguments ++++++++++ + +* Try and use state present and state absent like other modules +* You need to check that all your mandatory args are present. You can do this using the builtin Get-AnsibleParam function. +* Required arguments: + +.. code-block:: powershell $package = Get-AnsibleParam -obj $params -name name -failifempty $true - * Required arguments with name validation:: +Required arguments with name validation: + +.. code-block:: powershell $state = Get-AnsibleParam -obj $params -name "State" -ValidateSet "Present","Absent" -resultobj $resultobj -failifempty $true - * Optional arguments with name validation:: +Optional arguments with name validation ++++++++++++++++++++++++++++++++++++++++ + +.. code-block:: powershell $state = Get-AnsibleParam -obj $params -name "State" -default "Present" -ValidateSet "Present","Absent" - * the If "FailIfEmpty" is true, the resultobj parameter is used to specify the object returned to fail-json. You can also override the default message - using $emptyattributefailmessage (for missing required attributes) and $ValidateSetErrorMessage (for attribute validation errors) - * Look at existing modules for more examples of argument checking. +* If the "FailIfEmpty" is true, the resultobj parameter is used to specify the object returned to fail-json. You can also override the default message + using $emptyattributefailmessage (for missing required attributes) and $ValidateSetErrorMessage (for attribute validation errors) +* Look at existing modules for more examples of argument checking. + +Results ++++++++ +* The result object should always contain an attribute called changed set to either $true or $false +* Create your result object like this -* Results - * The result object should always contain an attribute called changed set to either $true or $false - * Create your result object like this:: +.. code-block:: powershell $result = New-Object psobject @{ changed = $false @@ -783,10 +856,10 @@ Windows modules checklist If all is well, exit with a Exit-Json $result - * Ensure anything you return, including errors can be converted to json. - * Be aware that because exception messages could contain almost anything. - * ConvertTo-Json will fail if it encounters a trailing \ in a string. - * If all is not well use Fail-Json to exit. +* Ensure anything you return, including errors can be converted to json. +* Be aware that because exception messages could contain almost anything. +* ConvertTo-Json will fail if it encounters a trailing \ in a string. +* If all is not well use Fail-Json to exit. * Have you tested for powershell 3.0 and 4.0 compliance? @@ -808,7 +881,7 @@ This example allows the stat module to be called with fileinfo, making the follo .. seealso:: - :doc:`modules` + :doc:`../modules` Learn about available modules :doc:`developing_plugins` Learn about developing plugins diff --git a/docs/docsite/rst/dev_guide/developing_plugins.rst b/docs/docsite/rst/dev_guide/developing_plugins.rst index e74993bfc71..977e126e821 100644 --- a/docs/docsite/rst/dev_guide/developing_plugins.rst +++ b/docs/docsite/rst/dev_guide/developing_plugins.rst @@ -76,17 +76,21 @@ You can only have one plugin be the main manager of your console output. If you Developing Callback Plugins +++++++++++++++++++++++++++ -Callback plugins are created by creating a new class with the Base(Callbacks) class as the parent:: +Callback plugins are created by creating a new class with the Base(Callbacks) class as the parent: + +.. code-block:: python from ansible.plugins.callback import CallbackBase from ansible import constants as C - - class CallbackModule(CallbackBase): + + class CallbackModule(CallbackBase): From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`. For a complete list of methods that you can override, please see ``__init__.py`` in the `lib/ansible/plugins/callback `_ directory. -The following example shows how Ansible's timer plugin is implemented:: +The following example shows how Ansible's timer plugin is implemented: + +.. code-block:: python # Make coding more python3-ish from __future__ import (absolute_import, division, print_function) @@ -105,21 +109,21 @@ The following example shows how Ansible's timer plugin is implemented:: CALLBACK_TYPE = 'aggregate' CALLBACK_NAME = 'timer' CALLBACK_NEEDS_WHITELIST = True - + def __init__(self): - + super(CallbackModule, self).__init__() - + self.start_time = datetime.now() - + def days_hours_minutes_seconds(self, runtime): minutes = (runtime.seconds // 60) % 60 r_seconds = runtime.seconds - (minutes * 60) return runtime.days, runtime.seconds // 3600, minutes, r_seconds - + def playbook_on_stats(self, stats): self.v2_playbook_on_stats(stats) - + def v2_playbook_on_stats(self, stats): end_time = datetime.now() runtime = end_time - self.start_time @@ -132,9 +136,8 @@ Note that the CALLBACK_VERSION and CALLBACK_NAME definitions are required for pr Connection Plugins ------------------ -By default, ansible ships with a 'paramiko' SSH, native ssh (just called 'ssh'), 'local' connection type, and there are also some minor players like 'chroot' and 'jail'. All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines. The basics of these connection types -are covered in the :doc:`intro_getting_started` section. Should you want to extend Ansible to support other transports (SNMP? Message bus? -Carrier Pigeon?) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins +By default, Ansible ships with a 'paramiko' SSH, native ssh (just called 'ssh'), 'local' connection type, and there are also some minor players like 'chroot' and 'jail'. All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines. The basics of these connection types +are covered in the :doc:`../intro_getting_started` section. Should you want to extend Ansible to support other transports (SNMP, Message bus, etc) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins directory. The value of 'smart' for a connection allows selection of paramiko or openssh based on system capabilities, and chooses 'ssh' if OpenSSH supports ControlPersist, in Ansible 1.2.1 and later. Previous versions did not support 'smart'. @@ -145,9 +148,11 @@ More documentation on writing connection plugins is pending, though you can jump Lookup Plugins -------------- -Lookup plugins are used to pull in data from external data stores. Lookup plugins can be used within playbooks for both looping - playbook language constructs like "with_fileglob" and "with_items" are implemented via lookup plugins - and to return values into a variable or parameter. +Lookup plugins are used to pull in data from external data stores. Lookup plugins can be used within playbooks for both looping - playbook language constructs like "with_fileglob" and "with_items" are implemented via lookup plugins - and to return values into a variable or parameter. + +Here's a simple lookup plugin implementation - this lookup returns the contents of a text file as a variable: -Here's a simple lookup plugin implementation - this lookup returns the contents of a text file as a variable:: +.. code-block:: python from ansible.errors import AnsibleError, AnsibleParserError from ansible.plugins.lookup import LookupBase @@ -193,7 +198,9 @@ An example of how this lookup is called:: - debug: msg="the value of foo.txt is {{ contents }} as seen today {{ lookup('pipe', 'date +"%Y-%m-%d"') }}" -Errors encountered during execution should be returned by raising AnsibleError() with a message describing the error. Any strings returned by your lookup plugin implementation that could ever contain non-ASCII characters must be converted into Python's unicode type because the strings will be run through jinja2. To do this, you can use:: +Errors encountered during execution should be returned by raising AnsibleError() with a message describing the error. Any strings returned by your lookup plugin implementation that could ever contain non-ASCII characters must be converted into Python's unicode type because the strings will be run through jinja2. To do this, you can use: + +.. code-block:: python from ansible.module_utils._text import to_text result_string = to_text(result_string) @@ -241,7 +248,7 @@ Distributing Plugins Plugins are loaded from the library installed path and the configured plugins directory (check your `ansible.cfg`). The location can vary depending on how you installed Ansible (pip, rpm, deb, etc) or by the OS/Distribution/Packager. -Plugins are automatically loaded when you have one of the following subfolders adjacent to your playbook or inside a role:: +Plugins are automatically loaded when you have one of the following subfolders adjacent to your playbook or inside a role: * action_plugins * lookup_plugins @@ -257,7 +264,7 @@ When shipped as part of a role, the plugin will be available as soon as the role .. seealso:: - :doc:`modules` + :doc:`../modules` List of built-in modules :doc:`developing_api` Learn about the Python API for task execution diff --git a/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst b/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst index 5f5572bebb5..ef4cde79b84 100644 --- a/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst +++ b/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst @@ -86,11 +86,15 @@ JSONARGS Scripts can arrange for an argument string to be placed within them by placing the string ``<>`` somewhere inside of the -file. The module typically sets a variable to that value like this:: python +file. The module typically sets a variable to that value like this: + +.. code-block:: python json_arguments = """<>""" -Which is expanded as:: python +Which is expanded as: + +.. code-block:: python json_arguments = """{"param1": "test's quotes", "param2": "\"To be or not to be\" - Hamlet"}""" @@ -458,7 +462,7 @@ this. If a module has to use this on its own, it should instantiate an :attr:`AnsibleModule._syslog_facility`. The code will look slightly different than it did under :ref:`module_replacer` due to how hacky the old way was -:: python +.. code-block:: python # Old way import syslog diff --git a/docs/docsite/rst/dev_guide/developing_test_pr.rst b/docs/docsite/rst/dev_guide/developing_test_pr.rst index 1d5a67c5213..72d85a6860b 100644 --- a/docs/docsite/rst/dev_guide/developing_test_pr.rst +++ b/docs/docsite/rst/dev_guide/developing_test_pr.rst @@ -116,7 +116,9 @@ For Those About To Test, We Salute You At this point, you should be ready to begin testing! If the PR is a bug-fix pull request, the first things to do are to run the suite of unit and integration tests, to ensure -the pull request does not break current functionality:: +the pull request does not break current functionality: + +.. code-block:: shell-session # Unit Tests make tests @@ -139,16 +141,20 @@ We encourage users to provide playbook examples for bugs that show how to reprod the issue if available. You may wish to also do your own review to poke the corners of the change. Since some reproducers can be quite involved, you might wish to create a testing directory with the issue # as a sub- -directory to keep things organized:: +directory to keep things organized: + +.. code-block:: shell-session mkdir -p testing/XXXX # where XXXX is again the issue # for the original issue or PR cd testing/XXXX - + # create files or git clone example playbook repo While it should go without saying, be sure to read any playbooks before you run them. VMs help with running untrusted content greatly, though a playbook could still do something to your computing resources that you'd rather not like. -Once the files are in place, you can run the provided playbook (if there is one) to test the functionality:: +Once the files are in place, you can run the provided playbook (if there is one) to test the functionality: + +.. code-block:: shell-session ansible-playbook -vvv playbook_name.yml @@ -164,30 +170,29 @@ If the pull request resolves the issue, please leave a comment on the pull reque In some cases, you may wish to share playbook output from the test run as well. -Example!:: - - Works for me! Tested on `Ansible 1.7.1`. I verified this on CentOS 6.5 and also Ubuntu 14.04. - -If the PR does not resolve the issue, or if you see any failures from the unit/integration tests, just include that output instead:: +Example! - This doesn't work for me. + | Works for me! Tested on `Ansible 1.7.1`. I verified this on CentOS 6.5 and also Ubuntu 14.04. - When I ran this my toaster started making loud noises! +If the PR does not resolve the issue, or if you see any failures from the unit/integration tests, just include that output instead: - Output from the toaster looked like this: + | This doesn't work for me. + | + | When I ran this Ubuntu 16.04 it failed with the following: + | + | BLARG + | StrackTrace + | RRRARRGGG - ``` - BLARG - StrackTrace - RRRARRGGG - ``` +When you are done testing a feature branch, you can remove it with the following command: -When you are done testing a feature branch, you can remove it with the following command:: +.. code-block:: shell-session - git branch -D someuser-feature_branch_name + $ git branch -D someuser-feature_branch_name -We understand some users may be inexperienced with git, or other aspects of the above procedure, so feel free to stop by ansible-devel -list for questions and we'd be happy to help answer them. +We understand some users may be inexperienced with git, or other aspects of +the above procedure, so feel free to stop by ansible-devel list for questions +and we'd be happy to help answer them. diff --git a/docs/docsite/rst/dev_guide/index.rst b/docs/docsite/rst/dev_guide/index.rst index b7d0b183256..fe0aa1bb81d 100644 --- a/docs/docsite/rst/dev_guide/index.rst +++ b/docs/docsite/rst/dev_guide/index.rst @@ -15,16 +15,15 @@ To get started, select one of the following topics. :maxdepth: 1 overview_architecture - overview_components - overview_contributing developing_modules + developing_modules_python3 developing_plugins developing_inventory developing_api - - - + developing_module_utilities developing_core developing_test_pr developing_rebasing + repomerge developing_releases + ../committer_guidelines diff --git a/docs/docsite/rst/dev_guide/repomerge.rst b/docs/docsite/rst/dev_guide/repomerge.rst index d8831f52bef..92152af29b4 100644 --- a/docs/docsite/rst/dev_guide/repomerge.rst +++ b/docs/docsite/rst/dev_guide/repomerge.rst @@ -30,6 +30,7 @@ The documentation pages for modules will be updated to reflect the above informa .. _PRMover: + Move Issues and PRs to new Repo ------------------------------- A tool has been developed to move a PR from the old repos to `ansible/ansible` this can be found at `prmover tool `_ @@ -41,3 +42,4 @@ To move issues please use `GitHub Issue Mover `_. There's also a core -repo for more established and widely used modules. "Extras" modules may be promoted to core periodically, -but there's no fundamental difference in the end - both ship with Ansible, all in one package, regardless -of how you acquire Ansible. - -.. _module_dev_tutorial: - -Tutorial -```````` - -Let's build a very-basic module to get and set the system time. For starters, let's build -a module that just outputs the current time. - -We are going to use Python here but any language is possible. Only File I/O and outputting to standard -out are required. So, bash, C++, clojure, Python, Ruby, whatever you want -is fine. - -Now Python Ansible modules contain some extremely powerful shortcuts (that all the core modules use) -but first we are going to build a module the very hard way. The reason we do this is because modules -written in any language OTHER than Python are going to have to do exactly this. We'll show the easy -way later. - -So, here's an example. You would never really need to build a module to set the system time, -the 'command' module could already be used to do this. - -Reading the modules that come with Ansible (linked above) is a great way to learn how to write -modules. Keep in mind, though, that some modules in Ansible's source tree are internalisms, -so look at :ref:`service` or :ref:`yum`, and don't stare too close into things like ``async_wrapper`` or -you'll turn to stone. Nobody ever executes ``async_wrapper`` directly. - -Ok, let's get going with an example. We'll use Python. For starters, save this as a file named :file:`timetest.py`:: - - #!/usr/bin/python - - import datetime - import json - - date = str(datetime.datetime.now()) - print json.dumps({ - "time" : date - }) - -.. _module_testing: - -Testing Modules -```````````````` - -There's a useful test script in the source checkout for Ansible:: - - git clone git://github.com/ansible/ansible.git --recursive - source ansible/hacking/env-setup - -For instructions on setting up Ansible from source, please see -:doc:`intro_installation`. - -Let's run the script you just wrote with that:: - - ansible/hacking/test-module -m ./timetest.py - -You should see output that looks something like this:: - - {'time': '2012-03-14 22:13:48.539183'} - -If you did not, you might have a typo in your module, so recheck it and try again. - -.. _reading_input: - -Reading Input -````````````` - - -Let's modify the module to allow setting the current time. We'll do this by seeing -if a key value pair in the form `time=` is passed in to the module. - -Ansible internally saves arguments to an arguments file. So we must read the file -and parse it. The arguments file is just a string, so any form of arguments are legal. -Here we'll do some basic parsing to treat the input as key=value. - -The example usage we are trying to achieve to set the time is:: - - time time="March 14 22:10" - -If no time parameter is set, we'll just leave the time as is and return the current time. - -.. note:: - This is obviously an unrealistic idea for a module. You'd most likely just - use the command module. However, it makes for a decent tutorial. - -Let's look at the code. Read the comments as we'll explain as we go. Note that this -is highly verbose because it's intended as an educational example. You can write modules -a lot shorter than this:: - - #!/usr/bin/python - - # import some python modules that we'll use. These are all - # available in Python's core - - import datetime - import sys - import json - import os - import shlex - - # read the argument string from the arguments file - args_file = sys.argv[1] - args_data = file(args_file).read() - - # For this module, we're going to do key=value style arguments. - # Modules can choose to receive json instead by adding the string: - # WANT_JSON - # Somewhere in the file. - # Modules can also take free-form arguments instead of key-value or json - # but this is not recommended. - - arguments = shlex.split(args_data) - for arg in arguments: - - # ignore any arguments without an equals in it - if "=" in arg: - - (key, value) = arg.split("=") - - # if setting the time, the key 'time' - # will contain the value we want to set the time to - - if key == "time": - - # now we'll affect the change. Many modules - # will strive to be idempotent, generally - # by not performing any actions if the current - # state is the same as the desired state. - # See 'service' or 'yum' in the main git tree - # for an illustrative example. - - rc = os.system("date -s \"%s\"" % value) - - # always handle all possible errors - # - # when returning a failure, include 'failed' - # in the return data, and explain the failure - # in 'msg'. Both of these conventions are - # required however additional keys and values - # can be added. - - if rc != 0: - print json.dumps({ - "failed" : True, - "msg" : "failed setting the time" - }) - sys.exit(1) - - # when things do not fail, we do not - # have any restrictions on what kinds of - # data are returned, but it's always a - # good idea to include whether or not - # a change was made, as that will allow - # notifiers to be used in playbooks. - - date = str(datetime.datetime.now()) - print json.dumps({ - "time" : date, - "changed" : True - }) - sys.exit(0) - - # if no parameters are sent, the module may or - # may not error out, this one will just - # return the time - - date = str(datetime.datetime.now()) - print json.dumps({ - "time" : date - }) - -Let's test that module:: - - ansible/hacking/test-module -m ./timetest.py -a "time=\"March 14 12:23\"" - -This should return something like:: - - {"changed": true, "time": "2012-03-14 12:23:00.000307"} - -.. _binary_module_reading_input: - -Binary Modules Input -++++++++++++++++++++ - -Support for binary modules was added in Ansible 2.2. When Ansible detects a binary module, it will proceed to -supply the argument input as a file on ``argv[1]`` that is formatted as JSON. The JSON contents of that file -would resemble something similar to the following payload for a module accepting the same arguments as the -``ping`` module:: - - { - "data": "pong", - "_ansible_verbosity": 4, - "_ansible_diff": false, - "_ansible_debug": false, - "_ansible_check_mode": false, - "_ansible_no_log": false - } - -.. _module_provided_facts: - -Module Provided 'Facts' -```````````````````````` - -The :ref:`setup` module that ships with Ansible provides many variables about a system that can be used in playbooks -and templates. However, it's possible to also add your own facts without modifying the system module. To do -this, just have the module return a `ansible_facts` key, like so, along with other return data:: - - { - "changed" : True, - "rc" : 5, - "ansible_facts" : { - "leptons" : 5000, - "colors" : { - "red" : "FF0000", - "white" : "FFFFFF" - } - } - } - -These 'facts' will be available to all statements called after that module (but not before) in the playbook. -A good idea might be to make a module called 'site_facts' and always call it at the top of each playbook, though -we're always open to improving the selection of core facts in Ansible as well. - -.. _common_module_boilerplate: - -Common Module Boilerplate -````````````````````````` - -As mentioned, if you are writing a module in Python, there are some very powerful shortcuts you can use. -Modules are still transferred as one file, but an arguments file is no longer needed, so these are not -only shorter in terms of code, they are actually FASTER in terms of execution time. - -Rather than mention these here, the best way to learn is to read some of the `source of the modules `_ that come with Ansible. - -The 'group' and 'user' modules are reasonably non-trivial and showcase what this looks like. - -Key parts include always importing the boilerplate code from -:mod:`ansible.module_utils.basic` like this:: - - from ansible.module_utils.basic import AnsibleModule - if __name__ == '__main__': - main() - -.. note:: - Prior to Ansible-2.1.0, importing only what you used from - :mod:`ansible.module_utils.basic` did not work. You needed to use - a wildcard import like this:: - - from ansible.module_utils.basic import * - -And instantiating the module class like:: - - def main(): - module = AnsibleModule( - argument_spec = dict( - state = dict(default='present', choices=['present', 'absent']), - name = dict(required=True), - enabled = dict(required=True, type='bool'), - something = dict(aliases=['whatever']) - ) - ) - -The :class:`AnsibleModule` provides lots of common code for handling returns, parses your arguments -for you, and allows you to check inputs. - -Successful returns are made like this:: - - module.exit_json(changed=True, something_else=12345) - -And failures are just as simple (where `msg` is a required parameter to explain the error):: - - module.fail_json(msg="Something fatal happened") - -There are also other useful functions in the module class, such as :func:`module.sha1(path)`. See -:file:`lib/ansible/module_utils/basic.py` in the source checkout for implementation details. - -Again, modules developed this way are best tested with the :file:`hacking/test-module` script in the git -source checkout. Because of the magic involved, this is really the only way the scripts -can function outside of Ansible. - -If submitting a module to Ansible's core code, which we encourage, use of -:class:`AnsibleModule` is required. - -.. _developing_for_check_mode: - -Check Mode -`````````` -.. versionadded:: 1.1 - -Modules may optionally support check mode. If the user runs Ansible in check -mode, the module should try to predict whether changes will occur. - -For your module to support check mode, you must pass ``supports_check_mode=True`` -when instantiating the AnsibleModule object. The AnsibleModule.check_mode attribute -will evaluate to True when check mode is enabled. For example:: - - module = AnsibleModule( - argument_spec = dict(...), - supports_check_mode=True - ) - - if module.check_mode: - # Check if any changes would be made but don't actually make those changes - module.exit_json(changed=check_if_system_state_would_be_changed()) - -Remember that, as module developer, you are responsible for ensuring that no -system state is altered when the user enables check mode. - -If your module does not support check mode, when the user runs Ansible in check -mode, your module will simply be skipped. - -.. _module_dev_pitfalls: - -Common Pitfalls -``````````````` - -You should also never do this in a module:: - - print "some status message" - -Because the output is supposed to be valid JSON. - -Modules must not output anything on standard error, because the system will merge -standard out with standard error and prevent the JSON from parsing. Capturing standard -error and returning it as a variable in the JSON on standard out is fine, and is, in fact, -how the command module is implemented. - -If a module returns stderr or otherwise fails to produce valid JSON, the actual output -will still be shown in Ansible, but the command will not succeed. - -Don't write to files directly; use a temporary file and then use the `atomic_move` function from `ansibile.module_utils.basic` to move the updated temporary file into place. This prevents data corruption and ensures that the correct context for the file is kept. - -Avoid creating a module that does the work of other modules; this leads to code duplication and divergence, and makes things less uniform, unpredictable and harder to maintain. Modules should be the building blocks. Instead of creating a module that does the work of other modules, use Plays and Roles instead. - -Avoid creating 'caches'. Ansible is designed without a central server or authority, so you cannot guarantee it will not run with different permissions, options or locations. If you need a central authority, have it on top of Ansible (for example, using bastion/cm/ci server or tower); do not try to build it into modules. - -Always use the hacking/test-module script when developing modules and it will warn -you about these kind of things. - -.. _module_dev_conventions: - -Conventions/Recommendations -``````````````````````````` - -As a reminder from the example code above, here are some basic conventions -and guidelines: - -* If the module is addressing an object, the parameter for that object should be called 'name' whenever possible, or accept 'name' as an alias. - -* If you have a company module that returns facts specific to your installations, a good name for this module is `site_facts`. - -* Modules accepting boolean status should generally accept 'yes', 'no', 'true', 'false', or anything else a user may likely throw at them. The AnsibleModule common code supports this with "type='bool'". - -* Include a minimum of dependencies if possible. If there are dependencies, document them at the top of the module file, and have the module raise JSON error messages when the import fails. - -* Modules must be self-contained in one file to be auto-transferred by ansible. - -* If packaging modules in an RPM, they only need to be installed on the control machine and should be dropped into /usr/share/ansible. This is entirely optional and up to you. - -* Modules must output valid JSON only. The toplevel return type must be a hash (dictionary) although they can be nested. Lists or simple scalar values are not supported, though they can be trivially contained inside a dictionary. - -* In the event of failure, a key of 'failed' should be included, along with a string explanation in 'msg'. Modules that raise tracebacks (stacktraces) are generally considered 'poor' modules, though Ansible can deal with these returns and will automatically convert anything unparseable into a failed result. If you are using the AnsibleModule common Python code, the 'failed' element will be included for you automatically when you call 'fail_json'. - -* Return codes from modules are actually not significant, but continue on with 0=success and non-zero=failure for reasons of future proofing. - -* As results from many hosts will be aggregated at once, modules should return only relevant output. Returning the entire contents of a log file is generally bad form. - -.. _module_documenting: - -Documenting Your Module -``````````````````````` - -All modules included in the CORE distribution must have a -``DOCUMENTATION`` string. This string MUST be a valid YAML document -which conforms to the schema defined below. You may find it easier to -start writing your ``DOCUMENTATION`` string in an editor with YAML -syntax highlighting before you include it in your Python file. - -.. _module_doc_example: - -Example -+++++++ - -See an example documentation string in the checkout under `examples/DOCUMENTATION.yml `_. - -Include it in your module file like this:: - - #!/usr/bin/python - # Copyright header.... - - DOCUMENTATION = ''' - --- - module: modulename - short_description: This is a sentence describing the module - # ... snip ... - ''' - -The ``description``, and ``notes`` fields -support formatting with some special macros. - -These formatting functions are ``U()``, ``M()``, ``I()``, and ``C()`` -for URL, module, italic, and constant-width respectively. It is suggested -to use ``C()`` for file and option names, and ``I()`` when referencing -parameters; module names should be specified as ``M(module)``. - -Examples should be written in YAML format in plain text in an -``EXAMPLES`` string within the module like this:: - - EXAMPLES = ''' - - modulename: - opt1: arg1 - opt2: arg2 - ''' - -The EXAMPLES section, just like the documentation section, is required in -all module pull requests for new modules. - -The RETURN section documents what the module returns. For each value returned, -provide a ``description``, in what circumstances the value is ``returned``, -the ``type`` of the value and a ``sample``. For example, from -the ``copy`` module:: - - RETURN = ''' - dest: - description: destination file/path - returned: success - type: string - sample: /path/to/file.txt - src: - description: source file used for the copy on the target machine - returned: changed - type: string - sample: /home/httpd/.ansible/tmp/ansible-tmp-1423796390.97-147729857856000/source - md5sum: - description: md5 checksum of the file after running copy - returned: when supported - type: string - sample: 2a5aeecc61dc98c4d780b14b330e3282 - ... - ''' - -.. _module_dev_testing: - -Building & Testing -++++++++++++++++++ - -Put your completed module file into the 'library' directory and then -run the command: ``make webdocs``. The new 'modules.html' file will be -built and appear in the 'docsite/' directory. - -.. tip:: - - If you're having a problem with the syntax of your YAML you can - validate it on the `YAML Lint `_ website. - -.. tip:: - - You can set the environment variable ANSIBLE_KEEP_REMOTE_FILES=1 on the controlling host to prevent ansible from - deleting the remote files so you can debug your module. - -.. _debugging_ansiblemodule_based_modules: - -Debugging AnsibleModule-based modules -````````````````````````````````````` - -.. tip:: - - If you're using the :file:`hacking/test-module` script then most of this - is taken care of for you. If you need to do some debugging of the module - on the remote machine that the module will actually run on or when the - module is used in a playbook then you may need to use this information - instead of relying on test-module. - -Starting with Ansible-2.1.0, AnsibleModule-based modules are put together as -a zip file consisting of the module file and the various python module -boilerplate inside of a wrapper script instead of as a single file with all of -the code concatenated together. Without some help, this can be harder to -debug as the file needs to be extracted from the wrapper in order to see -what's actually going on in the module. Luckily the wrapper script provides -some helper methods to do just that. - -If you are using Ansible with the :envvar:`ANSIBLE_KEEP_REMOTE_FILES` -environment variables to keep the remote module file, here's a sample of how -your debugging session will start:: - - $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible localhost -m ping -a 'data=debugging_session' -vvv - <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: badger - <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" )' - <127.0.0.1> PUT /var/tmp/tmpjdbJ1w TO /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping - <127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping' - localhost | SUCCESS => { - "changed": false, - "invocation": { - "module_args": { - "data": "debugging_session" - }, - "module_name": "ping" - }, - "ping": "debugging_session" - } - -Setting :envvar:`ANSIBLE_KEEP_REMOTE_FILES` to ``1`` tells Ansible to keep the -remote module files instead of deleting them after the module finishes -executing. Giving Ansible the ``-vvv`` optin makes Ansible more verbose. -That way it prints the file name of the temporary module file for you to see. - -If you want to examine the wrapper file you can. It will show a small python -script with a large, base64 encoded string. The string contains the module -that is going to be executed. Run the wrapper's explode command to turn the -string into some python files that you can work with:: - - $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping explode - Module expanded into: - /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/debug_dir - -When you look into the debug_dir you'll see a directory structure like this:: - - ├── ansible_module_ping.py - ├── args - └── ansible - ├── __init__.py - └── module_utils - ├── basic.py - └── __init__.py - -* :file:`ansible_module_ping.py` is the code for the module itself. The name - is based on the name of the module with a prefix so that we don't clash with - any other python module names. You can modify this code to see what effect - it would have on your module. - -* The :file:`args` file contains a JSON string. The string is a dictionary - containing the module arguments and other variables that Ansible passes into - the module to change it's behaviour. If you want to modify the parameters - that are passed to the module, this is the file to do it in. - -* The :file:`ansible` directory contains code from - :mod:`ansible.module_utils` that is used by the module. Ansible includes - files for any :`module:`ansible.module_utils` imports in the module but not - no files from any other module. So if your module uses - :mod:`ansible.module_utils.url` Ansible will include it for you, but if - your module includes :mod:`requests` then you'll have to make sure that - the python requests library is installed on the system before running the - module. You can modify files in this directory if you suspect that the - module is having a problem in some of this boilerplate code rather than in - the module code you have written. - -Once you edit the code or arguments in the exploded tree you need some way to -run it. There's a separate wrapper subcommand for this:: - - $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping execute - {"invocation": {"module_args": {"data": "debugging_session"}}, "changed": false, "ping": "debugging_session"} - -This subcommand takes care of setting the PYTHONPATH to use the exploded -:file:`debug_dir/ansible/module_utils` directory and invoking the script using -the arguments in the :file:`args` file. You can continue to run it like this -until you understand the problem. Then you can copy it back into your real -module file and test that the real module works via :command:`ansible` or -:command:`ansible-playbook`. - -.. note:: - - The wrapper provides one more subcommand, ``excommunicate``. This - subcommand is very similar to ``execute`` in that it invokes the exploded - module on the arguments in the :file:`args`. The way it does this is - different, however. ``excommunicate`` imports the :func:`main` - function from the module and then calls that. This makes excommunicate - execute the module in the wrapper's process. This may be useful for - running the module under some graphical debuggers but it is very different - from the way the module is executed by Ansible itself. Some modules may - not work with ``excommunicate`` or may behave differently than when used - with Ansible normally. Those are not bugs in the module; they're - limitations of ``excommunicate``. Use at your own risk. - -.. _module_paths: - -Module Paths -```````````` - -If you are having trouble getting your module "found" by ansible, be -sure it is in the :envvar:`ANSIBLE_LIBRARY` environment variable. - -If you have a fork of one of the ansible module projects, do something like this:: - - ANSIBLE_LIBRARY=~/ansible-modules-core:~/ansible-modules-extras - -And this will make the items in your fork be loaded ahead of what ships with Ansible. Just be sure -to make sure you're not reporting bugs on versions from your fork! - -To be safe, if you're working on a variant on something in Ansible's normal distribution, it's not -a bad idea to give it a new name while you are working on it, to be sure you know you're pulling -your version. - -.. _module_contribution: - -Getting Your Module Into Ansible -```````````````````````````````` - -High-quality modules with minimal dependencies -can be included in Ansible, but modules (just due to the programming -preferences of the developers) will need to be implemented in Python and use -the AnsibleModule common code, and should generally use consistent arguments with the rest of -the program. Stop by the mailing list to inquire about requirements if you like, and submit -a github pull request to the `extras `_ project. -Included modules will ship with ansible, and also have a chance to be promoted to 'core' status, which -gives them slightly higher development priority (though they'll work in exactly the same way). - -Module checklist -```````````````` - -The following checklist items are important guidelines for people who want to contribute to the development of modules to Ansible on GitHub. Please read the guidelines before you submit your PR/proposal. - -* The shebang should always be ``#!/usr/bin/python``, this allows ansible_python_interpreter to work -* Modules must be written to support Python 2.4. If this is not possible, required minimum python version and rationale should be explained in the requirements section in DOCUMENTATION. -* Modules must be written to use proper Python-3 syntax. At some point in the future we'll come up with rules for running on Python-3 but we're not there yet. See :doc:`developing_modules_python3` for help on how to do this. -* Documentation: Make sure it exists - * Module documentation should briefly and accurately define what each module and option does, and how it works with others in the underlying system. Documentation should be written for broad audience--readable both by experts and non-experts. This documentation is not meant to teach a total novice, but it also should not be reserved for the Illuminati (hard balance). - * Descriptions should always start with a capital letter and end with a full stop. Consistency always helps. - * The `required` setting is only required when true, otherwise it is assumed to be false. - * If `required` is false/missing, `default` may be specified (assumed 'null' if missing). Ensure that the default parameter in docs matches default parameter in code. - * Documenting `default` is not needed for `required: true`. - * Remove unnecessary doc like `aliases: []` or `choices: []`. - * Do not use Boolean values in a choice list . For example, in the list `choices: ['no', 'verify', 'always]`, 'no' will be interpreted as a Boolean value (you can check basic.py for BOOLEANS_* constants to see the full list of Boolean keywords). If your option actually is a boolean, just use `type=bool`; there is no need to populate 'choices'. - * For new modules or options in a module add version_added. The version should match the value of the current development version and is a string (not a float), so be sure to enclose it in quotes. - * Verify that arguments in doc and module spec dict are identical. - * For password / secret arguments no_log=True should be set. - * Requirements should be documented, using the `requirements=[]` field. - * Author should be set, with their name and their github id, at the least. - * Ensure that you make use of U() for urls, C() for files and options, I() for params, M() for modules. - * If an optional parameter is sometimes required this need to be reflected in the documentation, e.g. "Required when C(state=present)." - * Verify that a GPL 3 License header is included. - * Does module use check_mode? Could it be modified to use it? Document it. Documentation is everyone's friend. - * Examples--include them whenever possible and make sure they are reproducible. - * Document the return structure of the module. Refer to :ref:`common_return_values` and :ref:`module_documenting` for additional information. -* Predictable user interface: This is a particularly important section as it is also an area where we need significant improvements. - * Name consistency across modules (we’ve gotten better at this, but we still have many deviations). - * Declarative operation (not CRUD)--this makes it easy for a user not to care what the existing state is, just about the final state. ``started/stopped``, ``present/absent``--don't overload options too much. It is preferable to add a new, simple option than to add choices/states that don't fit with existing ones. - * Keep options small, having them take large data structures might save us a few tasks, but adds a complex requirement that we cannot easily validate before passing on to the module. - * Allow an "expert mode". This may sound like the absolute opposite of the previous one, but it is always best to let expert users deal with complex data. This requires different modules in some cases, so that you end up having one (1) expert module and several 'piecemeal' ones (ec2_vpc_net?). The reason for this is not, as many users express, because it allows a single task and keeps plays small (which just moves the data complexity into vars files, leaving you with a slightly different structure in another YAML file). It does, however, allow for a more 'atomic' operation against the underlying APIs and services. -* Informative responses: Please note, that for >= 2.0, it is required that return data to be documented. - * Always return useful data, even when there is no change. - * Be consistent about returns (some modules are too random), unless it is detrimental to the state/action. - * Make returns reusable--most of the time you don't want to read it, but you do want to process it and re-purpose it. - * Return diff if in diff mode. This is not required for all modules, as it won't make sense for certain ones, but please attempt to include this when applicable). -* Code: This applies to all code in general, but often seems to be missing from modules, so please keep the following in mind as you work. - * Validate upfront--fail fast and return useful and clear error messages. - * Defensive programming--modules should be designed simply enough that this should be easy. Modules should always handle errors gracefully and avoid direct stacktraces. Ansible deals with this better in 2.0 and returns them in the results. - * Fail predictably--if we must fail, do it in a way that is the most expected. Either mimic the underlying tool or the general way the system works. - * Modules should not do the job of other modules, that is what roles are for. Less magic is more. - * Don't reinvent the wheel. Part of the problem is that code sharing is not that easy nor documented, we also need to expand our base functions to provide common patterns (retry, throttling, etc). - * Support check mode. This is not required for all modules, as it won't make sense for certain ones, but please attempt to include this when applicable). For more information, refer to :ref:`check_mode_drift` and :ref:`check_mode_dry`. -* Exceptions: The module must handle them. (exceptions are bugs) - * Give out useful messages on what you were doing and you can add the exception message to that. - * Avoid catchall exceptions, they are not very useful unless the underlying API gives very good error messages pertaining the attempted action. -* Module-dependent guidelines: Additional module guidelines may exist for certain families of modules. - * Be sure to check out the modules themselves for additional information. - * Amazon: https://github.com/ansible/ansible-modules-extras/blob/devel/cloud/amazon/GUIDELINES.md - * Modules should make use of the "extends_documentation_fragment" to ensure documentation available. For example, the AWS module should include:: - - extends_documentation_fragment: - - aws - - ec2 - -* The module must not use sys.exit() --> use fail_json() from the module object. -* Import custom packages in try/except and handled with fail_json() in main() e.g.:: - - try: - import foo - HAS_LIB=True - except: - HAS_LIB=False - -* The return structure should be consistent, even if NA/None are used for keys normally returned under other options. -* Are module actions idempotent? If not document in the descriptions or the notes. -* Import module snippets `from ansible.module_utils.basic import *` at the bottom, conserves line numbers for debugging. -* The module must have a `main` function that wraps the normal execution. -* Call your :func:`main` from a conditional so that it would be possible to - import them into unittests in the future example:: - - if __name__ == '__main__': - main() - -* Try to normalize parameters with other modules, you can have aliases for when user is more familiar with underlying API name for the option -* Being pep8 compliant is nice, but not a requirement. Specifically, the 80 column limit now hinders readability more that it improves it -* Avoid '`action`/`command`', they are imperative and not declarative, there are other ways to express the same thing -* Do not add `list` or `info` state options to an existing module - create a new `_facts` module. -* If you are asking 'how can I have a module execute other modules' ... you want to write a role -* Return values must be able to be serialized as json via the python stdlib - json library. basic python types (strings, int, dicts, lists, etc) are - serializable. A common pitfall is to try returning an object via - exit_json(). Instead, convert the fields you need from the object into the - fields of a dictionary and return the dictionary. -* When fetching URLs, please use either fetch_url or open_url from ansible.module_utils.urls - rather than urllib2; urllib2 does not natively verify TLS certificates and so is insecure for https. -* facts modules must return facts in the ansible_facts field of the result - dictionary. :ref:`module_provided_facts` -* modules that are purely about fact gathering need to implement check_mode. - they should not cause any changes anyway so it should be as simple as adding - check_mode=True when instantiating AnsibleModule. (The reason is that - playbooks which conditionalize based on fact information will only - conditionalize correctly in check_mode if the facts are returned in - check_mode). -* Basic auth: module_utils.api has some helpers for doing basic auth with - module_utils.urls.fetch_url(). If you use those you may find you also want - to fallback on environment variables for default values. If you do that, - be sure to use non-generic environment variables (like - :envvar:`API__USERNAME`). Using generic environment variables - like :envvar:`API_USERNAME` would conflict between modules. - -Windows modules checklist -````````````````````````` -* Favour native powershell and .net ways of doing things over calls to COM libraries or calls to native executables which may or may not be present in all versions of windows -* modules are in powershell (.ps1 files) but the docs reside in same name python file (.py) -* look at ansible/lib/ansible/module_utils/powershell.ps1 for common code, avoid duplication -* Ansible uses strictmode version 2.0 so be sure to test with that enabled -* start with:: - - #!powershell - - then:: - - - - then:: - - # WANT_JSON - # POWERSHELL_COMMON - - then, to parse all arguments into a variable modules generally use:: - - $params = Parse-Args $args - -* Arguments: - * Try and use state present and state absent like other modules - * You need to check that all your mandatory args are present. You can do this using the builtin Get-AnsibleParam function. - * Required arguments:: - - $package = Get-AnsibleParam -obj $params -name name -failifempty $true - - * Required arguments with name validation:: - - $state = Get-AnsibleParam -obj $params -name "State" -ValidateSet "Present","Absent" -resultobj $resultobj -failifempty $true - - * Optional arguments with name validation:: - - $state = Get-AnsibleParam -obj $params -name "State" -default "Present" -ValidateSet "Present","Absent" - - * the If "FailIfEmpty" is true, the resultobj parameter is used to specify the object returned to fail-json. You can also override the default message - using $emptyattributefailmessage (for missing required attributes) and $ValidateSetErrorMessage (for attribute validation errors) - * Look at existing modules for more examples of argument checking. - -* Results - * The result object should always contain an attribute called changed set to either $true or $false - * Create your result object like this:: - - $result = New-Object psobject @{ - changed = $false - other_result_attribute = $some_value - }; - - If all is well, exit with a - Exit-Json $result - - * Ensure anything you return, including errors can be converted to json. - * Be aware that because exception messages could contain almost anything. - * ConvertTo-Json will fail if it encounters a trailing \ in a string. - * If all is not well use Fail-Json to exit. - -* Have you tested for powershell 3.0 and 4.0 compliance? - -Deprecating and making module aliases -`````````````````````````````````````` - -Starting in 1.8, you can deprecate modules by renaming them with a preceding _, i.e. old_cloud.py to -_old_cloud.py. This keeps the module available, but hides it from the primary docs and listing. - -You can also rename modules and keep an alias to the old name by using a symlink that starts with _. -This example allows the stat module to be called with fileinfo, making the following examples equivalent:: - - EXAMPLES = ''' - ln -s stat.py _fileinfo.py - ansible -m stat -a "path=/tmp" localhost - ansible -m fileinfo -a "path=/tmp" localhost - ''' - - -.. seealso:: - - :doc:`modules` - Learn about available modules - :doc:`developing_plugins` - Learn about developing plugins - :doc:`developing_api` - Learn about the Python API for playbook and task execution - `GitHub Core modules directory `_ - Browse source of core modules - `Github Extras modules directory `_ - Browse source of extras modules. - `Mailing List `_ - Development mailing list - `irc.freenode.net `_ - #ansible IRC chat channel diff --git a/docs/docsite/rst/faq.rst b/docs/docsite/rst/faq.rst index d567fd8af9d..7881d693c13 100644 --- a/docs/docsite/rst/faq.rst +++ b/docs/docsite/rst/faq.rst @@ -25,19 +25,22 @@ Setting inventory variables in the inventory file is the easiest way. .. include:: ../rst_common/ansible_ssh_changes_note.rst -For instance, suppose these hosts have different usernames and ports:: +For instance, suppose these hosts have different usernames and ports: + +.. code-block:: ini [webservers] asdf.example.com ansible_port=5000 ansible_user=alice jkl.example.com ansible_port=5001 ansible_user=bob -You can also dictate the connection type to be used, if you want:: +You can also dictate the connection type to be used, if you want: + +.. code-block:: ini [testcluster] localhost ansible_connection=local /path/to/chroot1 ansible_connection=chroot - foo.example.com - bar.example.com + foo.example.com ansible_connection=paramiko You may also wish to keep these in group variables instead, or file them in a group_vars/ file. See the rest of the documentation for more information about how to organize variables. @@ -67,7 +70,9 @@ How do I configure a jump host to access servers that I have no direct access to With Ansible 2, you can set a `ProxyCommand` in the `ansible_ssh_common_args` inventory variable. Any arguments specified in this variable are added to the sftp/scp/ssh command line when connecting -to the relevant host(s). Consider the following inventory group:: +to the relevant host(s). Consider the following inventory group: + +.. code-block:: ini [gatewayed] foo ansible_host=192.0.2.1 @@ -147,6 +152,8 @@ How do I disable cowsay? If cowsay is installed, Ansible takes it upon itself to make your day happier when running playbooks. If you decide that you would like to work in a professional cow-free environment, you can either uninstall cowsay, or set an environment variable:: +.. code-block:: shell-session + export ANSIBLE_NOCOWS=1 .. _browse_facts: @@ -154,7 +161,9 @@ that you would like to work in a professional cow-free environment, you can eith How do I see a list of all of the ansible\_ variables? ++++++++++++++++++++++++++++++++++++++++++++++++++++++ -Ansible by default gathers "facts" about the machines under management, and these facts can be accessed in Playbooks and in templates. To see a list of all of the facts that are available about a machine, you can run the "setup" module as an ad-hoc action:: +Ansible by default gathers "facts" about the machines under management, and these facts can be accessed in Playbooks and in templates. To see a list of all of the facts that are available about a machine, you can run the "setup" module as an ad-hoc action: + +.. code-block:: shell-session ansible -m setup hostname @@ -165,7 +174,9 @@ This will print out a dictionary of all of the facts that are available for that How do I see all the inventory vars defined for my host? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -You can see the resulting vars you define in inventory running the following command:: +You can see the resulting vars you define in inventory running the following command: + +.. code-block:: shell-session ansible -m debug -a "var=hostvars['hostname']" localhost @@ -189,7 +200,9 @@ If you need to access facts about these hosts, for instance, the IP address of e tasks: - debug: msg="doesn't matter what you do, just that they were talked to previously." -Then you can use the facts inside your template, like this:: +Then you can use the facts inside your template, like this: + +.. code-block:: jinja {% for host in groups['db_servers'] %} {{ hostvars[host]['ansible_eth0']['ipv4']['address'] }} @@ -201,7 +214,9 @@ How do I access a variable name programmatically? +++++++++++++++++++++++++++++++++++++++++++++++++ An example may come up where we need to get the ipv4 address of an arbitrary interface, where the interface to be used may be supplied -via a role parameter or other input. Variable names can be built by adding strings together, like so:: +via a role parameter or other input. Variable names can be built by adding strings together, like so: + +.. code-block:: jinja {{ hostvars[inventory_hostname]['ansible_' + which_interface]['ipv4']['address'] }} @@ -218,7 +233,9 @@ are using dynamic inventory, which host is the 'first' may not be consistent, so was static and predictable. (If you are using :doc:`tower`, it will use database order, so this isn't a problem even if you are using cloud based inventory scripts). -Anyway, here's the trick:: +Anyway, here's the trick: + +.. code-block:: jinja {{ hostvars[groups['webservers'][0]]['ansible_eth0']['ipv4']['address'] }} @@ -253,7 +270,9 @@ environment variable on management machine:: If you need to set environment variables, see the Advanced Playbooks section about environments. -Ansible 1.4 will also make remote environment variables available via facts in the 'ansible_env' variable:: +Ansible 1.4 will also make remote environment variables available via facts in the 'ansible_env' variable: + +.. code-block:: jinja {{ ansible_env.SOME_VARIABLE }} @@ -262,17 +281,23 @@ Ansible 1.4 will also make remote environment variables available via facts in t How do I generate crypted passwords for the user module? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -The mkpasswd utility that is available on most Linux systems is a great option:: +The mkpasswd utility that is available on most Linux systems is a great option: + +.. code-block:: shell-session mkpasswd --method=sha-512 If this utility is not installed on your system (e.g. you are using OS X) then you can still easily generate these passwords using Python. First, ensure that the `Passlib `_ -password hashing library is installed:: +password hashing library is installed: + +.. code-block:: shell-session pip install passlib -Once the library is ready, SHA512 password values can then be generated as follows:: +Once the library is ready, SHA512 password values can then be generated as follows: + +.. code-block:: shell-session python -c "from passlib.hash import sha512_crypt; import getpass; print sha512_crypt.encrypt(getpass.getpass())" @@ -343,11 +368,15 @@ so `when:`, `failed_when:` and `changed_when:` are always templated and you shou In most other cases you should always use the brackets, even if previouslly you could use variables without specifying (like `with_` clauses), as this made it hard to distinguish between an undefined variable and a string. -Another rule is 'moustaches don't stack'. We often see this:: +Another rule is 'moustaches don't stack'. We often see this: + +.. code-block:: jinja {{ somevar_{{other_var}} }} -The above DOES NOT WORK, if you need to use a dynamic variable use the hostvars or vars dictionary as appropriate:: +The above DOES NOT WORK, if you need to use a dynamic variable use the hostvars or vars dictionary as appropriate: + +.. code-block:: jinja {{ hostvars[inventory_hostname]['somevar_' + other_var] }} diff --git a/docs/docsite/rst/guide_cloudstack.rst b/docs/docsite/rst/guide_cloudstack.rst index 7cb22f04789..c181651c218 100644 --- a/docs/docsite/rst/guide_cloudstack.rst +++ b/docs/docsite/rst/guide_cloudstack.rst @@ -144,7 +144,7 @@ Our CloudStack cloud has an advanced networking setup, we would like to provisio This is how our inventory looks like: -.. code-block:: ini +.. code-block:: none [cloud-vm:children] webserver @@ -249,7 +249,7 @@ A basic networking CloudStack setup is slightly different: Every VM gets a publi This is how our inventory looks like: -.. code-block:: ini +.. code-block:: none [cloud-vm:children] webserver diff --git a/docs/docsite/rst/guide_rolling_upgrade.rst b/docs/docsite/rst/guide_rolling_upgrade.rst index 760d839ad30..c4728029266 100644 --- a/docs/docsite/rst/guide_rolling_upgrade.rst +++ b/docs/docsite/rst/guide_rolling_upgrade.rst @@ -156,7 +156,9 @@ You can also use these variables in templates, like this, in ``roles/common/temp You can see that the variable substitution syntax of {{ and }} is the same for both templates and variables. The syntax inside the curly braces is Jinja2, and you can do all sorts of operations and apply different filters to the data inside. In templates, you can also use for loops and if statements to handle more complex situations, -like this, in ``roles/common/templates/iptables.j2``:: +like this, in ``roles/common/templates/iptables.j2``: + +.. code-block:: jinja {% if inventory_hostname in groups['dbservers'] %} -A INPUT -p tcp --dport 3306 -j ACCEPT @@ -165,7 +167,9 @@ like this, in ``roles/common/templates/iptables.j2``:: This is testing to see if the inventory name of the machine we're currently operating on (``inventory_hostname``) exists in the inventory group ``dbservers``. If so, that machine will get an iptables ACCEPT line for port 3306. -Here's another example, from the same template:: +Here's another example, from the same template: + +.. code-block:: jinja {% for host in groups['monitoring'] %} -A INPUT -p tcp -s {{ hostvars[host].ansible_default_ipv4.address }} --dport 5666 -j ACCEPT diff --git a/docs/docsite/rst/intro_bsd.rst b/docs/docsite/rst/intro_bsd.rst index d977c9eeadb..1fdd9d2ec71 100644 --- a/docs/docsite/rst/intro_bsd.rst +++ b/docs/docsite/rst/intro_bsd.rst @@ -82,7 +82,7 @@ Please feel free to report any issues or incompatibilities you discover with BSD Examples of basic commands :doc:`playbooks` Learning ansible's configuration management language - :doc:`developing_modules` + :doc:`dev_guide/developing_modules` How to write modules `Mailing List `_ Questions? Help? Ideas? Stop by the list on Google Groups diff --git a/docs/docsite/rst/intro_configuration.rst b/docs/docsite/rst/intro_configuration.rst index d008cf4a96a..10663f3cfdf 100644 --- a/docs/docsite/rst/intro_configuration.rst +++ b/docs/docsite/rst/intro_configuration.rst @@ -82,7 +82,7 @@ different locations:: action_plugins = ~/.ansible/plugins/action_plugins/:/usr/share/ansible_plugins/action_plugins -Most users will not need to use this feature. See :doc:`developing_plugins` for more details. +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details. .. _allow_world_readable_tmpfiles: @@ -208,7 +208,7 @@ different locations:: callback_plugins = ~/.ansible/plugins/callback:/usr/share/ansible/plugins/callback -Most users will not need to use this feature. See :doc:`developing_plugins` for more details +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details .. _callback_whitelist: @@ -259,7 +259,7 @@ different locations:: connection_plugins = ~/.ansible/plugins/connection_plugins/:/usr/share/ansible_plugins/connection_plugins -Most users will not need to use this feature. See :doc:`developing_plugins` for more details +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details .. _deprecation_warnings: @@ -349,7 +349,7 @@ different locations:: filter_plugins = ~/.ansible/plugins/filter_plugins/:/usr/share/ansible_plugins/filter_plugins -Most users will not need to use this feature. See :doc:`developing_plugins` for more details +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details .. _force_color: @@ -555,7 +555,7 @@ different locations:: lookup_plugins = ~/.ansible/plugins/lookup_plugins/:/usr/share/ansible_plugins/lookup_plugins -Most users will not need to use this feature. See :doc:`developing_plugins` for more details +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details .. _merge_multiple_cli_tags: @@ -779,7 +779,7 @@ different locations:: strategy_plugins = ~/.ansible/plugins/strategy_plugins/:/usr/share/ansible_plugins/strategy_plugins -Most users will not need to use this feature. See :doc:`developing_plugins` for more details +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details .. _cfg_strategy: @@ -868,7 +868,7 @@ different locations:: vars_plugins = ~/.ansible/plugins/vars_plugins/:/usr/share/ansible_plugins/vars_plugins -Most users will not need to use this feature. See :doc:`developing_plugins` for more details +Most users will not need to use this feature. See :doc:`dev_guide/developing_plugins` for more details .. _vault_password_file: diff --git a/docs/docsite/rst/intro_patterns.rst b/docs/docsite/rst/intro_patterns.rst index 31411731722..8b86955624a 100644 --- a/docs/docsite/rst/intro_patterns.rst +++ b/docs/docsite/rst/intro_patterns.rst @@ -59,7 +59,9 @@ is uncommonly used:: webservers:!{{excluded}}:&{{required}} You also don't have to manage by strictly defined groups. Individual host names, IPs and groups, can also be referenced using -wildcards:: +wildcards + +.. code-block:: none *.example.com *.com diff --git a/docs/docsite/rst/intro_windows.rst b/docs/docsite/rst/intro_windows.rst index 2cb1d7e5d41..53becd1b936 100644 --- a/docs/docsite/rst/intro_windows.rst +++ b/docs/docsite/rst/intro_windows.rst @@ -191,7 +191,7 @@ a version that is 3 or higher. You'll run this command again later though, to make sure everything is working. -Since 2.0, the following custom inventory variables are also supported for additional configuration of WinRM connections:: +Since 2.0, the following custom inventory variables are also supported for additional configuration of WinRM connections * ``ansible_winrm_scheme``: Specify the connection scheme (``http`` or ``https``) to use for the WinRM connection. Ansible uses ``https`` by default unless the port is 5985. * ``ansible_winrm_path``: Specify an alternate path to the WinRM endpoint. Ansible uses ``/wsman`` by default. @@ -397,7 +397,7 @@ form of new modules, tweaks to existing modules, documentation, or something els .. seealso:: - :doc:`developing_modules` + :doc:`dev_guide/developing_modules` How to write modules :doc:`playbooks` Learning Ansible's configuration management language diff --git a/docs/docsite/rst/modules.rst b/docs/docsite/rst/modules.rst index 6988fac8dce..11473b7c2a2 100644 --- a/docs/docsite/rst/modules.rst +++ b/docs/docsite/rst/modules.rst @@ -23,9 +23,9 @@ like services, packages, or files (anything really), or handle executing system Examples of using modules in /usr/bin/ansible :doc:`playbooks` Examples of using modules with /usr/bin/ansible-playbook - :doc:`developing_modules` + :doc:`dev_guide/developing_modules` How to write your own modules - :doc:`developing_api` + :doc:`dev_guide/developing_api` Examples of using modules with the Python API `Mailing List `_ Questions? Help? Ideas? Stop by the list on Google Groups diff --git a/docs/docsite/rst/modules_extra.rst b/docs/docsite/rst/modules_extra.rst index bcfdc629596..18816695605 100644 --- a/docs/docsite/rst/modules_extra.rst +++ b/docs/docsite/rst/modules_extra.rst @@ -18,5 +18,5 @@ Development oriented topics should instead use the similar `ansible-devel google Documentation updates for this module can also be edited directly in the module and by submitting a pull request to the module source code, just look for the "DOCUMENTATION" block in the source tree. -For help in developing on modules, should you be so inclined, please read :doc:`community`, :doc:`developing_test_pr` and :doc:`developing_modules`. +For help in developing on modules, should you be so inclined, please read :doc:`community`, :doc:`dev_guide/developing_test_pr` and :doc:`dev_guide/developing_modules`. diff --git a/docs/docsite/rst/modules_intro.rst b/docs/docsite/rst/modules_intro.rst index 8b9027b5223..a657eb513c8 100644 --- a/docs/docsite/rst/modules_intro.rst +++ b/docs/docsite/rst/modules_intro.rst @@ -55,9 +55,9 @@ A list of all installed modules is also available:: Examples of using modules in /usr/bin/ansible :doc:`playbooks` Examples of using modules with /usr/bin/ansible-playbook - :doc:`developing_modules` + :doc:`dev_guide/developing_modules` How to write your own modules - :doc:`developing_api` + :doc:`dev_guide/developing_api` Examples of using modules with the Python API `Mailing List `_ Questions? Help? Ideas? Stop by the list on Google Groups diff --git a/docs/docsite/rst/playbooks.rst b/docs/docsite/rst/playbooks.rst index 97a9f40464a..8afff023e6c 100644 --- a/docs/docsite/rst/playbooks.rst +++ b/docs/docsite/rst/playbooks.rst @@ -22,6 +22,7 @@ It is recommended to look at `Example Playbooks 'c' + "{{ ['a','b','c']|random }}" + # => 'c' To get a random number from 0 to supplied end:: - {{ 59 |random}} * * * * root /script/from/cron + "{{ 59 |random}} * * * * root /script/from/cron" + # => '21 * * * * root /script/from/cron' Get a random number from 0 to 100 but in steps of 10:: - {{ 100 |random(step=10) }} => 70 + {{ 100 |random(step=10) }} + # => 70 Get a random number from 1 to 100 but in steps of 10:: - {{ 100 |random(1, 10) }} => 31 - {{ 100 |random(start=1, step=10) }} => 51 + {{ 100 |random(1, 10) }} + # => 31 + {{ 100 |random(start=1, step=10) }} + # => 51 As of Ansible version 2.3, it's also possible to initialize the random number generator from a seed. This way, you can create random-but-idempotent numbers:: - {{ 59 |random(seed=inventory_hostname) }} * * * * root /script/from/cron + "{{ 59 |random(seed=inventory_hostname) }} * * * * root /script/from/cron" Shuffle Filter @@ -182,12 +187,15 @@ This filter will randomize an existing list, giving a different order every invo To get a random list from an existing list:: - {{ ['a','b','c']|shuffle }} => ['c','a','b'] - {{ ['a','b','c']|shuffle }} => ['b','c','a'] + {{ ['a','b','c']|shuffle }} + # => ['c','a','b'] + {{ ['a','b','c']|shuffle }} + # => ['b','c','a'] As of Ansible version 2.3, it's also possible to shuffle a list idempotent. All you need is a seed.:: - {{ ['a','b','c']|shuffle(seed=inventory_hostname) }} => ['b','a','c'] + {{ ['a','b','c']|shuffle(seed=inventory_hostname) }} + # => ['b','a','c'] note that when used with a non 'listable' item it is a noop, otherwise it always returns a list @@ -360,7 +368,9 @@ The resulting hash would be:: The filter also accepts an optional `recursive=True` parameter to not only override keys in the first hash, but also recurse into nested -hashes and merge their keys too:: +hashes and merge their keys too + +.. code-block:: jinja {{ {'a':{'foo':1, 'bar':2}, 'b':2}|combine({'a':{'bar':3, 'baz':4}}, recursive=True) }} @@ -441,7 +451,9 @@ It is also possible to fully customize the comment style:: {{ "Custom style" | comment('plain', prefix='#######\n#', postfix='#\n#######\n ###\n #') }} -That will create the following output:: +That will create the following output: + +.. code-block:: sh ####### # @@ -453,7 +465,9 @@ That will create the following output:: The filter can also be applied to any Ansible variable. For example to make the output of the ``ansible_managed`` variable more readable, we can -change the definition in the ``ansible.cfg`` file to this:: +change the definition in the ``ansible.cfg`` file to this: + +.. code-block:: jinja [defaults] @@ -467,7 +481,9 @@ and then use the variable with the `comment` filter:: {{ ansible_managed | comment }} -which will produce this output:: +which will produce this output: + +.. code-block:: sh # # This file is managed by Ansible. @@ -478,6 +494,17 @@ which will produce this output:: # host: myhost # +ipaddr() filters +---------------- + +A number of filters are available, including: + +* `ipaddr` +* `ipv4` +* `ipv6` + +See :doc:`playbooks_filters_ipaddr` + .. _other_useful_filters: Other Useful Filters diff --git a/docs/docsite/rst/playbooks_filters_ipaddr.rst b/docs/docsite/rst/playbooks_filters_ipaddr.rst index d5f9391db81..c76b921c76e 100644 --- a/docs/docsite/rst/playbooks_filters_ipaddr.rst +++ b/docs/docsite/rst/playbooks_filters_ipaddr.rst @@ -29,7 +29,9 @@ Basic tests ``ipaddr()`` is designed to return the input value if a query is True, and ``False`` if query is False. This way it can be very easily used in chained -filters. To use the filter, pass a string to it:: +filters. To use the filter, pass a string to it + +.. code-block:: none {{ '192.0.2.0' | ipaddr }} @@ -238,7 +240,9 @@ just subnets or single IP addresses:: # {{ test_list | ipaddr('host/prefix') }} ['2001:db8:deaf:be11::ef3/64', '192.0.2.48/24'] -In Debian-based systems, network configuration stored in ``/etc/network/interfaces`` file uses combination of IP address, network address, netmask and broadcast address to configure IPv4 network interface. We can get these values from single 'host/prefix' combination:: +In Debian-based systems, network configuration stored in ``/etc/network/interfaces`` file uses combination of IP address, network address, netmask and broadcast address to configure IPv4 network interface. We can get these values from a single 'host/prefix' combination: + +.. code-block:: jinja # Jinja2 template {% set ipv4_host = host_prefix | unique | ipv4('host/prefix') | first %} diff --git a/docs/docsite/rst/playbooks_intro.rst b/docs/docsite/rst/playbooks_intro.rst index 957b7f1ccd0..6be2c1826ec 100644 --- a/docs/docsite/rst/playbooks_intro.rst +++ b/docs/docsite/rst/playbooks_intro.rst @@ -478,7 +478,7 @@ can do this:: Hop back to the documentation index for a lot of special topics about playbooks :doc:`modules` Learn about available modules - :doc:`developing_modules` + :doc:`dev_guide/developing_modules` Learn how to extend Ansible by writing your own modules :doc:`intro_patterns` Learn about how to select hosts diff --git a/docs/docsite/rst/playbooks_lookups.rst b/docs/docsite/rst/playbooks_lookups.rst index 09c4c966028..3014555f877 100644 --- a/docs/docsite/rst/playbooks_lookups.rst +++ b/docs/docsite/rst/playbooks_lookups.rst @@ -65,7 +65,7 @@ This length can be changed by passing an extra parameter:: password="{{ lookup('password', 'credentials/' + client + '/' + tier + '/' + role + '/mysqlpassword length=15') }}" priv={{ client }}_{{ tier }}_{{ role }}.*:ALL - (...) + # (...) .. note:: If the file already exists, no data will be written to it. If the file has contents, those contents will be read in as the password. Empty files cause the password to return as an empty string. @@ -93,7 +93,7 @@ Starting in version 1.4, password accepts a "chars" parameter to allow defining password="{{ lookup('password', '/tmp/passwordfile chars=ascii_letters,digits,hexdigits,punctuation') }}" priv={{ client }}_{{ tier }}_{{ role }}.*:ALL - (...) + # (...) To enter comma use two commas ',,' somewhere - preferably at the end. Quotes and double quotes are not supported. @@ -156,7 +156,9 @@ This plugin retrieve the value on the right side after the equal sign ('=') of a given section ([section]). You can also read a property file which - in this case - does not contain section. -Here's a simple example of an INI file with user/password configuration:: +Here's a simple example of an INI file with user/password configuration: + +.. code-block:: ini [production] # My production information @@ -175,7 +177,9 @@ We can use the ``ini`` plugin to lookup user configuration:: - debug: msg="User in production is {{ lookup('ini', 'user section=production file=users.ini') }}" Another example for this plugin is for looking for a value on java properties. -Here's a simple properties we'll take as an example:: +Here's a simple properties we'll take as an example: + +.. code-block:: ini user.name=robert user.pass=somerandompassword @@ -213,12 +217,13 @@ The Credstash Lookup Credstash is a small utility for managing secrets using AWS's KMS and DynamoDB: https://github.com/LuminalOSS/credstash -First, you need to store your secrets with credstash:: +First, you need to store your secrets with credstash: +.. code-block:: shell-session - $ credstash put my-github-password secure123 + credstash put my-github-password secure123 - my-github-password has been stored + # my-github-password has been stored Example usage:: @@ -239,8 +244,8 @@ You can specify regions or tables to fetch secrets from:: - name: "Test credstash lookup plugin -- get the company's github password" debug: msg="Credstash lookup! {{ lookup('credstash', 'company-github-password', table='company-passwords') }}" - - + + If you use the context feature when putting your secret, you can get it by passing a dictionary to the context option like this:: --- @@ -401,7 +406,9 @@ sort no list [pymongo default] Sortin Please check https://api.mongodb.org/python/current/api/pymongo/collection.html?highlight=find#pymongo.collection.Collection.find for more detais. -Since there are too many parameters for this lookup method, below is a sample playbook which shows its usage and a nice way to feed the parameters:: +Since there are too many parameters for this lookup method, below is a sample playbook which shows its usage and a nice way to feed the parameters: + +.. code-block:: YAML+Jinja --- - hosts: all @@ -434,9 +441,10 @@ Since there are too many parameters for this lookup method, below is a sample pl -Sample output:: +Sample output: + +.. code-block:: shell-session - --- mdiez@batman:~/ansible$ ansible-playbook m.yml -i localhost.ini PLAY [all] ********************************************************************* diff --git a/docs/docsite/rst/playbooks_loops.rst b/docs/docsite/rst/playbooks_loops.rst index 35dccc380ea..68cbfa52d1b 100644 --- a/docs/docsite/rst/playbooks_loops.rst +++ b/docs/docsite/rst/playbooks_loops.rst @@ -112,7 +112,9 @@ Looping over Files - first_example_file - second_example_file -Assuming that ``first_example_file`` contained the text "hello" and ``second_example_file`` contained the text "world", this would result in:: +Assuming that ``first_example_file`` contained the text "hello" and ``second_example_file`` contained the text "world", this would result in: + +.. code-block:: shell-session TASK [debug msg={{ item }}] ****************************************************** ok: [localhost] => (item=hello) => { @@ -384,7 +386,9 @@ Using ini file with a loop `````````````````````````` .. versionadded:: 2.0 -The ini plugin can use regexp to retrieve a set of keys. As a consequence, we can loop over this set. Here is the ini file we'll use:: +The ini plugin can use regexp to retrieve a set of keys. As a consequence, we can loop over this set. Here is the ini file we'll use: + +.. code-block:: ini [section1] value1=section1/value1 @@ -640,7 +644,7 @@ for `item`:: Writing Your Own Iterators `````````````````````````` -While you ordinarily shouldn't have to, should you wish to write your own ways to loop over arbitrary data structures, you can read :doc:`developing_plugins` for some starter +While you ordinarily shouldn't have to, should you wish to write your own ways to loop over arbitrary data structures, you can read :doc:`dev_guide/developing_plugins` for some starter information. Each of the above features are implemented as plugins in ansible, so there are many implementations to reference. .. seealso:: diff --git a/docs/docsite/rst/playbooks_roles.rst b/docs/docsite/rst/playbooks_roles.rst index 9043e76c513..1ed043434d4 100644 --- a/docs/docsite/rst/playbooks_roles.rst +++ b/docs/docsite/rst/playbooks_roles.rst @@ -441,7 +441,7 @@ Embedding Modules and Plugins In Roles This is an advanced topic that should not be relevant for most users. -If you write a custom module (see :doc:`developing_modules`) or a plugin (see :doc:`developing_plugins`), you may wish to distribute it as part of a role. +If you write a custom module (see :doc:`dev_guide/developing_modules`) or a plugin (see :doc:`dev_guide/developing_plugins`), you may wish to distribute it as part of a role. Generally speaking, Ansible as a project is very interested in taking high-quality modules into ansible core for inclusion, so this shouldn't be the norm, but it's quite easy to do. A good example for this is if you worked at a company called AcmeWidgets, and wrote an internal module that helped configure your internal software, and you wanted other @@ -508,7 +508,7 @@ Read the "About" page on the Galaxy site for more information. Loops in playbooks :doc:`modules` Learn about available modules - :doc:`developing_modules` + :doc:`dev_guide/developing_modules` Learn how to extend Ansible by writing your own modules `GitHub Ansible examples `_ Complete playbook files from the GitHub project source diff --git a/docs/docsite/rst/playbooks_variables.rst b/docs/docsite/rst/playbooks_variables.rst index fdf7681056b..2fa2a8a8d8a 100644 --- a/docs/docsite/rst/playbooks_variables.rst +++ b/docs/docsite/rst/playbooks_variables.rst @@ -172,53 +172,55 @@ To see what information is available, try the following:: ansible hostname -m setup -This will return a ginormous amount of variable data, which may look like this, as taken from Ansible 1.4 on a Ubuntu 12.04 system:: +This will return a ginormous amount of variable data, which may look like this, as taken from Ansible 1.4 on a Ubuntu 12.04 system + +.. code-block: none "ansible_all_ipv4_addresses": [ "REDACTED IP ADDRESS" ], "ansible_all_ipv6_addresses": [ "REDACTED IPV6 ADDRESS" - ], - "ansible_architecture": "x86_64", - "ansible_bios_date": "09/20/2012", - "ansible_bios_version": "6.00", + ], + "ansible_architecture": "x86_64", + "ansible_bios_date": "09/20/2012", + "ansible_bios_version": "6.00", "ansible_cmdline": { - "BOOT_IMAGE": "/boot/vmlinuz-3.5.0-23-generic", - "quiet": true, - "ro": true, - "root": "UUID=4195bff4-e157-4e41-8701-e93f0aec9e22", + "BOOT_IMAGE": "/boot/vmlinuz-3.5.0-23-generic", + "quiet": true, + "ro": true, + "root": "UUID=4195bff4-e157-4e41-8701-e93f0aec9e22", "splash": true - }, + }, "ansible_date_time": { - "date": "2013-10-02", - "day": "02", - "epoch": "1380756810", - "hour": "19", - "iso8601": "2013-10-02T23:33:30Z", - "iso8601_micro": "2013-10-02T23:33:30.036070Z", - "minute": "33", - "month": "10", - "second": "30", - "time": "19:33:30", - "tz": "EDT", + "date": "2013-10-02", + "day": "02", + "epoch": "1380756810", + "hour": "19", + "iso8601": "2013-10-02T23:33:30Z", + "iso8601_micro": "2013-10-02T23:33:30.036070Z", + "minute": "33", + "month": "10", + "second": "30", + "time": "19:33:30", + "tz": "EDT", "year": "2013" }, "ansible_default_ipv4": { - "address": "REDACTED", - "alias": "eth0", - "gateway": "REDACTED", - "interface": "eth0", - "macaddress": "REDACTED", - "mtu": 1500, - "netmask": "255.255.255.0", - "network": "REDACTED", + "address": "REDACTED", + "alias": "eth0", + "gateway": "REDACTED", + "interface": "eth0", + "macaddress": "REDACTED", + "mtu": 1500, + "netmask": "255.255.255.0", + "network": "REDACTED", "type": "ether" }, - "ansible_default_ipv6": {}, + "ansible_default_ipv6": {}, "ansible_devices": { "fd0": { - "holders": [], + "holders": [], "host": "", "model": null, "partitions": {}, @@ -604,7 +606,7 @@ While it's mentioned elsewhere in that document too, here's a quick syntax examp Registered variables are valid on the host the remainder of the playbook run, which is the same as the lifetime of "facts" in Ansible. Effectively registered variables are just like facts. -When using ``register`` with a loop the data structure placed in the variable during a loop, will contain a ``results`` attribute, that is a list of all responses from the module. For a more in-depth example of how this works, see the :doc:`playbook_loops` section on using register with a loop. +When using ``register`` with a loop the data structure placed in the variable during a loop, will contain a ``results`` attribute, that is a list of all responses from the module. For a more in-depth example of how this works, see the :doc:`playbooks_loops` section on using register with a loop. .. note:: If a task fails or is skipped, the variable still is registered with a failure or skipped status, the only way to avoid registering a variable is using tags. @@ -646,20 +648,27 @@ assigned to another node, it's easy to do so within a template or even an action {{ hostvars['test.example.com']['ansible_distribution'] }} -Additionally, ``group_names`` is a list (array) of all the groups the current host is in. This can be used in templates using Jinja2 syntax to make template source files that vary based on the group membership (or role) of the host:: +Additionally, ``group_names`` is a list (array) of all the groups the current host is in. This can be used in templates using Jinja2 syntax to make template source files that vary based on the group membership (or role) of the host + +.. code-block:: jinja {% if 'webserver' in group_names %} # some part of a configuration file that only applies to webservers {% endif %} + ``groups`` is a list of all the groups (and hosts) in the inventory. This can be used to enumerate all hosts within a group. -For example:: +For example + +.. code-block:: jinja {% for host in groups['app_servers'] %} # something that applies to all app servers. {% endfor %} -A frequently used idiom is walking a group to find all IP addresses in that group:: +A frequently used idiom is walking a group to find all IP addresses in that group + +.. code-block:: jinja {% for host in groups['app_servers'] %} {{ hostvars[host]['ansible_eth0']['ipv4']['address'] }} diff --git a/docs/docsite/rst/porting_guide_2.0.rst b/docs/docsite/rst/porting_guide_2.0.rst index 21f4dd0249c..8ca817e6081 100644 --- a/docs/docsite/rst/porting_guide_2.0.rst +++ b/docs/docsite/rst/porting_guide_2.0.rst @@ -232,7 +232,9 @@ Although Ansible 2.0 provides a new callback API the old one continues to work for most callback plugins. However, if your callback plugin makes use of :attr:`self.playbook`, :attr:`self.play`, or :attr:`self.task` then you will have to store the values for these yourself as ansible no longer automatically -populates the callback with them. Here's a short snippet that shows you how:: +populates the callback with them. Here's a short snippet that shows you how + +.. code-block:: python import os from ansible.plugins.callback import CallbackBase @@ -289,7 +291,9 @@ You may find the following tips useful: Lookup plugins ~~~~~~~~~~~~~~ -As a simple example we are going to make a hybrid ``fileglob`` lookup plugin. The ``fileglob`` lookup plugin is pretty simple to understand:: +As a simple example we are going to make a hybrid ``fileglob`` lookup plugin. The ``fileglob`` lookup plugin is pretty simple to understand + +.. code-block:: python from __future__ import (absolute_import, division, print_function) __metaclass__ = type @@ -376,6 +380,5 @@ Connection plugins Porting custom scripts ---------------------- -Custom scripts that used the ``ansible.runner.Runner`` API in 1.x have to be ported in 2.x. Please refer to: -https://github.com/ansible/ansible/blob/devel/docsite/rst/developing_api.rst +Custom scripts that used the ``ansible.runner.Runner`` API in 1.x have to be ported in 2.x. Please refer to: :doc:`dev_guide//developing_api` diff --git a/docs/docsite/rst/roadmap/ROADMAP_2_1.rst b/docs/docsite/rst/roadmap/ROADMAP_2_1.rst index 79c03cafdd6..aa025edcd2b 100644 --- a/docs/docsite/rst/roadmap/ROADMAP_2_1.rst +++ b/docs/docsite/rst/roadmap/ROADMAP_2_1.rst @@ -1,6 +1,10 @@ -========================================== -2.1 Roadmap, Targeted for the End of April -========================================== +**************** +Ansible Core 2.1 +**************** +************* +Target: April +************* + - Windows, General - Figuring out privilege escalation (runas w/ username/password) - Implement kerberos encryption over http diff --git a/docs/docsite/rst/roadmap/ROADMAP_2_2.rst b/docs/docsite/rst/roadmap/ROADMAP_2_2.rst index de92f393b3a..d1718c41b5c 100644 --- a/docs/docsite/rst/roadmap/ROADMAP_2_2.rst +++ b/docs/docsite/rst/roadmap/ROADMAP_2_2.rst @@ -4,6 +4,7 @@ Ansible Core 2.2 ********************** Target: September 2016 ********************** + - **Docker** (lead by Chris Houseknecht) - Docker_network: **done** @@ -109,37 +110,42 @@ Target: September 2016 a critical task and one that has so much work and so many moving parts that we don’t expect this to be complete by the 2.2 release.  Toshio will lead this overall effort. + - Motivation: - Ubuntu LTS (16.04) already ships without python2.  RHEL8 is coming which is also expected to be python3 based. These considerations make this high priority. - Ansible users are getting restless: https://groups.google.com/forum/#!topic/ansible-project/DUKzTho3OCI - This is probably going to take multiple releases to complete; need to get started now + - Baselines: - We're targeting Python-3.5 and above. - Goals for 2.2: + - Tech preview level of support - Controller-side code can run on Python3 - - Update: Essential features have been shown to work on Python3. - Currently all unittests and all but three integration tests are - passing on Python3. Code has not been line-by-line audited so bugs - remain but can be treated as bugs, not as massive, invasive new features. - - Almost all of our deps have been ported: - - The base deps in setup.py are ported: ['paramiko', 'jinja2', "PyYAML", 'setuptools', 'pycrypto >= 2.6'] - - python-six from the rpm spec file has been ported - - Python-keyczar from the rpm spec file is not. - - Strategy: removing keyczar when we drop accelerate for 2.3. Print deprecation in 2.1. + - Update: Essential features have been shown to work on Python3. + Currently all unittests and all but three integration tests are + passing on Python3. Code has not been line-by-line audited so bugs + remain but can be treated as bugs, not as massive, invasive new features. + - Almost all of our deps have been ported: + - The base deps in setup.py are ported: ['paramiko', 'jinja2', "PyYAML", 'setuptools', 'pycrypto >= 2.6'] + - python-six from the rpm spec file has been ported + - Python-keyczar from the rpm spec file is not. + - Strategy: removing keyczar when we drop accelerate for 2.3. Print deprecation in 2.1. - Module_utils ported to dual python3/python2(2.4 for much of it, python2.6 for specific things) - - Update: Mostly done. Also not line-by-line audited but the unittests - and integration tests do show that the most use functionality is working. + **Mostly done:** Also not line-by-line audited but the unittests + and integration tests do show that the most use functionality is working. - Add module_utils files to help port - Update: copy of the six library (v1.4.1 for python2.4 compat) and unicode helpers are here (ansible.module_utils._text.{to_bytes,to_text,to_native}) - A few basic modules ported to python3 - Stat module best example module since it’s essential. - - Update: A handful of modules like stat have been line-by-line ported. They should work reliably with few python3-specific bugs. All but three integration tests pass which means that most essential modules are working to some extent on Python3. - - The three failing tests are: service, hg, and uri. - - Note, large swaths of the modules are not tested. The status of - these is unknown + - Update: + + - A handful of modules like stat have been line-by-line ported. They should work reliably with few python3-specific bugs. All but three integration tests pass which means that most essential modules are working to some extent on Python3. + - The three failing tests are: service, hg, and uri. + - Note, large swaths of the modules are not tested. The status of + these is unknown - All code should compile under Python3. - lib/ansible/* and all modules now compile under Python-3.5 @@ -160,20 +166,34 @@ Target: September 2016 - **CI Performance** Reduce time spent waiting on CI for PRs. Combination of optimizing existing Travis setup and offloading work to other services. Will be impacted by available budget. + **Done:** Most tests have been migrated from Travis to Shippable. + - **Core Module Test Organization** Relocate core module tests to ansible-modules-core to encourage inclusion of tests in core module PRs. + **Deferred:** Relocation of core module tests has been deferred due to proposed changes in `modules management `_. + - **Documentation** Expand documentation on setting up a development and test environment, as well as writing tests. The goal is to ease development for new contributors and encourage more testing, particularly with module contributions. - **Test Coverage** - Expand test coverage, particularly for CI. Being testing, this is open ended. Will be impacted by available budget. + + - Expand test coverage, particularly for CI. Being testing, this is open ended. Will be impacted by available budget. + **Done:** Module PRs now run integration tests for the module(s) being changed. + - Python 3 - Run integration tests using Python 3 on CI with tagging for those which should pass, so we can track progress and detect regressions. + **Done:** Integration tests now run on Shippable using a Ubuntu 16.04 docker image with only Python 3 installed. + - Windows - Create framework for running Windows integration tests, ideally both locally and on CI. + **Done:** Windows integration tests now run on Shippable. + - FreeBSD - Include FreeBSD in CI coverage. Not originally on the roadmap, this is an intermediary step for CI coverage for OS X. + **Done:** FreeBSD integration tests now run on Shippable. + - OS X - Include OS X in CI coverage. + **Done:** OS X integration tests now run on Shippable. diff --git a/docs/docsite/rst/roadmap/ROADMAP_2_3.rst b/docs/docsite/rst/roadmap/ROADMAP_2_3.rst index 2974dfd5d95..27584bf6ec4 100644 --- a/docs/docsite/rst/roadmap/ROADMAP_2_3.rst +++ b/docs/docsite/rst/roadmap/ROADMAP_2_3.rst @@ -1,6 +1,6 @@ -***************************** -Ansible by Red Hat, Core 2.3 -***************************** +**************************** +Ansible by Red Hat, Core 2.3 +**************************** *************************** Target: February/March 2017 *************************** diff --git a/docs/docsite/rst/roadmap/ROADMAP_2_4.rst b/docs/docsite/rst/roadmap/ROADMAP_2_4.rst index 7a6cf80f5ea..ee40d9f5f92 100644 --- a/docs/docsite/rst/roadmap/ROADMAP_2_4.rst +++ b/docs/docsite/rst/roadmap/ROADMAP_2_4.rst @@ -1,7 +1,9 @@ +**************************** Ansible by Red Hat, Core 2.4 -============================ - -**Target Delivery: June/July 2017** +**************************** +********************** +Target: June/July 2017 +********************** This is meant to be a living document, and is by no means **FINAL** until stated otherwise in the document. @@ -11,7 +13,9 @@ stated otherwise in the document. - Extend the ability of the current config system by adding creating an ansible-config command and add the following: - Dump existing config settings + - Update / write a config entry + - Show available options (ini entry, yaml, env var, etc) - Proposal found in ansible/proposals issue `#35 `_.