From 8d03c2ddc4d3deaf65954fea8b02411943d94022 Mon Sep 17 00:00:00 2001 From: Alicia Cozine <879121+acozine@users.noreply.github.com> Date: Wed, 28 Apr 2021 11:10:29 -0500 Subject: [PATCH] Backport/2.11/docs1 (#74457) * Fix issue with version 3 in docs version list (#74089) Previously would subsitute the "3" in "s3" instead of the version location in the URL (cherry picked from commit 325ccf22fee9a56eb6bd0ed148bd2cffc7449922) * corrected epmhasis line (#74254) need to be careful when adding lines as you create a different offset (cherry picked from commit 7b39ee361df95df54ddcdda9fae687b7bdb28c6e) * correct how to install Docker SDK for Python (#74288) * docs: correct guide for the latest Docker SDK for Python Co-authored-by: Felix Fontein Co-authored-by: Sandra McCann Co-authored-by: Daeseok Youn Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com> (cherry picked from commit 9369bd6ca174f9f29d423d2879eeaaa55c1f1be1) * Provide results to examples (#73984) (cherry picked from commit 99a2b5f3006ebe1845565071099c4fe54b227590) * remove deprecated ansible.module_utils._text from documentation (#73211) According to comment in ansible.module_utils._text it is deprecated and should not be used. This is now reflected in the documentation. (cherry picked from commit 5e5bfa811666fbbf5326e68c1615861034521be7) * Update playbooks_filters.rst (#74242) ##### SUMMARY Make the `random` filter description more clear. (cherry picked from commit 5f391a72eec600421b3d597503217168141331c1) * Docs: Fix k8s_config_resource_name YAML example (#74129) The `name` key should be beneath `metadata`: (cherry picked from commit c9c84594d893dc8188a76d0c5b40d836a08dc29a) * Update lookup.rst (#73716) Document that users must pass `allow_unsafe=True` as an option in the lookup to allow templating, with a note about security implications. Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com> Co-authored-by: Sandra McCann (cherry picked from commit c0cc574e6562f872583bc5eca518f84b4f4cad84) * vmware: Add a note about known issue (#73273) Signed-off-by: Abhijeet Kasurde Co-authored-by: Sandra McCann Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com> (cherry picked from commit 6e56e72d9966999911b572fc2856a66beb48276f) Co-authored-by: Scott Sinclair <252082+pwae@users.noreply.github.com> Co-authored-by: Brian Coca Co-authored-by: daeseokyoun Co-authored-by: Baptiste Mille-Mathias Co-authored-by: schurzi Co-authored-by: yuri <1969yuri1969@gmail.com> Co-authored-by: ml <6209465+ml-@users.noreply.github.com> Co-authored-by: sry9681 Co-authored-by: Abhijeet Kasurde --- .../sphinx_rtd_theme/ansible_versions.html | 6 +- .../rst/dev_guide/developing_collections.rst | 2 +- .../rst/dev_guide/developing_plugins.rst | 4 +- .../rst/dev_guide/developing_python_3.rst | 10 ++-- .../docsite/rst/dev_guide/migrating_roles.rst | 2 +- .../testing/sanity/no-basestring.rst | 4 +- .../rst/dev_guide/testing_units_modules.rst | 4 +- docs/docsite/rst/plugins/lookup.rst | 1 + .../rst/scenario_guides/guide_docker.rst | 10 +++- .../vmware_scenarios/vmware_requirements.rst | 2 + .../user_guide/complex_data_manipulation.rst | 6 +- .../rst/user_guide/playbooks_filters.rst | 60 ++++++++++++++----- 12 files changed, 76 insertions(+), 35 deletions(-) diff --git a/docs/docsite/_themes/sphinx_rtd_theme/ansible_versions.html b/docs/docsite/_themes/sphinx_rtd_theme/ansible_versions.html index 3d3d7fc682d..6fa067f54dc 100644 --- a/docs/docsite/_themes/sphinx_rtd_theme/ansible_versions.html +++ b/docs/docsite/_themes/sphinx_rtd_theme/ansible_versions.html @@ -13,10 +13,10 @@ if ( "{{ slug }}" == "{{ current_version }}" ) { option.selected = true; } - if (current_url.search("{{ current_version }}") > -1) { - option.value = current_url.replace("{{ current_version }}","{{ slug }}"); + if (current_url.search("/{{ current_version }}/") > -1) { + option.value = current_url.replace("/{{ current_version }}/","/{{ slug }}/"); } else { - option.value = current_url.replace("latest","{{ slug }}"); + option.value = current_url.replace("/latest/","/{{ slug }}/"); } x.add(option); diff --git a/docs/docsite/rst/dev_guide/developing_collections.rst b/docs/docsite/rst/dev_guide/developing_collections.rst index 61067e60c76..8504f7e58f6 100644 --- a/docs/docsite/rst/dev_guide/developing_collections.rst +++ b/docs/docsite/rst/dev_guide/developing_collections.rst @@ -100,7 +100,7 @@ In the Python example the ``module_util`` in question is called ``qradar`` such .. code-block:: python from ansible.module_utils.basic import AnsibleModule - from ansible.module_utils._text import to_text + from ansible.module_utils.common.text.converters import to_text from ansible.module_utils.six.moves.urllib.parse import urlencode, quote_plus from ansible.module_utils.six.moves.urllib.error import HTTPError diff --git a/docs/docsite/rst/dev_guide/developing_plugins.rst b/docs/docsite/rst/dev_guide/developing_plugins.rst index 0e3ce3c5a94..38825fb36dc 100644 --- a/docs/docsite/rst/dev_guide/developing_plugins.rst +++ b/docs/docsite/rst/dev_guide/developing_plugins.rst @@ -29,7 +29,7 @@ You should return errors encountered during plugin execution by raising ``Ansibl .. code-block:: python - from ansible.module_utils._text import to_native + from ansible.module_utils.common.text.converters import to_native try: cause_an_exception() @@ -45,7 +45,7 @@ You must convert any strings returned by your plugin into Python's unicode type. .. code-block:: python - from ansible.module_utils._text import to_text + from ansible.module_utils.common.text.converters import to_text result_string = to_text(result_string) Plugin configuration & documentation standards diff --git a/docs/docsite/rst/dev_guide/developing_python_3.rst b/docs/docsite/rst/dev_guide/developing_python_3.rst index b9b80174a03..577e487acb1 100644 --- a/docs/docsite/rst/dev_guide/developing_python_3.rst +++ b/docs/docsite/rst/dev_guide/developing_python_3.rst @@ -116,7 +116,7 @@ to yield text but instead do the conversion explicitly ourselves. For example: .. code-block:: python - from ansible.module_utils._text import to_text + from ansible.module_utils.common.text.converters import to_text with open('filename-with-utf8-data.txt', 'rb') as my_file: b_data = my_file.read() @@ -136,7 +136,7 @@ Writing to files is the opposite process: .. code-block:: python - from ansible.module_utils._text import to_bytes + from ansible.module_utils.common.text.converters import to_bytes with open('filename.txt', 'wb') as my_file: my_file.write(to_bytes(some_text_string)) @@ -160,7 +160,7 @@ works on both versions: import os.path - from ansible.module_utils._text import to_bytes + from ansible.module_utils.common.text.converters import to_bytes filename = u'/var/tmp/くらとみ.txt' f = open(to_bytes(filename), 'wb') @@ -246,9 +246,9 @@ In ``module_utils`` code: * Functions that return strings **must** document whether they return strings of the same type as they were given or native strings. Module-utils functions are therefore often very defensive in nature. -They convert their string parameters into text (using ``ansible.module_utils._text.to_text``) +They convert their string parameters into text (using ``ansible.module_utils.common.text.converters.to_text``) at the beginning of the function, do their work, and then convert -the return values into the native string type (using ``ansible.module_utils._text.to_native``) +the return values into the native string type (using ``ansible.module_utils.common.text.converters.to_native``) or back to the string type that their parameters received. Tips, tricks, and idioms for Python 2/Python 3 compatibility diff --git a/docs/docsite/rst/dev_guide/migrating_roles.rst b/docs/docsite/rst/dev_guide/migrating_roles.rst index 2ed5f53e848..5ca134e558f 100644 --- a/docs/docsite/rst/dev_guide/migrating_roles.rst +++ b/docs/docsite/rst/dev_guide/migrating_roles.rst @@ -225,7 +225,7 @@ In the Python example the ``module_utils`` is ``helper`` and the :abbr:`FQCN (Fu .. code-block:: text from ansible.module_utils.basic import AnsibleModule - from ansible.module_utils._text import to_text + from ansible.module_utils.common.text.converters import to_text from ansible.module_utils.six.moves.urllib.parse import urlencode from ansible.module_utils.six.moves.urllib.error import HTTPError from ansible_collections.ansible_example.community.plugins.module_utils.helper import HelperRequest diff --git a/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst b/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst index f1b6ba92c94..f2fea137ebd 100644 --- a/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst +++ b/docs/docsite/rst/dev_guide/testing/sanity/no-basestring.rst @@ -7,5 +7,5 @@ from ``ansible.module_utils.six`` and then use ``isinstance(s, string_types)`` or ``isinstance(s, (binary_type, text_type))`` instead. If this is part of code to convert a string to a particular type, -``ansible.module_utils._text`` contains several functions that may be even -better for you: ``to_text``, ``to_bytes``, and ``to_native``. +``ansible.module_utils.common.text.converters`` contains several functions +that may be even better for you: ``to_text``, ``to_bytes``, and ``to_native``. diff --git a/docs/docsite/rst/dev_guide/testing_units_modules.rst b/docs/docsite/rst/dev_guide/testing_units_modules.rst index 88763eb0a14..97f0f1771cf 100644 --- a/docs/docsite/rst/dev_guide/testing_units_modules.rst +++ b/docs/docsite/rst/dev_guide/testing_units_modules.rst @@ -296,7 +296,7 @@ variable is set it will be treated as if the input came on ``STDIN`` to the modu import json from units.modules.utils import set_module_args - from ansible.module_utils._text import to_bytes + from ansible.module_utils.common.text.converters import to_bytes def test_already_registered(self): set_module_args({ @@ -388,7 +388,7 @@ mock for :meth:`Ansible.get_bin_path`:: from units.compat import unittest from units.compat.mock import patch from ansible.module_utils import basic - from ansible.module_utils._text import to_bytes + from ansible.module_utils.common.text.converters import to_bytes from ansible.modules.namespace import my_module diff --git a/docs/docsite/rst/plugins/lookup.rst b/docs/docsite/rst/plugins/lookup.rst index 21845d9357c..1ddbd32fdc1 100644 --- a/docs/docsite/rst/plugins/lookup.rst +++ b/docs/docsite/rst/plugins/lookup.rst @@ -13,6 +13,7 @@ Lookup plugins are an Ansible-specific extension to the Jinja2 templating langua - Lookups are executed with a working directory relative to the role or play, as opposed to local tasks, which are executed relative the executed script. - Pass ``wantlist=True`` to lookups to use in Jinja2 template "for" loops. + - By default, lookup return values are marked as unsafe for security reasons. If you trust the outside source your lookup accesses, pass ``allow_unsafe=True`` to allow Jinja2 templates to evaluate lookup values. .. warning:: - Some lookups pass arguments to a shell. When using variables from a remote/untrusted source, use the `|quote` filter to ensure safe usage. diff --git a/docs/docsite/rst/scenario_guides/guide_docker.rst b/docs/docsite/rst/scenario_guides/guide_docker.rst index c3f019bd6c8..b4be0ef2198 100644 --- a/docs/docsite/rst/scenario_guides/guide_docker.rst +++ b/docs/docsite/rst/scenario_guides/guide_docker.rst @@ -15,13 +15,19 @@ Most of the modules and plugins in community.docker require the `Docker SDK for Note that plugins (inventory plugins and connection plugins) are always executed in the context of Ansible itself. If you use a plugin that requires the Docker SDK for Python, you need to install it on the machine running ``ansible`` or ``ansible-playbook`` and for the same Python interpreter used by Ansible. To see which Python is used, run ``ansible --version``. -You can install the Docker SDK for Python for Python 2.7 or Python 3 as follows: +You can install the Docker SDK for Python for Python 3.6 or later as follows: .. code-block:: bash $ pip install docker -For Python 2.6, you need a version before 2.0. For these versions, the SDK was called ``docker-py``, so you need to install it as follows: +For Python 2.7, you need to use a version between 2.0.0 and 4.4.4 since the Python package for Docker removed support for Python 2.7 on 5.0.0. You can install the specific version of the Docker SDK for Python as follows: + +.. code-block:: bash + + $ pip install 'docker==4.4.4' + +For Python 2.6, you need a version before 2.0.0. For these versions, the SDK was called ``docker-py``, so you need to install it as follows: .. code-block:: bash diff --git a/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_requirements.rst b/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_requirements.rst index f6cb8459c17..e3eb4e746f7 100644 --- a/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_requirements.rst +++ b/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_requirements.rst @@ -60,3 +60,5 @@ For example, if ``/var/vmware/certs/vcenter1.crt`` is the SSL certificate for yo password: '{{ vcenter_password }}' environment: REQUESTS_CA_BUNDLE: /var/vmware/certs/vcenter1.crt + +There is a `known issue `_ in ``requests`` library (version 2) which you may want to consider when using this environment variable. Basically, setting ``REQUESTS_CA_BUNDLE`` environment variable on managed nodes overrides the ``validate_certs`` value. This may result in unexpected behavior while running the playbook. Please see `community.vmware issue 601 `_ and `vmware issue 254 `_ for more information. diff --git a/docs/docsite/rst/user_guide/complex_data_manipulation.rst b/docs/docsite/rst/user_guide/complex_data_manipulation.rst index 3f6da7b8051..fc4a16275e0 100644 --- a/docs/docsite/rst/user_guide/complex_data_manipulation.rst +++ b/docs/docsite/rst/user_guide/complex_data_manipulation.rst @@ -55,7 +55,7 @@ The Python equivalent code would be: There are several ways to do it in Ansible, this is just one example: .. code-block:: YAML+Jinja - :emphasize-lines: 3 + :emphasize-lines: 4 :caption: Way to extract matching keys from a list of dictionaries tasks: @@ -110,7 +110,7 @@ In this case, we want to find the mount point for a given path across our machin .. code-block:: YAML+Jinja :caption: Use selectattr to filter mounts into list I can then sort and select the last from - :emphasize-lines: 7 + :emphasize-lines: 8 - hosts: all gather_facts: True @@ -131,7 +131,7 @@ The special ``omit`` variable ONLY works with module options, but we can still u .. code-block:: YAML+Jinja :caption: Inline list filtering when feeding a module option - :emphasize-lines: 3, 7 + :emphasize-lines: 3, 6 - name: Enable a list of Windows features, by name ansible.builtin.set_fact: diff --git a/docs/docsite/rst/user_guide/playbooks_filters.rst b/docs/docsite/rst/user_guide/playbooks_filters.rst index 984de55e4b3..39672232bb4 100644 --- a/docs/docsite/rst/user_guide/playbooks_filters.rst +++ b/docs/docsite/rst/user_guide/playbooks_filters.rst @@ -293,18 +293,24 @@ To get a list combining the elements of other lists use ``zip``:: - name: Give me list combo of two lists ansible.builtin.debug: - msg: "{{ [1,2,3,4,5] | zip(['a','b','c','d','e','f']) | list }}" + msg: "{{ [1,2,3,4,5,6] | zip(['a','b','c','d','e','f']) | list }}" + + # => [[1, "a"], [2, "b"], [3, "c"], [4, "d"], [5, "e"], [6, "f"]] - name: Give me shortest combo of two lists ansible.builtin.debug: msg: "{{ [1,2,3] | zip(['a','b','c','d','e','f']) | list }}" + # => [[1, "a"], [2, "b"], [3, "c"]] + To always exhaust all lists use ``zip_longest``:: - name: Give me longest combo of three lists , fill with X ansible.builtin.debug: msg: "{{ [1,2,3] | zip_longest(['a','b','c','d','e','f'], [21, 22, 23], fillvalue='X') | list }}" + # => [[1, "a", 21], [2, "b", 22], [3, "c", 23], ["X", "d", "X"], ["X", "e", "X"], ["X", "f", "X"]] + Similarly to the output of the ``items2dict`` filter mentioned above, these filters can be used to construct a ``dict``:: {{ dict(keys_list | zip(values_list)) }} @@ -640,11 +646,11 @@ To get permutations of a list:: - name: Give me largest permutations (order matters) ansible.builtin.debug: - msg: "{{ [1,2,3,4,5] | permutations | list }}" + msg: "{{ [1,2,3,4,5] | ansible.builtin.permutations | list }}" - name: Give me permutations of sets of three ansible.builtin.debug: - msg: "{{ [1,2,3,4,5] | permutations(3) | list }}" + msg: "{{ [1,2,3,4,5] | ansible.builtin.permutations(3) | list }}" combinations ^^^^^^^^^^^^ @@ -652,7 +658,7 @@ Combinations always require a set size:: - name: Give me combinations for sets of two ansible.builtin.debug: - msg: "{{ [1,2,3,4,5] | combinations(2) | list }}" + msg: "{{ [1,2,3,4,5] | ansible.builtin.combinations(2) | list }}" Also see the :ref:`zip_filter` @@ -848,7 +854,7 @@ To get a random item from a list:: "{{ ['a','b','c'] | random }}" # => 'c' -To get a random number between 0 and a specified number:: +To get a random number between 0 (inclusive) and a specified integer (exclusive):: "{{ 60 | random }} * * * * root /script/from/cron" # => '21 * * * * root /script/from/cron' @@ -921,10 +927,12 @@ To get the maximum value in a list of objects:: Flatten a list (same thing the `flatten` lookup does):: {{ [3, [4, 2] ] | flatten }} + # => [3, 4, 2] Flatten only the first level of a list (akin to the `items` lookup):: {{ [3, [4, [2]] ] | flatten(levels=1) }} + # => [3, 4, [2]] .. versionadded:: 2.11 @@ -932,7 +940,7 @@ Flatten only the first level of a list (akin to the `items` lookup):: Preserve nulls in a list, by default flatten removes them. :: {{ [3, None, [4, [2]] ] | flatten(levels=1, skip_nulls=False) }} - + # => [3, None, 4, [2]] .. _set_theory_filters: @@ -989,21 +997,26 @@ You can calculate logs, powers, and roots of numbers with Ansible filters. Jinja Get the logarithm (default is e):: - {{ myvar | log }} + {{ 8 | log }} + # => 2.0794415416798357 Get the base 10 logarithm:: - {{ myvar | log(10) }} + {{ 8 | log(10) }} + # => 0.9030899869919435 Give me the power of 2! (or 5):: - {{ myvar | pow(2) }} - {{ myvar | pow(5) }} + {{ 8 | pow(5) }} + # => 32768.0 Square root, or the 5th:: - {{ myvar | root }} - {{ myvar | root(5) }} + {{ 8 | root }} + # => 2.8284271247461903 + + {{ 8 | root(5) }} + # => 1.5157165665103982 Managing network interactions @@ -1035,6 +1048,7 @@ IP address filter can also be used to extract specific information from an IP address. For example, to get the IP address itself from a CIDR, you can use:: {{ '192.0.2.1/24' | ansible.netcommon.ipaddr('address') }} + # => 192.168.0.1 More information about ``ipaddr`` filter and complete usage guide can be found in :ref:`playbooks_filters_ipaddr`. @@ -1281,14 +1295,17 @@ Encrypting and checksumming strings and passwords To get the sha1 hash of a string:: {{ 'test1' | hash('sha1') }} + # => "b444ac06613fc8d63795be9ad0beaf55011936ac" To get the md5 hash of a string:: {{ 'test1' | hash('md5') }} + # => "5a105e8b9d40e1329780d62ea2265d8a" Get a string checksum:: {{ 'test2' | checksum }} + # => "109f4b3c50d7b0df729d299bc6f8e9ef9066971f" Other hashes (platform dependent):: @@ -1297,14 +1314,17 @@ Other hashes (platform dependent):: To get a sha512 password hash (random salt):: {{ 'passwordsaresecret' | password_hash('sha512') }} + # => "$6$UIv3676O/ilZzWEE$ktEfFF19NQPF2zyxqxGkAceTnbEgpEKuGBtk6MlU4v2ZorWaVQUMyurgmHCh2Fr4wpmQ/Y.AlXMJkRnIS4RfH/" To get a sha256 password hash with a specific salt:: {{ 'secretpassword' | password_hash('sha256', 'mysecretsalt') }} + # => "$5$mysecretsalt$ReKNyDYjkKNqRVwouShhsEqZ3VOE8eoVO4exihOfvG4" An idempotent method to generate unique hashes per system is to use a salt that is consistent between runs:: {{ 'secretpassword' | password_hash('sha512', 65534 | random(seed=inventory_hostname) | string) }} + # => "$6$43927$lQxPKz2M2X.NWO.gK.t7phLwOKQMcSq72XxDZQ0XzYV6DlL1OD72h417aj16OnHTGxNzhftXJQBcjbunLEepM0" Hash types available depend on the control system running Ansible, 'hash' depends on hashlib, password_hash depends on passlib (https://passlib.readthedocs.io/en/stable/lib/passlib.hash.html). @@ -1313,6 +1333,7 @@ Hash types available depend on the control system running Ansible, 'hash' depend Some hash types allow providing a rounds parameter:: {{ 'secretpassword' | password_hash('sha256', 'mysecretsalt', rounds=10000) }} + # => "$5$rounds=10000$mysecretsalt$Tkm80llAxD4YHll6AgNIztKn0vzAACsuuEfYeGP7tm7" .. _other_useful_filters: @@ -1468,9 +1489,11 @@ To search a string with a regex, use the "regex_search" filter:: # search for "foo" in "foobar" {{ 'foobar' | regex_search('(foo)') }} + # => "foo" # will return empty if it cannot find a match {{ 'ansible' | regex_search('(foobar)') }} + # => "" # case insensitive search in multiline mode {{ 'foo\nBAR' | regex_search("^bar", multiline=True, ignorecase=True) }} @@ -1486,18 +1509,23 @@ To replace text in a string with regex, use the "regex_replace" filter:: # convert "ansible" to "able" {{ 'ansible' | regex_replace('^a.*i(.*)$', 'a\\1') }} + # => 'able' # convert "foobar" to "bar" {{ 'foobar' | regex_replace('^f.*o(.*)$', '\\1') }} + # => 'bar' # convert "localhost:80" to "localhost, 80" using named groups {{ 'localhost:80' | regex_replace('^(?P.+):(?P\\d+)$', '\\g, \\g') }} + # => 'localhost, 80' # convert "localhost:80" to "localhost" {{ 'localhost:80' | regex_replace(':80') }} + # => 'localhost' # change a multiline string - {{ var | regex_replace('^', '#CommentThis#', multiline=True) }} + {{ 'var' | regex_replace('^', '#CommentThis#', multiline=True) }} + # => '#CommentThis#var' .. note:: If you want to match the whole string and you are using ``*`` make sure to always wraparound your regular expression with the start/end anchors. For example ``^(.*)$`` will always match only one result, while ``(.*)`` on some Python versions will match the whole string and an empty string at the end, which means it will make two replacements:: @@ -1683,12 +1711,15 @@ To format a date using a string (like with the shell date command), use the "str # Display year-month-day {{ '%Y-%m-%d' | strftime }} + # => "2021-03-19" # Display hour:min:sec {{ '%H:%M:%S' | strftime }} + # => "21:51:04" # Use ansible_date_time.epoch fact {{ '%Y-%m-%d %H:%M:%S' | strftime(ansible_date_time.epoch) }} + # => "2021-03-19 21:54:09" # Use arbitrary epoch value {{ '%Y-%m-%d' | strftime(0) }} # => 1970-01-01 @@ -1712,7 +1743,8 @@ This can then be used to reference hashes in Pod specifications:: my_secret: kind: Secret - name: my_secret_name + metadata: + name: my_secret_name deployment_resource: kind: Deployment