Backportapalooza 06 16 (#78072)

* Docsite: Updated style guide resources (#78048)

(cherry picked from commit 561cf1956d)

* Create reusable role layout snippet for documentation (#78057)

* reuse role structure
* reuse role snippet in role docs

(cherry picked from commit f4ad1c771e)

* Docsite: update links to community docs (#78043)

(cherry picked from commit 46d86a882d)

* Windows - incorrect python-devel package in WinRM docs (#78037)

Co-authored-by: Emanuele Barbato <manu@Giuseppes-MacBook-Air.local>
(cherry picked from commit 681dc6eab9)

* issue #72449 winrm script rm (#77931)

(cherry picked from commit 3cd2c494bd)

* reword the regex note (#75393)

Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
(cherry picked from commit 66c0882314)

* Document debconf interaction with dpkg-reconfigure (#74196)

Signed-off-by: David Greaves <david@dgreaves.com>
(cherry picked from commit 5b90601ca4)

Co-authored-by: Lewis Brogan <ilewisbrogan@gmail.com>
Co-authored-by: Andrew Klychkov <aklychko@redhat.com>
Co-authored-by: Emanuele Barbato <30394740+emanuelebarbato@users.noreply.github.com>
Co-authored-by: Don Naro <dnaro@redhat.com>
Co-authored-by: David Greaves <david@dgreaves.com>

docs/docsite/rst/community/maintainers_workflow.rst

@@ -8,8 +8,8 @@ Each collection community can set its own rules and workflow for managing pull r
 
 * :ref:`code_of_conduct`
 * :ref:`maintainer_requirements`
-* :ref:`Committer guidelines <committer_general_rules>`.
-* `PR review checklist <https://github.com/ansible/community-docs/blob/main/review_checklist.rst>`_
+* :ref:`Committer guidelines <committer_general_rules>`
+* :ref:`PR review checklist<review_checklist>`
 
 There can be two kinds of maintainers: :ref:`collection_maintainers` and :ref:`module_maintainers`.
 

docs/docsite/rst/dev_guide/style_guide/resources.rst

@@ -7,4 +7,5 @@ Resources
   * `AP Stylebook <https://www.apstylebook.com>`_
   * `Chicago Manual of Style <https://www.chicagomanualofstyle.org/home.html>`_
   * `Strunk and White's Elements of Style <https://www.crockford.com/wrrrld/style.html>`_
+  * `Google developer documentation style guide <https://developers.google.com/style/highlights>`_
 

docs/docsite/rst/reference_appendices/faq.rst

@@ -811,6 +811,31 @@ and backups, which most file based modules also support:
              path: "{{ updated['backup_file'] }}"
              state: absent
 
+.. _jinja2_faqs:
+
+Why does the ``regex_search`` filter return `None` instead of an empty string?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+Until the jinja2 2.10 release, Jinja was only able to return strings, but Ansible needed Python objects in some cases. Ansible uses ``safe_eval`` and  only sends strings that look like certain types of Python objects through this function. With ``regex_search`` that does not find a match, the result (``None``) is converted to the string "None" which is not useful in non-native jinja2.
+
+The following example of a single templating action shows this behavior:
+
+.. code-block:: Jinja
+
+  {{ 'ansible' | regex_search('foobar') }}
+
+This example does not result in a Python ``None``, so Ansible historically converted it to "" (empty string).
+
+The native jinja2 functionality actually allows us to return full Python objects, that are always represented as Python objects everywhere, and as such the result of a single templating action with ``regex_search`` can result in the Python ``None``.
+
+.. note::
+
+  Native jinja2 functionality is not needed when ``regex_search`` is used as an intermediate result that is then compared to the jinja2 ``none`` test.
+
+  .. code-block:: Jinja
+
+     {{ 'ansible' | regex_search('foobar') is none }}
+
 
 .. _docs_contributions:
 

docs/docsite/rst/user_guide/playbooks_filters.rst

@@ -1826,16 +1826,20 @@ The ``regex_search`` filter returns an empty string if it cannot find a match:
     {{ 'ansible' | regex_search('foobar') }}
     # => ''
 
-Note that due to historic behavior and custom re-implementation of some of the Jinja internals in Ansible there is an exception to the behavior. When used in a Jinja expression (for example in conjunction with operators, other filters, and so on) the return value differs, in those situations the return value is ``none``. See the two examples below:
 
-.. code-block:: yaml+jinja
+.. note::
+
+
+  The ``regex_search`` filter returns ``None`` when used in a Jinja expression (for example in conjunction with operators, other filters, and so on). See the two examples below.
+
+  .. code-block:: Jinja
 
     {{ 'ansible' | regex_search('foobar') == '' }}
     # => False
-    {{ 'ansible' | regex_search('foobar') == none }}
+    {{ 'ansible' | regex_search('foobar') is none }}
     # => True
 
-When ``jinja2_native`` setting is enabled, the ``regex_search`` filter always returns ``none`` if it cannot find a match.
+  This is due to historic behavior and the custom re-implementation of some of the Jinja internals in Ansible. Enable the ``jinja2_native`` setting if you want the ``regex_search`` filter to always return ``None`` if it cannot find a match. See :ref:`jinja2_faqs` for details.
 
 To extract all occurrences of regex matches in a string, use the ``regex_findall`` filter:
 
@@ -2038,7 +2042,7 @@ To split a string into a list:
 .. code-block:: yaml+jinja
 
     {{ csv_string | split(",") }}
-    
+
 .. versionadded:: 2.11
 
 To work with Base64 encoded strings:

docs/docsite/rst/user_guide/playbooks_reuse_roles.rst

@@ -20,20 +20,8 @@ An Ansible role has a defined directory structure with eight main standard direc
     site.yml
     webservers.yml
     fooservers.yml
-    roles/
-        common/
-            tasks/
-            handlers/
-            library/
-            files/
-            templates/
-            vars/
-            defaults/
-            meta/
-        webservers/
-            tasks/
-            defaults/
-            meta/
+.. include:: shared_snippets/role_directory.txt
+
 
 By default Ansible will look in each directory within a role for a ``main.yml`` file for relevant content (also ``main.yaml`` and ``main``):
 

docs/docsite/rst/user_guide/sample_setup.rst

@@ -16,7 +16,7 @@ Sample directory layout
 
 This layout organizes most tasks in roles, with a single inventory file for each environment and a few playbooks in the top-level directory:
 
-  .. code-block:: console
+.. code-block:: console
 
     production                # inventory file for production servers
     staging                   # inventory file for staging environment
@@ -37,31 +37,7 @@ This layout organizes most tasks in roles, with a single inventory file for each
     dbservers.yml             # playbook for dbserver tier
     tasks/                    # task files included from playbooks
         webservers-extra.yml  # <-- avoids confusing playbook with task files
-
-    roles/
-        common/               # this hierarchy represents a "role"
-            tasks/            #
-                main.yml      #  <-- tasks file can include smaller files if warranted
-            handlers/         #
-                main.yml      #  <-- handlers file
-            templates/        #  <-- files for use with the template resource
-                ntp.conf.j2   #  <------- templates end in .j2
-            files/            #
-                bar.txt       #  <-- files for use with the copy resource
-                foo.sh        #  <-- script files for use with the script resource
-            vars/             #
-                main.yml      #  <-- variables associated with this role
-            defaults/         #
-                main.yml      #  <-- default lower priority variables for this role
-            meta/             #
-                main.yml      #  <-- role dependencies
-            library/          # roles can also include custom modules
-            module_utils/     # roles can also include custom module_utils
-            lookup_plugins/   # or other types of plugins, like lookup in this case
-
-        webtier/              # same kind of structure as "common" was above, done for the webtier role
-        monitoring/           # ""
-        fooapp/               # ""
+.. include:: shared_snippets/role_directory.txt
 
 .. note:: By default, Ansible assumes your playbooks are stored in one directory with roles stored in a sub-directory called ``roles/``. As you use Ansible to automate more tasks, you may want to move your playbooks into a sub-directory called ``playbooks/``. If you do this, you must configure the path to your ``roles/`` directory using the ``roles_path`` setting in ansible.cfg.
 

docs/docsite/rst/user_guide/shared_snippets/role_directory.txt

@@ -0,0 +1,26 @@
+.. code-block:: yaml
+
+  roles/
+      common/               # this hierarchy represents a "role"
+          tasks/            #
+              main.yml      #  <-- tasks file can include smaller files if warranted
+          handlers/         #
+              main.yml      #  <-- handlers file
+          templates/        #  <-- files for use with the template resource
+              ntp.conf.j2   #  <------- templates end in .j2
+          files/            #
+              bar.txt       #  <-- files for use with the copy resource
+              foo.sh        #  <-- script files for use with the script resource
+          vars/             #
+              main.yml      #  <-- variables associated with this role
+          defaults/         #
+              main.yml      #  <-- default lower priority variables for this role
+          meta/             #
+              main.yml      #  <-- role dependencies
+          library/          # roles can also include custom modules
+          module_utils/     # roles can also include custom module_utils
+          lookup_plugins/   # or other types of plugins, like lookup in this case
+
+      webtier/              # same kind of structure as "common" was above, done for the webtier role
+      monitoring/           # ""
+      fooapp/               # ""

docs/docsite/rst/user_guide/windows_setup.rst

@@ -107,39 +107,11 @@ For more details, please refer to the `Hotfix document <https://support.microsof
 
 WinRM Setup
 ```````````
-Once Powershell has been upgraded to at least version 3.0, the final step is for the
-WinRM service to be configured so that Ansible can connect to it. There are two
+Once Powershell has been upgraded to at least version 3.0, the final step is to
+configure the WinRM service so that Ansible can connect to it. There are two
 main components of the WinRM service that governs how Ansible can interface with
 the Windows host: the ``listener`` and the ``service`` configuration settings.
 
-Details about each component can be read below, but the script
-`ConfigureRemotingForAnsible.ps1 <https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1>`_
-can be used to set up the basics. This script sets up both HTTP and HTTPS
-listeners with a self-signed certificate and enables the ``Basic``
-authentication option on the service.
-
-To use this script, run the following in PowerShell:
-
-.. code-block:: powershell
-
-    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
-    $url = "https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1"
-    $file = "$env:temp\ConfigureRemotingForAnsible.ps1"
-
-    (New-Object -TypeName System.Net.WebClient).DownloadFile($url, $file)
-
-    powershell.exe -ExecutionPolicy ByPass -File $file
-
-There are different switches and parameters (like ``-EnableCredSSP`` and
-``-ForceNewSSLCert``) that can be set alongside this script. The documentation
-for these options are located at the top of the script itself.
-
-.. Note:: The ConfigureRemotingForAnsible.ps1 script is intended for training and
-    development purposes only and should not be used in a
-    production environment, since it enables settings (like ``Basic`` authentication)
-    that can be inherently insecure. Kerberos is considered a safer production setup. See :ref:`winrm_kerberos` for details.
-
-
 WinRM Listener
 --------------
 The WinRM services listens for requests on one or more ports. Each of these ports must have a
@@ -467,7 +439,6 @@ this problems is to either:
 
 See `KB4076842 <https://support.microsoft.com/en-us/help/4076842>`_ for more information on this problem.
 
-
 Windows SSH Setup
 `````````````````
 Ansible 2.8 has added an experimental SSH connection for Windows managed nodes.

docs/docsite/rst/user_guide/windows_winrm.rst

@@ -341,9 +341,12 @@ Some system dependencies that must be installed prior to using Kerberos. The scr
 
 .. code-block:: shell
 
-    # Via Yum (RHEL/Centos/Fedora)
+    # Via Yum (RHEL/Centos/Fedora for the older version)
     yum -y install gcc python-devel krb5-devel krb5-libs krb5-workstation
 
+    # Via DNF (RHEL/Centos/Fedora for the newer version)
+    dnf -y install gcc python3-devel krb5-devel krb5-libs krb5-workstation
+
     # Via Apt (Ubuntu)
     sudo apt-get install python-dev libkrb5-dev krb5-user
 
@@ -628,9 +631,8 @@ The WinRM protocol considers the channel to be encrypted if using TLS over HTTP
 recommended option as it works with all authentication options, but requires
 a certificate to be created and used on the WinRM listener.
 
-The ``ConfigureRemotingForAnsible.ps1`` creates a self-signed certificate and
-creates the listener with that certificate. If in a domain environment, ADCS
-can also create a certificate for the host that is issued by the domain itself.
+If in a domain environment, ADCS can create a certificate for the host that 
+is issued by the domain itself.
 
 If using HTTPS is not an option, then HTTP can be used when the authentication
 option is ``NTLM``, ``Kerberos`` or ``CredSSP``. These protocols will encrypt

examples/scripts/ConfigureRemotingForAnsible.ps1

@@ -7,6 +7,21 @@
 # the necessary changes to allow Ansible to connect, authenticate and
 # execute PowerShell commands.
 #
+# IMPORTANT: This script uses self-signed certificates and authentication mechanisms
+# that are intended for development environments and evaluation purposes only.
+# Production environments and deployments that are exposed on the network should
+# use CA-signed certificates and secure authentication mechanisms such as Kerberos.
+#
+# To run this script in Powershell:
+#
+# [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
+# $url = "https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1"
+# $file = "$env:temp\ConfigureRemotingForAnsible.ps1"
+#
+# (New-Object -TypeName System.Net.WebClient).DownloadFile($url, $file)
+#
+# powershell.exe -ExecutionPolicy ByPass -File $file
+#
 # All events are logged to the Windows EventLog, useful for unattended runs.
 #
 # Use option -Verbose in order to see the verbose output messages.

lib/ansible/modules/debconf.py

@@ -32,6 +32,25 @@ notes:
       installed to see questions/settings available.
     - Some distros will always record tasks involving the setting of passwords as changed. This is due to debconf-get-selections masking passwords.
     - It is highly recommended to add I(no_log=True) to task while handling sensitive information using this module.
+    - The debconf module does not reconfigure packages, it just updates the debconf database.
+      An additional step is needed (typically with I(notify) if debconf makes a change)
+      to reconfigure the package and apply the changes.
+      debconf is extensively used for pre-seeding configuration prior to installation
+      rather than modifying configurations.
+      So, while dpkg-reconfigure does use debconf data, it is not always authoritative
+      and you may need to check how your package is handled.
+    - Also note dpkg-reconfigure is a 3-phase process. It invokes the
+      control scripts from the C(/var/lib/dpkg/info) directory with the
+      C(<package>.prerm  reconfigure <version>),
+      C(<package>.config reconfigure <version>) and C(<package>.postinst control <version>) arguments.
+    - The main issue is that the C(<package>.config reconfigure) step for many packages
+      will first reset the debconf database (overriding changes made by this module) by
+      checking the on-disk configuration. If this is the case for your package then
+      dpkg-reconfigure will effectively ignore changes  made by debconf.
+    - However as dpkg-reconfigure only executes the C(<package>.config) step if the file
+      exists, it is possible to rename it to C(/var/lib/dpkg/info/<package>.config.ignore)
+      before executing C(dpkg-reconfigure -f noninteractive <package>) and then restore it.
+      This seems to be compliant with Debian policy for the .config file.
 requirements:
 - debconf
 - debconf-utils

test/lib/ansible_test/_util/target/setup/ConfigureRemotingForAnsible.ps1

@@ -7,6 +7,21 @@
 # the necessary changes to allow Ansible to connect, authenticate and
 # execute PowerShell commands.
 #
+# IMPORTANT: This script uses self-signed certificates and authentication mechanisms
+# that are intended for development environments and evaluation purposes only.
+# Production environments and deployments that are exposed on the network should
+# use CA-signed certificates and secure authentication mechanisms such as Kerberos.
+#
+# To run this script in Powershell:
+#
+# [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
+# $url = "https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1"
+# $file = "$env:temp\ConfigureRemotingForAnsible.ps1"
+#
+# (New-Object -TypeName System.Net.WebClient).DownloadFile($url, $file)
+#
+# powershell.exe -ExecutionPolicy ByPass -File $file
+#
 # All events are logged to the Windows EventLog, useful for unattended runs.
 #
 # Use option -Verbose in order to see the verbose output messages.