Add docs detailing how to convert many with_X style loops to use loop and filters (#40964)

* Add docs detailing how to convert many with_X style loops to use loop and filters. Fixes #40727

* Switch lookup used in query vs lookup comparison, to not recommend use of nested lookup

* Improve docs based on feedback

docs/docsite/rst/porting_guides/porting_guide_2.5.rst

@@ -91,6 +91,11 @@ You will run into errors because Ansible reads name in this context as a keyword
 
 For a full list of keywords see ::ref::`Playbook Keywords`.
 
+Migrating from with_X to loop
+-----------------------------
+
+.. include:: ../user_guide/shared_snippets/with2loop.txt
+
 
 Deprecated
 ==========

docs/docsite/rst/user_guide/playbooks_loops.rst

@@ -105,7 +105,7 @@ For example, using the 'nested' lookup, you can combine lists::
         priv: "{{ item[1] }}.*:ALL"
         append_privs: yes
         password: "foo"
-      loop: "{{ query('nested', [ 'alice', 'bob' ], [ 'clientdb', 'employeedb', 'providerdb' ]) }}"
+      loop: "{{ ['alice', 'bob'] |product(['clientdb', 'employeedb', 'providerdb'])|list }}"
 
 .. note:: ``with_`` loops are actually a combination of things ``with_`` + ``lookup()``, even ``items`` is a lookup. ``loop`` can be used in the same way as shown above.
 
@@ -121,9 +121,9 @@ In certain situations the ``lookup`` function may not return a list which ``loop
 
 The following invocations are equivalent, using ``wantlist=True`` with ``lookup`` to ensure a return type of a list::
 
-    loop: "{{ query('nested', ['alice', 'bob'], ['clientdb', 'employeedb', 'providerdb']) }}"
+    loop: "{{ query('inventory_hostnames', 'all') }}"
 
-    loop: "{{ lookup('nested', ['alice', 'bob'], ['clientdb', 'employeedb', 'providerdb'], wantlist=True) }}"
+    loop: "{{ lookup('inventory_hostnames', 'all', wantlist=True) }}"
 
 
 .. _do_until_loops:
@@ -330,6 +330,11 @@ If you need to keep track of where you are in a loop, you can use the ``index_va
       loop_control:
         index_var: my_idx
 
+Migrating from with_X to loop
+`````````````````````````````
+
+.. include:: shared_snippets/with2loop.txt
+
 
 .. seealso::
 

docs/docsite/rst/user_guide/shared_snippets/with2loop.txt

@@ -0,0 +1,193 @@
+With the release of Ansible 2.5, the recommended way to perform loops is the use the new ``loop`` keyword instead of ``with_X`` style loops.
+
+In many cases, ``loop`` syntax is better expressed using filters instead of more complex use of ``query`` or ``lookup``.
+
+The following examples will show how to convert many common ``with_`` style loops to ``loop`` and filters.
+
+with_list
+---------
+
+``with_list`` is directly replaced by ``loop``.
+
+.. code-block:: yaml+jinja
+
+    - name: with_list
+      debug:
+        msg: "{{ item }}"
+      with_list:
+        - one
+        - two
+
+    - name: with_list -> loop
+      debug:
+        msg: "{{ item }}"
+      loop:
+        - one
+        - two
+
+with_items
+----------
+
+``with_items`` is replaced by ``loop`` and the ``flatten`` filter.
+
+.. code-block:: yaml+jinja
+
+    - name: with_items
+      debug:
+        msg: "{{ item }}"
+      with_items: "{{ items }}"
+
+    - name: with_items -> loop
+      debug:
+        msg: "{{ item }}"
+      loop: "{{ items|flatten(levels=1) }}"
+
+with_indexed_items
+------------------
+
+``with_indexed_items`` is replaced by ``loop``, the ``flatten`` filter and ``loop_control.index_var``.
+
+.. code-block:: yaml+jinja
+
+    - name: with_indexed_items
+      debug:
+        msg: "{{ item.0 }} - {{ item.1 }}"
+      with_indexed_items: "{{ items }}"
+
+    - name: with_indexed_items -> loop
+      debug:
+        msg: "{{ index }} - {{ item }}"
+      loop: "{{ items|flatten(levels=1) }}"
+      loop_control:
+        index_var: index
+
+with_flattened
+--------------
+
+``with_flattened`` is replaced by ``loop`` and the ``flatten`` filter.
+
+.. code-block:: yaml+jinja
+
+    - name: with_flattened
+      debug:
+        msg: "{{ item }}"
+      with_flattened: "{{ items }}"
+
+    - name: with_flattened -> loop
+      debug:
+        msg: "{{ item }}"
+      loop: "{{ items|flatten }}"
+
+with_together
+-------------
+
+``with_together`` is replaced by ``loop`` and the ``zip`` filter.
+
+.. code-block:: yaml+jinja
+
+    - name: with_together
+      debug:
+        msg: "{{ item.0 }} - {{ item.1 }}"
+      with_together:
+        - "{{ list_one }}"
+        - "{{ list_two }}"
+
+    - name: with_together -> loop
+      debug:
+        msg: "{{ item.0 }} - {{ item.1 }}"
+      loop: "{{ list_one|zip(list_two)|list }}"
+
+with_dict
+---------
+
+``with_dict`` can be substituted by ``loop`` and either the ``dictsort`` or ``dict2items`` filters.
+
+.. code-block:: yaml+jinja
+
+    - name: with_dict
+      debug:
+        msg: "{{ item.key }} - {{ item.value }}"
+      with_dict: "{{ dictionary }}"
+
+    - name: with_dict -> loop (option 1)
+      debug:
+        msg: "{{ item.key }} - {{ item.value }}"
+      loop: "{{ dictionary|dict2items }}"
+
+    - name: with_dict -> loop (option 2)
+      debug:
+        msg: "{{ item.0 }} - {{ item.1 }}"
+      loop: "{{ dictionary|dictsort }}"
+
+with_sequence
+-------------
+
+``with_sequence`` is replaced by ``loop`` and the ``range`` function, and potentially the ``format`` filter.
+
+.. code-block:: yaml+jinja
+
+    - name: with_sequence
+      debug:
+        msg: "{{ item }}"
+      with_sequence: start=0 end=4 stride=2 format=testuser%02x
+
+    - name: with_sequence -> loop
+      debug:
+        msg: "{{ 'testuser%02x' | format(item) }}"
+      # range is exclusive of the end point
+      loop: "{{ range(0, 4 + 1, 2)|list }}"
+
+with_subelements
+----------------
+
+``with_subelements`` is replaced by ``loop`` and the ``subelements`` filter.
+
+.. code-block:: yaml+jinja
+
+    - name: with_subelements
+      debug:
+        msg: "{{ item.0.name }} - {{ item.1 }}"
+      with_subelements:
+        - "{{ users }}"
+        - mysql.hosts
+
+    - name: with_subelements -> loop
+      debug:
+        msg: "{{ item.0.name }} - {{ item.1 }}"
+      loop: "{{ users|subelements('mysql.hosts') }}"
+
+with_nested/with_cartesian
+--------------------------
+
+``with_nested`` and ``with_cartesian`` are replaced by loop and the ``product`` filter.
+
+.. code-block:: yaml+jinja
+
+    - name: with_nested
+      debug:
+        msg: "{{ item.0 }} - {{ item.1 }}"
+      with_nested:
+        - "{{ list_one }}"
+        - "{{ list_two }}"
+
+    - name: with_nested -> loop
+      debug:
+        msg: "{{ item.0 }} - {{ item.1 }}"
+      loop: "{{ list_one|product(list_two)|list }}"
+
+with_random_choice
+------------------
+
+``with_random_choice`` is replaced by just use of the ``random`` filter, without need of ``loop``.
+
+.. code-block:: yaml+jinja
+
+    - name: with_random_choice
+      debug:
+        msg: "{{ item }}"
+      with_random_choice: "{{ my_list }}"
+
+    - name: with_random_choice -> loop (No loop is needed here)
+      debug:
+        msg: "{{ my_list|random }}"
+      tags: random