Add pymarkdown sanity test for core (#81220)

* Clean up markdown

* Add pymarkdown sanity test for core

* Update unit test SHA

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,23 +1,29 @@
 ##### SUMMARY
+
 <!--- Describe the change below, including rationale and design decisions -->
 
 <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
 
 ##### ISSUE TYPE
+
 <!--- Pick one below and delete the rest -->
+
 - Bugfix Pull Request
 - Docs Pull Request
 - Feature Pull Request
 - Test Pull Request
 
 ##### COMPONENT NAME
+
 <!--- Write the short name of the module, plugin, task or feature below -->
 
 ##### ADDITIONAL INFORMATION
+
 <!--- Include additional information to help people understand the change here -->
 <!--- A step-by-step reproduction of the problem is helpful if there is no related issue -->
 
 <!--- Paste verbatim command output below, e.g. before and after your change -->
+
 ```paste below
 
 ```

hacking/README.md

@@ -9,14 +9,18 @@ ansible from a git checkout using python >= 3.10.
 
 First, set up your environment to run from the checkout:
 
-    $ source ./hacking/env-setup
+```shell
+source ./hacking/env-setup
+```
 
 You will need some basic prerequisites installed.  If you do not already have them
 and do not wish to install them from your operating system package manager, you
 can install them from pip
 
-    $ easy_install pip               # if pip is not already available
-    $ pip install -r requirements.txt
+```shell
+easy_install pip  # if pip is not already available
+pip install -r requirements.txt
+```
 
 From there, follow ansible instructions on docs.ansible.com as normal.
 
@@ -28,7 +32,9 @@ a module outside of the ansible program, locally, on the current machine.
 
 Example:
 
-    $ ./hacking/test-module.py -m lib/ansible/modules/command.py -a "echo hi"
+```shell
+./hacking/test-module.py -m lib/ansible/modules/command.py -a "echo hi"
+```
 
 This is a good way to insert a breakpoint into a module, for instance.
 
@@ -45,8 +51,10 @@ parent:
 
 Use:
 
-    $ ./hacking/test-module.py -m module \
+```shell
+./hacking/test-module.py -m module \
   -a '{"parent": {"child": [{"item": "first", "val": "foo"}, {"item": "second", "val": "bar"}]}}'
+```
 
 return_skeleton_generator.py
 ----------------------------

hacking/azp/README.md

@@ -34,25 +34,31 @@ Reducing incidental test coverage, and eventually removing incidental tests invo
 1. Run the entire test suite with code coverage enabled.
    This is done automatically each day on Azure Pipelines.
    The URLs and statuses of the most recent such test runs can be found with:
+
    ```shell
    hacking/azp/get_recent_coverage_runs.py <optional branch name>
    ```
+
    The branch name defaults to `devel`.
 2. Download code coverage data from Azure Pipelines for local analysis.
    Example:
+
    ```shell
    # download results to ansible/ansible directory under cwd
    # substitute the correct run number for the Azure Pipelines coverage run you want to download
    hacking/azp/download.py 14075 --artifacts --run-metadata -v
    ```
+
 3. Analyze code coverage data to see which portions of the code are covered by each test.
    Example:
-   ```shell script
+
+   ```shell
    # make sure ansible-test is in $PATH
    source hacking/env-setup
    # run the script using whichever directory results were downloaded into
    hacking/azp/incidental.py 14075/
    ```
+
 4. Create new intentional tests, or extend existing ones, to cover code that is currently covered by incidental tests.
    Reports are created by default in a ``test/results/.tmp/incidental/{hash}/reports/`` directory.
    The ``{hash}`` value is based on the input files used to generate the report.
@@ -90,7 +96,7 @@ If the coverage is for Python code a comment on the right side will indicate the
 
 Below is an example of a report:
 
-```
+```text
 Target: incidental_win_psexec
 GitHub: https://github.com/ansible/ansible/blob/6994ef0b554a816f02e0771cb14341a421f7cead/test/integration/targets/incidental_win_psexec
 

hacking/backport/README.md

@@ -6,26 +6,25 @@ variable called `GITHUB_TOKEN`.
 
 To generate a Github token, go to https://github.com/settings/tokens/new
 
-### `backport_of_line_adder.py`
+## `backport_of_line_adder.py`
 
 This script will attempt to add a reference line ("Backport of ...") to a new
 backport PR.
 
 It is called like this:
 
-```
+```shell
 ./backport_of_line_adder.py <backport> <original PR>
 ```
 
 However, it contains some logic to try to automatically deduce the original PR
 for you. You can trigger that logic by making the second argument be `auto`.
 
-```
+```shell
 ./backport_of_line_adder.py 12345 auto
 ```
 
-... for example, will look for an appropriate reference to add to backport PR
-#12345.
+... for example, will look for an appropriate reference to add to backport PR #12345.
 
 The script will prompt you before making any changes, and give you a chance to
 review the PR that it is about to reference.

hacking/deprecated_issue_template.md

@@ -1,34 +1,43 @@
 ##### SUMMARY
+
 %(component)s contains call to Display.deprecated or AnsibleModule.deprecate and is scheduled for removal
 
-```
+```text
 %(line)s
 ```
 
 ##### ISSUE TYPE
+
 - Bug Report
 
 ##### COMPONENT NAME
-```
+
+```text
 %(path)s
 ```
 
 ##### ANSIBLE VERSION
-```
+
+```text
 %(version)s
 ```
 
 ##### CONFIGURATION
+
 N/A
 
 ##### OS / ENVIRONMENT
+
 N/A
 
 ##### STEPS TO REPRODUCE
+
 N/A
 
 ##### EXPECTED RESULTS
+
 N/A
 
 ##### ACTUAL RESULTS
+
 N/A

hacking/ticket_stubs/bug_wrong_repo.md

@@ -34,5 +34,3 @@ See  this page for a complete list of communication channels and their purposes:
 * https://docs.ansible.com/ansible/latest/community/communication.html
 
 Thank you once again!
-
-

hacking/ticket_stubs/no_thanks.md

@@ -16,4 +16,3 @@ If you or anyone else has any further questions, please let us know by using any
 In the future, sometimes starting a discussion on the development list prior to implementing a feature can make getting things included a little easier, but it's not always necessary.
 
 Thank you once again for this and your interest in Ansible!
-

hacking/ticket_stubs/pr_duplicate.md

@@ -18,4 +18,3 @@ If you or anyone else has any further questions, please let us know by using any
 * https://docs.ansible.com/ansible/latest/community/communication.html
 
 Thank you once again for this and your interest in Ansible!
-

hacking/ticket_stubs/proposal.md

@@ -11,7 +11,6 @@ The mailing list and irc are great ways to ask questions, or post if you don't t
 * #ansible-devel on [irc.libera.chat](https://libera.chat/)
 * https://groups.google.com/forum/#!forum/ansible-devel
 
-
 Or check this page for a more complete list of communication channels and their purposes:
 
 * https://docs.ansible.com/ansible/latest/community/communication.html

hacking/ticket_stubs/question_not_bug.md

@@ -16,4 +16,3 @@ If don't you think this particular issue is resolved, you should still stop by t
 This allows us to keep the issue tracker for bugs, pull requests, RFEs and the like.
 
 Thank you once again and we look forward to seeing you on the list or IRC.  Thanks!
-

hacking/ticket_stubs/wider_discussion.md

@@ -19,4 +19,3 @@ For other alternatives, check this page for a more complete list of communicatio
 * https://docs.ansible.com/ansible/latest/community/communication.html
 
 Thank you once again for this and your interest in Ansible!
-

lib/ansible/galaxy/data/container/README.md

@@ -3,7 +3,7 @@
 Adds a <SERVICE_NAME> service to your [Ansible Container](https://github.com/ansible/ansible-container) project. Run the following commands
 to install the service:
 
-```
+```shell
 # Set the working directory to your Ansible Container project root
 $ cd myproject
 
@@ -15,7 +15,8 @@ $ ansible-container install <USERNAME.ROLE_NAME>
 
 - [Ansible Container](https://github.com/ansible/ansible-container)
 - An existing Ansible Container project. To create a project, simply run the following:
-    ```
+
+    ```shell
     # Create an empty project directory
     $ mkdir myproject
 
@@ -28,7 +29,6 @@ $ ansible-container install <USERNAME.ROLE_NAME>
 
 - Continue listing any prerequisites here...
 
-
 ## Role Variables
 
 A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set
@@ -45,5 +45,3 @@ BSD
 ## Author Information
 
 An optional section for the role authors to include contact information, or a website (HTML is not allowed).
-
-

test/integration/targets/ansible-test-sanity-validate-modules/ansible_collections/ns/failure/README.md

@@ -1,3 +1,4 @@
 README
 ------
+
 This is a simple collection used to test failures with ``ansible-test sanity --test validate-modules``.

test/integration/targets/ansible-test-sanity-validate-modules/ansible_collections/ns/ps_only/README.md

@@ -1,3 +1,4 @@
 README
 ------
+
 This is a simple PowerShell-only collection used to verify that ``ansible-test`` works on a collection.

test/integration/targets/ansible-test-sanity/ansible_collections/ns/col/README.md

@@ -1,3 +1,4 @@
 README
 ------
+
 This is a simple collection used to verify that ``ansible-test`` works on a collection.

test/lib/ansible_test/_internal/commands/sanity/__init__.py

@@ -878,6 +878,7 @@ class SanityCodeSmellTest(SanitySingleVersion):
             self.__include_directories: bool = self.config.get('include_directories')
             self.__include_symlinks: bool = self.config.get('include_symlinks')
             self.__py2_compat: bool = self.config.get('py2_compat', False)
+            self.__error_code: str | None = self.config.get('error_code', None)
         else:
             self.output = None
             self.extensions = []
@@ -893,6 +894,7 @@ class SanityCodeSmellTest(SanitySingleVersion):
             self.__include_directories = False
             self.__include_symlinks = False
             self.__py2_compat = False
+            self.__error_code = None
 
         if self.no_targets:
             mutually_exclusive = (
@@ -911,6 +913,11 @@ class SanityCodeSmellTest(SanitySingleVersion):
             if problems:
                 raise ApplicationError('Sanity test "%s" option "no_targets" is mutually exclusive with options: %s' % (self.name, ', '.join(problems)))
 
+    @property
+    def error_code(self) -> t.Optional[str]:
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return self.__error_code
+
     @property
     def all_targets(self) -> bool:
         """True if test targets will not be filtered using includes, excludes, requires or changes. Mutually exclusive with no_targets."""
@@ -995,6 +1002,8 @@ class SanityCodeSmellTest(SanitySingleVersion):
                 pattern = '^(?P<path>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+): (?P<message>.*)$'
             elif self.output == 'path-message':
                 pattern = '^(?P<path>[^:]*): (?P<message>.*)$'
+            elif self.output == 'path-line-column-code-message':
+                pattern = '^(?P<path>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+): (?P<code>[^:]*): (?P<message>.*)$'
             else:
                 raise ApplicationError('Unsupported output type: %s' % self.output)
 
@@ -1024,6 +1033,7 @@ class SanityCodeSmellTest(SanitySingleVersion):
                     path=m['path'],
                     line=int(m.get('line', 0)),
                     column=int(m.get('column', 0)),
+                    code=m.get('code'),
                 ) for m in matches]
 
                 messages = settings.process_errors(messages, paths)

test/sanity/code-smell/pymarkdown.config.json

@@ -0,0 +1,11 @@
+{
+    "plugins": {
+        "line-length": {
+            "line_length": 160,
+            "code_block_line_length": 160
+        },
+        "first-line-heading": {
+            "enabled": false
+        }
+    }
+}

test/sanity/code-smell/pymarkdown.json

@@ -0,0 +1,7 @@
+{
+    "output": "path-line-column-code-message",
+    "error_code": "ansible-test",
+    "extensions": [
+        ".md"
+    ]
+}

test/sanity/code-smell/pymarkdown.py

@@ -0,0 +1,64 @@
+"""Sanity test for Markdown files."""
+from __future__ import annotations
+
+import pathlib
+import re
+import subprocess
+import sys
+
+import typing as t
+
+
+def main() -> None:
+    paths = sys.argv[1:] or sys.stdin.read().splitlines()
+
+    cmd = [
+        sys.executable,
+        '-m', 'pymarkdown',
+        '--config', pathlib.Path(__file__).parent / 'pymarkdown.config.json',
+        '--strict-config',
+        'scan',
+    ] + paths
+
+    process = subprocess.run(
+        cmd,
+        stdin=subprocess.DEVNULL,
+        capture_output=True,
+        check=False,
+        text=True,
+    )
+
+    if process.stderr:
+        print(process.stderr.strip(), file=sys.stderr)
+        sys.exit(1)
+
+    if not (stdout := process.stdout.strip()):
+        return
+
+    pattern = re.compile(r'^(?P<path_line_column>[^:]*:[0-9]+:[0-9]+): (?P<code>[^:]*): (?P<message>.*) \((?P<aliases>.*)\)$')
+    matches = parse_to_list_of_dict(pattern, stdout)
+    results = [f"{match['path_line_column']}: {match['aliases'].split(', ')[0]}: {match['message']}" for match in matches]
+
+    print('\n'.join(results))
+
+
+def parse_to_list_of_dict(pattern: re.Pattern, value: str) -> list[dict[str, t.Any]]:
+    matched = []
+    unmatched = []
+
+    for line in value.splitlines():
+        match = re.search(pattern, line)
+
+        if match:
+            matched.append(match.groupdict())
+        else:
+            unmatched.append(line)
+
+    if unmatched:
+        raise Exception('Pattern {pattern!r} did not match values:\n' + '\n'.join(unmatched))
+
+    return matched
+
+
+if __name__ == '__main__':
+    main()

test/sanity/code-smell/pymarkdown.requirements.in

@@ -0,0 +1 @@
+pymarkdownlnt

test/sanity/code-smell/pymarkdown.requirements.txt

@@ -0,0 +1,7 @@
+# edit "pymarkdown.requirements.in" and generate with: hacking/update-sanity-requirements.py --test pymarkdown
+application-properties==0.6.0
+Columnar==1.4.1
+pymarkdownlnt==0.9.11
+toolz==0.12.0
+typing_extensions==4.5.0
+wcwidth==0.2.6

test/sanity/ignore.txt

@@ -178,3 +178,26 @@ test/units/utils/collection_loader/fixtures/collections_masked/ansible_collectio
 test/units/utils/collection_loader/fixtures/collections_masked/ansible_collections/testns/__init__.py empty-init  # testing that collections don't need inits
 test/units/utils/collection_loader/fixtures/collections_masked/ansible_collections/testns/testcoll/__init__.py empty-init  # testing that collections don't need inits
 test/units/utils/collection_loader/test_collection_loader.py pylint:undefined-variable  # magic runtime local var splatting
+.github/CONTRIBUTING.md pymarkdown:line-length
+hacking/backport/README.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/bug_internal_api.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/bug_wrong_repo.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/collections.md pymarkdown:line-length
+hacking/ticket_stubs/collections.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/guide_newbie_about_gh_and_contributing_to_ansible.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/no_thanks.md pymarkdown:line-length
+hacking/ticket_stubs/no_thanks.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/pr_duplicate.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/pr_merged.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/proposal.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/question_not_bug.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/resolved.md pymarkdown:no-bare-urls
+hacking/ticket_stubs/wider_discussion.md pymarkdown:no-bare-urls
+lib/ansible/galaxy/data/apb/README.md pymarkdown:line-length
+lib/ansible/galaxy/data/container/README.md pymarkdown:line-length
+lib/ansible/galaxy/data/default/role/README.md pymarkdown:line-length
+lib/ansible/galaxy/data/network/README.md pymarkdown:line-length
+README.md pymarkdown:line-length
+test/integration/targets/ansible-vault/invalid_format/README.md pymarkdown:no-bare-urls
+test/support/README.md pymarkdown:no-bare-urls
+test/units/cli/test_data/role_skeleton/README.md pymarkdown:line-length

test/support/README.md

@@ -1,4 +1,4 @@
-# IMPORTANT!
+# IMPORTANT
 
 Files under this directory are not actual plugins and modules used by Ansible
 and as such should **not be modified**. They are used for testing purposes

test/units/galaxy/test_collection.py

@@ -824,7 +824,7 @@ def test_build_with_symlink_inside_collection(collection_input):
         actual_file = secure_hash_s(linked_file_obj.read())
         linked_file_obj.close()
 
-        assert actual_file == '63444bfc766154e1bc7557ef6280de20d03fcd81'
+        assert actual_file == '08f24200b9fbe18903e7a50930c9d0df0b8d7da3'  # shasum test/units/cli/test_data/collection_skeleton/README.md
 
 
 def test_publish_no_wait(galaxy_server, collection_artifact, monkeypatch):