Use consolidated ignore.txt in ansible-test. (#59618)

docs/docsite/rst/dev_guide/testing/sanity/ignores.rst

@@ -0,0 +1,99 @@
+Sanity Tests » ignores
+======================
+
+Sanity tests for individual files can be skipped, and specific errors can be ignored.
+
+When to Ignore Errors
+---------------------
+
+Sanity tests are designed to improve code quality and identify common issues with content.
+When issues are identified during development, those issues should be corrected.
+
+As development of Ansible continues, sanity tests are expanded to detect issues that previous releases could not.
+To allow time for existing content to be updated to pass newer tests, ignore entries can be added.
+New content should not use ignores for existing sanity tests.
+
+When code is fixed to resolve sanity test errors, any relevant ignores must also be removed.
+If the ignores are not removed, this will be reported as an unnecessary ignore error.
+This is intended to prevent future regressions due to the same error recurring after being fixed.
+
+When to Skip Tests
+------------------
+
+Although rare, there are reasons for skipping a sanity test instead of ignoring the errors it reports.
+
+If a sanity test results in a traceback when processing content, that error cannot be ignored.
+If this occurs, open a new `bug report <https://github.com/ansible/ansible/issues/new?template=bug_report.md>`_ for the issue so it can be fixed.
+If the traceback occurs due to an issue with the content, that issue should be fixed.
+If the content is correct, the test will need to be skipped until the bug in the sanity test is fixed.
+
+    Caution should be used when skipping sanity tests instead of ignoring them.
+    Since the test is skipped entirely, resolution of the issue will not be automatically detected.
+    This will prevent prevent regression detection from working once the issue has been resolved.
+    For this reason it is a good idea to periodically review skipped entries manually to verify they are required.
+
+Ignore File Location
+--------------------
+
+The location of the ignore file depends on the type of content being tested.
+
+Ansible Collections
+~~~~~~~~~~~~~~~~~~~
+
+Since sanity tests change between Ansible releases, a separate ignore file is needed for each Ansible major release.
+
+The filename is ``test/sanity/ignore-X.Y.txt`` where ``X.Y`` is the Ansible release being used to test the collection.
+
+Maintaining a separate file for each Ansible release allows a collection to pass tests for multiple versions of Ansible.
+
+Ansible
+~~~~~~~
+
+When testing Ansible, all ignores are placed in the ``test/sanity/ignore.txt`` file.
+
+Only a single file is needed because ``ansible-test`` is developed and released as a part of Ansible itself.
+
+Ignore File Format
+------------------
+
+The ignore file contains one entry per line.
+Each line consists of two columns, separated by a single space.
+Comments may be added at the end of an entry, started with a hash (``#``) character, which can be proceeded by zero or more spaces.
+Blank and comment only lines are not allowed.
+
+The first column specifies the file path that the entry applies to.
+File paths must be relative to the root of the content being tested.
+This is either the Ansible source or an Ansible collection.
+File paths cannot contain a space or the hash (``#``) character.
+
+The second column specifies the sanity test that the entry applies to.
+This will be the name of the sanity test.
+If the sanity test is specific to a version of Python, the name will include a dash (``-``) and the relevant Python version.
+If the named test uses error codes then the error code to ignore must be appended to the name of the test, separated by a colon (``:``).
+
+Below are some example ignore entries for an Ansible collection::
+
+    roles/my_role/files/my_script.sh shellcheck:SC2154 # ignore undefined variable
+    plugins/modules/my_module.py validate-modules:E105 # ignore license check
+    plugins/modules/my_module.py import-3.8 # needs update to support collections.abc on Python 3.8+
+
+It is also possible to skip a sanity test for a specific file.
+This is done by adding ``!skip`` after the sanity test name in the second column.
+When this is done, no error code is included, even if the sanity test uses error codes.
+
+Below are some example skip entries for an Ansible collection::
+
+    plugins/module_utils/my_util.py validate-modules!skip # waiting for bug fix in module validator
+    plugins/lookup/my_plugin.py compile-2.6!skip # Python 2.6 is not supported on the controller
+
+Ignore File Errors
+------------------
+
+There are various errors that can be reported for the ignore file itself:
+
+- syntax errors parsing the ignore file
+- references a file path that does not exist
+- references to a sanity test that does not exist
+- ignoring an error that does not occur
+- ignoring a file which is skipped
+- duplicate entries

test/runner/lib/sanity/__init__.py

@@ -57,13 +57,16 @@ from lib.test import (
     TestSkipped,
     TestMessage,
     calculate_best_confidence,
-    calculate_confidence,
 )
 
 from lib.data import (
     data_context,
 )
 
+from lib.env import (
+    get_ansible_version,
+)
+
 COMMAND = 'sanity'
 
 
@@ -174,6 +177,258 @@ def sanity_get_tests():
     return SANITY_TESTS
 
 
+class SanityIgnoreParser:
+    """Parser for the consolidated sanity test ignore file."""
+    NO_CODE = '_'
+
+    def __init__(self, args):  # type: (SanityConfig) -> None
+        if data_context().content.collection:
+            ansible_version = '%s.%s' % tuple(get_ansible_version(args).split('.')[:2])
+
+            ansible_label = 'Ansible %s' % ansible_version
+            file_name = 'ignore-%s.txt' % ansible_version
+        else:
+            ansible_label = 'Ansible'
+            file_name = 'ignore.txt'
+
+        self.args = args
+        self.relative_path = os.path.join('test/sanity', file_name)
+        self.path = os.path.join(data_context().content.root, self.relative_path)
+        self.ignores = collections.defaultdict(lambda: collections.defaultdict(dict))  # type: t.Dict[str, t.Dict[str, t.Dict[str, int]]]
+        self.skips = collections.defaultdict(lambda: collections.defaultdict(int))  # type: t.Dict[str, t.Dict[str, int]]
+        self.parse_errors = []  # type: t.List[t.Tuple[int, int, str]]
+        self.file_not_found_errors = []  # type: t.List[t.Tuple[int, str]]
+
+        lines = read_lines_without_comments(self.path, optional=True)
+        paths = set(data_context().content.all_files())
+        tests_by_name = {}  # type: t.Dict[str, SanityTest]
+        versioned_test_names = set()  # type: t.Set[str]
+        unversioned_test_names = {}  # type: t.Dict[str, str]
+
+        display.info('Read %d sanity test ignore line(s) for %s from: %s' % (len(lines), ansible_label, self.relative_path), verbosity=1)
+
+        for test in sanity_get_tests():
+            if isinstance(test, SanityMultipleVersion):
+                versioned_test_names.add(test.name)
+                tests_by_name.update(dict(('%s-%s' % (test.name, python_version), test) for python_version in SUPPORTED_PYTHON_VERSIONS))
+            else:
+                unversioned_test_names.update(dict(('%s-%s' % (test.name, python_version), test.name) for python_version in SUPPORTED_PYTHON_VERSIONS))
+                tests_by_name[test.name] = test
+
+        for line_no, line in enumerate(lines, start=1):
+            if not line:
+                self.parse_errors.append((line_no, 1, "Line cannot be empty or contain only a comment"))
+                continue
+
+            parts = line.split(' ')
+            path = parts[0]
+            codes = parts[1:]
+
+            if not path:
+                self.parse_errors.append((line_no, 1, "Line cannot start with a space"))
+                continue
+
+            if path not in paths:
+                self.file_not_found_errors.append((line_no, path))
+                continue
+
+            if not codes:
+                self.parse_errors.append((line_no, len(path), "Error code required after path"))
+                continue
+
+            code = codes[0]
+
+            if not code:
+                self.parse_errors.append((line_no, len(path) + 1, "Error code after path cannot be empty"))
+                continue
+
+            if len(codes) > 1:
+                self.parse_errors.append((line_no, len(path) + len(code) + 2, "Error code cannot contain spaces"))
+                continue
+
+            parts = code.split('!')
+            code = parts[0]
+            commands = parts[1:]
+
+            parts = code.split(':')
+            test_name = parts[0]
+            error_codes = parts[1:]
+
+            test = tests_by_name.get(test_name)
+
+            if not test:
+                unversioned_name = unversioned_test_names.get(test_name)
+
+                if unversioned_name:
+                    self.parse_errors.append((line_no, len(path) + len(unversioned_name) + 2, "Sanity test '%s' cannot use a Python version like '%s'" % (
+                        unversioned_name, test_name)))
+                elif test_name in versioned_test_names:
+                    self.parse_errors.append((line_no, len(path) + len(test_name) + 1, "Sanity test '%s' requires a Python version like '%s-%s'" % (
+                        test_name, test_name, args.python_version)))
+                else:
+                    self.parse_errors.append((line_no, len(path) + 2, "Sanity test '%s' does not exist" % test_name))
+
+                continue
+
+            if commands and error_codes:
+                self.parse_errors.append((line_no, len(path) + len(test_name) + 2, "Error code cannot contain both '!' and ':' characters"))
+                continue
+
+            if commands:
+                command = commands[0]
+
+                if len(commands) > 1:
+                    self.parse_errors.append((line_no, len(path) + len(test_name) + len(command) + 3, "Error code cannot contain multiple '!' characters"))
+                    continue
+
+                if command == 'skip':
+                    if not test.can_skip:
+                        self.parse_errors.append((line_no, len(path) + len(test_name) + 2, "Sanity test '%s' cannot be skipped" % test_name))
+                        continue
+
+                    existing_line_no = self.skips.get(test_name, {}).get(path)
+
+                    if existing_line_no:
+                        self.parse_errors.append((line_no, 1, "Duplicate '%s' skip for path '%s' first found on line %d" % (test_name, path, existing_line_no)))
+                        continue
+
+                    self.skips[test_name][path] = line_no
+                    continue
+
+                self.parse_errors.append((line_no, len(path) + len(test_name) + 2, "Command '!%s' not recognized" % command))
+                continue
+
+            if not test.can_ignore:
+                self.parse_errors.append((line_no, len(path) + 1, "Sanity test '%s' cannot be ignored" % test_name))
+                continue
+
+            if test.error_code:
+                if not error_codes:
+                    self.parse_errors.append((line_no, len(path) + len(test_name) + 1, "Sanity test '%s' requires an error code" % test_name))
+                    continue
+
+                error_code = error_codes[0]
+
+                if len(error_codes) > 1:
+                    self.parse_errors.append((line_no, len(path) + len(test_name) + len(error_code) + 3, "Error code cannot contain multiple ':' characters"))
+                    continue
+            else:
+                if error_codes:
+                    self.parse_errors.append((line_no, len(path) + len(test_name) + 2, "Sanity test '%s' does not support error codes" % test_name))
+                    continue
+
+                error_code = self.NO_CODE
+
+            existing = self.ignores.get(test_name, {}).get(path, {}).get(error_code)
+
+            if existing:
+                if test.error_code:
+                    self.parse_errors.append((line_no, 1, "Duplicate '%s' ignore for error code '%s' for path '%s' first found on line %d" % (
+                        test_name, error_code, path, existing)))
+                else:
+                    self.parse_errors.append((line_no, 1, "Duplicate '%s' ignore for path '%s' first found on line %d" % (
+                        test_name, path, existing)))
+
+                continue
+
+            self.ignores[test_name][path][error_code] = line_no
+
+    @staticmethod
+    def load(args):  # type: (SanityConfig) -> SanityIgnoreParser
+        """Return the current SanityIgnore instance, initializing it if needed."""
+        try:
+            return SanityIgnoreParser.instance
+        except AttributeError:
+            pass
+
+        SanityIgnoreParser.instance = SanityIgnoreParser(args)
+        return SanityIgnoreParser.instance
+
+
+class SanityIgnoreProcessor:
+    """Processor for sanity test ignores for a single run of one sanity test."""
+    def __init__(self,
+                 args,  # type: SanityConfig
+                 name,  # type: str
+                 code,  # type: t.Optional[str]
+                 python_version,  # type: t.Optional[str]
+                 ):  # type: (...) -> None
+        if python_version:
+            full_name = '%s-%s' % (name, python_version)
+        else:
+            full_name = name
+
+        self.args = args
+        self.code = code
+        self.parser = SanityIgnoreParser.load(args)
+        self.ignore_entries = self.parser.ignores.get(full_name, {})
+        self.skip_entries = self.parser.skips.get(full_name, {})
+        self.used_line_numbers = set()  # type: t.Set[int]
+
+    def filter_skipped_paths(self, paths):  # type: (t.List[str]) -> t.List[str]
+        """Return the given paths, with any skipped paths filtered out."""
+        return sorted(set(paths) - set(self.skip_entries.keys()))
+
+    def filter_skipped_targets(self, targets):  # type: (t.List[TestTarget]) -> t.List[TestTarget]
+        """Return the given targets, with any skipped paths filtered out."""
+        return sorted(target for target in targets if target.path not in self.skip_entries)
+
+    def process_errors(self, errors, paths):  # type: (t.List[SanityMessage], t.List[str]) -> t.List[SanityMessage]
+        """Return the given errors filtered for ignores and with any settings related errors included."""
+        errors = self.filter_messages(errors)
+        errors.extend(self.get_errors(paths))
+
+        errors = sorted(set(errors))
+
+        return errors
+
+    def filter_messages(self, messages):  # type: (t.List[SanityMessage]) -> t.List[SanityMessage]
+        """Return a filtered list of the given messages using the entries that have been loaded."""
+        filtered = []
+
+        for message in messages:
+            path_entry = self.ignore_entries.get(message.path)
+
+            if path_entry:
+                code = message.code if self.code else SanityIgnoreParser.NO_CODE
+                line_no = path_entry.get(code)
+
+                if line_no:
+                    self.used_line_numbers.add(line_no)
+                    continue
+
+            filtered.append(message)
+
+        return filtered
+
+    def get_errors(self, paths):  # type: (t.List[str]) -> t.List[SanityMessage]
+        """Return error messages related to issues with the file."""
+        messages = []
+
+        # unused errors
+
+        unused = []  # type: t.List[t.Tuple[int, str, str]]
+
+        for path in paths:
+            path_entry = self.ignore_entries.get(path)
+
+            if not path_entry:
+                continue
+
+            unused.extend((line_no, path, code) for code, line_no in path_entry.items() if line_no not in self.used_line_numbers)
+
+        messages.extend(SanityMessage(
+            code=self.code,
+            message="Ignoring '%s' on '%s' is unnecessary" % (code, path) if self.code else "Ignoring '%s' is unnecessary" % path,
+            path=self.parser.relative_path,
+            line=line,
+            column=1,
+            confidence=calculate_best_confidence(((self.parser.path, line), (path, 0)), self.args.metadata) if self.args.metadata.changes else None,
+        ) for line, path, code in unused)
+
+        return messages
+
+
 class SanitySuccess(TestSuccess):
     """Sanity test success."""
     def __init__(self, test, python_version=None):
@@ -233,6 +488,21 @@ class SanityTest(ABC):
         self.name = name
         self.enabled = True
 
+    @property
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return None
+
+    @property
+    def can_ignore(self):  # type: () -> bool
+        """True if the test supports ignore entries."""
+        return True
+
+    @property
+    def can_skip(self):  # type: () -> bool
+        """True if the test supports skip entries."""
+        return True
+
 
 class SanityCodeSmellTest(SanityTest):
     """Sanity test script."""
@@ -277,6 +547,10 @@ class SanityCodeSmellTest(SanityTest):
         pattern = None
         data = None
 
+        settings = self.load_processor(args)
+
+        paths = []
+
         if self.config:
             output = self.config.get('output')
             extensions = self.config.get('extensions')
@@ -317,6 +591,8 @@ class SanityCodeSmellTest(SanityTest):
             if files:
                 paths = [p for p in paths if os.path.basename(p) in files]
 
+            paths = settings.filter_skipped_paths(paths)
+
             if not paths and not always:
                 return SanitySkipped(self.name)
 
@@ -344,14 +620,28 @@ class SanityCodeSmellTest(SanityTest):
                     column=int(m.get('column', 0)),
                 ) for m in matches]
 
+                messages = settings.process_errors(messages, paths)
+
+                if not messages:
+                    return SanitySuccess(self.name)
+
                 return SanityFailure(self.name, messages=messages)
 
         if stderr or status:
             summary = u'%s' % SubprocessError(cmd=cmd, status=status, stderr=stderr, stdout=stdout)
             return SanityFailure(self.name, summary=summary)
 
+        messages = settings.process_errors([], paths)
+
+        if messages:
+            return SanityFailure(self.name, messages=messages)
+
         return SanitySuccess(self.name)
 
+    def load_processor(self, args):  # type: (SanityConfig) -> SanityIgnoreProcessor
+        """Load the ignore processor for this sanity test."""
+        return SanityIgnoreProcessor(args, self.name, self.error_code, None)
+
 
 class SanityFunc(SanityTest):
     """Base class for sanity test plugins."""
@@ -373,9 +663,9 @@ class SanitySingleVersion(SanityFunc):
         :rtype: TestResult
         """
 
-    def load_settings(self, args, code):  # type: (SanityConfig, t.Optional[str]) -> SanitySettings
+    def load_processor(self, args):  # type: (SanityConfig) -> SanityIgnoreProcessor
-        """Load settings for this sanity test."""
+        """Load the ignore processor for this sanity test."""
-        return SanitySettings(args, self.name, code, None)
+        return SanityIgnoreProcessor(args, self.name, self.error_code, None)
 
 
 class SanityMultipleVersion(SanityFunc):
@@ -389,222 +679,9 @@ class SanityMultipleVersion(SanityFunc):
         :rtype: TestResult
         """
 
-    def load_settings(self, args, code, python_version):  # type: (SanityConfig, t.Optional[str], t.Optional[str]) -> SanitySettings
+    def load_processor(self, args, python_version):  # type: (SanityConfig, str) -> SanityIgnoreProcessor
-        """Load settings for this sanity test."""
+        """Load the ignore processor for this sanity test."""
-        return SanitySettings(args, self.name, code, python_version)
+        return SanityIgnoreProcessor(args, self.name, self.error_code, python_version)
-
-
-class SanitySettings:
-    """Settings for sanity tests."""
-    def __init__(self,
-                 args,  # type: SanityConfig
-                 name,  # type: str
-                 code,  # type: t.Optional[str]
-                 python_version,  # type: t.Optional[str]
-                 ):  # type: (...) -> None
-        self.args = args
-        self.code = code
-        self.ignore_settings = SanitySettingsFile(args, name, 'ignore', code, python_version)
-        self.skip_settings = SanitySettingsFile(args, name, 'skip', code, python_version)
-
-    def filter_skipped_paths(self, paths):  # type: (t.List[str]) -> t.List[str]
-        """Return the given paths, with any skipped paths filtered out."""
-        return sorted(set(paths) - set(self.skip_settings.entries.keys()))
-
-    def filter_skipped_targets(self, targets):  # type: (t.List[TestTarget]) -> t.List[TestTarget]
-        """Return the given targets, with any skipped paths filtered out."""
-        return sorted(target for target in targets if target.path not in self.skip_settings.entries)
-
-    def process_errors(self, errors, paths):  # type: (t.List[SanityMessage], t.List[str]) -> t.List[SanityMessage]
-        """Return the given errors filtered for ignores and with any settings related errors included."""
-        errors = self.ignore_settings.filter_messages(errors)
-        errors.extend(self.ignore_settings.get_errors(paths))
-        errors.extend(self.skip_settings.get_errors([]))
-
-        for ignore_path, ignore_entry in self.ignore_settings.entries.items():
-            skip_entry = self.skip_settings.entries.get(ignore_path)
-
-            if not skip_entry:
-                continue
-
-            skip_line_no = skip_entry[SanitySettingsFile.NO_CODE]
-
-            for ignore_line_no in ignore_entry.values():
-                candidates = ((self.ignore_settings.path, ignore_line_no), (self.skip_settings.path, skip_line_no))
-
-                errors.append(SanityMessage(
-                    code=self.code,
-                    message="Ignoring '%s' is unnecessary due to skip entry on line %d of '%s'" % (ignore_path, skip_line_no, self.skip_settings.relative_path),
-                    path=self.ignore_settings.relative_path,
-                    line=ignore_line_no,
-                    column=1,
-                    confidence=calculate_best_confidence(candidates, self.args.metadata) if self.args.metadata.changes else None,
-                ))
-
-        errors = sorted(set(errors))
-
-        return errors
-
-
-class SanitySettingsFile:
-    """Interface to sanity ignore or sanity skip file settings."""
-    NO_CODE = '_'
-
-    def __init__(self,
-                 args,  # type: SanityConfig
-                 name,  # type: str
-                 mode,  # type: str
-                 code,  # type: t.Optional[str]
-                 python_version,  # type: t.Optional[str]
-                 ):  # type: (...) -> None
-        """
-        :param mode: must be either "ignore" or "skip"
-        :param code: a code for ansible-test to use for internal errors, using a style that matches codes used by the test, or None if codes are not used
-        """
-        if mode == 'ignore':
-            self.parse_codes = bool(code)
-        elif mode == 'skip':
-            self.parse_codes = False
-        else:
-            raise Exception('Unsupported mode: %s' % mode)
-
-        if name == 'compile':
-            filename = 'python%s-%s' % (python_version, mode)
-        else:
-            filename = '%s-%s' % (mode, python_version) if python_version else mode
-
-        self.args = args
-        self.code = code
-        self.relative_path = 'test/sanity/%s/%s.txt' % (name, filename)
-        self.path = os.path.join(data_context().content.root, self.relative_path)
-        self.entries = collections.defaultdict(dict)  # type: t.Dict[str, t.Dict[str, int]]
-        self.parse_errors = []  # type: t.List[t.Tuple[int, int, str]]
-        self.file_not_found_errors = []  # type: t.List[t.Tuple[int, str]]
-        self.used_line_numbers = set()  # type: t.Set[int]
-
-        lines = read_lines_without_comments(self.path, optional=True)
-        paths = set(data_context().content.all_files())
-
-        for line_no, line in enumerate(lines, start=1):
-            if not line:
-                continue
-
-            if line.startswith(' '):
-                self.parse_errors.append((line_no, 1, 'Line cannot start with a space'))
-                continue
-
-            if line.endswith(' '):
-                self.parse_errors.append((line_no, len(line), 'Line cannot end with a space'))
-                continue
-
-            parts = line.split(' ')
-            path = parts[0]
-
-            if path not in paths:
-                self.file_not_found_errors.append((line_no, path))
-                continue
-
-            if self.parse_codes:
-                if len(parts) < 2:
-                    self.parse_errors.append((line_no, len(line), 'Code required after path'))
-                    continue
-
-                code = parts[1]
-
-                if not code:
-                    self.parse_errors.append((line_no, len(path) + 1, 'Code after path cannot be empty'))
-                    continue
-
-                if len(parts) > 2:
-                    self.parse_errors.append((line_no, len(path) + len(code) + 2, 'Code cannot contain spaces'))
-                    continue
-
-                existing = self.entries.get(path, {}).get(code)
-
-                if existing:
-                    self.parse_errors.append((line_no, 1, "Duplicate code '%s' for path '%s' first found on line %d" % (code, path, existing)))
-                    continue
-            else:
-                if len(parts) > 1:
-                    self.parse_errors.append((line_no, len(path) + 1, 'Path cannot contain spaces'))
-                    continue
-
-                code = self.NO_CODE
-                existing = self.entries.get(path)
-
-                if existing:
-                    self.parse_errors.append((line_no, 1, "Duplicate path '%s' first found on line %d" % (path, existing[code])))
-                    continue
-
-            self.entries[path][code] = line_no
-
-    def filter_messages(self, messages):  # type: (t.List[SanityMessage]) -> t.List[SanityMessage]
-        """Return a filtered list of the given messages using the entries that have been loaded."""
-        filtered = []
-
-        for message in messages:
-            path_entry = self.entries.get(message.path)
-
-            if path_entry:
-                code = message.code if self.code else self.NO_CODE
-                line_no = path_entry.get(code)
-
-                if line_no:
-                    self.used_line_numbers.add(line_no)
-                    continue
-
-            filtered.append(message)
-
-        return filtered
-
-    def get_errors(self, paths):  # type: (t.List[str]) -> t.List[SanityMessage]
-        """Return error messages related to issues with the file."""
-        messages = []
-
-        # parse errors
-
-        messages.extend(SanityMessage(
-            code=self.code,
-            message=message,
-            path=self.relative_path,
-            line=line,
-            column=column,
-            confidence=calculate_confidence(self.path, line, self.args.metadata) if self.args.metadata.changes else None,
-        ) for line, column, message in self.parse_errors)
-
-        # file not found errors
-
-        messages.extend(SanityMessage(
-            code=self.code,
-            message="File '%s' does not exist" % path,
-            path=self.relative_path,
-            line=line,
-            column=1,
-            confidence=calculate_best_confidence(((self.path, line), (path, 0)), self.args.metadata) if self.args.metadata.changes else None,
-        ) for line, path in self.file_not_found_errors)
-
-        # unused errors
-
-        unused = []  # type: t.List[t.Tuple[int, str, str]]
-
-        for path in paths:
-            path_entry = self.entries.get(path)
-
-            if not path_entry:
-                continue
-
-            unused.extend((line_no, path, code) for code, line_no in path_entry.items() if line_no not in self.used_line_numbers)
-
-        messages.extend(SanityMessage(
-            code=self.code,
-            message="Ignoring '%s' on '%s' is unnecessary" % (code, path) if self.code else "Ignoring '%s' is unnecessary" % path,
-            path=self.relative_path,
-            line=line,
-            column=1,
-            confidence=calculate_best_confidence(((self.path, line), (path, 0)), self.args.metadata) if self.args.metadata.changes else None,
-        ) for line, path, code in unused)
-
-        return messages
 
 
 SANITY_TESTS = (

test/runner/lib/sanity/ansible_doc.py

@@ -48,7 +48,7 @@ class AnsibleDocTest(SanitySingleVersion):
         :type targets: SanityTargets
         :rtype: TestResult
         """
-        settings = self.load_settings(args, None)
+        settings = self.load_processor(args)
 
         targets_include = [target for target in targets.include if os.path.splitext(target.path)[1] == '.py']
         targets_include = settings.filter_skipped_targets(targets_include)

test/runner/lib/sanity/compile.py

@@ -38,7 +38,7 @@ class CompileTest(SanityMultipleVersion):
         :type python_version: str
         :rtype: TestResult
         """
-        settings = self.load_settings(args, None, python_version)
+        settings = self.load_processor(args, python_version)
 
         paths = sorted(i.path for i in targets.include if os.path.splitext(i.path)[1] == '.py' or i.path.startswith('bin/'))
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/ignores.py

@@ -0,0 +1,87 @@
+"""Sanity test for the sanity ignore file."""
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+from lib.sanity import (
+    SanityFailure,
+    SanityIgnoreParser,
+    SanitySingleVersion,
+    SanitySuccess,
+    SanityMessage,
+)
+
+from lib.test import (
+    calculate_confidence,
+    calculate_best_confidence,
+)
+
+from lib.config import (
+    SanityConfig,
+)
+
+
+class IgnoresTest(SanitySingleVersion):
+    """Sanity test for sanity test ignore entries."""
+    @property
+    def can_ignore(self):  # type: () -> bool
+        """True if the test supports ignore entries."""
+        return False
+
+    @property
+    def can_skip(self):  # type: () -> bool
+        """True if the test supports skip entries."""
+        return False
+
+    # noinspection PyUnusedLocal
+    def test(self, args, targets):  # pylint: disable=locally-disabled, unused-argument
+        """
+        :type args: SanityConfig
+        :type targets: SanityTargets
+        :rtype: TestResult
+        """
+        sanity_ignore = SanityIgnoreParser.load(args)
+
+        messages = []
+
+        # parse errors
+
+        messages.extend(SanityMessage(
+            message=message,
+            path=sanity_ignore.relative_path,
+            line=line,
+            column=column,
+            confidence=calculate_confidence(sanity_ignore.path, line, args.metadata) if args.metadata.changes else None,
+        ) for line, column, message in sanity_ignore.parse_errors)
+
+        # file not found errors
+
+        messages.extend(SanityMessage(
+            message="File '%s' does not exist" % path,
+            path=sanity_ignore.relative_path,
+            line=line,
+            column=1,
+            confidence=calculate_best_confidence(((sanity_ignore.path, line), (path, 0)), args.metadata) if args.metadata.changes else None,
+        ) for line, path in sanity_ignore.file_not_found_errors)
+
+        # conflicting ignores and skips
+
+        for test_name, ignores in sanity_ignore.ignores.items():
+            for ignore_path, ignore_entry in ignores.items():
+                skip_line_no = sanity_ignore.skips.get(test_name, {}).get(ignore_path)
+
+                if not skip_line_no:
+                    continue
+
+                for ignore_line_no in ignore_entry.values():
+                    messages.append(SanityMessage(
+                        message="Ignoring '%s' is unnecessary due to skip entry on line %d" % (ignore_path, skip_line_no),
+                        path=sanity_ignore.relative_path,
+                        line=ignore_line_no,
+                        column=1,
+                        confidence=calculate_confidence(sanity_ignore.path, ignore_line_no, args.metadata) if args.metadata.changes else None,
+                    ))
+
+        if messages:
+            return SanityFailure(self.name, messages=messages)
+
+        return SanitySuccess(self.name)

test/runner/lib/sanity/import.py

@@ -58,7 +58,7 @@ class ImportTest(SanityMultipleVersion):
         :type python_version: str
         :rtype: TestResult
         """
-        settings = self.load_settings(args, None, python_version)
+        settings = self.load_processor(args, python_version)
 
         paths = sorted(
             i.path

test/runner/lib/sanity/integration_aliases.py

@@ -88,6 +88,16 @@ class IntegrationAliasesTest(SanitySingleVersion):
         self._shippable_yml_lines = []  # type: t.List[str]
         self._shippable_test_groups = {}  # type: t.Dict[str, t.Set[int]]
 
+    @property
+    def can_ignore(self):  # type: () -> bool
+        """True if the test supports ignore entries."""
+        return False
+
+    @property
+    def can_skip(self):  # type: () -> bool
+        """True if the test supports skip entries."""
+        return False
+
     @property
     def shippable_yml_lines(self):
         """

test/runner/lib/sanity/pep8.py

@@ -2,9 +2,10 @@
 from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
-import json
 import os
 
+import lib.types as t
+
 from lib.sanity import (
     SanitySingleVersion,
     SanityMessage,
@@ -30,6 +31,10 @@ from lib.config import (
 
 class Pep8Test(SanitySingleVersion):
     """Sanity test for PEP 8 style guidelines using pycodestyle."""
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return 'A100'
+
     def test(self, args, targets):
         """
         :type args: SanityConfig
@@ -39,7 +44,7 @@ class Pep8Test(SanitySingleVersion):
         current_ignore_file = os.path.join(ANSIBLE_ROOT, 'test/sanity/pep8/current-ignore.txt')
         current_ignore = sorted(read_lines_without_comments(current_ignore_file, remove_blank_lines=True))
 
-        settings = self.load_settings(args, 'A100')
+        settings = self.load_processor(args)
 
         paths = sorted(i.path for i in targets.include if os.path.splitext(i.path)[1] == '.py' or i.path.startswith('bin/'))
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/pslint.py

@@ -6,6 +6,8 @@ import json
 import os
 import re
 
+import lib.types as t
+
 from lib.sanity import (
     SanitySingleVersion,
     SanityMessage,
@@ -35,13 +37,17 @@ from lib.data import (
 
 class PslintTest(SanitySingleVersion):
     """Sanity test using PSScriptAnalyzer."""
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return 'AnsibleTest'
+
     def test(self, args, targets):
         """
         :type args: SanityConfig
         :type targets: SanityTargets
         :rtype: TestResult
         """
-        settings = self.load_settings(args, 'AnsibleTest')
+        settings = self.load_processor(args)
 
         paths = sorted(i.path for i in targets.include if os.path.splitext(i.path)[1] in ('.ps1', '.psm1', '.psd1'))
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/pylint.py

@@ -50,6 +50,10 @@ UNSUPPORTED_PYTHON_VERSIONS = (
 
 class PylintTest(SanitySingleVersion):
     """Sanity test using pylint."""
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return 'ansible-test'
+
     def test(self, args, targets):
         """
         :type args: SanityConfig
@@ -64,7 +68,7 @@ class PylintTest(SanitySingleVersion):
         plugin_names = sorted(p[0] for p in [
             os.path.splitext(p) for p in os.listdir(plugin_dir)] if p[1] == '.py' and p[0] != '__init__')
 
-        settings = self.load_settings(args, 'ansible-test')
+        settings = self.load_processor(args)
 
         paths = sorted(i.path for i in targets.include if os.path.splitext(i.path)[1] == '.py' or is_subdir(i.path, 'bin/'))
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/rstcheck.py

@@ -48,7 +48,7 @@ class RstcheckTest(SanitySingleVersion):
         ignore_file = os.path.join(ANSIBLE_ROOT, 'test/sanity/rstcheck/ignore-substitutions.txt')
         ignore_substitutions = sorted(set(read_lines_without_comments(ignore_file, remove_blank_lines=True)))
 
-        settings = self.load_settings(args, None)
+        settings = self.load_processor(args)
 
         paths = sorted(i.path for i in targets.include if os.path.splitext(i.path)[1] in ('.rst',))
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/sanity_docs.py

@@ -25,6 +25,16 @@ class SanityDocsTest(SanitySingleVersion):
     """Sanity test for documentation of sanity tests."""
     ansible_only = True
 
+    @property
+    def can_ignore(self):  # type: () -> bool
+        """True if the test supports ignore entries."""
+        return False
+
+    @property
+    def can_skip(self):  # type: () -> bool
+        """True if the test supports skip entries."""
+        return False
+
     # noinspection PyUnusedLocal
     def test(self, args, targets):  # pylint: disable=locally-disabled, unused-argument
         """

test/runner/lib/sanity/shellcheck.py

@@ -9,6 +9,8 @@ from xml.etree.ElementTree import (
     Element,
 )
 
+import lib.types as t
+
 from lib.sanity import (
     SanitySingleVersion,
     SanityMessage,
@@ -34,6 +36,10 @@ from lib.config import (
 
 class ShellcheckTest(SanitySingleVersion):
     """Sanity test using shellcheck."""
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return 'AT1000'
+
     def test(self, args, targets):
         """
         :type args: SanityConfig
@@ -43,7 +49,7 @@ class ShellcheckTest(SanitySingleVersion):
         exclude_file = os.path.join(ANSIBLE_ROOT, 'test/sanity/shellcheck/exclude.txt')
         exclude = set(read_lines_without_comments(exclude_file, remove_blank_lines=True, optional=True))
 
-        settings = self.load_settings(args, 'AT1000')
+        settings = self.load_processor(args)
 
         paths = sorted(i.path for i in targets.include if os.path.splitext(i.path)[1] == '.sh')
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/validate_modules.py

@@ -5,6 +5,8 @@ __metaclass__ = type
 import json
 import os
 
+import lib.types as t
+
 from lib.sanity import (
     SanitySingleVersion,
     SanityMessage,
@@ -43,6 +45,10 @@ UNSUPPORTED_PYTHON_VERSIONS = (
 
 class ValidateModulesTest(SanitySingleVersion):
     """Sanity test using validate-modules."""
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return 'A100'
+
     def test(self, args, targets):
         """
         :type args: SanityConfig
@@ -62,7 +68,7 @@ class ValidateModulesTest(SanitySingleVersion):
 
         env = ansible_environment(args, color=False)
 
-        settings = self.load_settings(args, 'A100')
+        settings = self.load_processor(args)
 
         paths = sorted(i.path for i in targets.include if i.module)
         paths = settings.filter_skipped_paths(paths)

test/runner/lib/sanity/yamllint.py

@@ -5,6 +5,8 @@ __metaclass__ = type
 import json
 import os
 
+import lib.types as t
+
 from lib.sanity import (
     SanitySingleVersion,
     SanityMessage,
@@ -35,13 +37,17 @@ from lib.data import (
 
 class YamllintTest(SanitySingleVersion):
     """Sanity test using yamllint."""
+    def error_code(self):  # type: () -> t.Optional[str]
+        """Error code for ansible-test matching the format used by the underlying test program, or None if the program does not use error codes."""
+        return 'ansible-test'
+
     def test(self, args, targets):
         """
         :type args: SanityConfig
         :type targets: SanityTargets
         :rtype: TestResult
         """
-        settings = self.load_settings(args, 'ansible-test')
+        settings = self.load_processor(args)
 
         paths = [i.path for i in targets.include if os.path.splitext(i.path)[1] in ('.yml', '.yaml')]
 

test/sanity/ansible-doc/skip.txt

@@ -1,42 +0,0 @@
-lib/ansible/modules/utilities/logic/async_wrapper.py
-lib/ansible/modules/utilities/helper/_accelerate.py
-lib/ansible/modules/network/aos/_aos_asn_pool.py
-lib/ansible/modules/network/aos/_aos_blueprint.py
-lib/ansible/modules/network/aos/_aos_blueprint_param.py
-lib/ansible/modules/network/aos/_aos_blueprint_virtnet.py
-lib/ansible/modules/network/aos/_aos_device.py
-lib/ansible/modules/network/aos/_aos_external_router.py
-lib/ansible/modules/network/aos/_aos_ip_pool.py
-lib/ansible/modules/network/aos/_aos_logical_device.py
-lib/ansible/modules/network/aos/_aos_logical_device_map.py
-lib/ansible/modules/network/aos/_aos_login.py
-lib/ansible/modules/network/aos/_aos_rack_type.py
-lib/ansible/modules/network/aos/_aos_template.py
-lib/ansible/modules/cloud/azure/_azure.py
-lib/ansible/modules/network/cumulus/_cl_bond.py
-lib/ansible/modules/network/cumulus/_cl_bridge.py
-lib/ansible/modules/network/cumulus/_cl_img_install.py
-lib/ansible/modules/network/cumulus/_cl_interface.py
-lib/ansible/modules/network/cumulus/_cl_interface_policy.py
-lib/ansible/modules/network/cumulus/_cl_license.py
-lib/ansible/modules/network/cumulus/_cl_ports.py
-lib/ansible/modules/cloud/cloudstack/_cs_nic.py
-lib/ansible/modules/cloud/docker/_docker.py
-lib/ansible/modules/cloud/amazon/_ec2_ami_find.py
-lib/ansible/modules/cloud/amazon/_ec2_ami_search.py
-lib/ansible/modules/cloud/amazon/_ec2_facts.py
-lib/ansible/modules/cloud/amazon/_ec2_remote_facts.py
-lib/ansible/modules/cloud/amazon/_ec2_vpc.py
-lib/ansible/modules/clustering/k8s/_kubernetes.py
-lib/ansible/modules/network/citrix/_netscaler.py
-lib/ansible/modules/network/nxos/_nxos_ip_interface.py
-lib/ansible/modules/network/nxos/_nxos_mtu.py
-lib/ansible/modules/network/nxos/_nxos_portchannel.py
-lib/ansible/modules/network/nxos/_nxos_switchport.py
-lib/ansible/modules/clustering/openshift/_oc.py
-lib/ansible/modules/cloud/openstack/_os_server_actions.py
-lib/ansible/modules/network/panos/_panos_nat_policy.py
-lib/ansible/modules/network/panos/_panos_security_policy.py
-lib/ansible/modules/cloud/amazon/_s3.py
-lib/ansible/modules/cloud/vmware/_vsphere_guest.py
-lib/ansible/modules/windows/_win_msi.py

test/sanity/compile/python2.6-skip.txt

@@ -1,6 +0,0 @@
-hacking/build_library/build_ansible/command_plugins/porting_guide.py # release process only, 3.6+ required
-hacking/build_library/build_ansible/command_plugins/release_announcement.py # release process only, 3.6+ required
-hacking/build_library/build_ansible/command_plugins/dump_config.py # docs build only, 2.7+ required
-hacking/build_library/build_ansible/command_plugins/dump_keywords.py # docs build only, 2.7+ required
-hacking/build_library/build_ansible/command_plugins/generate_man.py # docs build only, 2.7+ required
-hacking/build_library/build_ansible/command_plugins/plugin_formatter.py # docs build only, 2.7+ required

test/sanity/compile/python2.7-skip.txt

@@ -1,2 +0,0 @@
-hacking/build_library/build_ansible/command_plugins/porting_guide.py # release process only, 3.6+ required
-hacking/build_library/build_ansible/command_plugins/release_announcement.py # release process only, 3.6+ required

test/sanity/compile/python3.5-skip.txt

@@ -1,2 +0,0 @@
-hacking/build_library/build_ansible/command_plugins/porting_guide.py # release process only, 3.6+ required
-hacking/build_library/build_ansible/command_plugins/release_announcement.py # release process only, 3.6+ required

test/sanity/pep8/skip.txt

@@ -1 +0,0 @@
-lib/ansible/module_utils/distro/_distro.py # bundled code we don't want to modify

test/sanity/pslint/ignore.txt

@@ -1,106 +0,0 @@
-examples/scripts/ConfigureRemotingForAnsible.ps1 PSCustomUseLiteralPath
-examples/scripts/upgrade_to_ps3.ps1 PSCustomUseLiteralPath
-examples/scripts/upgrade_to_ps3.ps1 PSUseApprovedVerbs
-lib/ansible/executor/powershell/async_watchdog.ps1 PSCustomUseLiteralPath
-lib/ansible/executor/powershell/async_wrapper.ps1 PSCustomUseLiteralPath
-lib/ansible/executor/powershell/exec_wrapper.ps1 PSCustomUseLiteralPath
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.ArgvParser.psm1 PSUseApprovedVerbs
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.CommandUtil.psm1 PSProvideCommentHelp  # need to agree on best format for comment location
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.CommandUtil.psm1 PSUseApprovedVerbs
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.FileUtil.psm1 PSCustomUseLiteralPath
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.FileUtil.psm1 PSProvideCommentHelp
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.Legacy.psm1 PSCustomUseLiteralPath
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.Legacy.psm1 PSUseApprovedVerbs
-lib/ansible/module_utils/powershell/Ansible.ModuleUtils.LinkUtil.psm1 PSUseApprovedVerbs
-lib/ansible/modules/windows/async_status.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/setup.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_acl_inheritance.ps1 PSAvoidTrailingWhitespace
-lib/ansible/modules/windows/win_audit_rule.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_chocolatey.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_chocolatey_config.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_chocolatey_facts.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_chocolatey_source.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_copy.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_credential.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_dns_client.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_dns_client.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_domain.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_domain_controller.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_domain_controller.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_domain_membership.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_domain_membership.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_dotnet_ngen.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_dsc.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_eventlog.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_feature.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_file_version.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_firewall_rule.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_hotfix.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_hotfix.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_iis_virtualdirectory.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_iis_webapplication.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_iis_webapppool.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_iis_webbinding.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_iis_webbinding.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_iis_website.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_lineinfile.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_mapped_drive.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_package.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_package.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_pagefile.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_pagefile.ps1 PSUseSupportsShouldProcess
-lib/ansible/modules/windows/win_pester.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_product_facts.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_rabbitmq_plugin.ps1 PSAvoidUsingInvokeExpression
-lib/ansible/modules/windows/win_rabbitmq_plugin.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_rds_cap.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_rds_rap.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_rds_settings.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_regedit.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_region.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_regmerge.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_robocopy.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_security_policy.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_security_policy.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_share.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_shell.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_shortcut.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_snmp.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_say.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_unzip.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_unzip.ps1 PSUseApprovedVerbs
-lib/ansible/modules/windows/win_updates.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_user_profile.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_wait_for.ps1 PSCustomUseLiteralPath
-lib/ansible/modules/windows/win_webpicmd.ps1 PSAvoidUsingInvokeExpression
-lib/ansible/modules/windows/win_xml.ps1 PSCustomUseLiteralPath
-test/integration/targets/collections/collection_root_user/ansible_collections/testns/testcoll/plugins/module_utils/MyPSMU.psm1 PSUseApprovedVerbs
-test/integration/targets/win_audit_rule/library/test_get_audit_rule.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_chocolatey/files/tools/chocolateyUninstall.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_chocolatey_source/library/choco_source.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_csharp_utils/library/ansible_basic_tests.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_csharp_utils/library/ansible_basic_tests.ps1 PSUseDeclaredVarsMoreThanAssignments # test setup requires vars to be set globally and not referenced in the same scope
-test/integration/targets/win_csharp_utils/library/ansible_become_tests.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_exec_wrapper/library/test_fail.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_iis_webbinding/library/test_get_webbindings.ps1 PSUseApprovedVerbs
-test/integration/targets/win_module_utils/library/argv_parser_test.ps1 PSUseApprovedVerbs
-test/integration/targets/win_module_utils/library/backup_file_test.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_module_utils/library/command_util_test.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_psmodule/files/setup_modules.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_reboot/templates/post_reboot.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_script/files/test_script_creates_file.ps1 PSAvoidUsingCmdletAliases
-test/integration/targets/win_script/files/test_script_removes_file.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_stat/library/test_symlink_file.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_user_right/library/test_get_right.ps1 PSCustomUseLiteralPath
-test/runner/setup/windows-httptester.ps1 PSCustomUseLiteralPath
-test/integration/targets/win_script/files/test_script.ps1 PSAvoidUsingWriteHost # Keep
-test/integration/targets/win_script/files/test_script_with_args.ps1 PSAvoidUsingWriteHost # Keep
-test/integration/targets/win_script/files/test_script_with_splatting.ps1 PSAvoidUsingWriteHost # Keep
-lib/ansible/modules/windows/win_domain.ps1 PSAvoidUsingEmptyCatchBlock # Keep
-lib/ansible/modules/windows/win_dsc.ps1 PSAvoidUsingEmptyCatchBlock # Keep
-lib/ansible/modules/windows/win_find.ps1 PSAvoidUsingEmptyCatchBlock # Keep
-lib/ansible/modules/windows/win_region.ps1 PSAvoidUsingEmptyCatchBlock # Keep
-lib/ansible/modules/windows/win_uri.ps1 PSAvoidUsingEmptyCatchBlock # Keep
-lib/ansible/modules/windows/win_domain_membership.ps1 PSAvoidGlobalVars # New PR
-lib/ansible/modules/windows/win_domain_controller.ps1 PSAvoidGlobalVars # New PR
-lib/ansible/modules/windows/win_pagefile.ps1 PSUseDeclaredVarsMoreThanAssignments # New PR - bug test_path should be testPath

test/sanity/pslint/skip.txt

@@ -1,8 +0,0 @@
-test/integration/targets/win_ping/library/win_ping_syntax_error.ps1
-test/integration/targets/win_dsc/files/xTestDsc/1.0.0/DSCResources/ANSIBLE_xSetReboot/ANSIBLE_xSetReboot.psm1
-test/integration/targets/win_dsc/files/xTestDsc/1.0.0/xTestDsc.psd1
-test/integration/targets/win_dsc/files/xTestDsc/1.0.0/DSCResources/ANSIBLE_xTestResource/ANSIBLE_xTestResource.psm1
-test/integration/targets/win_dsc/files/xTestDsc/1.0.1/xTestDsc.psd1
-test/integration/targets/win_dsc/files/xTestDsc/1.0.1/DSCResources/ANSIBLE_xTestResource/ANSIBLE_xTestResource.psm1
-test/integration/targets/win_psmodule/files/module/template.psd1
-test/integration/targets/win_psmodule/files/module/template.psm1

test/sanity/pylint/ignore.txt

@@ -1,97 +0,0 @@
-contrib/inventory/gce.py blacklisted-name
-lib/ansible/cli/console.py blacklisted-name
-lib/ansible/compat/selectors/_selectors2.py blacklisted-name
-lib/ansible/executor/playbook_executor.py blacklisted-name
-lib/ansible/executor/task_queue_manager.py blacklisted-name
-lib/ansible/module_utils/facts/network/linux.py blacklisted-name
-lib/ansible/module_utils/network/edgeswitch/edgeswitch_interface.py duplicate-string-formatting-argument
-lib/ansible/module_utils/urls.py blacklisted-name
-lib/ansible/modules/cloud/amazon/data_pipeline.py blacklisted-name
-lib/ansible/modules/cloud/amazon/ec2_vpc_nat_gateway.py blacklisted-name
-lib/ansible/modules/cloud/amazon/ec2_vpc_vpn.py blacklisted-name
-lib/ansible/modules/cloud/amazon/efs.py blacklisted-name
-lib/ansible/modules/cloud/amazon/efs_facts.py blacklisted-name
-lib/ansible/modules/cloud/amazon/kinesis_stream.py blacklisted-name
-lib/ansible/modules/cloud/amazon/s3_sync.py blacklisted-name
-lib/ansible/modules/cloud/google/_gce.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_eip.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_img.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_instance_template.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_lb.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_mig.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_net.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_pd.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_snapshot.py blacklisted-name
-lib/ansible/modules/cloud/google/gce_tag.py blacklisted-name
-lib/ansible/modules/cloud/google/_gcp_backend_service.py blacklisted-name
-lib/ansible/modules/cloud/google/_gcp_healthcheck.py blacklisted-name
-lib/ansible/modules/cloud/lxc/lxc_container.py blacklisted-name
-lib/ansible/modules/files/copy.py blacklisted-name
-lib/ansible/modules/files/patch.py blacklisted-name
-lib/ansible/modules/files/synchronize.py blacklisted-name
-lib/ansible/modules/monitoring/statusio_maintenance.py blacklisted-name
-lib/ansible/modules/monitoring/zabbix/zabbix_maintenance.py blacklisted-name
-lib/ansible/modules/net_tools/basics/uri.py blacklisted-name
-lib/ansible/modules/network/cloudengine/ce_command.py blacklisted-name
-lib/ansible/modules/network/cloudengine/ce_dldp_interface.py blacklisted-name
-lib/ansible/modules/network/cloudengine/ce_mlag_interface.py blacklisted-name
-lib/ansible/modules/network/cloudvision/cv_server_provision.py blacklisted-name
-lib/ansible/modules/network/illumos/dladm_etherstub.py blacklisted-name
-lib/ansible/modules/network/illumos/dladm_iptun.py blacklisted-name
-lib/ansible/modules/network/illumos/dladm_linkprop.py blacklisted-name
-lib/ansible/modules/network/illumos/dladm_vlan.py blacklisted-name
-lib/ansible/modules/network/illumos/dladm_vnic.py blacklisted-name
-lib/ansible/modules/network/illumos/flowadm.py blacklisted-name
-lib/ansible/modules/network/illumos/ipadm_addr.py blacklisted-name
-lib/ansible/modules/network/illumos/ipadm_addrprop.py blacklisted-name
-lib/ansible/modules/network/illumos/ipadm_if.py blacklisted-name
-lib/ansible/modules/network/illumos/ipadm_ifprop.py blacklisted-name
-lib/ansible/modules/network/illumos/ipadm_prop.py blacklisted-name
-lib/ansible/modules/network/vyos/vyos_command.py blacklisted-name
-lib/ansible/modules/packaging/language/pip.py blacklisted-name
-lib/ansible/modules/packaging/os/yum.py blacklisted-name
-lib/ansible/modules/source_control/git.py blacklisted-name
-lib/ansible/modules/system/alternatives.py blacklisted-name
-lib/ansible/modules/system/beadm.py blacklisted-name
-lib/ansible/modules/system/cronvar.py blacklisted-name
-lib/ansible/modules/system/dconf.py blacklisted-name
-lib/ansible/modules/system/filesystem.py blacklisted-name
-lib/ansible/modules/system/gconftool2.py blacklisted-name
-lib/ansible/modules/system/interfaces_file.py blacklisted-name
-lib/ansible/modules/system/iptables.py blacklisted-name
-lib/ansible/modules/system/java_cert.py blacklisted-name
-lib/ansible/modules/system/lvg.py blacklisted-name
-lib/ansible/modules/system/lvol.py blacklisted-name
-lib/ansible/modules/system/parted.py blacklisted-name
-lib/ansible/modules/system/timezone.py blacklisted-name
-lib/ansible/modules/utilities/logic/wait_for.py blacklisted-name
-lib/ansible/modules/web_infrastructure/rundeck_acl_policy.py blacklisted-name
-lib/ansible/parsing/vault/__init__.py blacklisted-name
-lib/ansible/playbook/base.py blacklisted-name
-lib/ansible/playbook/helpers.py blacklisted-name
-lib/ansible/playbook/role/__init__.py blacklisted-name
-lib/ansible/plugins/callback/hipchat.py blacklisted-name
-lib/ansible/plugins/connection/lxc.py blacklisted-name
-lib/ansible/plugins/lookup/sequence.py blacklisted-name
-lib/ansible/plugins/strategy/__init__.py blacklisted-name
-lib/ansible/plugins/strategy/linear.py blacklisted-name
-lib/ansible/vars/hostvars.py blacklisted-name
-test/integration/targets/module_utils/module_utils/bar0/foo.py blacklisted-name
-test/integration/targets/module_utils/module_utils/foo.py blacklisted-name
-test/integration/targets/module_utils/module_utils/sub/bar/__init__.py blacklisted-name
-test/integration/targets/module_utils/module_utils/sub/bar/bar.py blacklisted-name
-test/integration/targets/module_utils/module_utils/yak/zebra/foo.py blacklisted-name
-test/legacy/cleanup_gce.py blacklisted-name
-test/legacy/gce_credentials.py blacklisted-name
-test/units/contrib/inventory/test_vmware_inventory.py blacklisted-name
-test/units/executor/test_play_iterator.py blacklisted-name
-test/units/module_utils/basic/test_run_command.py blacklisted-name
-test/units/modules/cloud/amazon/test_ec2_vpc_nat_gateway.py blacklisted-name
-test/units/modules/cloud/amazon/test_ec2_vpc_vpn.py blacklisted-name
-test/units/modules/packaging/os/test_apt.py blacklisted-name
-test/units/modules/system/interfaces_file/test_interfaces_file.py blacklisted-name
-test/units/modules/system/test_known_hosts.py ansible-bad-function
-test/units/parsing/vault/test_vault.py blacklisted-name
-test/units/playbook/role/test_role.py blacklisted-name
-test/units/plugins/test_plugins.py blacklisted-name
-test/units/template/test_templar.py blacklisted-name

test/sanity/validate-modules/skip.txt

@@ -1 +0,0 @@
-lib/ansible/modules/utilities/logic/async_status.py