Add ansible-galaxy collection list command (#65022)

* Start adding ansible-galaxy collection list options

* Working list all collections and list a specific collection

* Nuke debugging cruft

* Use to_text to get a string of the FQCN for sorting

* Improve collection output formatting

- add header
- display collection name and version in separate columns
- width of columns is dynamic based on collection name and version length

* Make role list output match collection output list

- add header
- add columns for role name and version
- make column width dynamic based on name and version length

* remove debug statemnt and extra header

* Revert "Make role list output match collection output list"

This reverts commit a0b3db47bb3b198aafd34c1f1be5b6561af2f928.

* Add validate_collection_path function

Utility function for ensuring a collection target ends with 'ansible_collection'

* Use validate_collection_path

* Do not warn if a specific collection in found in any search path

* Fix extraneous warning and remove duplicate code

Do not warn when listing a specific collection and it does not exist
in other collection paths.

Restructure the code that loops through collection paths to remove
duplicate conditional code.

* Indicate role path was found

* Use new function name

* WIP Save Point

* Use separate functions for role and collection list

* Wrap error message

There may be a better way to do this besides hard coding a line break, but this
does make the message a lot more readable.

* Add validate_collection_path function (#66441)

* Add validate_collection_path function

Utility function for ensuring a collection target ends with 'ansible_collection'

* Fix bad syntax

* Correct docstring

* Bikeshed the names

* Properly list a single role

* Simplify _display_warnings()

Only display warnings. Move exception raise back to each caller.

* Move private methods to private functions

They don't need self, so it makes sense to have them as functions
Get rid of _display_warnings() function since it doesn't do anything worthy of
an independent function.

* Add integration tests for ansible-galaxy collection list

* Fix docs sanity test

* Fix bug where ansible_collections dir does not exist

The path may exist, but if there is no ansible_collections dir inside that path,
an exception was raised in find_existing_collections().

Add integration test for this scenario

* Put execute_list() method back

* Add some informational messages for debugging

* Add unit tests

Units tests for the various private methods in support of collection list

* Start adding unit tests for test_execute_list

* Display collection path when listing specific collection

* Add unit tests for listing all collections and specific collection

- Create fixture for creating test objects
- Add function for controlling os.path.isdir results

* Set defaults for minimum collection widths

Ensure that collections with small FQCNs display correctly.
Add unit tests

* Split up unit tests and fix fixtures

Add more fixtures for mocking objects during the specific collection tests

* Change help message for -p in list subcommand

Give accurate description of what it actually does rather than trying to use language shared between sub commands.

* Disable colorized output in unit test

* Add docs for collection list

* Fix integration test on macOS

The temp file path is really long on macOS, so the warning message gets wrapped
across multiple lines. That make seth grep fail. Switch to matching on a smaller
part of the warning.

* Recreate common path options for collections

Improve help about what the '-p' option does and how it works.

* Remove unnecessary elif after continue statements

* Account for duplicate paths in collections_searh_paths

If someone specifies the same path via '-p' that is the COLLECTIONS_PATHS,
do not list the collections twice.

* Docs updates

changelogs/fragments/galaxy-collections-add-list.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-galaxy - add ``collection list`` command for listing installed collections (https://github.com/ansible/ansible/pull/65022)

docs/docsite/rst/galaxy/user_guide.rst

@@ -78,6 +78,11 @@ Install multiple collections with a requirements file
 
 .. include:: ../shared_snippets/installing_multiple_collections.txt
 
+Listing installed collections
+-----------------------------
+
+To list installed collections, run ``ansible-galaxy collection list``. See :ref:`collections_listing` for more details.
+
 
 Configuring the ``ansible-galaxy`` client
 ------------------------------------------

docs/docsite/rst/user_guide/collections_using.rst

@@ -44,6 +44,80 @@ Configuring the ``ansible-galaxy`` client
 
 .. include:: ../shared_snippets/galaxy_server_list.txt
 
+.. _collections_listing:
+
+Listing collections
+===================
+
+To list installed collections, run ``ansible-galaxy collection list``. This shows all of the installed collections found in the configured collections search paths. The path where the collections are located are displayed as well as version information. If no version information is available, a ``*`` is displayed for the version number.
+
+.. code-block:: shell
+
+      # /home/astark/.ansible/collections/ansible_collections
+      Collection                 Version
+      -------------------------- -------
+      cisco.aci                  0.0.5
+      cisco.mso                  0.0.4
+      sandwiches.ham             *
+      splunk.enterprise_security 0.0.5
+
+      # /usr/share/ansible/collections/ansible_collections
+      Collection        Version
+      ----------------- -------
+      fortinet.fortios  1.0.6
+      pureport.pureport 0.0.8
+      sensu.sensu_go    1.3.0
+
+Run with ``-vvv`` to display more detailed information.
+
+To list a specific collection, pass a valid fully qualified collection name (FQCN) to the command ``ansible-galaxy collection list``. All instances of the collection will be listed.
+
+.. code-block:: shell
+
+      > ansible-galaxy collection list fortinet.fortios
+
+      # /home/astark/.ansible/collections/ansible_collections
+      Collection       Version
+      ---------------- -------
+      fortinet.fortios 1.0.1
+
+      # /usr/share/ansible/collections/ansible_collections
+      Collection       Version
+      ---------------- -------
+      fortinet.fortios 1.0.6
+
+To search other paths for collections, use the ``-p`` option. Specify multiple search paths by separating them with a ``:``. The list of paths specified on the command line will be added to the beginning of the configured collections search paths.
+
+.. code-block:: shell
+
+      > ansible-galaxy collection list -p '/opt/ansible/collections:/etc/ansible/collections'
+
+      # /opt/ansible/collections/ansible_collections
+      Collection      Version
+      --------------- -------
+      sandwiches.club 1.7.2
+
+      # /etc/ansible/collections/ansible_collections
+      Collection     Version
+      -------------- -------
+      sandwiches.pbj 1.2.0
+
+      # /home/astark/.ansible/collections/ansible_collections
+      Collection                 Version
+      -------------------------- -------
+      cisco.aci                  0.0.5
+      cisco.mso                  0.0.4
+      fortinet.fortios           1.0.1
+      sandwiches.ham             *
+      splunk.enterprise_security 0.0.5
+
+      # /usr/share/ansible/collections/ansible_collections
+      Collection        Version
+      ----------------- -------
+      fortinet.fortios  1.0.6
+      pureport.pureport 0.0.8
+      sensu.sensu_go    1.3.0
+
 
 .. _using_collections:
 

lib/ansible/cli/galaxy.py

@@ -24,6 +24,8 @@ from ansible.galaxy import Galaxy, get_collections_galaxy_meta_info
 from ansible.galaxy.api import GalaxyAPI
 from ansible.galaxy.collection import (
     build_collection,
+    CollectionRequirement,
+    find_existing_collections,
     install_collections,
     publish_collection,
     validate_collection_name,
@@ -34,6 +36,7 @@ from ansible.galaxy.login import GalaxyLogin
 from ansible.galaxy.role import GalaxyRole
 from ansible.galaxy.token import BasicAuthToken, GalaxyToken, KeycloakToken, NoTokenSentinel
 from ansible.module_utils.ansible_release import __version__ as ansible_version
+from ansible.module_utils.common.collections import is_iterable
 from ansible.module_utils._text import to_bytes, to_native, to_text
 from ansible.module_utils import six
 from ansible.parsing.yaml.loader import AnsibleLoader
@@ -45,6 +48,51 @@ display = Display()
 urlparse = six.moves.urllib.parse.urlparse
 
 
+def _display_header(path, h1, h2, w1=10, w2=7):
+    display.display('\n# {0}\n{1:{cwidth}} {2:{vwidth}}\n{3} {4}\n'.format(
+        path,
+        h1,
+        h2,
+        '-' * max([len(h1), w1]),  # Make sure that the number of dashes is at least the width of the header
+        '-' * max([len(h2), w2]),
+        cwidth=w1,
+        vwidth=w2,
+    ))
+
+
+def _display_role(gr):
+    install_info = gr.install_info
+    version = None
+    if install_info:
+        version = install_info.get("version", None)
+    if not version:
+        version = "(unknown version)"
+    display.display("- %s, %s" % (gr.name, version))
+
+
+def _display_collection(collection, cwidth=10, vwidth=7, min_cwidth=10, min_vwidth=7):
+    display.display('{fqcn:{cwidth}} {version:{vwidth}}'.format(
+        fqcn=to_text(collection),
+        version=collection.latest_version,
+        cwidth=max(cwidth, min_cwidth),  # Make sure the width isn't smaller than the header
+        vwidth=max(vwidth, min_vwidth)
+    ))
+
+
+def _get_collection_widths(collections):
+    if is_iterable(collections):
+        fqcn_set = set(to_text(c) for c in collections)
+        version_set = set(to_text(c.latest_version) for c in collections)
+    else:
+        fqcn_set = set([to_text(collections)])
+        version_set = set([collections.latest_version])
+
+    fqcn_length = len(max(fqcn_set, key=len))
+    version_length = len(max(version_set, key=len))
+
+    return fqcn_length, version_length
+
+
 class GalaxyCLI(CLI):
     '''command to manage Ansible roles in shared repositories, the default of which is Ansible Galaxy *https://galaxy.ansible.com*.'''
 
@@ -99,12 +147,12 @@ class GalaxyCLI(CLI):
                                 help='The path to the directory containing your roles. The default is the first '
                                      'writable one configured via DEFAULT_ROLES_PATH: %s ' % default_roles_path)
 
-        default_collections_path = C.config.get_configuration_definition('COLLECTIONS_PATHS').get('default', '')
         collections_path = opt_help.argparse.ArgumentParser(add_help=False)
-        collections_path.add_argument('-p', '--collections-path', dest='collections_path', type=opt_help.unfrack_path(pathsep=True),
+        collections_path.add_argument('-p', '--collection-path', dest='collections_path', type=opt_help.unfrack_path(pathsep=True),
                                       default=C.COLLECTIONS_PATHS, action=opt_help.PrependListAction,
-                                      help='The path to the directory containing your collections. The default is the first '
+                                      help="One or more directories to search for collections in addition "
-                                      'writable one configured via COLLECTIONS_PATHS: %s ' % default_collections_path)
+                                      "to the default COLLECTIONS_PATHS. Separate multiple paths "
+                                      "with '{0}'.".format(os.path.pathsep))
 
         # Add sub parser for the Galaxy role type (role or collection)
         type_parser = self.parser.add_subparsers(metavar='TYPE', dest='type')
@@ -118,6 +166,7 @@ class GalaxyCLI(CLI):
         self.add_build_options(collection_parser, parents=[common, force])
         self.add_publish_options(collection_parser, parents=[common])
         self.add_install_options(collection_parser, parents=[common, force])
+        self.add_list_options(collection_parser, parents=[common, collections_path])
         self.add_verify_options(collection_parser, parents=[common, collections_path])
 
         # Add sub parser for the Galaxy role actions
@@ -127,7 +176,7 @@ class GalaxyCLI(CLI):
         self.add_init_options(role_parser, parents=[common, force, offline])
         self.add_remove_options(role_parser, parents=[common, roles_path])
         self.add_delete_options(role_parser, parents=[common, github])
-        self.add_list_options(role_parser, parents=[common, roles_path])
+        self.add_list_options(role_parser, parents=[common])
         self.add_search_options(role_parser, parents=[common])
         self.add_import_options(role_parser, parents=[common, github])
         self.add_setup_options(role_parser, parents=[common, roles_path])
@@ -175,11 +224,16 @@ class GalaxyCLI(CLI):
         delete_parser.set_defaults(func=self.execute_delete)
 
     def add_list_options(self, parser, parents=None):
+        galaxy_type = 'role'
+        if parser.metavar == 'COLLECTION_ACTION':
+            galaxy_type = 'collection'
+
         list_parser = parser.add_parser('list', parents=parents,
-                                        help='Show the name and version of each role installed in the roles_path.')
+                                        help='Show the name and version of each {0} installed in the {0}s_path.'.format(galaxy_type))
+
         list_parser.set_defaults(func=self.execute_list)
 
-        list_parser.add_argument('role', help='Role', nargs='?', metavar='role')
+        list_parser.add_argument(galaxy_type, help=galaxy_type.capitalize(), nargs='?', metavar=galaxy_type)
 
     def add_search_options(self, parser, parents=None):
         search_parser = parser.add_parser('search', parents=parents,
@@ -619,9 +673,9 @@ class GalaxyCLI(CLI):
                 requirements.append((name, requirement or '*', None))
         return requirements
 
-############################
+    ############################
-# execute actions
+    # execute actions
-############################
+    ############################
 
     def execute_role(self):
         """
@@ -759,7 +813,8 @@ class GalaxyCLI(CLI):
 
                 if any(r.match(os.path.join(rel_root, f)) for r in skeleton_ignore_re):
                     continue
-                elif galaxy_type == 'collection' and own_skeleton and rel_root == '.' and f == 'galaxy.yml.j2':
+
+                if galaxy_type == 'collection' and own_skeleton and rel_root == '.' and f == 'galaxy.yml.j2':
                     # Special use case for galaxy.yml.j2 in our own default collection skeleton. We build the options
                     # dynamically which requires special options to be set.
 
@@ -863,6 +918,13 @@ class GalaxyCLI(CLI):
             no_deps = context.CLIARGS['no_deps']
             force_deps = context.CLIARGS['force_with_deps']
 
+            if collections and requirements_file:
+                raise AnsibleError("The positional collection_name arg and --requirements-file are mutually exclusive.")
+            elif not collections and not requirements_file:
+                raise AnsibleError("You must specify a collection name or a requirements file.")
+
+            if requirements_file:
+                requirements_file = GalaxyCLI._resolve_path(requirements_file)
             requirements = self._require_one_of_collections_requirements(collections, requirements_file)
 
             output_path = GalaxyCLI._resolve_path(output_path)
@@ -1004,51 +1066,156 @@ class GalaxyCLI(CLI):
 
     def execute_list(self):
         """
-        lists the roles installed on the local system or matches a single role passed as an argument.
+        List installed collections or roles
         """
 
-        def _display_role(gr):
+        if context.CLIARGS['type'] == 'role':
-            install_info = gr.install_info
+            self.execute_list_role()
-            version = None
+        elif context.CLIARGS['type'] == 'collection':
-            if install_info:
+            self.execute_list_collection()
-                version = install_info.get("version", None)
+
-            if not version:
+    def execute_list_role(self):
-                version = "(unknown version)"
+        """
-            display.display("- %s, %s" % (gr.name, version))
+        List all roles installed on the local system or a specific role
-
+        """
-        if context.CLIARGS['role']:
+
-            # show the requested role, if it exists
+        path_found = False
-            name = context.CLIARGS['role']
+        role_found = False
-            gr = GalaxyRole(self.galaxy, self.api, name)
+        warnings = []
-            if gr.metadata:
+        roles_search_paths = context.CLIARGS['roles_path']
-                display.display('# %s' % os.path.dirname(gr.path))
+        role_name = context.CLIARGS['role']
-                _display_role(gr)
+
+        for path in roles_search_paths:
+            role_path = GalaxyCLI._resolve_path(path)
+            if os.path.isdir(path):
+                path_found = True
+            else:
+                warnings.append("- the configured path {0} does not exist.".format(path))
+                continue
+
+            if role_name:
+                # show the requested role, if it exists
+                gr = GalaxyRole(self.galaxy, self.api, role_name, path=os.path.join(role_path, role_name))
+                if os.path.isdir(gr.path):
+                    role_found = True
+                    display.display('# %s' % os.path.dirname(gr.path))
+                    _display_role(gr)
+                    break
+                warnings.append("- the role %s was not found" % role_name)
             else:
-                display.display("- the role %s was not found" % name)
-        else:
-            # show all valid roles in the roles_path directory
-            roles_path = context.CLIARGS['roles_path']
-            path_found = False
-            warnings = []
-            for path in roles_path:
-                role_path = os.path.expanduser(path)
                 if not os.path.exists(role_path):
                     warnings.append("- the configured path %s does not exist." % role_path)
                     continue
-                elif not os.path.isdir(role_path):
+
+                if not os.path.isdir(role_path):
                     warnings.append("- the configured path %s, exists, but it is not a directory." % role_path)
                     continue
+
                 display.display('# %s' % role_path)
                 path_files = os.listdir(role_path)
-                path_found = True
                 for path_file in path_files:
                     gr = GalaxyRole(self.galaxy, self.api, path_file, path=path)
                     if gr.metadata:
                         _display_role(gr)
-            for w in warnings:
+
-                display.warning(w)
+        # Do not warn if the role was found in any of the search paths
-            if not path_found:
+        if role_found and role_name:
-                raise AnsibleOptionsError("- None of the provided paths was usable. Please specify a valid path with --roles-path")
+            warnings = []
+
+        for w in warnings:
+            display.warning(w)
+
+        if not path_found:
+            raise AnsibleOptionsError("- None of the provided paths were usable. Please specify a valid path with --{0}s-path".format(context.CLIARGS['type']))
+
+        return 0
+
+    def execute_list_collection(self):
+        """
+        List all collections installed on the local system
+        """
+
+        collections_search_paths = set(context.CLIARGS['collections_path'])
+        collection_name = context.CLIARGS['collection']
+        default_collections_path = C.config.get_configuration_definition('COLLECTIONS_PATHS').get('default')
+
+        warnings = []
+        path_found = False
+        collection_found = False
+        for path in collections_search_paths:
+            collection_path = GalaxyCLI._resolve_path(path)
+            if not os.path.exists(path):
+                if path in default_collections_path:
+                    # don't warn for missing default paths
+                    continue
+                warnings.append("- the configured path {0} does not exist.".format(collection_path))
+                continue
+
+            if not os.path.isdir(collection_path):
+                warnings.append("- the configured path {0}, exists, but it is not a directory.".format(collection_path))
+                continue
+
+            path_found = True
+
+            if collection_name:
+                # list a specific collection
+
+                validate_collection_name(collection_name)
+                namespace, collection = collection_name.split('.')
+
+                collection_path = validate_collection_path(collection_path)
+                b_collection_path = to_bytes(os.path.join(collection_path, namespace, collection), errors='surrogate_or_strict')
+
+                if not os.path.exists(b_collection_path):
+                    warnings.append("- unable to find {0} in collection paths".format(collection_name))
+                    continue
+
+                if not os.path.isdir(collection_path):
+                    warnings.append("- the configured path {0}, exists, but it is not a directory.".format(collection_path))
+                    continue
+
+                collection_found = True
+                collection = CollectionRequirement.from_path(b_collection_path, False)
+                fqcn_width, version_width = _get_collection_widths(collection)
+
+                _display_header(collection_path, 'Collection', 'Version', fqcn_width, version_width)
+                _display_collection(collection, fqcn_width, version_width)
+
+            else:
+                # list all collections
+                collection_path = validate_collection_path(path)
+                if os.path.isdir(collection_path):
+                    display.vvv("Searching {0} for collections".format(collection_path))
+                    collections = find_existing_collections(collection_path)
+                else:
+                    # There was no 'ansible_collections/' directory in the path, so there
+                    # or no collections here.
+                    display.vvv("No 'ansible_collections' directory found at {0}".format(collection_path))
+                    continue
+
+                if not collections:
+                    display.vvv("No collections found at {0}".format(collection_path))
+                    continue
+
+                # Display header
+                fqcn_width, version_width = _get_collection_widths(collections)
+                _display_header(collection_path, 'Collection', 'Version', fqcn_width, version_width)
+
+                # Sort collections by the namespace and name
+                collections.sort(key=to_text)
+                for collection in collections:
+                    _display_collection(collection, fqcn_width, version_width)
+
+        # Do not warn if the specific collection was found in any of the search paths
+        if collection_found and collection_name:
+            warnings = []
+
+        for w in warnings:
+            display.warning(w)
+
+        if not path_found:
+            raise AnsibleOptionsError("- None of the provided paths were usable. Please specify a valid path with --{0}s-path".format(context.CLIARGS['type']))
+
         return 0
 
     def execute_publish(self):

lib/ansible/galaxy/collection.py

@@ -535,7 +535,7 @@ def validate_collection_name(name):
         return name
 
     raise AnsibleError("Invalid collection name '%s', "
-                       "name must be in the format <namespace>.<collection>. "
+                       "name must be in the format <namespace>.<collection>. \n"
                        "Please make sure namespace and collection name contains "
                        "characters from [a-zA-Z0-9_] only." % name)
 

test/integration/targets/ansible-galaxy/cleanup.yml

@@ -9,3 +9,11 @@
         - "cleanup-default.yml"
       loop_control:
         loop_var: cleanup_filename
+
+    - name: Remove default collection directories
+      file:
+        path: "{{ item }}"
+        state: absent
+      loop:
+        - "~/.ansible/collections/ansible_collections"
+        - /usr/share/ansible/collections/ansible_collections

test/integration/targets/ansible-galaxy/runme.sh

@@ -4,7 +4,7 @@ set -eux -o pipefail
 
 ansible-playbook setup.yml "$@"
 
-trap 'ansible-playbook cleanup.yml' EXIT
+trap 'ansible-playbook ${ANSIBLE_PLAYBOOK_DIR}/cleanup.yml' EXIT
 
 # Very simple version test
 ansible-galaxy --version
@@ -39,7 +39,6 @@ popd # "${galaxy_local_test_role_dir}"
 # Status message function (f_ to designate that it's a function)
 f_ansible_galaxy_status()
 {
-
     printf "\n\n\n### Testing ansible-galaxy: %s\n" "${@}"
 }
 
@@ -208,6 +207,88 @@ f_ansible_galaxy_status \
     [[ -f "${galaxy_testdir}/install/ansible_collections/ansible_test/my_collection/MANIFEST.json" ]]
     grep "Installing 'ansible_test.my_collection:1.0.0' to .*" out.txt
 
+
+## ansible-galaxy collection list tests
+
+# Create more collections and put them in various places
+f_ansible_galaxy_status \
+    "setting up for collection list tests"
+
+rm -rf ansible_test/* install/*
+
+NAMES=(zoo museum airport)
+for n in "${NAMES[@]}"; do
+    ansible-galaxy collection init "ansible_test.$n"
+    ansible-galaxy collection build "ansible_test/$n"
+done
+
+ansible-galaxy collection install ansible_test-zoo-1.0.0.tar.gz
+ansible-galaxy collection install ansible_test-museum-1.0.0.tar.gz -p ./install
+ansible-galaxy collection install ansible_test-airport-1.0.0.tar.gz -p ./local
+
+# Change the collection version and install to another location
+sed -i -e 's#^version:.*#version: 2.5.0#' ansible_test/zoo/galaxy.yml
+ansible-galaxy collection build ansible_test/zoo
+ansible-galaxy collection install ansible_test-zoo-2.5.0.tar.gz -p ./local
+
+export ANSIBLE_COLLECTIONS_PATHS=~/.ansible/collections:${galaxy_testdir}/local
+
+f_ansible_galaxy_status \
+    "collection list all collections"
+
+    ansible-galaxy collection list -p ./install | tee out.txt
+
+    [[ $(grep -c ansible_test out.txt) -eq 4 ]]
+
+f_ansible_galaxy_status \
+    "collection list specific collection"
+
+    ansible-galaxy collection list -p ./install ansible_test.airport | tee out.txt
+
+    [[ $(grep -c 'ansible_test\.airport' out.txt) -eq 1 ]]
+
+f_ansible_galaxy_status \
+    "collection list specific collection found in multiple places"
+
+    ansible-galaxy collection list -p ./install ansible_test.zoo | tee out.txt
+
+    [[ $(grep -c 'ansible_test\.zoo' out.txt) -eq 2 ]]
+
+f_ansible_galaxy_status \
+    "collection list all with duplicate paths"
+
+    ansible-galaxy collection list -p ~/.ansible/collections | tee out.txt
+
+    [[ $(grep -c '# /root/.ansible/collections/ansible_collections' out.txt) -eq 1 ]]
+
+f_ansible_galaxy_status \
+    "collection list invalid collection name"
+
+    ansible-galaxy collection list -p ./install dirty.wraughten.name "$@" 2>&1 | tee out.txt || echo "expected failure"
+
+    grep 'ERROR! Invalid collection name' out.txt
+
+f_ansible_galaxy_status \
+    "collection list path not found"
+
+    ansible-galaxy collection list -p ./nope "$@" 2>&1 | tee out.txt || echo "expected failure"
+
+    grep '\[WARNING\]: - the configured path' out.txt
+
+f_ansible_galaxy_status \
+    "collection list missing ansible_collections dir inside path"
+
+    mkdir emptydir
+
+    ansible-galaxy collection list -p ./emptydir "$@"
+
+    rmdir emptydir
+
+unset ANSIBLE_COLLECTIONS_PATHS
+
+## end ansible-galaxy collection list
+
+
 popd # ${galaxy_testdir}
 
 rm -fr "${galaxy_testdir}"

test/units/cli/galaxy/test_display_collection.py

@@ -0,0 +1,47 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2020 Ansible Project
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import pytest
+
+from ansible.cli.galaxy import _display_collection
+
+
+@pytest.fixture
+def collection_object(mocker):
+    def _cobj(fqcn='sandwiches.ham'):
+        cobj = mocker.MagicMock(latest_version='1.5.0')
+        cobj.__str__.return_value = fqcn
+        return cobj
+    return _cobj
+
+
+def test_display_collection(capsys, collection_object):
+    _display_collection(collection_object())
+    out, err = capsys.readouterr()
+
+    assert out == 'sandwiches.ham 1.5.0  \n'
+
+
+def test_display_collections_small_max_widths(capsys, collection_object):
+    _display_collection(collection_object(), 1, 1)
+    out, err = capsys.readouterr()
+
+    assert out == 'sandwiches.ham 1.5.0  \n'
+
+
+def test_display_collections_large_max_widths(capsys, collection_object):
+    _display_collection(collection_object(), 20, 20)
+    out, err = capsys.readouterr()
+
+    assert out == 'sandwiches.ham       1.5.0               \n'
+
+
+def test_display_collection_small_minimum_widths(capsys, collection_object):
+    _display_collection(collection_object('a.b'), min_cwidth=0, min_vwidth=0)
+    out, err = capsys.readouterr()
+
+    assert out == 'a.b        1.5.0  \n'

test/units/cli/galaxy/test_display_header.py

@@ -0,0 +1,41 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2020 Ansible Project
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+from ansible.cli.galaxy import _display_header
+
+
+def test_display_header_default(capsys):
+    _display_header('/collections/path', 'h1', 'h2')
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /collections/path'
+    assert out_lines[2] == 'h1         h2     '
+    assert out_lines[3] == '---------- -------'
+
+
+def test_display_header_widths(capsys):
+    _display_header('/collections/path', 'Collection', 'Version', 18, 18)
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /collections/path'
+    assert out_lines[2] == 'Collection         Version           '
+    assert out_lines[3] == '------------------ ------------------'
+
+
+def test_display_header_small_widths(capsys):
+    _display_header('/collections/path', 'Col', 'Ver', 1, 1)
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /collections/path'
+    assert out_lines[2] == 'Col Ver'
+    assert out_lines[3] == '--- ---'

test/units/cli/galaxy/test_display_role.py

@@ -0,0 +1,28 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2020 Ansible Project
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+from ansible.cli.galaxy import _display_role
+
+
+def test_display_role(mocker, capsys):
+    mocked_galaxy_role = mocker.Mock(install_info=None)
+    mocked_galaxy_role.name = 'testrole'
+    _display_role(mocked_galaxy_role)
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert out_lines[0] == '- testrole, (unknown version)'
+
+
+def test_display_role_known_version(mocker, capsys):
+    mocked_galaxy_role = mocker.Mock(install_info={'version': '1.0.0'})
+    mocked_galaxy_role.name = 'testrole'
+    _display_role(mocked_galaxy_role)
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert out_lines[0] == '- testrole, 1.0.0'

test/units/cli/galaxy/test_execute_list.py

@@ -0,0 +1,40 @@
+# -*- coding: utf-8 -*-
+# -*- coding: utf-8 -*-
+# Copyright (c) 2020 Ansible Project
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import pytest
+
+from ansible import context
+from ansible.cli.galaxy import GalaxyCLI
+
+
+def test_execute_list_role_called(mocker):
+    """Make sure the correct method is called for a role"""
+
+    gc = GalaxyCLI(['ansible-galaxy', 'role', 'list'])
+    context.CLIARGS._store = {'type': 'role'}
+    execute_list_role_mock = mocker.patch('ansible.cli.galaxy.GalaxyCLI.execute_list_role', side_effect=AttributeError('raised intentionally'))
+    execute_list_collection_mock = mocker.patch('ansible.cli.galaxy.GalaxyCLI.execute_list_collection', side_effect=AttributeError('raised intentionally'))
+    with pytest.raises(AttributeError):
+        gc.execute_list()
+
+    assert execute_list_role_mock.call_count == 1
+    assert execute_list_collection_mock.call_count == 0
+
+
+def test_execute_list_collection_called(mocker):
+    """Make sure the correct method is called for a collection"""
+
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list'])
+    context.CLIARGS._store = {'type': 'collection'}
+    execute_list_role_mock = mocker.patch('ansible.cli.galaxy.GalaxyCLI.execute_list_role', side_effect=AttributeError('raised intentionally'))
+    execute_list_collection_mock = mocker.patch('ansible.cli.galaxy.GalaxyCLI.execute_list_collection', side_effect=AttributeError('raised intentionally'))
+    with pytest.raises(AttributeError):
+        gc.execute_list()
+
+    assert execute_list_role_mock.call_count == 0
+    assert execute_list_collection_mock.call_count == 1

test/units/cli/galaxy/test_execute_list_collection.py

@@ -0,0 +1,276 @@
+# Copyright (c) 2020 Ansible Project
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import pytest
+
+from ansible import context
+from ansible.cli.galaxy import GalaxyCLI
+from ansible.errors import AnsibleError, AnsibleOptionsError
+from ansible.galaxy.collection import CollectionRequirement
+from ansible.module_utils._text import to_native
+
+
+def path_exists(path):
+    if to_native(path) == '/root/.ansible/collections/ansible_collections/sandwiches/ham':
+        return False
+    elif to_native(path) == '/usr/share/ansible/collections/ansible_collections/sandwiches/reuben':
+        return False
+    elif to_native(path) == 'nope':
+        return False
+    else:
+        return True
+
+
+def isdir(path):
+    if to_native(path) == 'nope':
+        return False
+    else:
+        return True
+
+
+def cliargs(collections_paths=None, collection_name=None):
+    if collections_paths is None:
+        collections_paths = ['~/root/.ansible/collections', '/usr/share/ansible/collections']
+
+    context.CLIARGS._store = {
+        'collections_path': collections_paths,
+        'collection': collection_name,
+        'type': 'collection',
+    }
+
+
+@pytest.fixture
+def mock_collection_objects(mocker):
+    mocker.patch('ansible.cli.galaxy.GalaxyCLI._resolve_path', side_effect=['/root/.ansible/collections', '/usr/share/ansible/collections'])
+    mocker.patch('ansible.cli.galaxy.validate_collection_path',
+                 side_effect=['/root/.ansible/collections/ansible_collections', '/usr/share/ansible/collections/ansible_collections'])
+
+    collection_args = (
+        (
+            'sandwiches',
+            'pbj',
+            b'/usr/share/ansible/collections/ansible_collections/sandwiches/pbj',
+            mocker.Mock(),
+            ['1.0.0', '1.5.0'],
+            '1.0.0',
+            False,
+        ),
+        (
+            'sandwiches',
+            'pbj',
+            b'/root/.ansible/collections/ansible_collections/sandwiches/pbj',
+            mocker.Mock(),
+            ['1.0.0', '1.5.0'],
+            '1.5.0',
+            False,
+        ),
+        (
+            'sandwiches',
+            'ham',
+            b'/usr/share/ansible/collections/ansible_collections/sandwiches/ham',
+            mocker.Mock(),
+            ['1.0.0'],
+            '1.0.0',
+            False,
+        ),
+        (
+            'sandwiches',
+            'reuben',
+            b'/root/.ansible/collections/ansible_collections/sandwiches/reuben',
+            mocker.Mock(),
+            ['1.0.0', '2.5.0'],
+            '2.5.0',
+            False,
+        ),
+    )
+
+    collections_path_1 = [CollectionRequirement(*cargs) for cargs in collection_args if to_native(cargs[2]).startswith('/root')]
+    collections_path_2 = [CollectionRequirement(*cargs) for cargs in collection_args if to_native(cargs[2]).startswith('/usr/share')]
+    mocker.patch('ansible.cli.galaxy.find_existing_collections', side_effect=[collections_path_1, collections_path_2])
+
+
+@pytest.fixture
+def mock_from_path(mocker):
+    def _from_path(collection_name='pbj'):
+        collection_args = {
+            'sandwiches.pbj': (
+                (
+                    'sandwiches',
+                    'pbj',
+                    b'/root/.ansible/collections/ansible_collections/sandwiches/pbj',
+                    mocker.Mock(),
+                    ['1.0.0', '1.5.0'],
+                    '1.5.0',
+                    False,
+                ),
+                (
+                    'sandwiches',
+                    'pbj',
+                    b'/usr/share/ansible/collections/ansible_collections/sandwiches/pbj',
+                    mocker.Mock(),
+                    ['1.0.0', '1.5.0'],
+                    '1.0.0',
+                    False,
+                ),
+            ),
+            'sandwiches.ham': (
+                (
+                    'sandwiches',
+                    'ham',
+                    b'/usr/share/ansible/collections/ansible_collections/sandwiches/ham',
+                    mocker.Mock(),
+                    ['1.0.0'],
+                    '1.0.0',
+                    False,
+                ),
+            ),
+        }
+
+        from_path_objects = [CollectionRequirement(*args) for args in collection_args[collection_name]]
+        mocker.patch('ansible.galaxy.collection.CollectionRequirement.from_path', side_effect=from_path_objects)
+
+    return _from_path
+
+
+def test_execute_list_collection_all(mocker, capsys, mock_collection_objects):
+    """Test listing all collections from multiple paths"""
+
+    cliargs()
+
+    mocker.patch('os.path.exists', return_value=True)
+    mocker.patch('os.path.isdir', return_value=True)
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list'])
+    gc.execute_list_collection()
+
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert len(out_lines) == 12
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /root/.ansible/collections/ansible_collections'
+    assert out_lines[2] == 'Collection        Version'
+    assert out_lines[3] == '----------------- -------'
+    assert out_lines[4] == 'sandwiches.pbj    1.5.0  '
+    assert out_lines[5] == 'sandwiches.reuben 2.5.0  '
+    assert out_lines[6] == ''
+    assert out_lines[7] == '# /usr/share/ansible/collections/ansible_collections'
+    assert out_lines[8] == 'Collection     Version'
+    assert out_lines[9] == '-------------- -------'
+    assert out_lines[10] == 'sandwiches.ham 1.0.0  '
+    assert out_lines[11] == 'sandwiches.pbj 1.0.0  '
+
+
+def test_execute_list_collection_specific(mocker, capsys, mock_collection_objects, mock_from_path):
+    """Test listing a specific collection"""
+
+    collection_name = 'sandwiches.ham'
+    mock_from_path(collection_name)
+
+    cliargs(collection_name=collection_name)
+    mocker.patch('os.path.exists', path_exists)
+    mocker.patch('os.path.isdir', return_value=True)
+    mocker.patch('ansible.galaxy.collection.validate_collection_name', collection_name)
+    mocker.patch('ansible.cli.galaxy._get_collection_widths', return_value=(14, 5))
+
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list', collection_name])
+    gc.execute_list_collection()
+
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert len(out_lines) == 5
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /usr/share/ansible/collections/ansible_collections'
+    assert out_lines[2] == 'Collection     Version'
+    assert out_lines[3] == '-------------- -------'
+    assert out_lines[4] == 'sandwiches.ham 1.0.0  '
+
+
+def test_execute_list_collection_specific_duplicate(mocker, capsys, mock_collection_objects, mock_from_path):
+    """Test listing a specific collection that exists at multiple paths"""
+
+    collection_name = 'sandwiches.pbj'
+    mock_from_path(collection_name)
+
+    cliargs(collection_name=collection_name)
+    mocker.patch('os.path.exists', path_exists)
+    mocker.patch('os.path.isdir', return_value=True)
+    mocker.patch('ansible.galaxy.collection.validate_collection_name', collection_name)
+
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list', collection_name])
+    gc.execute_list_collection()
+
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert len(out_lines) == 10
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /root/.ansible/collections/ansible_collections'
+    assert out_lines[2] == 'Collection     Version'
+    assert out_lines[3] == '-------------- -------'
+    assert out_lines[4] == 'sandwiches.pbj 1.5.0  '
+    assert out_lines[5] == ''
+    assert out_lines[6] == '# /usr/share/ansible/collections/ansible_collections'
+    assert out_lines[7] == 'Collection     Version'
+    assert out_lines[8] == '-------------- -------'
+    assert out_lines[9] == 'sandwiches.pbj 1.0.0  '
+
+
+def test_execute_list_collection_specific_invalid_fqcn(mocker):
+    """Test an invalid fully qualified collection name (FQCN)"""
+
+    collection_name = 'no.good.name'
+
+    cliargs(collection_name=collection_name)
+    mocker.patch('os.path.exists', return_value=True)
+    mocker.patch('os.path.isdir', return_value=True)
+
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list', collection_name])
+    with pytest.raises(AnsibleError, match='Invalid collection name'):
+        gc.execute_list_collection()
+
+
+def test_execute_list_collection_no_valid_paths(mocker, capsys):
+    """Test listing collections when no valid paths are given"""
+
+    cliargs()
+
+    mocker.patch('os.path.exists', return_value=True)
+    mocker.patch('os.path.isdir', return_value=False)
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list'])
+
+    with pytest.raises(AnsibleOptionsError, match=r'None of the provided paths were usable.'):
+        gc.execute_list_collection()
+
+    out, err = capsys.readouterr()
+
+    assert '[WARNING]: - the configured path' in err
+    assert 'exists, but it is not a directory.' in err
+
+
+def test_execute_list_collection_one_invalid_path(mocker, capsys, mock_collection_objects):
+    """Test listing all collections when one invalid path is given"""
+
+    cliargs()
+    mocker.patch('os.path.exists', return_value=True)
+    mocker.patch('os.path.isdir', isdir)
+    mocker.patch('ansible.cli.galaxy.GalaxyCLI._resolve_path', side_effect=['/root/.ansible/collections', 'nope'])
+    mocker.patch('ansible.utils.color.ANSIBLE_COLOR', False)
+
+    gc = GalaxyCLI(['ansible-galaxy', 'collection', 'list', '-p', 'nope'])
+    gc.execute_list_collection()
+
+    out, err = capsys.readouterr()
+    out_lines = out.splitlines()
+
+    assert out_lines[0] == ''
+    assert out_lines[1] == '# /root/.ansible/collections/ansible_collections'
+    assert out_lines[2] == 'Collection        Version'
+    assert out_lines[3] == '----------------- -------'
+    assert out_lines[4] == 'sandwiches.pbj    1.5.0  '
+    # Only a partial test of the output
+
+    assert err == '[WARNING]: - the configured path nope, exists, but it is not a directory.\n'

test/units/cli/galaxy/test_get_collection_widths.py

@@ -0,0 +1,37 @@
+# -*- coding: utf-8 -*-
+# Copyright (c) 2020 Ansible Project
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import pytest
+
+from ansible.cli.galaxy import _get_collection_widths
+
+
+@pytest.fixture
+def collection_objects(mocker):
+    collection_ham = mocker.MagicMock(latest_version='1.5.0')
+    collection_ham.__str__.return_value = 'sandwiches.ham'
+
+    collection_pbj = mocker.MagicMock(latest_version='2.5')
+    collection_pbj.__str__.return_value = 'sandwiches.pbj'
+
+    collection_reuben = mocker.MagicMock(latest_version='4')
+    collection_reuben.__str__.return_value = 'sandwiches.reuben'
+
+    return [collection_ham, collection_pbj, collection_reuben]
+
+
+def test_get_collection_widths(collection_objects):
+    assert _get_collection_widths(collection_objects) == (17, 5)
+
+
+def test_get_collection_widths_single_collection(mocker):
+    mocked_collection = mocker.MagicMock(latest_version='3.0.0')
+    mocked_collection.__str__.return_value = 'sandwiches.club'
+    # Make this look like it is not iterable
+    mocker.patch('ansible.cli.galaxy.is_iterable', return_value=False)
+
+    assert _get_collection_widths(mocked_collection) == (15, 5)