ansible-galaxy - add configuration options for more flexible collection signature verification (#77026)

* Add a toggle to control the number of signatures required to verify the authenticity of a collection

* Make the default number of required valid signatures 1

* Add option to make signature verification strict and fail if there are no valid signatures (e.g. "+1")

* Use a regex to validate --required-valid-signature-count

* Add a toggle to limit the gpg status codes that are considered a failure

* Update documentation and changelog

* Add unit and integration tests for the new options

* Fixes #77146

Fix using user-provided signatures when running 'ansible-galaxy collection verify ns.coll --offline'

Add a test for a user-provided signature when running ansible-galaxy collection verify with --offline

Fix displaying overall gpg failure without extra verbosity

Add a test for displaying gpg failure without verbosity

Improve documentation to be more clear that signature verification only currently applies to collections directly sourced from Galaxy servers
pull/77342/head
Sloane Hertel 2 years ago committed by GitHub
parent 21827522dc
commit f96a661ada
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -22,3 +22,16 @@ minor_changes:
verification with ``ansible-galaxy collection verify --offline``.
- >-
``ansible-galaxy collection install`` - Add a global toggle to turn off GPG signature verification.
- >-
``ansible-galaxy collection install`` - Add a global configuration to modify the required number of
signatures that must verify the authenticity of the collection. By default, the number of required
successful signatures is 1.
Set this option to ``all`` to require all signatures verify the collection.
To ensure signature verification fails if there are no valid signatures, prepend the value with '+',
such as ``+all`` or ``+1``.
- >-
``ansible-galaxy collection install`` - Add a global ignore list for gpg signature errors.
This can be used to ignore certain signatures when the number of required successful signatures is all.
When the required number of successful signatures is a positive integer, the only effect this has is to
display fewer errors to the user on failure (success is determined by having the minimum number of
successful signatures, in which case all errors are disregarded).

@ -70,10 +70,34 @@ You can also keep a collection adjacent to the current playbook, under a ``colle
See :ref:`collection_structure` for details on the collection directory structure.
Collections signed by a Galaxy server can be verified during installation with GnuPG. To opt into signature verification, configure a keyring for ``ansible-galaxy`` with native GnuPG tooling and provide the file path with the ``--keyring`` CLI option. Signatures provided by the Galaxy server will be used to verify the collection's ``MANIFEST.json``. If verification is unsuccessful, the collection will not be installed. GnuPG signature verification can be disabled with ``--disable-gpg-verify`` or by configuring :ref:`GALAXY_DISABLE_GPG_VERIFY`.
Collections signed by a Galaxy server can be verified during installation with GnuPG. To opt into signature verification, configure a keyring for ``ansible-galaxy`` with native GnuPG tooling and provide the file path with the ``--keyring`` CLI option or ref:`GALAXY_GPG_KEYRING`. Signatures provided by the Galaxy server will be used to verify the collection's ``MANIFEST.json``.
Use the ``--signature`` option to verify the collection's ``MANIFEST.json`` with additional signatures to those provided by the Galaxy server. Supplemental signatures should be provided as URIs.
.. code-block:: bash
ansible-galaxy collection install my_namespace.my_collection --signature https://examplehost.com/detached_signature.asc --signature file:///path/to/local/detached_signature.asc --keyring ~/.ansible/pubring.kbx
ansible-galaxy collection install my_namespace.my_collection --signature https://examplehost.com/detached_signature.asc --keyring ~/.ansible/pubring.kbx
GnuPG verification only occurs for collections installed from a Galaxy server. User-provided signatures are not used to verify collections installed from git repositories, source directories, or URLs/paths to tar.gz files.
By default, verification is considered successful if a minimum of 1 signature successfully verifies the collection. The number of required signatures can be configured with ``--required-valid-signature-count`` or :ref:`GALAXY_REQUIRED_VALID_SIGNATURE_COUNT`. All signatures can be required by setting the option to ``all``. To fail signature verification if no valid signatures are found, prepend the value with ``+``, such as ``+all`` or ``+1``.
.. code-block:: bash
export ANSIBLE_GALAXY_GPG_KEYRING=~/.ansible/pubring.kbx
export ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT=2
ansible-galaxy collection install my_namespace.my_collection --signature https://examplehost.com/detached_signature.asc --signature file:///path/to/local/detached_signature.asc
Certain GnuPG errors can be ignored with ``--ignore-signature-status-code`` or :ref:`GALAXY_REQUIRED_VALID_SIGNATURE_COUNT`. :ref:`GALAXY_REQUIRED_VALID_SIGNATURE_COUNT` should be a list, and ``--ignore-signature-status-code`` can be provided multiple times to ignore multiple additional error status codes.
This example requires any signatures provided by the Galaxy server to verify the collection except if they fail due to NO_PUBKEY:
.. code-block:: bash
export ANSIBLE_GALAXY_GPG_KEYRING=~/.ansible/pubring.kbx
export ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT=all
ansible-galaxy collection install my_namespace.my_collection --ignore-signature-status-code NO_PUBKEY
If verification fails for the example above, only errors other than NO_PUBKEY will be displayed.
If verification is unsuccessful, the collection will not be installed. GnuPG signature verification can be disabled with ``--disable-gpg-verify`` or by configuring :ref:`GALAXY_DISABLE_GPG_VERIFY`.

@ -23,7 +23,9 @@ You can specify the following keys for each collection entry:
The ``version`` key uses the same range identifier format documented in :ref:`collections_older_version`.
The ``signatures`` key accepts a list of signature sources that are used to supplement those found on the Galaxy server during collection installation and ``ansible-galaxy collection verify``. Signature sources should be URIs that contain the detached signature. These are only used for collections on Galaxy servers. The ``--keyring`` CLI option must be provided if signatures are specified.
The ``signatures`` key accepts a list of signature sources that are used to supplement those found on the Galaxy server during collection installation and ``ansible-galaxy collection verify``. Signature sources should be URIs that contain the detached signature. The ``--keyring`` CLI option must be provided if signatures are specified.
Signatures are only used to verify collections on Galaxy servers. User-provided signatures are not used to verify collections installed from git repositories, source directories, or URLs/paths to tar.gz files.
.. code-block:: yaml

@ -34,11 +34,13 @@ from ansible.galaxy.collection import (
publish_collection,
validate_collection_name,
validate_collection_path,
verify_collections
verify_collections,
SIGNATURE_COUNT_RE,
)
from ansible.galaxy.collection.concrete_artifact_manager import (
ConcreteArtifactsManager,
)
from ansible.galaxy.collection.gpg import GPG_ERROR_MAP
from ansible.galaxy.dependency_resolution.dataclasses import Requirement
from ansible.galaxy.role import GalaxyRole
@ -82,14 +84,19 @@ def with_collection_artifacts_manager(wrapped_method):
if 'artifacts_manager' in kwargs:
return wrapped_method(*args, **kwargs)
artifacts_manager_kwargs = {'validate_certs': not context.CLIARGS['ignore_certs']}
keyring = context.CLIARGS.get('keyring', None)
if keyring is not None:
keyring = GalaxyCLI._resolve_path(keyring)
artifacts_manager_kwargs.update({
'keyring': GalaxyCLI._resolve_path(keyring),
'required_signature_count': context.CLIARGS.get('required_valid_signature_count', None),
'ignore_signature_errors': context.CLIARGS.get('ignore_gpg_errors', None),
})
with ConcreteArtifactsManager.under_tmpdir(
C.DEFAULT_LOCAL_TMP,
validate_certs=not context.CLIARGS['ignore_certs'],
keyring=keyring,
**artifacts_manager_kwargs
) as concrete_artifact_cm:
kwargs['artifacts_manager'] = concrete_artifact_cm
return wrapped_method(*args, **kwargs)
@ -140,6 +147,15 @@ def _get_collection_widths(collections):
return fqcn_length, version_length
def validate_signature_count(value):
match = re.match(SIGNATURE_COUNT_RE, value)
if match is None:
raise ValueError("{value} is not a valid signature count value")
return value
class GalaxyCLI(CLI):
'''command to manage Ansible roles in shared repositories, the default of which is Ansible Galaxy *https://galaxy.ansible.com*.'''
@ -396,6 +412,17 @@ class GalaxyCLI(CLI):
help='An additional signature source to verify the authenticity of the MANIFEST.json before using '
'it to verify the rest of the contents of a collection from a Galaxy server. Use in '
'conjunction with a positional collection name (mutually exclusive with --requirements-file).')
valid_signature_count_help = 'The number of signatures that must successfully verify the collection. This should be a positive integer ' \
'or all to signify that all signatures must be used to verify the collection. ' \
'Prepend the value with + to fail if no valid signatures are found for the collection (e.g. +all).'
ignore_gpg_status_help = 'A status code to ignore during signature verification (for example, NO_PUBKEY). ' \
'Provide this option multiple times to ignore a list of status codes. ' \
'Descriptions for the choices can be seen at L(https://github.com/gpg/gnupg/blob/master/doc/DETAILS#general-status-codes).'
verify_parser.add_argument('--required-valid-signature-count', dest='required_valid_signature_count', type=validate_signature_count,
help=valid_signature_count_help, default=C.GALAXY_REQUIRED_VALID_SIGNATURE_COUNT)
verify_parser.add_argument('--ignore-signature-status-code', dest='ignore_gpg_errors', type=str, action='append',
help=ignore_gpg_status_help, default=C.GALAXY_IGNORE_INVALID_SIGNATURE_STATUS_CODES,
choices=list(GPG_ERROR_MAP.keys()))
def add_install_options(self, parser, parents=None):
galaxy_type = 'collection' if parser.metavar == 'COLLECTION_ACTION' else 'role'
@ -426,6 +453,13 @@ class GalaxyCLI(CLI):
help="Force overwriting an existing {0} and its "
"dependencies.".format(galaxy_type))
valid_signature_count_help = 'The number of signatures that must successfully verify the collection. This should be a positive integer ' \
'or -1 to signify that all signatures must be used to verify the collection. ' \
'Prepend the value with + to fail if no valid signatures are found for the collection (e.g. +all).'
ignore_gpg_status_help = 'A status code to ignore during signature verification (for example, NO_PUBKEY). ' \
'Provide this option multiple times to ignore a list of status codes. ' \
'Descriptions for the choices can be seen at L(https://github.com/gpg/gnupg/blob/master/doc/DETAILS#general-status-codes).'
if galaxy_type == 'collection':
install_parser.add_argument('-p', '--collections-path', dest='collections_path',
default=self._get_default_collection_path(),
@ -445,6 +479,11 @@ class GalaxyCLI(CLI):
help='An additional signature source to verify the authenticity of the MANIFEST.json before '
'installing the collection from a Galaxy server. Use in conjunction with a positional '
'collection name (mutually exclusive with --requirements-file).')
install_parser.add_argument('--required-valid-signature-count', dest='required_valid_signature_count', type=validate_signature_count,
help=valid_signature_count_help, default=C.GALAXY_REQUIRED_VALID_SIGNATURE_COUNT)
install_parser.add_argument('--ignore-signature-status-code', dest='ignore_gpg_errors', type=str, action='append',
help=ignore_gpg_status_help, default=C.GALAXY_IGNORE_INVALID_SIGNATURE_STATUS_CODES,
choices=list(GPG_ERROR_MAP.keys()))
else:
install_parser.add_argument('-r', '--role-file', dest='requirements',
help='A file containing a list of roles to be installed.')
@ -455,6 +494,11 @@ class GalaxyCLI(CLI):
install_parser.add_argument('--disable-gpg-verify', dest='disable_gpg_verify', action='store_true',
default=C.GALAXY_DISABLE_GPG_VERIFY,
help='Disable GPG signature verification when installing collections from a Galaxy server')
install_parser.add_argument('--required-valid-signature-count', dest='required_valid_signature_count', type=validate_signature_count,
help=valid_signature_count_help, default=C.GALAXY_REQUIRED_VALID_SIGNATURE_COUNT)
install_parser.add_argument('--ignore-signature-status-code', dest='ignore_gpg_errors', type=str, action='append',
help=ignore_gpg_status_help, default=C.GALAXY_IGNORE_INVALID_SIGNATURE_STATUS_CODES,
choices=list(GPG_ERROR_MAP.keys()))
install_parser.add_argument('-g', '--keep-scm-meta', dest='keep_scm_meta', action='store_true',
default=False,

@ -1461,6 +1461,47 @@ GALAXY_GPG_KEYRING:
description:
- Configure the keyring used for GPG signature verification during collection installation and verification.
version_added: '2.13'
GALAXY_IGNORE_INVALID_SIGNATURE_STATUS_CODES:
type: list
env:
- name: ANSIBLE_GALAXY_IGNORE_SIGNATURE_STATUS_CODES
ini:
- section: galaxy
key: ignore_signature_status_codes
description:
- A list of GPG status codes to ignore during GPG signature verfication.
See L(https://github.com/gpg/gnupg/blob/master/doc/DETAILS#general-status-codes) for status code descriptions.
- If fewer signatures successfully verify the collection than `GALAXY_REQUIRED_VALID_SIGNATURE_COUNT`,
signature verification will fail even if all error codes are ignored.
choices:
- EXPSIG
- EXPKEYSIG
- REVKEYSIG
- BADSIG
- ERRSIG
- NO_PUBKEY
- MISSING_PASSPHRASE
- BAD_PASSPHRASE
- NODATA
- UNEXPECTED
- ERROR
- FAILURE
- BADARMOR
- KEYEXPIRED
- KEYREVOKED
- NO_SECKEY
GALAXY_REQUIRED_VALID_SIGNATURE_COUNT:
type: str
default: 1
env:
- name: ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT
ini:
- section: galaxy
key: required_valid_signature_count
description:
- The number of signatures that must be successful during GPG signature verification while installing or verifying collections.
- This should be a positive integer or all to indicate all signatures must successfully validate the collection.
- Prepend + to the value to fail if no valid signatures are found for the collection.
HOST_KEY_CHECKING:
# note: constant not in use by ssh plugin anymore
# TODO: check non ssh connection plugins for use/migration

@ -12,6 +12,7 @@ import functools
import json
import os
import queue
import re
import shutil
import stat
import sys
@ -75,7 +76,8 @@ from ansible.galaxy.collection.galaxy_api_proxy import MultiGalaxyAPIProxy
from ansible.galaxy.collection.gpg import (
run_gpg_verify,
parse_gpg_errors,
get_signature_from_source
get_signature_from_source,
GPG_ERROR_MAP,
)
from ansible.galaxy.dependency_resolution import (
build_collection_dependency_resolver,
@ -103,12 +105,15 @@ MANIFEST_FILENAME = 'MANIFEST.json'
ModifiedContent = namedtuple('ModifiedContent', ['filename', 'expected', 'installed'])
SIGNATURE_COUNT_RE = r"^(?P<strict>\+)?(?:(?P<count>\d+)|(?P<all>all))$"
class CollectionSignatureError(Exception):
def __init__(self, reasons=None, stdout=None, rc=None):
def __init__(self, reasons=None, stdout=None, rc=None, ignore=False):
self.reasons = reasons
self.stdout = stdout
self.rc = rc
self.ignore = ignore
self._reason_wrapper = None
@ -197,11 +202,11 @@ def verify_local_collection(
return result
manifest_file = os.path.join(to_text(b_collection_path, errors='surrogate_or_strict'), MANIFEST_FILENAME)
signatures = []
signatures = list(local_collection.signatures)
if verify_local_only and local_collection.source_info is not None:
signatures = [info["signature"] for info in local_collection.source_info["signatures"]]
signatures = [info["signature"] for info in local_collection.source_info["signatures"]] + signatures
elif not verify_local_only and remote_collection.signatures:
signatures = remote_collection.signatures
signatures = list(remote_collection.signatures) + signatures
keyring_configured = artifacts_manager.keyring is not None
if not keyring_configured and signatures:
@ -213,17 +218,18 @@ def verify_local_collection(
"the origin of the collection. "
"Skipping signature verification."
)
else:
for signature in signatures:
try:
verify_file_signature(manifest_file, to_text(signature, errors='surrogate_or_strict'), artifacts_manager.keyring)
except CollectionSignatureError as error:
display.vvvv(error.report(local_collection.fqcn))
result.success = False
if not result.success:
elif keyring_configured:
if not verify_file_signatures(
local_collection.fqcn,
manifest_file,
signatures,
artifacts_manager.keyring,
artifacts_manager.required_successful_signature_count,
artifacts_manager.ignore_signature_errors,
):
result.success = False
return result
elif signatures:
display.vvvv(f"GnuPG signature verification succeeded, verifying contents of {local_collection}")
display.vvvv(f"GnuPG signature verification succeeded, verifying contents of {local_collection}")
if verify_local_only:
# since we're not downloading this, just seed it with the value from disk
@ -324,8 +330,58 @@ def verify_local_collection(
return result
def verify_file_signature(manifest_file, detached_signature, keyring):
# type: (str, str, str) -> None
def verify_file_signatures(fqcn, manifest_file, detached_signatures, keyring, required_successful_count, ignore_signature_errors):
# type: (str, str, list[str], str, str, list[str]) -> bool
successful = 0
error_messages = []
signature_count_requirements = re.match(SIGNATURE_COUNT_RE, required_successful_count).groupdict()
strict = signature_count_requirements['strict'] or False
require_all = signature_count_requirements['all']
require_count = signature_count_requirements['count']
if require_count is not None:
require_count = int(require_count)
for signature in detached_signatures:
signature = to_text(signature, errors='surrogate_or_strict')
try:
verify_file_signature(manifest_file, signature, keyring, ignore_signature_errors)
except CollectionSignatureError as error:
if error.ignore:
# Do not include ignored errors in either the failed or successful count
continue
error_messages.append(error.report(fqcn))
else:
successful += 1
if require_all:
continue
if successful == require_count:
break
if strict and not successful:
verified = False
display.display(f"Signature verification failed for '{fqcn}': no successful signatures")
elif require_all:
verified = not error_messages
if not verified:
display.display(f"Signature verification failed for '{fqcn}': some signatures failed")
else:
verified = not detached_signatures or require_count == successful
if not verified:
display.display(f"Signature verification failed for '{fqcn}': fewer successful signatures than required")
if not verified:
for msg in error_messages:
display.vvvv(msg)
return verified
def verify_file_signature(manifest_file, detached_signature, keyring, ignore_signature_errors):
# type: (str, str, str, list[str]) -> None
"""Run the gpg command and parse any errors. Raises CollectionSignatureError on failure."""
gpg_result, gpg_verification_rc = run_gpg_verify(manifest_file, detached_signature, keyring, display)
@ -336,8 +392,19 @@ def verify_file_signature(manifest_file, detached_signature, keyring):
except StopIteration:
pass
else:
reasons = set(error.get_gpg_error_description() for error in chain([error], errors))
raise CollectionSignatureError(reasons=reasons, stdout=gpg_result, rc=gpg_verification_rc)
reasons = []
ignored_reasons = 0
for error in chain([error], errors):
# Get error status (dict key) from the class (dict value)
status_code = list(GPG_ERROR_MAP.keys())[list(GPG_ERROR_MAP.values()).index(error.__class__)]
if status_code in ignore_signature_errors:
ignored_reasons += 1
reasons.append(error.get_gpg_error_description())
ignore = len(reasons) == ignored_reasons
raise CollectionSignatureError(reasons=set(reasons), stdout=gpg_result, rc=gpg_verification_rc, ignore=ignore)
if gpg_verification_rc:
raise CollectionSignatureError(stdout=gpg_result, rc=gpg_verification_rc)
@ -756,6 +823,18 @@ def verify_collections(
local_collection = Candidate.from_dir_path(
b_search_path, artifacts_manager,
)
supplemental_signatures = [
get_signature_from_source(source, display)
for source in collection.signature_sources or []
]
local_collection = Candidate(
local_collection.fqcn,
local_collection.ver,
local_collection.src,
local_collection.type,
signatures=frozenset(supplemental_signatures),
)
break
else:
raise AnsibleError(message=default_err)
@ -764,9 +843,6 @@ def verify_collections(
remote_collection = None
else:
signatures = api_proxy.get_signatures(local_collection)
# NOTE: If there are no Galaxy server signatures, only user-provided signature URLs,
# NOTE: those alone validate the MANIFEST.json and the remote collection is not downloaded.
# NOTE: The remote MANIFEST.json is only used in verification if there are no signatures.
signatures.extend([
get_signature_from_source(source, display)
for source in collection.signature_sources or []
@ -784,7 +860,10 @@ def verify_collections(
try:
# NOTE: If there are no signatures, trigger the lookup. If found,
# NOTE: it'll cache download URL and token in artifact manager.
if not signatures:
# NOTE: If there are no Galaxy server signatures, only user-provided signature URLs,
# NOTE: those alone validate the MANIFEST.json and the remote collection is not downloaded.
# NOTE: The remote MANIFEST.json is only used in verification if there are no signatures.
if not signatures and not collection.signature_sources:
api_proxy.get_collection_version_metadata(
remote_collection,
)
@ -1211,7 +1290,9 @@ def install(collection, path, artifacts_manager): # FIXME: mv to dataclasses?
b_collection_path,
artifacts_manager._b_working_directory,
collection.signatures,
artifacts_manager.keyring
artifacts_manager.keyring,
artifacts_manager.required_successful_signature_count,
artifacts_manager.ignore_signature_errors,
)
if (collection.is_online_index_pointer and isinstance(collection.src, GalaxyAPI)):
write_source_metadata(
@ -1249,23 +1330,17 @@ def write_source_metadata(collection, b_collection_path, artifacts_manager):
raise
def verify_artifact_manifest(manifest_file, signatures, keyring):
# type: (str, str, str) -> None
def verify_artifact_manifest(manifest_file, signatures, keyring, required_signature_count, ignore_signature_errors):
# type: (str, list[str], str, str, list[str]) -> None
failed_verify = False
coll_path_parts = to_text(manifest_file, errors='surrogate_or_strict').split(os.path.sep)
collection_name = '%s.%s' % (coll_path_parts[-3], coll_path_parts[-2]) # get 'ns' and 'coll' from /path/to/ns/coll/MANIFEST.json
for signature in signatures:
try:
verify_file_signature(manifest_file, to_text(signature, errors='surrogate_or_strict'), keyring)
except CollectionSignatureError as error:
display.vvvv(error.report(collection_name))
failed_verify = True
if failed_verify:
if not verify_file_signatures(collection_name, manifest_file, signatures, keyring, required_signature_count, ignore_signature_errors):
raise AnsibleError(f"Not installing {collection_name} because GnuPG signature verification failed.")
display.vvvv(f"GnuPG signature verification succeeded for {collection_name}")
def install_artifact(b_coll_targz_path, b_collection_path, b_temp_path, signatures, keyring):
def install_artifact(b_coll_targz_path, b_collection_path, b_temp_path, signatures, keyring, required_signature_count, ignore_signature_errors):
"""Install a collection from tarball under a given path.
:param b_coll_targz_path: Collection tarball to be installed.
@ -1273,15 +1348,17 @@ def install_artifact(b_coll_targz_path, b_collection_path, b_temp_path, signatur
:param b_temp_path: Temporary dir path.
:param signatures: frozenset of signatures to verify the MANIFEST.json
:param keyring: The keyring used during GPG verification
:param required_signature_count: The number of signatures that must successfully verify the collection
:param ignore_signature_errors: GPG errors to ignore during signature verification
"""
try:
with tarfile.open(b_coll_targz_path, mode='r') as collection_tar:
# Verify the signature on the MANIFEST.json before extracting anything else
_extract_tar_file(collection_tar, MANIFEST_FILENAME, b_collection_path, b_temp_path)
if signatures and keyring is not None:
if keyring is not None:
manifest_file = os.path.join(to_text(b_collection_path, errors='surrogate_or_strict'), MANIFEST_FILENAME)
verify_artifact_manifest(manifest_file, signatures, keyring)
verify_artifact_manifest(manifest_file, signatures, keyring, required_signature_count, ignore_signature_errors)
files_member_obj = collection_tar.getmember('FILES.json')
with _tarfile_extract(collection_tar, files_member_obj) as (dummy, files_obj):

@ -56,9 +56,8 @@ class ConcreteArtifactsManager:
* caching all of above
* retrieving the metadata out of the downloaded artifacts
"""
def __init__(self, b_working_directory, validate_certs=True, keyring=None, timeout=60):
# type: (bytes, bool, str, int) -> None
def __init__(self, b_working_directory, validate_certs=True, keyring=None, timeout=60, required_signature_count=None, ignore_signature_errors=None):
# type: (bytes, bool, str, int, str, list[str]) -> None
"""Initialize ConcreteArtifactsManager caches and costraints."""
self._validate_certs = validate_certs # type: bool
self._artifact_cache = {} # type: dict[bytes, bytes]
@ -70,11 +69,23 @@ class ConcreteArtifactsManager:
self._supplemental_signature_cache = {} # type: dict[str, str]
self._keyring = keyring # type: str
self.timeout = timeout # type: int
self._required_signature_count = required_signature_count # type: str
self._ignore_signature_errors = ignore_signature_errors # type: list[str]
@property
def keyring(self):
return self._keyring
@property
def required_successful_signature_count(self):
return self._required_signature_count
@property
def ignore_signature_errors(self):
if self._ignore_signature_errors is None:
return []
return self._ignore_signature_errors
def get_galaxy_artifact_source_info(self, collection):
# type: (Candidate) -> dict[str, str | list[dict[str, str]]]
server = collection.src.api_server
@ -321,6 +332,8 @@ class ConcreteArtifactsManager:
temp_dir_base, # type: str
validate_certs=True, # type: bool
keyring=None, # type: str
required_signature_count=None, # type: str
ignore_signature_errors=None, # type: list[str]
): # type: (...) -> t.Iterator[ConcreteArtifactsManager]
"""Custom ConcreteArtifactsManager constructor with temp dir.
@ -335,7 +348,13 @@ class ConcreteArtifactsManager:
)
b_temp_path = to_bytes(temp_path, errors='surrogate_or_strict')
try:
yield cls(b_temp_path, validate_certs, keyring=keyring)
yield cls(
b_temp_path,
validate_certs,
keyring=keyring,
required_signature_count=required_signature_count,
ignore_signature_errors=ignore_signature_errors
)
finally:
rmtree(b_temp_path)

@ -441,9 +441,9 @@
- name: namespace7.name
version: "1.0.0"
signatures:
- "file://{{ gpg_homedir }}/namespace7-name-1.0.0-MANIFEST.json.asc"
- "{{ not_mine }}"
- "{{ also_not_mine }}"
- "file://{{ gpg_homedir }}/namespace7-name-1.0.0-MANIFEST.json.asc"
- namespace8.name
- name: namespace9.name
signatures:
@ -495,6 +495,7 @@
keyring: "{{ gpg_homedir }}/pubring.kbx"
environment:
ANSIBLE_COLLECTIONS_PATH: '{{ galaxy_dir }}/ansible_collections'
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: all
- name: assert invalid signature is fatal with ansible-galaxy install - {{ test_name }}
assert:
@ -517,6 +518,7 @@
keyring: "{{ gpg_homedir }}/pubring.kbx"
environment:
ANSIBLE_COLLECTIONS_PATH: '{{ galaxy_dir }}/ansible_collections'
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: all
- name: get result of install collections with ansible-galaxy install - {{ test_name }}
slurp:
@ -556,6 +558,106 @@
- namespace8
- namespace9
- name: install collections with only one valid signature using ansible-galaxy install - {{ test_name }}
command: ansible-galaxy install -r {{ req_file }} {{ cli_opts }} {{ galaxy_verbosity }}
register: install_req
vars:
req_file: "{{ galaxy_dir }}/ansible_collections/requirements.yaml"
cli_opts: "-s {{ test_name }} --keyring {{ keyring }}"
keyring: "{{ gpg_homedir }}/pubring.kbx"
environment:
ANSIBLE_COLLECTIONS_PATH: '{{ galaxy_dir }}/ansible_collections'
- name: get result of install collections with ansible-galaxy install - {{ test_name }}
slurp:
path: '{{ galaxy_dir }}/ansible_collections/{{ collection }}/name/MANIFEST.json'
register: install_req_actual
loop_control:
loop_var: collection
loop:
- namespace7
- namespace8
- namespace9
- name: assert just one valid signature is not fatal with ansible-galaxy install - {{ test_name }}
assert:
that:
- install_req is success
- '"Installing ''namespace7.name:1.0.0'' to" in install_req.stdout'
- '"Signature verification failed for ''namespace7.name'' (return code 1)" not in install_req.stdout'
- '"Not installing namespace7.name because GnuPG signature verification failed." not in install_stderr'
- '"Installing ''namespace8.name:1.0.0'' to" in install_req.stdout'
- '"Installing ''namespace9.name:1.0.0'' to" in install_req.stdout'
- (install_req_actual.results[0].content | b64decode | from_json).collection_info.version == '1.0.0'
- (install_req_actual.results[1].content | b64decode | from_json).collection_info.version == '1.0.0'
- (install_req_actual.results[2].content | b64decode | from_json).collection_info.version == '1.0.0'
vars:
install_stderr: "{{ install_req.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
- name: clean up collections from last test
file:
path: '{{ galaxy_dir }}/ansible_collections/{{ collection }}/name'
state: absent
loop_control:
loop_var: collection
loop:
- namespace7
- namespace8
- namespace9
- name: install collections with only one valid signature by ignoring the other errors
command: ansible-galaxy install -r {{ req_file }} {{ cli_opts }} {{ galaxy_verbosity }} --ignore-signature-status-code FAILURE
register: install_req
vars:
req_file: "{{ galaxy_dir }}/ansible_collections/requirements.yaml"
cli_opts: "-s {{ test_name }} --keyring {{ keyring }}"
keyring: "{{ gpg_homedir }}/pubring.kbx"
environment:
ANSIBLE_COLLECTIONS_PATH: '{{ galaxy_dir }}/ansible_collections'
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: all
ANSIBLE_GALAXY_IGNORE_SIGNATURE_STATUS_CODES: BADSIG # cli option is appended and both status codes are ignored
- name: get result of install collections with ansible-galaxy install - {{ test_name }}
slurp:
path: '{{ galaxy_dir }}/ansible_collections/{{ collection }}/name/MANIFEST.json'
register: install_req_actual
loop_control:
loop_var: collection
loop:
- namespace7
- namespace8
- namespace9
- name: assert invalid signature is not fatal with ansible-galaxy install - {{ test_name }}
assert:
that:
- install_req is success
- '"Installing ''namespace7.name:1.0.0'' to" in install_req.stdout'
- '"Signature verification failed for ''namespace7.name'' (return code 1)" not in install_req.stdout'
- '"Not installing namespace7.name because GnuPG signature verification failed." not in install_stderr'
- '"Installing ''namespace8.name:1.0.0'' to" in install_req.stdout'
- '"Installing ''namespace9.name:1.0.0'' to" in install_req.stdout'
- (install_req_actual.results[0].content | b64decode | from_json).collection_info.version == '1.0.0'
- (install_req_actual.results[1].content | b64decode | from_json).collection_info.version == '1.0.0'
- (install_req_actual.results[2].content | b64decode | from_json).collection_info.version == '1.0.0'
vars:
install_stderr: "{{ install_req.stderr | regex_replace(reset_color) | regex_replace(color) | regex_replace('\\n', ' ') }}"
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
- name: clean up collections from last test
file:
path: '{{ galaxy_dir }}/ansible_collections/{{ collection }}/name'
state: absent
loop_control:
loop_var: collection
loop:
- namespace7
- namespace8
- namespace9
# Uncomment once pulp container is at pulp>=0.5.0
#- name: install cache.cache at the current latest version
# command: ansible-galaxy collection install cache.cache -s '{{ test_name }}' -vvv
@ -830,6 +932,33 @@
- ignore_invalid_signature is success
- '"Installing ''namespace1.name1:1.0.9'' to" in ignore_invalid_signature.stdout'
- name: use lenient signature verification (default) without providing signatures
command: ansible-galaxy collection install namespace1.name1:1.0.0 -vvvv --keyring {{ gpg_homedir }}/pubring.kbx --force
environment:
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: "all"
register: missing_signature
- assert:
that:
- missing_signature is success
- missing_signature.rc == 0
- '"namespace1.name1:1.0.0 was installed successfully" in missing_signature.stdout'
- '"Signature verification failed for ''namespace1.name1'': no successful signatures" not in missing_signature.stdout'
- name: use strict signature verification without providing signatures
command: ansible-galaxy collection install namespace1.name1:1.0.0 -vvvv --keyring {{ gpg_homedir }}/pubring.kbx --force
environment:
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: "+1"
ignore_errors: yes
register: missing_signature
- assert:
that:
- missing_signature is failed
- missing_signature.rc == 1
- '"Signature verification failed for ''namespace1.name1'': no successful signatures" in missing_signature.stdout'
- '"Not installing namespace1.name1 because GnuPG signature verification failed" in missing_signature.stderr'
- name: Remove the collection
file:
path: '{{ galaxy_dir }}/ansible_collections/namespace1'

@ -27,8 +27,9 @@
- name: install the collection from the server
command: ansible-galaxy collection install ansible_test.verify:1.0.0 -s {{ test_api_fallback }} {{ galaxy_verbosity }}
# This command is hardcoded with -vvvv purposefully to evaluate extra verbosity messages
- name: verify the collection against the first valid server
command: ansible-galaxy collection verify ansible_test.verify:1.0.0 -vvv {{ galaxy_verbosity }}
command: ansible-galaxy collection verify ansible_test.verify:1.0.0 -vvvv {{ galaxy_verbosity }}
register: verify
- assert:
@ -317,6 +318,32 @@
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
# This command is hardcoded with -vvvv purposefully to evaluate extra verbosity messages
- name: verify the installed collection with invalid detached signature offline
command: ansible-galaxy collection verify namespace1.name1:1.0.0 -vvvv {{ signature_options }} --offline
vars:
signature_options: "--signature {{ signature }} --keyring {{ keyring }}"
signature: "file://{{ gpg_homedir }}/namespace1-name1-1.0.9-MANIFEST.json.asc"
keyring: "{{ gpg_homedir }}/pubring.kbx"
register: verify
ignore_errors: yes
- assert:
that:
- verify.rc != 0
- '"Signature verification failed for ''namespace1.name1'' (return code 1)" in verify.stdout'
- expected_errors[0] in verify_stdout
- expected_errors[1] in verify_stdout
vars:
expected_errors:
- "* This is the counterpart to SUCCESS and used to indicate a program failure."
- "* The signature with the keyid has not been verified okay."
stdout_no_color: "{{ verify.stdout | regex_replace(reset_color) | regex_replace(color) }}"
# Remove formatting from the reason so it's one line
verify_stdout: "{{ stdout_no_color | regex_replace('\"') | regex_replace('\\n') | regex_replace(' ', ' ') }}"
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
- include_tasks: revoke_gpg_key.yml
# This command is hardcoded with -vvvv purposefully to evaluate extra verbosity messages
@ -345,6 +372,102 @@
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
# This command is hardcoded with no verbosity purposefully to evaluate overall gpg failure
- name: verify that ignoring the signature error and no successful signatures is not successful verification
command: ansible-galaxy collection verify namespace1.name1:1.0.0 {{ signature_options }}
vars:
signature_options: "--signature {{ signature }} --keyring {{ keyring }}"
signature: "file://{{ gpg_homedir }}/namespace1-name1-1.0.0-MANIFEST.json.asc"
keyring: "{{ gpg_homedir }}/pubring.kbx"
register: verify
ignore_errors: yes
environment:
ANSIBLE_GALAXY_IGNORE_SIGNATURE_STATUS_CODES: REVKEYSIG,KEYREVOKED
- assert:
that:
- verify.rc != 0
- '"Signature verification failed for ''namespace1.name1'': fewer successful signatures than required" in verify.stdout'
- ignored_errors[0] not in verify_stdout
- ignored_errors[1] not in verify_stdout
vars:
ignored_errors:
- "* The used key has been revoked by its owner."
- "* The signature with the keyid is good, but the signature was made by a revoked key."
stdout_no_color: "{{ verify.stdout | regex_replace(reset_color) | regex_replace(color) }}"
# Remove formatting from the reason so it's one line
verify_stdout: "{{ stdout_no_color | regex_replace('\"') | regex_replace('\\n') | regex_replace(' ', ' ') }}"
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
# This command is hardcoded with -vvvv purposefully to evaluate extra verbosity messages
- name: verify that ignoring the signature error and no successful signatures and required signature count all is successful verification
command: ansible-galaxy collection verify namespace1.name1:1.0.0 -vvvv {{ signature_options }}
vars:
signature_options: "--signature {{ signature }} --keyring {{ keyring }}"
signature: "file://{{ gpg_homedir }}/namespace1-name1-1.0.0-MANIFEST.json.asc"
keyring: "{{ gpg_homedir }}/pubring.kbx"
register: verify
ignore_errors: yes
environment:
ANSIBLE_GALAXY_IGNORE_SIGNATURE_STATUS_CODES: REVKEYSIG,KEYREVOKED
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: all
- assert:
that:
- verify is success
- verify.rc == 0
- '"Signature verification failed for ''namespace1.name1'': fewer successful signatures than required" not in verify.stdout'
- success_messages[0] in verify_stdout
- success_messages[1] in verify_stdout
- ignored_errors[0] not in verify_stdout
- ignored_errors[1] not in verify_stdout
vars:
success_messages:
- "GnuPG signature verification succeeded, verifying contents of namespace1.name1:1.0.0"
- "Successfully verified that checksums for 'namespace1.name1:1.0.0' match the remote collection."
ignored_errors:
- "* The used key has been revoked by its owner."
- "* The signature with the keyid is good, but the signature was made by a revoked key."
stdout_no_color: "{{ verify.stdout | regex_replace(reset_color) | regex_replace(color) }}"
# Remove formatting from the reason so it's one line
verify_stdout: "{{ stdout_no_color | regex_replace('\"') | regex_replace('\\n') | regex_replace(' ', ' ') }}"
reset_color: '\x1b\[0m'
color: '\x1b\[[0-9];[0-9]{2}m'
- name: use lenient signature verification (default) without providing signatures
command: ansible-galaxy collection verify namespace1.name1:1.0.0 -vvvv --keyring {{ gpg_homedir }}/pubring.kbx
environment:
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: "1"
register: verify
ignore_errors: yes
- assert:
that:
- verify is success
- verify.rc == 0
- error_message not in verify.stdout
- success_messages[0] in verify.stdout
- success_messages[1] in verify.stdout
vars:
error_message: "Signature verification failed for 'namespace1.name1': fewer successful signatures than required"
success_messages:
- "GnuPG signature verification succeeded, verifying contents of namespace1.name1:1.0.0"
- "Successfully verified that checksums for 'namespace1.name1:1.0.0' match the remote collection."
- name: use strict signature verification without providing signatures
command: ansible-galaxy collection verify namespace1.name1:1.0.0 -vvvv --keyring {{ gpg_homedir }}/pubring.kbx
environment:
ANSIBLE_GALAXY_REQUIRED_VALID_SIGNATURE_COUNT: "+1"
register: verify
ignore_errors: yes
- assert:
that:
- verify is failed
- verify.rc == 1
- '"Signature verification failed for ''namespace1.name1'': no successful signatures" in verify.stdout'
- name: empty installed collections
file:
path: "{{ galaxy_dir }}/ansible_collections"

@ -217,6 +217,42 @@ def server_config(monkeypatch):
return server1, server2, server3
@pytest.mark.parametrize(
'required_signature_count,valid',
[
("1", True),
("+1", True),
("all", True),
("+all", True),
("-1", False),
("invalid", False),
("1.5", False),
("+", False),
]
)
def test_cli_options(required_signature_count, valid, monkeypatch):
cli_args = [
'ansible-galaxy',
'collection',
'install',
'namespace.collection:1.0.0',
'--keyring',
'~/.ansible/pubring.kbx',
'--required-valid-signature-count',
required_signature_count
]
galaxy_cli = GalaxyCLI(args=cli_args)
mock_execute_install = MagicMock()
monkeypatch.setattr(galaxy_cli, '_execute_install_collection', mock_execute_install)
if valid:
galaxy_cli.run()
else:
with pytest.raises(SystemExit, match='2') as error:
galaxy_cli.run()
@pytest.mark.parametrize('global_ignore_certs', [True, False])
def test_validate_certs(global_ignore_certs, monkeypatch):
cli_args = [

@ -17,7 +17,7 @@ import tarfile
import yaml
from io import BytesIO, StringIO
from mock import MagicMock
from mock import MagicMock, patch
import ansible.module_utils.six.moves.urllib.error as urllib_error
@ -982,3 +982,50 @@ def test_install_collection_with_circular_dependency(collection_artifact, monkey
assert display_msgs[1] == "Starting collection install process"
assert display_msgs[2] == "Installing 'ansible_namespace.collection:0.1.0' to '%s'" % to_text(collection_path)
assert display_msgs[3] == "ansible_namespace.collection:0.1.0 was installed successfully"
@pytest.mark.parametrize(
"signatures,required_successful_count,ignore_errors,expected_success",
[
([], 'all', [], True),
(["good_signature"], 'all', [], True),
(["good_signature", collection.gpg.GpgBadArmor(status='failed')], 'all', [], False),
([collection.gpg.GpgBadArmor(status='failed')], 'all', [], False),
# This is expected to succeed because ignored does not increment failed signatures.
# "all" signatures is not a specific number, so all == no (non-ignored) signatures in this case.
([collection.gpg.GpgBadArmor(status='failed')], 'all', ["BADARMOR"], True),
([collection.gpg.GpgBadArmor(status='failed'), "good_signature"], 'all', ["BADARMOR"], True),
([], '+all', [], False),
([collection.gpg.GpgBadArmor(status='failed')], '+all', ["BADARMOR"], False),
([], '1', [], True),
([], '+1', [], False),
(["good_signature"], '2', [], False),
(["good_signature", collection.gpg.GpgBadArmor(status='failed')], '2', [], False),
# This is expected to fail because ignored does not increment successful signatures.
# 2 signatures are required, but only 1 is successful.
(["good_signature", collection.gpg.GpgBadArmor(status='failed')], '2', ["BADARMOR"], False),
(["good_signature", "good_signature"], '2', [], True),
]
)
def test_verify_file_signatures(signatures, required_successful_count, ignore_errors, expected_success):
# type: (List[bool], int, bool, bool) -> None
def gpg_error_generator(results):
for result in results:
if isinstance(result, collection.gpg.GpgBaseError):
yield result
fqcn = 'ns.coll'
manifest_file = 'MANIFEST.json'
keyring = '~/.ansible/pubring.kbx'
with patch.object(collection, 'run_gpg_verify', MagicMock(return_value=("somestdout", 0,))):
with patch.object(collection, 'parse_gpg_errors', MagicMock(return_value=gpg_error_generator(signatures))):
assert collection.verify_file_signatures(
fqcn,
manifest_file,
signatures,
keyring,
required_successful_count,
ignore_errors
) == expected_success

Loading…
Cancel
Save