You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
ansible/hacking/build-ansible.py

131 lines
3.7 KiB
Python

#!/usr/bin/env python
# coding: utf-8
# PYTHON_ARGCOMPLETE_OK
# Copyright: (c) 2019, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
# Make coding more python3-ish
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import argparse
import importlib
import inspect
import os.path
import pkgutil
import sys
import typing as t
try:
import argcomplete
except ImportError:
argcomplete = None
C = t.TypeVar('C')
def build_lib_path(this_script=__file__):
Collections docs generation (#59761) * Build documentation for Ansible-2.10 (formerly known as ACD). Builds plugin docs from collections whose source is on galaxy The new command downloads collections from galaxy, then finds the plugins inside of them to get the documentation for those plugins. * Update the python syntax checks * docs builds can now require python 3.6+. * Move plugin formatter code out to an external tool, antsibull-docs. Collection owners want to be able to extract docs for their own websites as well. * The jinja2 filters, tests, and other support code have moved to antsibull * Remove document_plugins as that has now been integrated into antsibull-docs * Cleanup and bugfix to other build script code: * The Commands class needed to have its metaclass set for abstractmethod to work correctly * Fix lint issues in some command plugins * Add the docs/docsite/rst/collections to .gitignore as everything in that directory will be generated so we don't want any of it saved in the git repository * gitignore the build dir and remove edit docs link on module pages * Add docs/rst/collections as a directory to remove on make clean * Split the collections docs from the main docs * remove version and edit on github * remove version banner for just collections * clarify examples need collection keyword defined * Remove references to plugin documentation locations that no longer exist. * Perhaps the pages in plugins/*.rst should be deprecated altogether and their content moved? * If not, perhaps we want to rephrase and link into the collection documentation? * Or perhaps we want to link to the plugins which are present in collections/ansible/builtin? * Remove PYTHONPATH from the build-ansible calls One of the design goals of the build-ansible.py script was for it to automatically set its library path to include the checkout of ansible and the library of code to implement itself. Because it automatically includes the checkout of ansible, we don't need to set PYTHONPATH in the Makefile any longer. * Create a command to only build ansible-base plugin docs * When building docs for devel, only build the ansible-base docs for now. This is because antsibull needs support for building a "devel tree" of docs. This can be changed once that is implemented * When building docs for the sanity tests, only build the ansible-base plugin docs for now. Those are the docs which are in this repo so that seems appropriate for now.
4 years ago
"""Return path to the common build library directory."""
hacking_dir = os.path.dirname(this_script)
libdir = os.path.abspath(os.path.join(hacking_dir, 'build_library'))
return libdir
Collections docs generation (#59761) * Build documentation for Ansible-2.10 (formerly known as ACD). Builds plugin docs from collections whose source is on galaxy The new command downloads collections from galaxy, then finds the plugins inside of them to get the documentation for those plugins. * Update the python syntax checks * docs builds can now require python 3.6+. * Move plugin formatter code out to an external tool, antsibull-docs. Collection owners want to be able to extract docs for their own websites as well. * The jinja2 filters, tests, and other support code have moved to antsibull * Remove document_plugins as that has now been integrated into antsibull-docs * Cleanup and bugfix to other build script code: * The Commands class needed to have its metaclass set for abstractmethod to work correctly * Fix lint issues in some command plugins * Add the docs/docsite/rst/collections to .gitignore as everything in that directory will be generated so we don't want any of it saved in the git repository * gitignore the build dir and remove edit docs link on module pages * Add docs/rst/collections as a directory to remove on make clean * Split the collections docs from the main docs * remove version and edit on github * remove version banner for just collections * clarify examples need collection keyword defined * Remove references to plugin documentation locations that no longer exist. * Perhaps the pages in plugins/*.rst should be deprecated altogether and their content moved? * If not, perhaps we want to rephrase and link into the collection documentation? * Or perhaps we want to link to the plugins which are present in collections/ansible/builtin? * Remove PYTHONPATH from the build-ansible calls One of the design goals of the build-ansible.py script was for it to automatically set its library path to include the checkout of ansible and the library of code to implement itself. Because it automatically includes the checkout of ansible, we don't need to set PYTHONPATH in the Makefile any longer. * Create a command to only build ansible-base plugin docs * When building docs for devel, only build the ansible-base docs for now. This is because antsibull needs support for building a "devel tree" of docs. This can be changed once that is implemented * When building docs for the sanity tests, only build the ansible-base plugin docs for now. Those are the docs which are in this repo so that seems appropriate for now.
4 years ago
def ansible_lib_path(this_script=__file__):
"""Return path to the common build library directory."""
hacking_dir = os.path.dirname(this_script)
libdir = os.path.abspath(os.path.join(hacking_dir, '..', 'lib'))
return libdir
sys.path.insert(0, ansible_lib_path())
sys.path.insert(0, build_lib_path())
Collections docs generation (#59761) * Build documentation for Ansible-2.10 (formerly known as ACD). Builds plugin docs from collections whose source is on galaxy The new command downloads collections from galaxy, then finds the plugins inside of them to get the documentation for those plugins. * Update the python syntax checks * docs builds can now require python 3.6+. * Move plugin formatter code out to an external tool, antsibull-docs. Collection owners want to be able to extract docs for their own websites as well. * The jinja2 filters, tests, and other support code have moved to antsibull * Remove document_plugins as that has now been integrated into antsibull-docs * Cleanup and bugfix to other build script code: * The Commands class needed to have its metaclass set for abstractmethod to work correctly * Fix lint issues in some command plugins * Add the docs/docsite/rst/collections to .gitignore as everything in that directory will be generated so we don't want any of it saved in the git repository * gitignore the build dir and remove edit docs link on module pages * Add docs/rst/collections as a directory to remove on make clean * Split the collections docs from the main docs * remove version and edit on github * remove version banner for just collections * clarify examples need collection keyword defined * Remove references to plugin documentation locations that no longer exist. * Perhaps the pages in plugins/*.rst should be deprecated altogether and their content moved? * If not, perhaps we want to rephrase and link into the collection documentation? * Or perhaps we want to link to the plugins which are present in collections/ansible/builtin? * Remove PYTHONPATH from the build-ansible calls One of the design goals of the build-ansible.py script was for it to automatically set its library path to include the checkout of ansible and the library of code to implement itself. Because it automatically includes the checkout of ansible, we don't need to set PYTHONPATH in the Makefile any longer. * Create a command to only build ansible-base plugin docs * When building docs for devel, only build the ansible-base docs for now. This is because antsibull needs support for building a "devel tree" of docs. This can be changed once that is implemented * When building docs for the sanity tests, only build the ansible-base plugin docs for now. Those are the docs which are in this repo so that seems appropriate for now.
4 years ago
from build_ansible import commands, errors
def create_arg_parser(program_name):
"""
Creates a command line argument parser
:arg program_name: The name of the script. Used in help texts
"""
parser = argparse.ArgumentParser(prog=program_name,
description="Implements utilities to build Ansible")
return parser
def load(package: str, subclasses: t.Type[C]) -> list[t.Type[C]]:
"""Load modules in the specified package and return concrete types that derive from the specified base class."""
for module in pkgutil.iter_modules(importlib.import_module(package).__path__, f'{package}.'):
try:
importlib.import_module(module.name)
except ImportError:
pass # ignore plugins which are missing dependencies
types: set[t.Type[C]] = set()
queue: list[t.Type[C]] = [subclasses]
while queue:
for child in queue.pop().__subclasses__():
queue.append(child)
if not inspect.isabstract(child):
types.add(child)
return sorted(types, key=lambda sc: sc.__name__)
def main():
"""
Collections docs generation (#59761) * Build documentation for Ansible-2.10 (formerly known as ACD). Builds plugin docs from collections whose source is on galaxy The new command downloads collections from galaxy, then finds the plugins inside of them to get the documentation for those plugins. * Update the python syntax checks * docs builds can now require python 3.6+. * Move plugin formatter code out to an external tool, antsibull-docs. Collection owners want to be able to extract docs for their own websites as well. * The jinja2 filters, tests, and other support code have moved to antsibull * Remove document_plugins as that has now been integrated into antsibull-docs * Cleanup and bugfix to other build script code: * The Commands class needed to have its metaclass set for abstractmethod to work correctly * Fix lint issues in some command plugins * Add the docs/docsite/rst/collections to .gitignore as everything in that directory will be generated so we don't want any of it saved in the git repository * gitignore the build dir and remove edit docs link on module pages * Add docs/rst/collections as a directory to remove on make clean * Split the collections docs from the main docs * remove version and edit on github * remove version banner for just collections * clarify examples need collection keyword defined * Remove references to plugin documentation locations that no longer exist. * Perhaps the pages in plugins/*.rst should be deprecated altogether and their content moved? * If not, perhaps we want to rephrase and link into the collection documentation? * Or perhaps we want to link to the plugins which are present in collections/ansible/builtin? * Remove PYTHONPATH from the build-ansible calls One of the design goals of the build-ansible.py script was for it to automatically set its library path to include the checkout of ansible and the library of code to implement itself. Because it automatically includes the checkout of ansible, we don't need to set PYTHONPATH in the Makefile any longer. * Create a command to only build ansible-base plugin docs * When building docs for devel, only build the ansible-base docs for now. This is because antsibull needs support for building a "devel tree" of docs. This can be changed once that is implemented * When building docs for the sanity tests, only build the ansible-base plugin docs for now. Those are the docs which are in this repo so that seems appropriate for now.
4 years ago
Start our run.
"It all starts here"
"""
subcommands = load('build_ansible.command_plugins', subclasses=commands.Command)
arg_parser = create_arg_parser(os.path.basename(sys.argv[0]))
Collections docs generation (#59761) * Build documentation for Ansible-2.10 (formerly known as ACD). Builds plugin docs from collections whose source is on galaxy The new command downloads collections from galaxy, then finds the plugins inside of them to get the documentation for those plugins. * Update the python syntax checks * docs builds can now require python 3.6+. * Move plugin formatter code out to an external tool, antsibull-docs. Collection owners want to be able to extract docs for their own websites as well. * The jinja2 filters, tests, and other support code have moved to antsibull * Remove document_plugins as that has now been integrated into antsibull-docs * Cleanup and bugfix to other build script code: * The Commands class needed to have its metaclass set for abstractmethod to work correctly * Fix lint issues in some command plugins * Add the docs/docsite/rst/collections to .gitignore as everything in that directory will be generated so we don't want any of it saved in the git repository * gitignore the build dir and remove edit docs link on module pages * Add docs/rst/collections as a directory to remove on make clean * Split the collections docs from the main docs * remove version and edit on github * remove version banner for just collections * clarify examples need collection keyword defined * Remove references to plugin documentation locations that no longer exist. * Perhaps the pages in plugins/*.rst should be deprecated altogether and their content moved? * If not, perhaps we want to rephrase and link into the collection documentation? * Or perhaps we want to link to the plugins which are present in collections/ansible/builtin? * Remove PYTHONPATH from the build-ansible calls One of the design goals of the build-ansible.py script was for it to automatically set its library path to include the checkout of ansible and the library of code to implement itself. Because it automatically includes the checkout of ansible, we don't need to set PYTHONPATH in the Makefile any longer. * Create a command to only build ansible-base plugin docs * When building docs for devel, only build the ansible-base docs for now. This is because antsibull needs support for building a "devel tree" of docs. This can be changed once that is implemented * When building docs for the sanity tests, only build the ansible-base plugin docs for now. Those are the docs which are in this repo so that seems appropriate for now.
4 years ago
arg_parser.add_argument('--debug', dest='debug', required=False, default=False,
action='store_true',
help='Show tracebacks and other debugging information')
subparsers = arg_parser.add_subparsers(title='Subcommands', dest='command',
help='for help use build-ansible.py SUBCOMMANDS -h')
for subcommand in subcommands:
subcommand.init_parser(subparsers.add_parser)
if argcomplete:
argcomplete.autocomplete(arg_parser)
args = arg_parser.parse_args(sys.argv[1:])
if args.command is None:
print('Please specify a subcommand to run')
sys.exit(1)
for subcommand in subcommands:
if subcommand.name == args.command:
command = subcommand
break
else:
# Note: We should never trigger this because argparse should shield us from it
print('Error: {0} was not a recognized subcommand'.format(args.command))
sys.exit(1)
try:
retval = command.main(args)
except (errors.DependencyError, errors.MissingUserInput, errors.InvalidUserInput) as e:
print(e)
if args.debug:
raise
sys.exit(2)
sys.exit(retval)
if __name__ == '__main__':
main()