* Split Ansible docs from core docs (#73616)
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build
This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs. Handle this via `make webdocs
ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
Handle this via `make coredocs`.
* `make webdocs` now always builds all the collection docs
* Use `make coredocs` to limit it to core plugins only
* The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit ccbfdec334)
* fix CI failures, correct version switcher and makefile logic, set args in all cases, allow 2.10 build
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
* Make Azure Pipelines resource_prefix lowercase.
* Make classification of CI files consistent.
* Update package-data sanity test for AZP.
(cherry picked from commit 92b66e3e31)
Co-authored-by: Matt Clay <mclay@redhat.com>
* Move 2.10.0rc1 release date a few days forward. (#71270)
At yesterday's meeting it was decided to have ansible-2.10.0 depend on
ansible-base-2.10.1 so that we can get several fixes for ansible-base's
routing (including adding the gluster.gluster collection).
ansible-base-2.10.1 will release on September 8th. So we will plan on
releasing ansible-2.10.0rc1 on the 10th.
https://meetbot.fedoraproject.org/ansible-community/2020-08-12/ansible_community_meeting.2020-08-12-18.00.html
(cherry picked from commit e507c127e5)
* a few writing style updates (#71212)
(cherry picked from commit 4f0bd5de38)
* Fix code markups and add link to CVE (#71082)
(cherry picked from commit 92d59a58c0)
* Fix 404 links (#71256)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit ecea018506)
* Writing style updates to Developing dynamic inventory topic (#71245)
* modified the writing style
* incorporated peer feedback
(cherry picked from commit ecd3b52ad7)
* Fix roadmap formatting. (#71275)
(cherry picked from commit ee48e0b0ad)
* Update password.py (#71295)
List md5_crypt, bcrypt, sha256_crypt, sha512_crypt as hash schemes in the password plugin.
(cherry picked from commit 1d1de2c6fd)
* Update ansible european IRC channel (#71326)
Signed-off-by: Rémi VERCHERE <remi@verchere.fr>
(cherry picked from commit 824cd4cbeb)
* Add warning about copyright year change (#71251)
To simplify project administration and avoid any legal issues,
add a warning in the docs. This reflects - https://github.com/ansible/ansible/issues/45989#issuecomment-423635622 and fixes: #45989
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 606604bb97)
* subelements: Clarify parameter docs (#71177)
skip_missing parameter in subelements lookup plugin is accepted from
inside the dictionary.
Fixes: #38182
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 6d17736ef4)
* Writing style updates to Using Variables topic (#71194)
* updated topic title, underline length for headings, and incorporated peer feedback
(cherry picked from commit 4d68efbe24)
* cron module defaults to current user, not root (#71337)
(cherry picked from commit 4792d83e13)
* Update Network Getting Started for FQCN/collection world (#71188)
* pull out network roles, cleanup, update first playbook examples, update gather facts section, some inventory conversion to .yml, update inventory and roles, simplify the navigation titles, fix tocs, feedback comments
(cherry picked from commit f79a7c5585)
* Add documentation about info/facts module development (#71250)
Fixes: #40151
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 4f993922c8)
* network: Correct documentation (#71246)
ini-style inventory does not support Ansible Vault password.
This fixes network_best_practices_2.5 doc.
Fixes: #69039
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit a1257d75aa)
* tidies up vars page (#71339)
(cherry picked from commit 02ea80f6d7)
* base.yml: Fix typos (#71346)
(cherry picked from commit 41d7d53573)
* quick fix to change main back to devel (#71342)
* quick fix to change main back to devel
* Update docs/docsite/rst/dev_guide/developing_collections.rst
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 74f88c56a5)
* Add note about integration tests for new modules to the dev guide (#71345)
(cherry picked from commit b82889eef5)
* update fest link (#71376)
(cherry picked from commit 80b8fde946)
* incorporate minimalism feedback on debugging page (#71272)
Co-authored-by: bobjohnsrh <50667510+bobjohnsrh@users.noreply.github.com>
(cherry picked from commit 5073cfc8bc)
* fix header problem
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Sayee <57951841+sayee-jadhav@users.noreply.github.com>
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: rovshango <rovshan.go@gmail.com>
Co-authored-by: Remi Verchere <rverchere@users.noreply.github.com>
Co-authored-by: Jake Howard <RealOrangeOne@users.noreply.github.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Per Lundberg <perlun@gmail.com>
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
* Build documentation for Ansible-2.10 (formerly known as ACD).
Builds plugin docs from collections whose source is on galaxy
The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.
* Update the python syntax checks
* docs builds can now require python 3.6+.
* Move plugin formatter code out to an external tool, antsibull-docs.
Collection owners want to be able to extract docs for their own
websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs
* Cleanup and bugfix to other build script code:
* The Commands class needed to have its metaclass set for abstractmethod
to work correctly
* Fix lint issues in some command plugins
* Add the docs/docsite/rst/collections to .gitignore as
everything in that directory will be generated so we don't want any of
it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages
* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs
* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined
* Remove references to plugin documentation locations that no longer exist.
* Perhaps the pages in plugins/*.rst should be deprecated
altogether and their content moved?
* If not, perhaps we want to rephrase and link into the collection
documentation?
* Or perhaps we want to link to the plugins which are present in
collections/ansible/builtin?
* Remove PYTHONPATH from the build-ansible calls
One of the design goals of the build-ansible.py script was for it to
automatically set its library path to include the checkout of ansible
and the library of code to implement itself. Because it automatically
includes the checkout of ansible, we don't need to set PYTHONPATH in
the Makefile any longer.
* Create a command to only build ansible-base plugin docs
* When building docs for devel, only build the ansible-base docs for
now. This is because antsibull needs support for building a "devel
tree" of docs. This can be changed once that is implemented
* When building docs for the sanity tests, only build the ansible-base
plugin docs for now. Those are the docs which are in this repo so
that seems appropriate for now.
* Docs: User guide overhaul, part 5 (#70307)
(cherry picked from commit db354c0300)
* Need to return any error code from running antsibull-docs (#70763)
This way we fail early if there's a problem
(cherry picked from commit 1e3989c9f7)
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
Some code-smell sanity tests for ansible-base use subprocess to invoke ansible commands.
Intercept these commands to make sure the correct script and python version are used.
* Split out sanity test requirements.
* Run each --venv test separately.
This provides verification that the requirements for each test are properly specified.
* Use a separate requirements file per sanity test.
* Skip setuptools/cryptography setup for sanity.
* Eliminate pyyaml missing warning.
* Eliminate more pip noise.
* Fix conflicting generate_pip_install commands.
* Add changelog fragment.
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances. There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code. (The most likely case of
this is when it's a global variable and some other code might import
that variable name).
Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement. At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
* remove azure extras and extras_require support
* Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now.
* Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history.
* Mark azure-requirements as orhpaned.
This keeps the docs around so that existing links from old test runs remain valid.
Co-authored-by: Matt Clay <matt@mystile.com>
* BOTMETA support migrated_to
Allow BOTMETA to define if this part of the codebase has moved
into a Collection on Galaxy.
See also https://github.com/ansible/ansible/pull/63935
* Enforce migrated_to URL format
* pep8
Add a list of previously used release names to make it easy to tell what
release names are no longer usable.
Add a test that new release names have been added to the used list.
Fixes#61616
* Remove pointless sanity tests on `bin` dir.
* Update action-plugin-docs test for collections.
* Update no-main-display test for collections.
* Update empty-init test for collections.
* Update no-assert test for collections.
* Move required-and-default-attributes test.
This test only applies to Ansible itself.
* Update use-argspec-type-path test for collections.
* Relocate ansible-test self tests outside package.
We don't want to include the tests for verifying ansible-test within the ansible-test package.
* Add `test/ansible_test/` to classification.py.
* Fix test invocation.
* Relocate tests in MANIFEST.in.
* Improve package-data sanity test error checking.
* Only use includes for ansible-test in MANIFEST.in.
* Improve readability of MANIFEST.in.
* Run no-unwanted-files sanity test only on Ansible.
Since collections should be able to use binary modules there is not really any limit on what could exist in a collection `plugins` directory.
* Add support for symlinks in sanity target lists:
- Sanity tests that need to analyze symlinks can do so using the supplied target list.
- Tests that analyze directories will now only look at symlinks if requested.
- Directory symlinks will now be seen as directories instead of files.
* Enable symlinks on filename based sanity tests.
Sanity tests that evalulate filenames instead of content should include symlinks.
* Update symlinks sanity test.
Use the sanity test target list now that it can include symlinks.
* Install ansible-test
Modify the install script to install ansible-test and its supporting
code. Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437
Also:
* No longer use package_data. Everything in the package dirs is going
to be installed. Anything that shouldn't be installed needs to be
moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
instead of same directory
* Add ansible_test files to package-data sanity test
* MANIFEST.in cleanups
* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
directories
* Change package-data test to be more complete
* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
everything in the install is in the sdist in addition to comparing that
everything in the repo that we want is in the install
* Leave out test artifacts
Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs
Remove test/utils files from the sdist as these are only needed for our CI
cleanup of docs in MANIFEST.in; getting rid of build files.
* Add the ability to output sdist and snapshot to specific directory
* Add a warning about modifying the heuristic to setup.py
* Address generated files
* Use make snapshot instead of sdist to generate changelog and man pages
and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity
* Add cli web docs to make clean
This was causing problems when attempting to test that the sdist didn't
have extra files
* Fix bug constructing python names from __pycache__ names
* Create a clean repo to work from
* Exclude test/legacy and be more explicit on extensions
* Exclude the legacy directory from sdist
This will help prevent accidental merging of content to recently obsoleted directories when adding new files.
It may also help contributors who have modified obsolete files understand where their changes should now be made.
* Relocate ansible-only sanity tests.
* Get "code smell" sanity tests from multiple dirs.
- `test/lib/ansible_test/_data/sanity/code-smell/` - General purpose tests used for both Ansible and Ansible Collections.
- `test/sanity/code-smell/` - Tests specific to Ansible, will not be used for Ansible Collections.
* Allow sanity tests to easily ignore themselves.
Useful for simple regex based sanity tests that error on their own script due to the regex or error message generated.
* Simplify no-smart-quotes sanity test.
Pruning of unversioned directories is handled by ansible-test.
* Remove directory pruning from boilerplate tests.
Files not passing these tests should be ignored instead.
* Add missing sanity ignore entries.
* Add sanity test to ensure all non-py files are installed
* Fix mode and regex
* Fix role skel inventory package_data
* Add docs
* Update package_data for inventory files
* Address pylint concerns
* Another tweak to package_data
* Address review feedback
* Change index to 1
* add to ansible-only.txt
* Initial ansible-test support for collections.
* Include cloud config in delegation payload.
* Add missing types import and fix `t` shadowing.
* Fix plugin traceback when config_path not set.
* Fix encoding issues.
* Remove unused imports.
* More encoding fixes.
* Handle delegation outside exception handler.
* Inject ssh keys only if not already in place.
* More defensive approach to getting remote pwd.
* Add missing string format var.
* Correct PowerShell require regex.
* Rename `is_install` and `INSTALL_ROOT`.
* Generate galaxy.yml based on single source of truth
* Fix up tests and align file names
* Minor Makefile tweak
* Remove link in galaxy.yml file and make it a template file
* Moved collections docs to dev_guide
* change Makefile clean path
* Added readme to example meta file
* review fixes
* Use newer style for doc generation script
* Fix mistake in dev_guide index
* removed uneeded file, fixed links and added preview banner
* Moved banner for sanity test
ansible-test only passes files which have the .py suffix for sanity
tests on python files. This change will allow sanity tests to run on
the Python files in hacking/
* Rename test-module to test-module.py
* Symlink test-module for backwards compat since end users may be using
test-module
* Fix test-module sanity errors that are now triggered
* Rename ansible_profile to ansible-profile.py
* Rename build-ansible
The metaclass boilerplate is safe to apply en masse. The future import
boilerplate needs code to be inspected to be sure that there aren't any
py2isms that need to be fixed. Split these two checks so that we can
fix them independently
Be explicit about which files are grandfathered so we can fix them up one by one
Alicia Cozine