Commit Graph

595 Commits (22538374535996da1fb61c7b58e981f2511ed9ef)

Author SHA1 Message Date
Matt Martz c1f2a9ea6c
No core irc meeting (#80811) 2 years ago
Matt Clay 2cd1744be3
Use ansible.module_utils.common.text.converters (#80704)
Replace use of old `ansible.module_utils._text` and add a unit test to maintain backwards compatibility.
2 years ago
Matt Clay 4d25e3d54f
Set the minimum setuptools to 45.2.0 (#80649)
Also update the package-data sanity test to use the minimum setuptools version.
2 years ago
Matt Clay 9026bc5d76
Remove obsolete pkg_resources usage (#80489) 2 years ago
Sviatoslav Sydorenko a84b3a4e72
Fix misrendered sections in manpage generation
This change fixes bugs in the manpage generator that existed since it
was first added.

It exposes CLI `ARGUMENTS` value to manpage templates.

Before this change, the code contained a typo, causing the `for`-loop
iterate over individual characters of the `'ARGUMENTS'` string rather
than iterating over a tuple. A missing comma was at fault.

The updated code gets rid of the `for`-loop and conditionals since it
seems to have been a premature complexity increase and no other things
than `'ARGUMENTS'` were ever added into the broken iterable.

The functional change is that `arguments` is now always present in the
Jinja2 context, unlike being missing sometimes because of the previous
design (not that it was ever present, because of the bug! sigh...)

The Jinja2 templates perform an `{% if arguments %}` check, letting
the template engine silently ignore the missing variable. The clause
was always falsy, meaning that the arguments section was not included
in the manpages for at least the last 6 years. With this fix, it will
be.

This patch also deduplicates calling `opt_doc_list` @ generate_man.

It was called late in the execution, more times than necessary. This
patch makes sure it happens once by putting it at the top of the scope.

It fixes rendering library and inventory in manpages.

The corresponding Jinja2 templates have blocks wrapped with
conditionals like `{% if inventory %}` and `{% if library %}` but said
variables were never injected into the context, nor were they even
deduced on the Python side of the generator. This means that the
conditional clauses were always falsy, never showing the portions of
the manpages.

The Python script has hints for how the `inventory` variable was to be
calculated, which is confirmed through the Git paleontology efforts.

The block of code that references to the `inventory` bit was
incorrectly checking a variable with a list of nested objects for the
presence of a string which was never going to work.

This patch fixes this check by verifying the CLI flag against the
correct variable containing a list of options and exposes it to the
Jinja2 templates.
It also exposes the `library` variable in a similar way.

The block displaying other binaries in Sphinx CLI docs has been
synchronized with the manpage template.
Previously, the current binary was displayed also. This patch gets rid
of the unwanted trailing comma there too.

Finally, the CLI executables list in the manpage template now reuses
the same variable as the RST template that doesn't need any
post-processing in Jinja2.
Before, it was already used in the RST template so this patch aligns
both templates to use the same logic as they got out-of-sync over time.

PR #80450.
2 years ago
Matt Clay 163f297d7f
Extend bulk issue creator (#80423)
It can now be used to create feature requests, not just deprecation bug reports.
2 years ago
Matt Clay 4f7f7d1c26
Remove old release announcement scripts (#80371)
These have been replaced by `packaging/release.py`
2 years ago
Matt Clay 2e413b74e4
Add work-around for old AZP jobs (#80339) 2 years ago
Matt Clay f587856beb
Remove straight.plugin dependency (#80084) 2 years ago
Matt Clay e69ef1b249
Clean up unused imports in `hacking` directory (#79894) 2 years ago
Matt Clay 868d721d8c
Flush output in sanity requirements update script. (#79774)
This keeps the subprocess output under the correct heading.
2 years ago
Matt Martz 56d142350d
Add support for importlib.resources (#78915)
* Add support for importlib.resources

* Remove the importlib.resources imports

* return the correct data

* Some code comments, and re-order for consistency

* Disallow traversing packages below an individual collection

* Add a traversable class for namespaces

* Re-use variable

* Utilize itertools.chain.from_iterable

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Simplify logic to check for packages from ansible loaders

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Just a generator expression, instead of a generator

* docstrings

* Add comment about find_spec for our namespaces

* Add some initial unit tests for importlib.resources

* normalize

* Utilize importlib.resources for listing collections

* collections_path is already in config, just use config

* install uses a different default for collections_path

* Remove unused import

* Remove duplicate __truediv__

* Bring back TraversableResources

* Apply some small suggestions from code review

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>

* Remove cross contamination between plugin loader code and CLI code

* Remove unused import

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Matt Davis <6775756+nitzmahone@users.noreply.github.com>
2 years ago
Felix Fontein 9545f2e0de
stable docs build: do not use YAML parsing/serialization for .deps files (#79233)
* Do not use YAML parsing/serialization for .deps files.

* Prevent crash.
2 years ago
Johanan Liebermann fbd828673d
Fix distro fact handling for Flatcar (#77635)
* Fix distro fact handling for Flatcar

The existence of the file /etc/flatcar/update.conf depends on
bootstrap configuration typically provided by the user. For that
reason this file is unsuitable for determining distro facts for
Flatcar Container Linux.

The distribution_release fact is meaningless in the case of Flatcar
since Flatcar doesn't have named releases. The distribution_version
fact, however, IS meaningful and should contain a number such as
"3139.2.0".

- Use /etc/os-release instead of /etc/flatcar/update.conf.
- Drop the distribution_release fact.
- Set the distribution_version fact.
- Update distro test fixture for Flatcar
- Generate the fixture using gen_distribution_version_testcase.py.
- Override result.distribution and result.os_family manually as the
  generator script gives wrong values.
- Use a recent Flatcar version.

Signed-off-by: Johanan Liebermann <jliebermann@microsoft.com>
2 years ago
Matt Clay 650befed37
Add missing space after keywords. (#78480)
Also remove unnecessary parenthesis.
2 years ago
Martin Krizek 43153c5831
`FieldAttribute`s as descriptors (#73908) 2 years ago
Diana S. Cardona 58b42abede
Update README.md (#78139)
Update Python version requirement to >= 3.8.
2 years ago
Matt Martz a415697d70
Allow result sha to be overriden with local sha (#77832) 3 years ago
Felix Fontein 836683a082
Stop making core docs build talk of (ansible-)'base' when meaning (ansible-)'core'. (#77728) 3 years ago
samccann 7471c036ec shift to ansible_core_version 3 years ago
Matt Clay b8793fa48d Fix use of deprecated antsibull-docs option. 3 years ago
Felix Fontein 841bdb74eb
Replace antsibull with antsibull-docs (#77504) 3 years ago
htol d7fbde4ea9
Make "~/.ansible" path configurable (#76114)
* replace hardcoded '~/.ansible' to C.ANSIBLE_HOME
* rename previously existing env ANSIBLE_HOME in env-setup script
* modify cache dir monkeypatching in galaxy api unit tests
* update "version_added" to 2.14 for ANSIBLE_HOME
* fix description of collections with proper use of ANSIBLE_HOME

Co-authored-by: htol <github@h-tol.net>
Co-authored-by: stefanwascoding <stefan@syntaxhelden.de>
3 years ago
Matt Clay 353511a900
Add script to handle more deprecations. (#77400)
* Add script to handle more deprecations.

This script currently supports deprecations from the following sanity tests:

* deprecated-config
* update-bundled

* Ignore script in package-data test.
3 years ago
Matt Clay 93d0ea0215 Update core porting guide generation. 3 years ago
Matt Clay 3e491db466
Porting guide fixes. (#77376)
* Update porting guide generation.
* Fix 2.13 porting guide link.
3 years ago
Cindy Park 1bdcb56249
Improve documentation to fix outdated file paths (#77341)
Co-authored-by: njthanhtrang <njthanhtrang@users.noreply.github.com>
3 years ago
Philip Douglass e6f63be7a4
Honor `--quiet` parameter properly in fish shell (#77180)
* The `^` stderr redirection is deprecated
* Setting `-q` for `setup.py` in `gen_egg_info` is redundant
3 years ago
Brian Coca 9a4ff01a4f
Cleanup of cannned responses (#76689)
* removed those made obsolete by bot
 * tried to rephrase to be more informative and require less updates in the future

Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
3 years ago
Pascal Höhnel 7cd94dd452
Fix unsafe handling of variable PYTHON_BIN (#76767)
The previous state lead to errors, when cloning the repo to a path with spaces in it.
3 years ago
Matt Clay bb63c97c16
ansible-test - Update import test and sanity requirements. (#76308)
* Add script to freeze sanity requirements.
* Declare sanity test requirements and freeze
* Use pinned requirements for import.plugin test.
* Expand scope of import test for ansible-core.
* Add ignores for galaxy import errors.
* Update test-constraints sanity test.
3 years ago
Matt Martz 66a83314b9
Modernize install (#76021)
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
Matt Martz 96b221b344
Update vendored copy of distro to 1.6.0 (#75732)
* Update vendored copy of distro to 1.6.0

* copy pasta mistake

* ci_complete
3 years ago
Matt Martz 208014422d
Fix markdown links that were errantly formatted as rst (#75697) 3 years ago
Sam Doran 5af0420cab
hacking - account for job names that contain separators (#75454)
Some job names contain '/' which causes the download job to fail.

Example job name: `Post-job: Checkout ansible/ansible@refs/pull/75445/merge to ansible.log`
3 years ago
Matt Clay a5b0626176
Cleanup hacking scripts to prepare for pylint update. (#75474)
* Use generator in get_recent_coverage_runs.py
* Use from import in test-module.py
3 years ago
Toshio Kuratomi a901ff2bc0 Allow docs-build to use a local ansible-build-data checkout.
This is useful for testing the effects on docs of changing what
collections are in a version of ansible.  ansible-build-data can be
cloned locally, modified, and then build-ansible.py docs-build can be
told to use the local checkout instead of the canonical one on github.
3 years ago
Sam Doran 93e72c96f0
Update hacking `env_setup` to prefer Python 3 (#75213)
* env_setup - Prefer Python3

Prefer the 'python3' executable over 'python'.

* Add shebang for syntax detection

While the script isn't executable and we recommend sourcing it, having a shebang allows
variuos text editors to detect that it is a shell script and apply proper syntax highlighting.
3 years ago
Felix Fontein 47c50c38bb
Use antsibull-docs devel for devel docs (#74988)
* Require antsibull 0.34.0.
* Remove Makefile comment about the devel docs building only the core
(formerly base) docs; the behavior was updated in ccbfdec334
Currently 'make coredocs' builds core-only docs; 'make webdocs' builds the full docs even on the devel
branch

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
3 years ago
Felix Fontein e6a7bc854c
Only consider directories which do have .deps files other than ancestor.deps. (#74987) 3 years ago
Toshio Kuratomi 80e7e1a17c
Due to the takeover of freenode we're moving to a different irc network. (#74775)
* Due to the takeover of freenode we're moving to a different irc network.

* Our channels updated to point at the same channel name on libera.chat
* Some links went to webchat.freenode.net.  At this time, libera.chat
  doesn't point you to an official webchat client so I changed these to
  https://libera.chat. (kiwi irc does work with libera.chat so that
  could be another option).
* In general, I used the name irc.libera.net for link names and
  https://libera.chat for link targets.  This is because the irc service
  is hosted on irc.libera.chat but the project web server is hosted on
  libera.chat.  (This appears to also be true for freenode but we were
  using http://irc.freenode.net which doesn't seem to work.  Oops).
* Removed http://irc.freenode.net from the linkcheck exceptions.
  linkcheck was actually correct to flag that as invalid (should have
  been http://frenode.net instead).

* Looks like hte important people in #yaml are now in libera.chat

* Link to where contributors should get help

Add a link target and then link to where contributors should get support
for developing groups of modules.

* Update docs/docsite/rst/dev_guide/developing_modules_in_groups.rst

Co-authored-by: Felix Fontein <felix@fontein.de>

Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
4 years ago
Matt Martz 8d1cf7f266
Vendor `distutils.version` (#74644)
* Vendor distutils.version

* Fix import order. ci_complete

* remove distutils warning filter

* Don't remove warnings filter from importer

* ci_complete

* Add pylint config for preventing distutils.version

* Add changelog fragment
4 years ago
Matt Clay c48f80d062 Rename hacking/shippable to hacking/azp.
References to Shippable were changed to Azure Pipelines.

Also remove rebalance.py as it does not work with Azure Pipelines due to the required data not being present.
4 years ago
Alexander Sowitzki 26214788ee Retrofit shippable scripts to work with AZP
Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Matt Clay ecc5a53288
Update default containers to 3.4.0. (#74415)
* Update default containers to 3.4.0.

The 3.4.0 containers use Python 3.6 (the system Python) for `/usr/bin/python3`.

Python 3.9 continues to be the default Python version selected by `ansible-test` for these containers.

* Fix shebang on build-ansible.py.

Using `python` instead of `python3` allows `ansible-test` python interception and requirements install to function.
4 years ago
Ikko Ashimine 176beddb3f
Fix typo in incidental.py (#73737)
successfull -> successful
4 years ago
Sandra McCann ccbfdec334
Split Ansible docs from core docs (#73616)
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build

This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs.  Handle this via `make webdocs
  ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
  Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
  ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
  Handle this via `make coredocs`.

* `make webdocs` now always builds all the collection docs
*  Use `make coredocs` to limit it to core plugins only
*  The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
 
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
4 years ago
Brian Coca 1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
4 years ago
Rick Elrod 7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
4 years ago
Vincent Bernat 4afc228108
hacking: replace use of "which" with "command -v" (#71278)
"command -v" is the POSIX compliant way to get the path to a command.
It returns the path to the command if it exists and exits with a
non-zero status when it does not, without any output. "which" is not
POSIX and it can have various different effects. With Zsh, it says
"python not found" on stdout.

See:
 - https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html

Signed-off-by: Vincent Bernat <vincent@bernat.ch>
4 years ago