Commit Graph

387 Commits (5635c50f9b2752bc9305d033b557fff2db076928)

Author SHA1 Message Date
John R Barker 04e816e13b Stricter module documentation validation (#22353)
Raise the bar for module `DOCUMENTAION`
This validator update was used to find the issues in https://github.com/ansible/ansible/pull/22297/files

**Validation**
* Updated Validation and docs to enforce more (items fixed in https://github.com/ansible/ansible/pull/22297/files)
* Use `suboptions` to document complex options 
* Validate module name
* Validate deprecated modules have correct ANSIBLE_METADATA

**Module Documentation Generation**
* Document `suboptions:` Example https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718
* Tidy up HTML generation (valid HTML, no empty lists, etc)
 
**Documentation**
* Clarify the steps for deprecating a module
* Use correct RST headings
* Document `suboptions:` (options)
* Document `contains:` (returns)


**Details**
The aim is to get this (and corresponding module updates) complete by the time `devel` becomes `2.4`, as this allows us to raise the bar for new modules

Example `suboptions` https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718

The aim is to get this PR integrated into `devel` *before* we branch `stable-2.3`, this will allows us to:
* Raise the bar for new modules in 2.4
* Ensure the generated module documentation for 2.3 and higher is improved, important as we will be doing versioned docs moving forward.
8 years ago
Matt Clay 48e82b4ca8 Add `test/runner` to $PATH in `hacking/env-setup`. (#22387) 8 years ago
Brian Coca b714f1fc4b made module notes render nicer on docsite 8 years ago
Matt Martz 5942de603d Legacy pep8 clean fixes for contrib and hacking (#21081) 8 years ago
Allan ba39d1158c Update test-module (#20737)
* Update test-module

Ensuring invoke is assigned

Traceback (most recent call last):
  File "ansible/hacking/test-module", line 267, in <module>
    main()
  File "ansible/hacking/test-module", line 263, in main
    runtest(modfile, argspath, modname, module_style, interpreters)
  File "ansible/hacking/test-module", line 207, in runtest
    invoke = "%s%s" % (invoke, modfile)
UnboundLocalError: local variable 'invoke' referenced before assignment

* Update test-module

Made the change to only require a single if, making the function more 'DRY'.
8 years ago
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
8 years ago
Matt Clay d0d1158c5e PEP 8 cleanup. (#20789)
* PEP 8 E703 cleanup.
* PEP 8 E701 cleanup.
* PEP 8 E711 cleanup.
* PEP 8 W191 and E101 cleanup.
8 years ago
Matt Clay 95789f3949 PEP 8 whitespace cleanup. (#20783)
* PEP 8 E271 whitespace cleanup.
* PEP 8 W293 whitespace cleanup.
* Fix whitespace issue from recent PR.
8 years ago
Matt Clay 63b1e0c277 Fix infrequent PEP 8 issues. 8 years ago
Brian Coca bcdfdc0cc3 minor cleanups
removed redundant print/exits
removed unused import and var
8 years ago
Adam Johnson fbfe66967d Doc module meta grammar fixes (#20446)
* Remove unnecessary 'the'
* End sentences with a period
8 years ago
Will Thames 082082857d Use the python used with test-module to run modules (#19591)
* Use ansible_python_interpreter to run modules

Use ansible_python_interpreter to run modules if
`-I ansible_python_interpreter` is set.

Remove unused default from `-I` help text.

* Update test-module to pep8 standards
8 years ago
Matt Clay cd3fdca540 Switch tests to pytest and ansible-test.
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
8 years ago
Brian Coca 3c4afd0470 Doc module meta (#20093)
* updated module docs to reflect unification/meta

* new doc page

* also remove module refs to old core/extra

* Update modules_support.rst

Light edit.
8 years ago
Brian Coca d138132bc2 show status/support in toc 8 years ago
Toshio Kuratomi 08d6990e67 Fix code-blocks to use correct syntax highlighting 8 years ago
Brian Coca f5c1aefca9 added symbol conversion to deprecated
fixes #19697
8 years ago
Toshio Kuratomi d001d250f0 Whitelist the hacking/cherrypick script's use of python3 8 years ago
Toshio Kuratomi 8f6009de0f Add initial script to aid in cherrypicking from devel to stable-2.2 8 years ago
Lujeni 021756a691 Update the module path from the hacking documentation (#19683)
- New path since the modules merge
8 years ago
Brian Coca 0f4ca877ac many doc fixes
removed a bunch of warnings
made nicer formatting of new module info
8 years ago
Sam Doran 73fc4e42f1 Correct type-o on support statement 8 years ago
Matt Clay bea8a3201d Remove obsolete files and instructions. (#19079)
* Remove obsolete `unify_repos.sh` script.
* Remove submodule update after checkout reference.
8 years ago
Matt Clay c709b22e5c Fix differences with devel. 8 years ago
Matt Davis 0b47c908eb quick and dirty module doc updates (#18944)
clean up core/extras, plumb in basic metadata display
8 years ago
Matt Davis cb1888125d add metadata to doc support (#18802)
fix broken module docs
change doc AST id extraction to use == instead of in
8 years ago
Matt Clay 75c281debc Fix compile errors in scripts. 8 years ago
Toshio Kuratomi ab43a26915 Additional reports (#18779)
* Additional reports
* Fix default status to be a list
8 years ago
Adrian Likins b5a63a556d fix docs on related to remove of core/extras 8 years ago
Toshio Kuratomi c53b73228f Update the metadata tool (#18765)
* Latest spreadsheet version uses different fields for support.
* Format the metadata to be a little nicer (multiple lines)
8 years ago
Toshio Kuratomi d3dd11b21c Update for latest spreadsheet fields 8 years ago
Toshio Kuratomi 39c2249e49 Update metadata for the rename of curated to committer 8 years ago
Brian Coca bfb43ed0b1 yes, im using bashisms 8 years ago
Brian Coca 2b13ad0ae8 dont skip github modules... 8 years ago
Brian Coca eef7b8b5a5 now also unifies plugins in same dir structure 8 years ago
Toshio Kuratomi c22dde615d Add a tool to work with module metadata (#18488)
* Add a tool to work with module metadata

* Don't hardcode async_wrapper
8 years ago
Toshio Kuratomi b8321c74c8 Fix indentation of multiline descriptions 8 years ago
Matt Clay 0d46805979 Clean up shebangs for various files.
- Remove shebangs from:
  - ini files
  - unit tests
  - module_utils
  - plugins
  - module_docs_fragments
  - non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.

Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.

A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
8 years ago
Brian Coca c16a88cac4 keep list as list in synopsis 8 years ago
Sam Doran cf8639ff62 Fish hacking setup fix (#18084)
* Remove old egg-info files before creating new ones

Currently, setup.py generates egg files then they are deleted. This change
fixes this behavior and matches that in env-setup.

* Do not try to move ansible*egg-info to lib/

setup.py creates the ansible.egg-info in lib/ so this step is unnecessary. Matches env-setup behavior.

* Better test for number of arguments in argv

This prevents an erronous error message from being thrown since set -q returns an error code with the number of variables not defined, resulting in a non-zero exit if no arguments are passed.

Indent case statement within switch statement.
8 years ago
Adrian Likins cf39a1abab test-module _ansible_selinux_special_fs arg added
modules need to have _ansible_selinux_special_fs passed in
as an arg, so add the default to the args.
8 years ago
Brian Coca 72e6ffad47 added script to unify repos
in case i forget how to do this, can be deleted afterwards
8 years ago
cinerama70 db276373e5 gen_distribution_version_testcase.py should fail if ansible run fails (#17693)
Currently, "ansible localhost -m setup" can fail silently during the
run of gen_distribution_version_testcase.py, resulting in incorrect
output. Use check_output() rather than communicate() and handle
the exception if we get a nonzero return value.
8 years ago
cinerama70 02cec7dca9 Add os_family to test_distribution_version (#17620)
As suggested in feedback on
https://github.com/ansible/ansible/pull/17575, add
os_family to test_distribution_version. Add the
correct os_family to the existing testcase data
entries.

Also add os_family to the output of
gen_distribution_version_testcase.py so any new
generated entries will contain this data.
8 years ago
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
8 years ago
Michael Scherer 619f3d13ef Add detection of python3 to hacking/env-setup (#17357)
People can still override the detection using PYTHON_BIN
8 years ago
scottb 235eab6609 Merge pull request #16316 from alikins/alikins_docsite_make
Fewer deps for 'make webdocs' from top level
8 years ago
Josh Smift 81d7e132f9 force deletion of .pyc files (#17109) 8 years ago
Matt Davis 525c0469d2 tweak hacking/env-setup to work under ash (#17054) 8 years ago
Toshio Kuratomi 48a2773463 Find places where ziploader is used and change them to ansiballz so that people aren't confused when they google for information.information (#16715) 8 years ago
Adrian Likins a1c21f8400 Fewer deps for 'make webdocs' from top level
Previously it also built the MANPAGES target
requiring asciidoc and libxml, before starting
a 'make docs' in docsite.

Also change the #! line in
hacking/dump_playbook_attributes.py to not specify
python2... yet.
9 years ago
André Pinto fa4011c6c3 Do not corrupt $MANPATH when it's not present (#16300)
Fixes #16299
9 years ago
Matt Davis ec2cb07988 Make 'required' optional in module docs (#15906)
Updated module dev docs, doc build, ansible-doc to match
9 years ago
Brian Coca 71a707fba5 quick yaml syntax checker 9 years ago
Brian Coca 478674cc57 typo fix 9 years ago
Brian Coca b7c874f81a enforce required 'required' in docs 9 years ago
Robin Roth 1d6608e84f Dist version fix for Red Hat and more tests (#15663)
* add tests for centos6, rhel6 and rhel7

* gen_distribution_version_testcase with python2.6

* remove unused imports

* fix redhat/vmware/... parsing

* add centos7 test case
9 years ago
Brett Inman 1f89a46b52 Make -q flag totally quiet for env-setup.fish
The first echo and setup.py steps are not quiet and result in output that is annoying if your Fish is configured to source the file.
9 years ago
Toshio Kuratomi 2762f12f7f Remove the duplicate modstyle parameter 9 years ago
Toshio Kuratomi bdd73e31dc Have test-module clean up the local temp dir when it exits
Get test-module's debugger switch to do something useful with ziploader modules
9 years ago
Toshio Kuratomi 7833b5bec4 Merge pull request #15420 from robinro/distribution_version_tests
Distribution version tests
9 years ago
Robin Roth 692bf51fde move gen_testcase to hacking/tests
* also use json instead of pprint
9 years ago
Toshio Kuratomi 669f3dc3a8 Cleanup. Since we no longer pass a lock, we no longer need to create it 9 years ago
Toshio Kuratomi d78ba34cf0 We switched away from passing the lock via the arguments to modify_module
Need to fix test-module to not pass the lock either
9 years ago
Toshio Kuratomi dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
9 years ago
Toshio Kuratomi b27c424fa1 Fixes to the documentation build (#15356)
* Could only have one alias before.  Subsequent aliases overrode the
  previous ones.  Now multiple aliases work.
* Fix BLACKLISTED_MODULES.   Previously, modules were listed in the
  generated documentation despite being blacklisted
* Deprecated modules form extras were showing the (E) tag and not the
  (D) tag. Reversed that now (Probably not necessary to also show the
  E tag).
* Sort the deprecated modules alphabetically in the Category docs as
  well as the list of all modules
* Optimization: Previously rendered the modules to rst twice once in all
  group and once in individual categories.  Fixed to only render them
  once.
* Add fireball to blacklist and remove async_status (as people need to
  use that).
9 years ago
Toshio Kuratomi a330a24ccc Python2.6 fix for test-module 9 years ago
Toshio Kuratomi 4b0aa1214c Ziploader
* Ziploader proof of concept (jimi-c)

* Cleanups to proof of concept ziploader branch:

* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
  systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
  we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
  is kept so that tracebacks show the real filenames that the errors
  appear in.

* Make sure we import modules that are used into the module_utils files that they are used in.

* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer

* Make it possible to set the module compression as an inventory var

This may be necessary on systems where python has been compiled without
zlib compression.

* Refactoring of module_common code:

* module replacer only replaces values that make sense for that type of
  file (example: don't attempt to replace python imports if we're in
  a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
  via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
  numbering)

* Don't repeat ourselves about detecting REPLACER

* Add an easy way to debug

* Port test-module to the ziploader-aware modify_module()

* strip comments and blank lines from the wrapper so we send less over the wire.

* Comments cleanup

* Remember to output write the module line itself in powershell modules

* for line in lines strips the newlines so we have to add them back in
9 years ago
Brian Coca 6a85da7e0c avoid private attributes
hardcode adding with_ for tasks
9 years ago
Brian Coca e1faa78722 fixed typo 9 years ago
Brian Coca cbc797a7ed added missing : 9 years ago
Brian Coca a3489408a5 fixes to playbooks_directives generation
order is now predictable
now correctly substitutes loop for with_
adds local_action to action
9 years ago
Brian Coca 3e28ee0fd4 avoid printing internal loop and loop_args 9 years ago
Brian Coca fbdcb22e36 now generate list of playbook ojbect directives
TODO: needs links/info and conditionals added
9 years ago
Brian Coca fe09f7ee49 clarify that requirements are on host that runs it
fixes http://github.com/ansible/ansible-modules-core/issues/3061
9 years ago
Brian Coca 6414c967e4 now check for description and listify if needed
fixes #14371
9 years ago
Toshio Kuratomi 5cd3f71792 Handle utf-8 in module short desc 9 years ago
Jeremy Audet 5cac8efd73 Make "make webdocs" compatible with Python 3
The `webdocs` make target fails under Python 3. It fails due to a variety of
syntax errors, such as the use of `except Foo, e` and `print 'foo'`. Fix #13463
by making code compatible with both Python 2 and 3.
9 years ago
Patrik Lundin 9f9944e355 Replace -delete with portable -exec rm {} \;
Needed on OpenBSD which does not support -delete.
9 years ago
Alberto Gireud 52ded67db5 Update check mode argument 9 years ago
Brian Coca 4a206cdde9 force egg deletion 9 years ago
Brian Coca 5719912e7f added a skip for the test dir in module repos 9 years ago
Toshio Kuratomi 4203850d1a Break apart a looped dependency to show a warning when parsing playbooks
Display a warning when a dict key is overwritten by pyyaml
Fixes #12888
9 years ago
Toshio Kuratomi baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 9 years ago
Brian Coca 3a0bf55ae3 better error catching for doc build 9 years ago
James Cammarata a431121f9f Tweak to hacking path fix to go back to subshell use 9 years ago
EC bcdb4e78ae Allow env-setup to use spaces in full path 9 years ago
Marius Gedminas 823677b490 Replace .iteritems() with six.iteritems()
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host).  And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
9 years ago
Luke Rohde cba4642d8d Use find ... -delete instead of non-portable globbing 9 years ago
Luke Rohde 647e48f776 Delete all compiled python files when running hacking/env-setup 9 years ago
Kevin Houdebert b8c9391d0c Change to python3 syntax 9 years ago
Jody Frankowski bc1e1d2cdc Fix rm call in hacking/env-setup. Since it is sourced from a user shell,
rm can have an alias to 'rm -i', which will make sourcing hang when '-q'
is enabled.
9 years ago
Abhijit Menon-Sen 114070c226 Add missing $ 9 years ago
Brian Coca 636f88fd31 Merge pull request #9438 from binarytemple/patch-1
export ANSIBLE_HOME so it can be used in scripts
9 years ago
Toshio Kuratomi 53ae326603 Port some things in test-module to v2.
In particular, fix arg parsing

Fixes #11820
9 years ago
Brian Coca f5c8055879 removed debug stuff 9 years ago
Brian Coca a78fdde32c better handling for weird doc corner cases (abscense of options, bad text objects, etc)
also updated extras to get some docfixes on the module side
9 years ago
bryan hunt 7a76fcb159 merged 9 years ago
Sheldon Hearn c800a1c68a Preserve OLDPWD 9 years ago
Brian Coca ae54792a35 Merge pull request #11688 from amenonsen/8935-rebase
8935 rebase: add «-o outputfile» and -n(oexec) options to hacking/test-module
9 years ago