Commit Graph

268 Commits (21546faa2afa42ae234f943cdf7ccb4e459ee1d5)

Author SHA1 Message Date
Matt Clay 3e778d3f8f Fix pycodestyle E117 issues. 6 years ago
Abhijeet Kasurde 5c992fcc3f
ansible-vault: handle utf-8 filename in vault (#50341)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
6 years ago
Matt Martz 8c08d03989
Fieldattribute inheritance with defaults (#50891)
* Add tests for check_mode at play and task level

These test inheritance of check_mode from the various levels (command
line, as a play attribute and as a task attribute) so they will be
useful for checking that the change to fieldattribute inheritance with
defaults works

* Add a sentinel object

The Sentinel object can be used in place of None when we need to mark an
entry as being special (usually used to mark something as not having
been set)

* Start of using a Sentinel object instead of None.

* Handle edge cases around use of Sentinel

* _get_parent_attribute needs to deal in Sentinel not None

* No need to special case any_errors_fatal in task.py any longer

* Handle more edge cases around Sentinel

* Use Sentinel instead of None in TaskInclude

* Update code to clarify the vars we are copying are class attrs

* Add changelog fragment

* Use a default of Sentinel for delegate_to, this also allows 'delegate_to: ~' now to unset inherited delegate_to

* Explain Sentinel stripping in _extend_value

* Fix ModuleArgsParser tests to compare with Sentinel

* Fixes for tasks inside of roles inheriting from play

* Remove incorrect note. ci_complete

* Remove commented code
6 years ago
Matt Clay b971ebd343 More path encoding fixes. 6 years ago
Abhijeet Kasurde 4817dcd0fc Minor typos (#50371)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
6 years ago
Toshio Kuratomi 3fba006207 Update bare exceptions to specify Exception.
This will keep us from accidentally catching program-exiting exceptions
like KeyboardInterupt and SystemExit.
6 years ago
Dag Wieers baf0ad2309 Docs: Add a "seealso" section to the module docs (#45949)
* Docs: Add a separate  "seealso" section to the module docs
to list related modules and/or related references. This clears up the notes
section for things that are actual notes.

So you can add a section in your module documentation and four types of
references are possible.

    seealso:

    # Reference by module name
    - module: aci_tenant

    # Reference by module name, including description
    - module: aci_tenant
      description: ACI module to create tenants on a Cisco ACI fabric.

    # Reference by rST documentation anchor
    - ref: aci_guide
      description: Detailed information on how to manage your ACI infrastructure using Ansible.

    # Reference by Internet resource
    - name: APIC Management Information Model reference
      description: Complete reference of the APIC object model.
      link: https://developer.cisco.com/docs/apic-mim-ref/

This PR also includes:

- Implements ansible-doc support
- Implements schema support for the seealso options
- Updates to the development documentation
- Rename filter convert_symbols_to_format to rst_ify, cfr the existing html_ify and tty_ify filters
  - This makes the existing template a lot easier to read and fixes the confusion I had myself rereading the template (again).
- We fixed the possible suboption types (which was limited to 'bool' only)

* Use latest stable instead of devel docs
6 years ago
Matt Martz c9040d7579
Don't just raise AnsibleError with the exception message (#49654)
* Don't just raise AnsibleError with the exception message. Fixes #49252

* Add changelog fragment
6 years ago
Brian Coca 597c258e0e
explain 'bare variables' (#49577) 6 years ago
Brian Coca 8c43ce491e unsafe cleanup 6 years ago
Matt Martz 9773a1f289
Add a Singleton metaclass, use it with Display (#48935)
* Add a Singleton class, use it with Display

* update six import

* Move remaining failes to display singleton

* Fix rebase issues

* Singleton improvements

* Add code-smell for 'from __main__ import display'. ci_complete

* s/self/cls/g

* Add docs for no-main-display

* Address linting issues

* Add changelog fragment. ci_complete

* Implement reentrant lock for class instantiation in Singleton

* Add Display singleton porting guide
6 years ago
Martin Krizek a80c25cbd9
Do not use str() on exceptions (#46950) 6 years ago
Toshio Kuratomi dd906e20f4 Set the default metadata 6 years ago
Toshio Kuratomi 61ae6424a3 An earlier optimization of ansible-doc -l caused failures. (#47012)
* An earlier optimization of ansible-doc -l caused failures.

The optimization quickly searches the plugin code for short_description
fields and then uses that in the -l output.  The searching was a bit too
naive and ended up pulling out malformed yaml.  This caused those
plugins to be omitted from the list of plugins of that type with
a warning that their documentation strings were wrong.

This change makes the documentation parser aware that the documentation
string could have a relative indent for all of its fields which makes it
robust in the face of this particular problem.

* Don't search for space after short_description:

Any whitespace would be valid.  In particular newline
6 years ago
Brian Coca b2ac745461 add note to inform awx team on change 6 years ago
Eitan Adler 063a9dd911 Reduce noise in warnings from ansible (#40754)
When python is compiled in debug mode, or certain command line flags are
passed, python issues helpful warnings to let users know of files opened
by not closed.

This can be fairly spammy on an ansible run.  This change reduces the
number of such warnings by a factor of 10.
6 years ago
Matt Clay a11f631ee4 Python 3.8 collections compatibility fixes.
Includes a new pylint blacklist plugin to prevent regressions.
6 years ago
Dag Wieers b72e989e18 Ensure _raw_params retain exact spaces
This fixes a few issues related to multi-line YAML strings that eat
whitespace and add whitespaces after newlines (that weren't there).
6 years ago
Brendan Jurd ab96bbdef9 Clean up Vault docs in User Guide. (#46188)
* Fix spelling of 'separate' throughout.

* Various cleanups in the User Guide for Vault.

- Fix spelling of 'algorithm'
- Fix indentation of nested list in payload format
- Fix mysterious refernce to 'b_pkey1'.
- Fix reference to newline as '\n': the backslash is lost when rendered
to the docs website. Specify the hex value for newline instead of the
backslash escape.

* Fix formatting

* Update vault.rst
6 years ago
Dag Wieers 771a3983d2 Docs: Keep the module index clean (#46155)
A lot of modules have a short_description with a trailing dot even
though we don't want trailing dots in the index. This change removes
it when creating the document index.
6 years ago
Matt Martz c0915e2f5a
Support nested JSON decoding in AnsibleJSONDecoder (#45924)
* Support nested JSON decoding in AnsibleJSONDecoder

* Add tests for vault portion of AnsibleJSONDecoder
6 years ago
Toshio Kuratomi 0873d46617 Fix ansible-doc and docsite generation for removed modules
* Fix ansible-doc wrt removed modules
  * Fix listing of modules ia ansible-doc to not complain about removed modules
    Removed modules are marked as such in the metadata but nowhere else.
    Need to retrieve the metadata when a module doesn't have a doc so that
    we can tell if it falls under this case.
  * omit removed modules from json dump
  * Print an error that the module has been removed if attempting to run
    ansible-doc on that specific module

* Get plugin_formatter to stop outputting removed modules
6 years ago
Matt Martz 81ca04512d
Raise a nicer error when we cannot execute the editor (#44423)
* Raise a nicer error when we cannot execute the editor. Fixes #44419

* Don't use to_bytes when constructing an exception

* Add changelog fragment
6 years ago
Matt Martz 617372f8c0
Mass nuke deprecated items that are easily removed. ci_complete (#44320) 6 years ago
Matt Martz c1c229c6d4
Remove use of simplejson throughout code base (#43548)
* Remove use of simplejson throughout code base. Fixes #42761

* Address failing tests

* Remove simplejson from contrib and other outlying files

* Add changelog fragment for simplejson removal
6 years ago
Joren Vrancken b954917761 Surround top-level function and class definitions with two blank lines. 6 years ago
Eric Rinish 551501f326 use faster method to get short_description from DOCUMENTATION (#42705)
* use faster method to get short_description from DOCUMENTATION

* fixed pep8 whitespace

* fixed blank line
6 years ago
Abhijeet Kasurde 4f1746ee1d
vault: check dir value before passing (#43184)
This fix checks if dirname is not equal to '' before proceeding
to create actual directory with name.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
6 years ago
Troy Murray 15ce7c5bab change OS X to macOS (#41294)
* change OS X to macOS

<!--- Your description here -->

+label: docsite_pr

* Update all Mac OS X references to be macOS

* Drop extra Mac
7 years ago
Andrew Gaffney 0ff04ad41b Fix wrong python keyword (followup to #36357) (#41067)
Also add tests around that code path
7 years ago
Tom Wassenberg 6097776dc0 Fix flag in debug message (#40753)
The debug message stated that the vault password client script was invoked as `example.py --vault-id=example`, while it's actually `example.py --vault-id example`.
7 years ago
Adrian Likins 1613a739ad fix decrypted vault utf8 values (#37539)
* Fix errors decrypted non-ascii vault vars

AnsibleVaultEncryptedUnicode was just using b"".decode()
instead of to_text() on the bytestrings returned from
vault.decrypt() and could cause errors on python2
if non-ascii since decode() defaults to ascii.
Use to_text() to default to decoding utf-8.

add intg and unit tests for value of vaulted vars
being non-ascii utf8

based on https://github.com/ansible/ansible/issues/37258

Fixes #37258

* yamllint fixups
7 years ago
Brian Coca 5852df57f4 allow dataloader to use descriptors (#40627)
also fixed improper uses of str()

fixes #40383
7 years ago
Martin Krizek dccb0d035b
vault: check dir existence before creating a file (#40010) 7 years ago
Brian Coca cbb6a7f4e8
Enabled unsafe and vault in JSON (#38759)
* allow to load json marked as unsafe or vault

 * centralized json code/decode, add vault support
 * use generics to allow for more varied inputs
 * allow inventory to dump vault w/o decrypting
 * override simplejson also
 * add entry for unsafe also
 * load vaulted and unsafe json, support unvaulting if secrets provided
7 years ago
Toshio Kuratomi 3e64036684 Fix encoding issue when parsing plugin examples
This caused a non-fatal traceback when building, for example, the module
documentation for the debug module.
7 years ago
Andrew Gaffney 95ce00ff00 Allow loading dirs from role defaults/vars (#36357)
This commit moves code to look for vars files/dirs to a common place and
uses it for loading role defaults/vars. This allows things such as
'defaults/main' or 'vars/main' being a directory in a role, allowing
splitting of defaults/vars into multiple files. This commit also fixes
the role loading unit tests for py3 when bytestrings are used for paths
instead of utf8 strings.

This fixes #14248 and #11639.
7 years ago
Adrian Likins 6e737c8cb6
Fix 'New Vault password' on vault 'edit' (#35923)
* Fix 'New Vault password' on vault 'edit'

ffe0ddea96 introduce a
change on 'ansible-vault edit' that tried to check
for --encrypt-vault-id in that mode. But '--encrypt-vault-id'
is not intended for 'edit' since the 'edit' should always
reuse the vault secret that was used to decrypt the text.

Change cli to not check for --encrypt-vault-id on 'edit'.

VaultLib.decrypt_and_get_vault_id() was change to return
the vault secret used to decrypt (in addition to vault_id
and the plaintext).

VaultEditor.edit_file() will now use 'vault_secret_used'
as returned from decrypt_and_get_vault_id() so that
an edited file always gets reencrypted with the same
secret, regardless of any vault id configuration or
cli options.

Fixes #35834
7 years ago
Toshio Kuratomi 9890ce47e8 Optimizations in mod_args parsing
* Make it less likely that we have to identify all the modules during
  a playbook run.  PluginLoader is optimized to look for modules one
  directory at a time.  If we find a module before we've examined all
  the directories we never have to touch the other directories.
  Reordering this conditional makes it so tasks which don't have
  a module file will not force us to examine all the module directories
  before moving on to other sources of task actions.
* Change several variables we consult to see if a task is in a certain
  category from lists/tuples to frozensets.  These are static lists
  which we only do containment tests on so frozensets should be faster
7 years ago
Toshio Kuratomi b945846814 Remove dead code
I wrote this code because we had a traceback at some point with pyyaml
not able to handle our AnsibleUnicode type (the C library was doing an
exact match to python's unicode type rather than testing with the object
inheritance hierarchy.) Neither jimi think that AnsibleUnicode gets
passed into this function anymore, though, so we think it is safe to remove.
7 years ago
Andrew Gaffney dae737c8b7 Only template each hostvars var on-demand (fixes #33259) 7 years ago
Matt Martz f659cc5963
Guard against assignments that are not ast.Name in docs parsing (#35513) 7 years ago
Michael Scherer 060001b08d Remove deprecated code (#34828)
* Remove compat code for to_unicode, to_str and to_bytes

Code was marked as deprecated and to be removed after 2.4

* Remove is_encrypted and is_encrypted_file

Code was marked as deprecated after 2.4 release.
7 years ago
Toshio Kuratomi 340a7be7c3 Implement plugin filtering 7 years ago
Toshio Kuratomi 14c3b4d8e5 Move yaml loading functions from DataLoader into a utility module. 7 years ago
Adrian Likins ffe0ddea96
add a vault --encrypt-vault-to specify vault id to use for encrypt (#31067)
Enforce that there can be only one --new-vault-id or
--new-vault-password-file and use this instead of
--encrypt-vault-id

* Add a config option for default vault encrypt id
7 years ago
Adrian Likins fc180a378a
Support using vault password files that are themselves vault encrypted (#27668)
Extract vault related bits of DataLoader._get_file_contents to DataLoader._decrypt_if_vault_data

When loading vault password files, detect if they are vault encrypted, and if so, try to decrypt with any already known vault secrets.

This implements the 'Allow vault password files to be vault encrypted'  (#31002) feature card from
the 2.5.0 project at https://github.com/ansible/ansible/projects/9

Fixes #31002
7 years ago
Abhijeet Kasurde 9f969a2176
Misc. typo is display log (#34388)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
7 years ago
Toshio Kuratomi 235bdfb996 Remove shell module specialcases
Shell is implemented via the command module.  There was a special case
in mod_args to do that.  Make shell into an action plugin to handle that
instead.

Also move the special case for the command nanny into a command module
action plugin.  This is more appropriate as we then do not have to send
a parameter that is only for the command module to every single module.
7 years ago
Toshio Kuratomi b06f7688cd Fix a unicodeerror so that we get the correct error message (#34058)
Fixes #33044
7 years ago