Commit Graph

69 Commits (a8b9d7f8be73379c4118704b9f25ff863e1bb220)

Author SHA1 Message Date
Matt Clay 74ac1c2e4a Test cleanup. 7 years ago
Matt Martz 70b31abf85
Fix tests as filters 5 (#34406)
* Address recent tests as filters in win_iis_webbinding tests

* Make no-tests-as-filters.py so that it is executed
7 years ago
Matt Martz 57575d1cfa
Fix tests as filters #4 (#33930)
* Resolve newly added tests as filters

* Add code smell to test for ansible provided jinja tests as filters syntax

* Add docs for no-tests-as-filters code smell test

* Address tests as filters in new integration tests

* Address feedback

* Address feedback 2
7 years ago
Toshio Kuratomi 7bd0af15d2
Enhance empty __init__ code-smell to check module_utils (#33438)
Empty __init__ will allow us to use python namespaces with all of these
files.  That may be something we want to take advantage of for allowign
them to be expanded by user dirs.  Also might be needed for AnsiballZ or
other wrapper enhancements in the future.
7 years ago
Matt Martz 99d4f5bab4 Remove uses of assert in production code (#32079)
* Remove uses of assert in production code

* Fix assertion

* Add code smell test for assertions, currently limited to lib/ansible

* Fix assertion

* Add docs for no-assert

* Remove new assert from enos

* Fix assert in module_utils.connection
7 years ago
Toshio Kuratomi 0b320ba15c Generate whitelist of current files using underscore variable and enable test 7 years ago
Matt Clay cf1337ca9a Update ansible-test sanity command. (#31958)
* Use correct pip version in ansible-test.
* Add git fallback for validate-modules.
* Run sanity tests in a docker container.
* Use correct python version for sanity tests.
* Pin docker completion images and add default.
* Split pylint execution into multiple contexts.
* Only test .py files in use-argspec-type-path test.
* Accept identical python interpeter name or binary.
* Switch cloud tests to default container.
* Remove unused extras from pip install.
* Filter out empty pip commands.
* Don't force running of pip list.
* Support delegation for windows and network tests.
* Fix ansible-test python version usage.
* Fix ansible-test python version skipping.
* Use absolute path for log in ansible-test.
* Run vyos_command test on python 3.
* Fix windows/network instance persistence.
* Add `test/cache` dir to classification.
* Enable more python versions for network tests.
* Fix cs_router test.
7 years ago
Matt Clay 36b13e3e3d Fix ansible-test default image. (#31966)
* Add openssh-client to default docker container.
* Include Azure requirements in default container.

To do so, handling of pip requirements was updated to install each
set of requirements separately and then run a verification pass to
make sure there are no conflicts between requirements.

* Add missing --docker-no-pull option.
* Add documentation for the azure-requirements test.
7 years ago
Matt Clay 6c3339402a Improve python 2/3 ABC fallback for pylint. (#31848)
* Improve python 2/3 ABC fallback for pylint.
* Allow longer method names in ansible-test.
7 years ago
Toshio Kuratomi b454115e9f Update test cases
* Remove cloudstack, f5, and nxos from the wildcard import whitelist
* Remove nxos and f5 from the get_exception whitelist
* Remove some files from the pep8 legacy whitelist
7 years ago
Toshio Kuratomi c81314dcb9 Add a hack to test whether type=path should be used in a module 7 years ago
Toshio Kuratomi 5e9384300d Add boilerplate and remove wildcard imports for openstack modules (#31461)
* Add boilerplate and remove wildcard imports for openstack modules
* Remove openstack files that are now pep8 compliant from the legacy list
7 years ago
Toshio Kuratomi 9c81257ce3 Remove wildcard imports from amazon modules (#31451)
* Remove wildcard imports from amazon modules
* Remove files from legacy pep8 that now pass
7 years ago
Hervé Beraud 0e9ae5b8cc [fix] no-unicode-literals sanity error on ansible.egg-info (#30446)
* [fix] no-unicode-literals sanity error on ansible.egg-info
7 years ago
Toshio Kuratomi c82cf791dd Add a code-smell test for smart quotes and remove smart quotes from all files 7 years ago
Toshio Kuratomi ff13d58c14 Remove use of unicode_literals as it is an anti-pattern
from __future__ unicode_literals leads to developer confusion as
developers no longer can tell whether a bare literal string is a byte
string or a unicode string.  Explicit marking as u"" or b"" is the way
to solve the same problem in the Ansbile codebase.
7 years ago
Toshio Kuratomi 7cd81b802d Fix wildcard imports, remove get_exception, add boilerplate
Fixed for monitoring modules
7 years ago
Toshio Kuratomi ac56a2f138 Remove wildcard imports and get_exception calls
Fixed module_utils
7 years ago
Toshio Kuratomi 44935a5db6 Add a (disabled) code-smell test for detecting _ variables
We are reserving the _ identifier for i18n work.  Code should use the
identifier dummy for dummy variables instead.

This test is currently skipped as someone needs to generate the list of
files which are currently out of compliance before this can be turned
on.
7 years ago
Toshio Kuratomi fbf6cb8fff Add boilerplate to fetch.py
This completes the modules/files directory so we can now check it for
regressions.
7 years ago
Toshio Kuratomi f4d7b9a596 code-smell test changes
* Create get_exception and wildcard import code-smell tests
* Add more detail to boilerplate and no-basestring descriptions
* Remove the no-list-cmp test as the pylint undefined-variable test covers it
7 years ago
Abhijeet Kasurde 97240a9ebc Fix for missing import and boilerplate
Fix adds missing imports and boilerplate for proxysql.
It also remove get_exception calls in-favor of native exception.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
7 years ago
Abhijeet Kasurde 5e2184c683 Fixing broken imports and get_exception for ipa
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
7 years ago
Robin Roth b134352d8c WIP: Replace boilerplate (#27554)
* Replace boilerplate

* Enable test on packaging/os
7 years ago
Toshio Kuratomi 0b9a78f0b3 Remove wildcard, add boilerplate and get rid of get_exception
* smaller collections of database modules
* Some of the smaller collections of network modules
7 years ago
Toshio Kuratomi 4109a82d0a remove wildcards and add boilerplate
* cloud/centurylink
* source_control
* cloud/google
* notification
* cloud/rackspace
* cloud/vmware
7 years ago
Toshio Kuratomi 0765ceb66d revise ordered list of boilerplate work 7 years ago
Toshio Kuratomi 8de6cff2b1 more wildcards 7 years ago
Toshio Kuratomi 1ee08c0f06 Get rid of more wildcard imports and add boilerplate to more modules
This commit cleans up the following module categories:
* messaging
* inventory
* crypto
* commands
* clustering
* cloud/webfaction
* cloud/docker
* cloud/digital_ocean
7 years ago
Toshio Kuratomi 8b7db55a94 Modify boilerplate code-smell test to check some of the module dirs 7 years ago
Toshio Kuratomi 225fa5d092 Fix undefined variables, basestring usage, and some associated python3 issues 7 years ago
Toshio Kuratomi 9f7b0dfc30 Remove automatic use of system six
* Enable the pylint no-name-in-module check.  Checks that identifiers in
  imports actually exist.  When we do this, we also have to ignore
  _MovedItems used in our bundled six.  This means pylint won't check
  for bad imports below ansible.module_utils.six.moves but that's
  something that pylint punts on with a system copy of six so this is
  still an improvement.
* Remove automatic use of system six.  The exec in the six code which
  tried to use a system library if available destroyed pylint's ability
  to check for imports of identifiers which did not exist (the
  no-name-in-module check).  That test is important enough that we
  should sacrifice the bundling detection in favour of the test.
  Distributions that want to unbundle six can replace the bundled six in
  ansible/module_utils/six/__init__.py to unbundle.  however, be aware
  that six is tricky to unbundle.  They may want to base their efforts
  off the code we were using:

  2fff690caa/lib/ansible/module_utils/six/__init__.py

* Update tests for new location of bundled six Several code-smell tests
  whitelist the bundled six library.  Update the path to the library so
  that they work.

* Also check for basestring in modules as the enabled pylint tests will
  also point out basestring usage for us.
7 years ago
Matt Clay 789218c215 Initial ansible-test sanity docs. (#26775)
* Rename no-iterkeys test for consistency.

* Require docs for all ansible-test sanity tests.

* Initial ansible-test sanity docs.

* Fix capitalization of Python.

* Fix sanity code smell test false positives.

* Fix another code-smell false positive.
7 years ago
Brian Coca 272f5fd03a avoid doing cmp on .git 8 years ago
Brian Coca 74842adc07 1st part of ansible config, adds ansible-config to view/manage configs (#12797)
* Start of ansible config project

moved configuration definitions to external yaml file vs hardcoded
 * updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
 * prints green for default/unchanged and yellow for those that have been overriden
 * added list action to show all configurable settings and their associated ini and env var names
 * allows specifying config file to see what result would look like
 * TBD update, edit and view options

removed test for functions that have been removed

env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling

updated as per feedback

* pep8
8 years ago
Chris Alfonso b937bad52e Narrowing the match for python 3 cmp check (#25220)
Allowing the yaml examples in python documentation.
8 years ago
Dag Wieers 4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
8 years ago
kkjang 5c43bd3bae Cmp python3 compat (#25008)
* Use double-quotes for expect integration tests

* Cast user input to string for expect integration tests

* Remove usage of cmp() for python3 compatibility

- Add code smell test to look for cmp usage

- Fixes #24756
8 years ago
Brian Coca 8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
8 years ago
Matt Clay a07d42e16d Add support for cloud tests to ansible-test. (#24315)
* Split out ansible-test docker functions.
* Add cloud support to ansible-test.
8 years ago
Toshio Kuratomi 3428f42120 Add integration test that modules are loaded from the expected locations (#24170)
* Add integration test that modules are loaded from the expected locations
* Fix ping module to pass PEP8
8 years ago
Matt Clay d662f6f0db Fix bugs in ansible-test units command. (#24044)
* Handle old versions of coverage.
* Handle old versions of setuptools.
* Detect python version for docker/remote units.
* Add sanity override for test constraints.
8 years ago
Matt Clay e7bb508ad6 Fix var precedence check to support python 3. (#23552)
* Fix var precedence check to support python 3.
* Run CI sanity tests using python 3.5.
* Disable pylint non-iterator-returned test to pass on python 3.5.
8 years ago
Lumír 'Frenzy' Balhar 9d41aefd71 Add Python 3 support for validate-modules test. (#21195)
* Improve PEP8 compatibility

* Fix Python 3 incompatibility

Is prohibited to mutate OrderedDict during iteration through it so
is better to add records with error or warning to empty dictionary
instead of delete records from copy of dictionary during iterating.

* Decode output of subprocess from bytes to unicode.

* Add Python 3 support for validate-modules test.

Fix #18367
8 years ago
Dag Wieers ac43a1bbbc Windows: Use the correct newline sequence for the platform (#21846)
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.

We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.

This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414

And also relates to issue #21128
8 years ago
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
8 years ago
James Cammarata a2599cab79 Fix variable precedence of INI/script vars to be in-line with docs.
This commit also adds a new test script (ansible-var-precedence-check.py in code-smell/)
to provide us with another line of defense against precedence bugs going forward.

The precedence docs state that the INI vars have a lower precedence than group/host
vars files for inventory and playbooks, however that has not been the case since 2.0
was released. This change fixes that in one way, though not exactly as the docs say.
The rules are:

1) INI/script < inventory dir < playbook dir
2) "all" group vars < other group_vars < host_vars

So the new order will be (from the test script mentioned above):

8. pb_host_vars_file - var in playbook/host_vars/host
9. ini_host_vars_file - var in inventory/host_vars/host
10. ini_host - host var inside the ini
11. pb_group_vars_file_child - var in playbook/group_vars/child
12. ini_group_vars_file_child - var in inventory/group_vars/child
13. pb_group_vars_file_parent - var in playbook/group_vars/parent
14. ini_group_vars_file_parent - var in inventory/group_vars/parent
15. pb_group_vars_file_all - var in playbook/group_vars/all
16. ini_group_vars_file_all - var in inventory/group_vars/all
17. ini_child - child group var inside the ini
18. ini_parent - parent group var inside the ini
19. ini_all - all group var inside the ini

Fixes #21845
8 years ago
Matt Clay d66ce40ecb Overhaul `ansible-test sanity` implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
8 years ago
Matt Clay cba91c7f44 Test placement (#21801)
* Fix test placement and update messaging.
* Add note about group selection.
8 years ago
Matt Clay b5e0eb0985 Fixes for missing aliases files sanity test. (#21798)
* Refine test for missing aliases files.
* Fix formatting of missing aliases message.
8 years ago