Commit Graph

120 Commits (fadf7a426fef8e61110d35e5b3d6c546a024a8dc)

Author SHA1 Message Date
Matt Clay 4063d58339 Use relative submodule status in ansible-test.
The `git submodule status` command is relative to the current git repository by default.
When running from a repository subdirectory paths can be returned above the current directory.
Specifying the current directory with `git submodule status` avoids listing submodules above that directory.

This will fix issues when testing a collection that is rooted below the repository root when that repository uses submodules.
5 years ago
Matt Clay 2215a62057 Fix ansible-test handling of submodules.
Resolves https://github.com/ansible/ansible/issues/61550
5 years ago
Toshio Kuratomi 68fa03138c Revert "New Module: NetApp Account (#59939)"
This reverts commit 6a8dcb8fcb.

Reverting this as it breaks CI for azure (more permissions would be
needed for our account)
5 years ago
Chris Archibald 6a8dcb8fcb New Module: NetApp Account (#59939)
* netapp account module for azure

* netapp account module for azure

* fixes

* fixes

* add requirements

* add requirements

* fix issues

* add units

* add units

* updates

* updates

* fixes

* fixes

* fies

* fixes
5 years ago
Matt Clay f90587069c
Update collections integration targets path. (#61523)
* Update collections integration targets path.

* Fix integration path handling.

* Add collections test target fallback.

Also add warnings and errors for common path mistakes.

* Improve role target detection.
5 years ago
Matt Clay 032bd3a030 Move the ansible-test output_dir.
This directory is currently a fixed location to make troubleshooting easier.
It is cleared before each test target runs, but is preserved when a test target finishes.
This allows the contents to be inspected when a test fails.

The previous location was `~/ansible_testing/`.
The new location is within the content root:

- `test/results/.tmp/output_dir` for Ansible
- `tests/output/.tmp/output_dir` for Ansible Collections

Moving the directory reduces the number of places on the filesystem where tests create output.
It also enables the results to be returned from delegated systems.
5 years ago
Matt Clay d92e2a6b0e
Simplify ansible-test target processing. (#61506)
* Clean up layout paths for integration tests.

* Remove "special" integration test target type.

* Remove unnecessary role detection logic.

* Remove support for non-sh runme scripts.

* Simplify reading of aliases.
5 years ago
Matt Clay ed46f83905 Fix paths in sanity tests. 5 years ago
Matt Clay 81ca44b7e5 Change location of collections tests.
The directories used for collections tests are changing as follows:

`test/` -> `tests/`
`test/results/` -> `tests/output/`

This is a breaking change for collections tests executed by ansible-test.
All collections will need to be updated to use the new directory.

The `tests/output/` directory should be added to the `.gitignore` or equivalent in each collection.

This change is being made before the first pre-release of Ansible 2.9 since ansible-test has not yet been shipped.
Using the `tests/` directory matches the other collections directories `plugins/` and `roles/`.

This resolves https://github.com/ansible/ansible/issues/60218
5 years ago
Matt Clay 460b583dff Improve ansible-test --venv error handling.
This allows it to fall back to virtualenv when venv is broken.
5 years ago
Sam Doran 8dd79fbbd2 Show doc URL for sanity test failures 5 years ago
Matt Clay 4afe757f93
More ansible-test path handling updates. (#61484)
* Move test results vars into layout.

* Add test_path to layout.

* Add sanity_path to layout.

* Clean up layout.

* Update change classification.

* Improve classification layout.

* Relocate common classification code.

* Use is_subdir for relocated code.

* Relocate ansible test/units/compat/ hack.

* Fix variable shadowing.

* Remove unused code.

* Fix ordering of path tests.

* Clean up ansible classification.

* Fix hard-coded plugin paths in classification.

* Relocate more common classification.

* Fix logic.

* Fix pylint issue.

* Add missing licenses classification.
5 years ago
Matt Clay 8521474f06 Revert "maven_artifact.py - add support for version ranges by using spec (#54309)"
This reverts commit 145a6a8756.
5 years ago
Sam Doran 73248bf27d validate-modules - Use error codes instead of numbers (#60711) 5 years ago
Matt Clay 1db78d4ba4 Attempt quick fix for CI failures. 5 years ago
Matt Clay 830f995ed4
Add a --venv option to ansible-test. (#61422)
* Add --venv delegation to ansible-test.

* Update import sanity test venv creation.

* Fix import test when using --venv on Python 2.x.

* Improve virtualenv setup overhead.

* Hide pip noise for import sanity test.

* Raise verbosity on venv info messages.

* Get rid of base branch noise for collections.

* Add missing --requirements check.
5 years ago
Matt Clay f5d829392a
Overhaul ansible-test test path handling. (#61416)
* Remove .keep files from test/results/ dirs.

* Remove classification of test/results/ dir.

* Add results_relative to data context.

* Use variables in delegation paths.

* Standardize file writing and results paths.

* Fix issues reported by PyCharm.

* Clean up invocation of coverage command.

It now runs through the injector.

* Hack to allow intercept_command in cover.py.

* Simplify git ignore for test results.

* Use test result tmp dir instead of cache dir.

* Remove old .pytest_cache reference.

* Fix unit test docker delegation.

* Show HTML report link.

* Clean up more results references.

* Move import sanity test output to .tmp dir.

* Exclude test results dir from coverage.

* Fix import sanity test lib paths.

* Fix hard-coded import test paths.

* Fix most hard-coded integration test paths.

* Fix PyCharm warnings.

* Fix import placement.

* Fix integration test dir path.

* Fix Shippable scripts.

* Fix Shippable matrix check.

* Overhaul key pair management.
5 years ago
Matt Clay f510d59943
Support relative imports in AnsiballZ. (#61196) 5 years ago
Jordan Borean 66f52b74b1 Change collection PS util import pattern (#61307)
* Change collection PS util import pattern

* Add changes for py2 compat

* fix up regex and doc errors

* fix up import analysis

* Sanity fix for 2.6 CI workers

* Get collection util path for coverage collection
5 years ago
Jordan Borean faaa669764
PowerShell - Added coverage collector (#59009)
* Added coverage collection for PowerShell - ci_complete ci_coverage

* uncomment out coverage uploader call

* Generate XML for PowerShell coverage

* Use whitelist to exclude coverage run on non content plugins

* Remove uneeded ignore entry

* Try to reduce diff in cover.py

* Fix up coverage report package - ci_complete ci_coverage
5 years ago
Gonéri Le Bouder e7a9d71ac0
ansible-test: async deployment w/ vcenter provider (#59984)
ansible-test: async deployment w/ vcenter provider

So far, `ansible-test`'s vcenter provider was trigger by a blocking POST
call. Since the depoyment take a lot of time (> 8m), we decided to use
an async call instead and poll the API server until the deployment
is ready.
5 years ago
Matt Clay 8a7b89c66f Support delegation testing collections with deps.
All collections in the same collection root as the collection being tested will be sent to the remote system/container.
5 years ago
Yuwei Zhou b4732dd2e6 Support Azure IoT hub and related module (#55121)
* Support iothub creation

* raise errordetailexception rather than clouderror

* add facts

* change requirement

* compare endpoint

* add documentation

* add documentation

* add iot device facts

* modify line ending

* add auth method

* add iot module

* add consumer group

* add the test

* enhencement of  doc

* add list consumer groups

* fix lint

* fix lint

* fix doc

* fix doc

* Update auzre_rm_iothub related document

* changed paramter's type

* update type

* rename facts -> info

* fixed sanity

* missed during merge
5 years ago
Matt Clay cca4eb71e3
Update sanity tests for collections. (#61352)
* Remove pointless sanity tests on `bin` dir.

* Update action-plugin-docs test for collections.

* Update no-main-display test for collections.

* Update empty-init test for collections.

* Update no-assert test for collections.

* Move required-and-default-attributes test.

This test only applies to Ansible itself.

* Update use-argspec-type-path test for collections.
5 years ago
Yuwei Zhou 1c06b302bc Support automation account managment (#56429)
* add automation account

* add test

* update sdk version

* fix doc

* Update azure_rm_automationaccount.py

* Update azure_rm_automationaccount_facts.py

* Update main.yml

* add type define (#64)

* add new (#67)

* change 'example' to 'sample'

* changed 'path' to 'ID'
5 years ago
Matt Clay e258ba703e
Improve ansible-test env display. (#61339)
* Improve ansible-test env display.
* Fix PyCharm warnings.
* Fix ansible-test ansible-doc sanity test with -e.
5 years ago
Matt Clay 02c38ed4fd
Relocate ansible-test self tests outside package. (#61255)
* Relocate ansible-test self tests outside package.

We don't want to include the tests for verifying ansible-test within the ansible-test package.

* Add `test/ansible_test/` to classification.py.

* Fix test invocation.

* Relocate tests in MANIFEST.in.

* Improve package-data sanity test error checking.

* Only use includes for ansible-test in MANIFEST.in.

* Improve readability of MANIFEST.in.
5 years ago
Matt Clay 1fbb930747 Add out-of-tree check to symlinks sanity test. 5 years ago
Matt Clay 2a9b996bd2 Use `.cfg` extension for pylint configs. 5 years ago
Matt Clay 6ec0b4ec86
Sanity test updates for collections support. (#61248)
* Run no-unwanted-files sanity test only on Ansible.

Since collections should be able to use binary modules there is not really any limit on what could exist in a collection `plugins` directory.

* Add support for symlinks in sanity target lists:

- Sanity tests that need to analyze symlinks can do so using the supplied target list.
- Tests that analyze directories will now only look at symlinks if requested.
- Directory symlinks will now be seen as directories instead of files.

* Enable symlinks on filename based sanity tests.

Sanity tests that evalulate filenames instead of content should include symlinks.

* Update symlinks sanity test.

Use the sanity test target list now that it can include symlinks.
5 years ago
Felix Fontein 4551965af1 ACME modules: bump test container version (#61192)
* Fix root cert path.

* Bump testing container version.

* Improve check (needed due to letsencrypt/pebble#257).
5 years ago
Matt Clay 2eff25aef8 Update ansible-test network inventory handling.
DCI uses an inventory directory instead of a file, along with --no-temp-workdir.

This changes the inventory check to keep that scenario working until all DCI jobs can be migrated to Zuul.
5 years ago
Matt Martz adb886e4ce
Speed up units using time.sleep by mocking or shortening sleep time (#61064)
* Speed up units using time.sleep by mocking or shortening sleep time

* Only use durations when not a collection
5 years ago
Matt Clay 3551d83289 Fix ansible-test payload bin creation. 5 years ago
Matt Clay 29ac0273d4
Update ansible-test collection inventory handling. (#61031)
* Update ansible-test collection inventory handling.

- The `windows-integration` command now supports the `--inventory` option.
- The incomplete support for host_vars and group_vars directories has been removed.
- The incomplete support for an inventory directory has been removed.
- The inventory specified by `--inventory` can now reside outside the install and content roots.
- Using `ansible_ssh_private_key_file` with `--docker` or `--remote` results in a warning about the combination being unsupported and likely to fail.

* Fix config handling.

* Fix payload handling of ssh keys.

* Disable pylint no-self-use rule for ansible-test.

* De-duplicate payload paths.
5 years ago
Pedro Magalhães 145a6a8756 maven_artifact.py - add support for version ranges by using spec (#54309) 5 years ago
Matt Davis bfa004930a
add subdir support to collection loading (#60682)
* add subdir support to collection loading

* collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py
* centralize parsing/validation in AnsibleCollectionRef class
* fix issues loading Jinja2 plugins from multiple sources 
* resolves #59462, #59890,

* sanity test fixes

* string fixes

* add changelog entry
5 years ago
Matt Clay 32b0a72547
Support windows-integration on collections. (#60984)
* Support windows-integration on collections.

* Update configure-remoting-ps1 sanity test.

* Add copy of ConfigureRemotingForAnsible.ps1.

* Add ignore entry for copied file.
5 years ago
Matt Clay 92380d7050
Update ansible-test --windows inventory creation. (#60911)
Host specific settings are created as part of the generated inventory instead of using a host_vars directory.

Also update connection tests to work properly with the generated inventory changes.
5 years ago
Matt Clay ebe55e83c3 Fix ansible-test bin creation.
Use symlink instead of link.
5 years ago
Matt Martz def3d1f815
validate-modules: support collections (#60247)
* Start of work to support collections

* remove version_added from base schema

* If a collection, pass that to validate-modules

* clean ups

* Allow version_added in a collection, just make it optional

* Don't traceback on missing doc_fragment

* Don't validate metadata in a collection
5 years ago
Matt Clay d9c44897a6 Fix ansible-test cloud vars overwrite. 5 years ago
Matt Clay 7968dc4c00 Load integration config test vars from content.
The `test/integration/integration_config.yml` vars file will now be loaded from the content under test and is now optional.

The `output_dir` and `win_output_dir` vars are now provided by ansible-test.
5 years ago
Matt Clay ba1b4565d3 Clean up ansible-test results path handling.
This brings us closer to being able to use `tests/` instead of `test/` in collections.
5 years ago
Matt Clay 1b8aa798df
Update ansible-test integration config handling. (#60448)
* Move Ansible test config into env vars.

This allows ansible-test to use an empty Ansible config file, leaving open the option for users to customize the one used to run tests (although such usage is discouraged).

* Use config from content under test when present.
5 years ago
Sam Doran a20848bf66
Use new version of default test image that contains Python 3.8.0b3 (#60139) 5 years ago
Matt Clay 25d396fdfc Fix testing of Windows binary modules. 5 years ago
Sam Doran c485a1b91e
Fix sanity tests based on newer version of shellcheck (#60423)
- change egrep to grep -E
- store exit codes and check them directly
- ignore SC1091
5 years ago
Matt Martz 697b566971
Update units to pass on macOS (#60435)
* Update units to pass on macOS. Fixes #27810

* raising=False
5 years ago
Gonéri Le Bouder a3407682ed ansible-test: fix the vcenter+vcsim support
The vcenter provider allow one to use the `VMWARE_TEST_PLATFORM` to
manually select a test platform. The test platforms are:

- `govcsim`
- `static`
- and `worldstream`.

Before this commit, the `govcsim` value was ignored.

Example:

```shell
$ VMWARE_TEST_PLATFORM=govcsim ansible-test integration --python 3.7 needs/target/prepare_vmware_tests/
```
5 years ago
Matt Clay 473bd6bb0a Relax boilerplate sanity tests.
This will avoid requiring boilerplate for documentation only python files.
5 years ago
Matt Clay 3fac6a0adb Fix ansible-test TypeVar type annotations. 5 years ago
Matt Clay cb68110296
Clean up ansible-test inventory templates. (#60321)
* Move ansible-test inventory templates.
* Remove obsolete inventoy.remote.template file.
5 years ago
Matt Clay 2a9964ede8
Code cleanup in ansible-test. (#60318)
* Move ansible-test units code into separate dir.
* Fix --explain on "code smell" sanity tests.
* Add --strict-markers to pytest invoctaion.
* Fix classification of units code changes.
5 years ago
Matt Clay df59f06d6a
Additional ansible-test path fixes. (#60300)
* Add missing ansible-test injector symlink.
* More ansible-test path fixes.
* Expand bin-symlinks docs.
5 years ago
Matt Clay 39b3fc0926 Update ansible-test layout and payload handling. 5 years ago
Abhijeet Kasurde 647b78a09c
VMware: Check if datastore is valid (#58872)
Check validation of datastore selected using autoselect_datastore API.

Fixes: #58541

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
5 years ago
Matt Clay 57dc7ec265
Prepare ansible-test for inclusion in setup.py (#60294)
* Cache ansible version lookup.
* Fix method of determining ANSIBLE_ROOT.
* Clean up based on PyCharm inspections.
* Generate minimal PKG-INFO without setup.py.
* Use ANSIBLE_LIB_ROOT where possible.
* Use import instead of subprocess to get version.
* Fix install layout type.
* Correct required paths message for installs.
* Update list of files copied during delegation.
* Fix ansible-test entry point.
* Fix pylint issue.
* Fix version lookup on Python 2.x.
* Fix pylint issue.
* Remove unwanted print statement.
5 years ago
Sam Doran 9cb47832d1 Pin openshift <=0.9.0 to prevent test failures (#60245) 5 years ago
Toshio Kuratomi b0ec91e69e Autointerpreter table (#60148)
* Some framework for docs

* Separate CSS file for our site-specific CSS.
* Override the read-the-docs theme for tables so that tables don't
  always horizontally scroll
* Add a |br| substitution that lets us line break inside of tables
* Add |_| non-breaking-space substitution which is also for formatting
  tables
* Configure rstcheck to ignore all substitutions which are being added
  by sphinx in the conf.py

* Fix table of auto interpreter options

The table was being hardcoded at a certain width to work around a
read-the-docs theme bug.  Fix the bug instead and format the table using
better sphinx practices.

* Remove unused substitutions

We had substitutions defined that were never used in our documentation.
Remove those.

Also add to the rstcheck whitelist three substitutions which are defined
by sphinx itself, version, release, and today.
5 years ago
Matt Clay 97a0810e0f
Move ansible-test entry point. (#60281) 5 years ago
Matt Clay 5f67b5a757 Move ansible-test coverage config to test/lib/ 5 years ago
Matt Clay 2e7d36a3f9
Fix placement and handling of ansible-test cloud config (#60253)
* Move cloud templates.
* Update paths for cloud config templates.
* Fix cloud config paths.
* Fix cloud-config classification.
5 years ago
Matt Clay 278044358c Split up ansible-test sanity test requirements. 5 years ago
Matt Clay a93a0c8fd6
Relocate validate-modules for easier testing. (#60214)
* Relocate module validator code and tests.
* Fix validate-modules entry point and imports.
* Fix paths and test entry points.
* Fix up unit tests.
* Fix shebang and execute bit.
5 years ago
Matt Clay 9e31cf7270 Avoid pylint error on Python 3.7. 5 years ago
Matt Clay 18867847f4
Split up "code smell" sanity tests. (#60174)
* Relocate ansible-only sanity tests.

* Get "code smell" sanity tests from multiple dirs.

- `test/lib/ansible_test/_data/sanity/code-smell/` - General purpose tests used for both Ansible and Ansible Collections.
- `test/sanity/code-smell/` - Tests specific to Ansible, will not be used for Ansible Collections.
5 years ago
Matt Clay 709a8c5026 Fix ansible-test classification of sanity changes. 5 years ago
Matt Clay 9da5908afb Fix ansible-test layout path generation. 5 years ago
Matt Clay d651bda123
Relocate ansible-test code. (#60147)
* Initial move of `test/runner/` content.

`test/runner/lib/` -> `test/lib/ansible_test/_internal/`
`test/runner/`     -> `test/lib/ansible_test/_internal/data/`

* Initial move of `test/sanity/` content.

`test/sanity/` -> `test/lib/ansible_test/_internal/data/sanity/` (except `test/sanity/ignore.txt`)

* Initial move of `test/units/pytest/` content.

`test/units/pytest/` -> `test/lib/ansible_test/_internal/data/pytest/`

* Follow-up move of `test/runner/unit/` content.

`test/lib/ansible_test/_internal/data/unit/` -> `test/lib/ansible_test/tests/unit/`

* Initial move of `ansible.cfg` content.

`test/units/ansible.cfg` -> `test/lib/ansible_test/_internal/data/units/ansible.cfg`
`test/env/ansible.cfg` -> `test/lib/ansible_test/_internal/data/env/ansible.cfg`

* Follow-up move of `data` directory.

`test/lib/ansible_test/_internal/data/` -> `test/lib/ansible_test/_data/`

* Update import statements.

* Add missing __init__.py for unit tests.

* Fix path references and miscellaneous issues.
5 years ago