Commit Graph

39 Commits (ce47785dd6aa72f8498436dec9f5b9058eb000f1)

Author SHA1 Message Date
Rick Elrod e2a57414f4 Remove `with` statement for pytest-mock unit tests
As per:
https://github.com/pytest-dev/pytest-mock#note-about-usage-as-context-manager
pytest-mock is not meant to be used within a `with` context or as a
decorator. Instead, pytest-mock will automatically unpatch the mocked
methods when each test is complete.

In newer pytest-mock, this use actually throws an exception and causes
the tests to fail.

This hasn't been hit in Ansible's CI yet, because the docker image
that the tests run in uses an older version of pytest-mock. However,
there is no constraint on the upper bound of pytest-mock in
test/lib/ansible_test/_data/requirements/constraints.txt which means
that when running the tests locally, outside of that docker image, the
tests never pass.

This patch removes the `with` context in each such case.

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Abhay Kadam 2fa8f9cfd8 Fix env lookup plugin error on utf8 values (#65541)
This commit fixes issue #65297.

The env lookup plugin used to fail when environment variable value
contained any UTF-8 characters (e.g., δ, ζ).
5 years ago
Mads Jensen 0f491c0289 Replace TestCase.assertEquals with TestCase.assertEqual. 5 years ago
Mads Jensen c5d61953b1 Remove empty setUp/tearDown/tearDownClass methods in test classes. 5 years ago
Mark Chappell ef7d060a3f AWS module_utils: Clear out Sanity Test issues (#63991) 5 years ago
Matt Martz c81a1057e1
Speed up a few more tests by mocking time.sleep (#61186) 5 years ago
Felix Fontein cd4736d893 missing_required_lib: add information how to change Python interpreter (#60736)
* Add information how to change Python interpreter used by Ansible.

* Update lib/ansible/module_utils/basic.py

Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>

* Make test less dependent on exact message.
5 years ago
Andrey Klychkov 21863d48f3 unit tests: remove unused imports (#60462) 5 years ago
Sandeep Bandi 1d82d25ea2 Adding Avi ansible lookup module (#58667)
* Adding Avi ansible lookup module

(cherry picked from commit 77b8951f68cbc889e6595b2a359ca27b84a43c0d)

* Added description for examples

* Added debug logs and unit tests

* Fix __builtin__ import and restting super

* Fix pep8 errors

* Updated as per review comments on IP address
5 years ago
Toshio Kuratomi 8acf71f78f
Skip tests with unsatisfied deps (#55853)
* Skip gitlab tests if dependencies aren't met

* Skip certain unittests if passlib is not installed

* Fix tests with deps on paramiko to skip if paramiko is not installed

* Use pytest to skip for cloudstack

If either on Python-2.6 or the cs library is not installed we cannot run
this test so skip it
6 years ago
Olivier Scherler 7ed7d374e4 Fix onepassword lookup plugin and onepassword_facts module when a field has no name. (#58308)
* Fix onepassword lookup plugin crashing on fields with no 'name' or 't' property.

* Fix onepassword_facts module crashing on fields with no 'name' or 't' property.

* Add unit test for onepassword lookup plugin failing on entries without a name.

* Add changelog fragment for onepassword lookup plugin and onepassword_facts module fixes on fields without a name.
6 years ago
bobdoah 33d0d36eaa AWS: Fix KeyError in aws_secret lookup (#54792) 6 years ago
Kyryl Galanov 4a5b9bd8eb New lookup module: manifold (#50435)
* New lookup module: manifold

Add Manifold.co integration. The plugin fetches resource credentials
from Manifold service.

* module manifold: fix ansible lint warnings

* module manifold: fix false warning - split test assertion

* manifold module: fix unittest import

* manifold module: fix unittest patch

* manifold module: fix python3 requests getheader error
6 years ago
Jordan Borean 9717bc4bcf
Add laps_password lookup for retrieving a Windows LAPS Password (#52012)
* Add laps_password lookup for retrieving a Windows LAPS Password

* Remove python-ldap from requirements and fix doc typos

* enable unit tests without ldap dependency

* use config options for lookup plugin

* Update docs and tests based on further distro testing
6 years ago
Yannig Perré 6b41588e93 Fix with_ini example and unittest
* Fix example in ini.py
* Fix unittest in test_ini.py to pass CI as latest ansible returns list in
  different order. To prevent such issues in future results are sorted
* PEP8 E501 styling improvements

Co-Authored-By: Sergii Golovatiuk <sgolovat@redhat.com>
6 years ago
Toshio Kuratomi 3fba006207 Update bare exceptions to specify Exception.
This will keep us from accidentally catching program-exiting exceptions
like KeyboardInterupt and SystemExit.
6 years ago
Matt Clay 37b013aca3
Enable additional pylint rules and resolve issues found. (#47221)
* Resolve unneeded-not.
* Resolve global-at-module-level.
* Resolve useless-import-alias.
* Resolve bad-whitespace.
* Resolve global-variable-not-assigned.
* Resolve logging-not-lazy.
* Resolve comparison-with-itself.
6 years ago
Matt Clay 3033fd96b0
Move unit test compat code out of `lib/ansible/`. (#46996)
* Move ansible.compat.tests to test/units/compat/.
* Fix unit test references to ansible.compat.tests.
* Move builtins compat to separate file.
* Fix classification of test/units/compat/ dir.
6 years ago
Sam Doran b60854357b
Unify terms and UI between 1Password lookups and facts module (#45427)
* Unify login behavior between 1Password lookup plugins and module

- Use the same names for all credential aspects
- Only require the minimal amount of information for each
- Add more examples

* Change parameter terms

- use terms in line with 1Password documentation.
- update examples
- update tests

* Improve error messages in lookup plugin

* Unify onepassword_facts with lookup plugins

- use same methods and logic for signing in or reusing existing session
- unify terms with lookup plugins

* Change rc test for determing login

An rc other than 1 can be returned when a current login session does not exist.

* Create AnsibleModuleError class

ansible.errors is not available to modules, so create an AnsibleModuleError class within the module

Do not user os.path.expanduser since this is already done by virtue of the type being "path" in the argument spec.

* Add note about risk with fact caching sensitive data

* Add note on op version that was used for testing
6 years ago
Sam Doran 8cd8d17980
Add ability to unlock 1Password vault to lookup plugins (#44923)
* Add ability to use login to 1Password vault to 1Password lookups

* Adjust unit tests

* Add changelog
6 years ago
Matthias Fuchs 7871027c9d Share the implementation of hashing for both vars_prompt and password_hash (#21215)
* Share the implementation of hashing for both vars_prompt and password_hash.
* vars_prompt with encrypt does not require passlib for the algorithms
  supported by crypt.
* Additional checks ensure that there is always a result.
  This works around issues in the crypt.crypt python function that returns
  None for algorithms it does not know.
  Some modules (like user module) interprets None as no password at all,
  which is misleading.
* The password_hash filter supports all parameters of passlib.
  This allows users to provide a rounds parameter, fixing #15326.
* password_hash is not restricted to the subset provided by crypt.crypt,
  fixing one half of #17266.
* Updated documentation fixes other half of #17266.
* password_hash does not hard-code the salt-length, which fixes bcrypt
  in connection with passlib.
  bcrypt requires a salt with length 22, which fixes #25347
* Salts are only generated by ansible when using crypt.crypt.
  Otherwise passlib generates them.
* Avoids deprecated functionality of passlib with newer library versions.
* When no rounds are specified for sha256/sha256_crypt and sha512/sha512_crypt
  always uses the default values used by crypt, i.e. 5000 rounds.
  Before when installed passlibs' defaults were used.
  passlib changes its defaults with newer library versions, leading to non
  idempotent behavior.

  NOTE: This will lead to the recalculation of existing hashes generated
        with passlib and without a rounds parameter.
        Yet henceforth the hashes will remain the same.
        No matter the installed passlib version.
        Making these hashes idempotent.

Fixes #15326
Fixes #17266
Fixes #25347 except bcrypt still uses 2a, instead of the suggested 2b.

* random_salt is solely handled by encrypt.py.
  There is no _random_salt function there anymore.
  Also the test moved to test_encrypt.py.
* Uses pytest.skip when passlib is not available, instead of a silent return.
* More checks are executed when passlib is not available.

* Moves tests that require passlib into their own test-function.

* Uses the six library to reraise the exception.

* Fixes integration test.

When no rounds are provided the defaults of crypt are used.
In that case the rounds are not part of the resulting MCF output.
6 years ago
Zhikang Zhang 0971a342d8
Solve race condition in password lookup (#42529)
NOTE:
1. use os.open() with os.O_CREAT|os.O_EXCL to check existence
and create a lock file if not exists, it's an atomic operation
2. the fastest process will create the lock file and others will
wait until the lock file is removed
3. after the writer finished writing to the password file, all the reading
operations use built-in open so processes can read the file parallel
6 years ago
Zhikang Zhang 3a3869f4c8
Clarify usage of encrypt arg (#43330)
According to the do_encrypt interface, encrypt arg should be the hash method name used for encrypting returning password. But in the doc and lookup code it's a boolean flag, correct it to string.
6 years ago
Scott Buchanan b12cf754f6 1Password lookup plugin (#37207)
* add pytest_cache to gitignore

* onepassword lookup plugin

* fix linter/style test complaints

* second pass at making pycodestyle happy

* use json module instead of jq

* update copyrights, license & version added

* fix python2 compatibility

* doh. fix spacing issue.

* use standard ansible exception

* remove potentially problematic stdin argument

* actually call assertion method

* add support for top-level fields

* make vault uuids pedantically consistent in fixture

* fix new style issues

* ability specify section & correct case handling

* improve error handling

* add onepassword_raw plugin

* Add maintainer info

* Move common code to module_utils/onepassword.py

* Load raw data JSON data for easier use in Ansible

* Put OnePass class back inside lookup plugin

There is no good place for sharing code across lookups currently.

* Remove debugging code in unit tests

* Patche proper module in raw unit tests

* Add changelog entry

Co-authored-by: Scott Buchanan <sbuchanan@ri.pn>
7 years ago
mikedlr 52f2057472 aws ssm parameter lookup - change to return Nones for missing variables (#36456)
* aws ssm parameter lookup - change to return Nones for missing variables

* aws ssm parameter lookup - fix error case message to dump response

* aws ssm parameter lookup - fix integration test cases
7 years ago
mikedlr d31ded47fb Aws ssm multiple fixes (#35569)
* aws ssm parameter lookup test case - fails demonstrating no exception when parameter missing

* aws ssm parameter lookup - fail in case parameter doesn't exist

* aws ssm parameter lookup test case - failing case for nice return from path lookup

* aws ssm parameter lookup - convert incoming taglist to a key-value dictionary

* aws ssm parameter lookup - pep8 / style clean up

* aws_ssm lookup plugin rewrite for more standard interface

* aws_ssm module and lookup - introduce integration test and fix:

* aws_ssm module and lookup - error case integraton test and many PEP8 and other cleanups

* aws ssm parameter lookup - Various fixes in response to review + recursive fix & test

* aws ssm parameter lookup - more in response to review - shertel/abadger

* aws ssm parameter lookup unit test - move to mocker according to abadger

* aws ssm parameter lookup - integrate with new documentation fragment

* aws ssm parameter lookup - accept either aws_profile or boto_profile

* aws ssm parameter lookup - eliminate lookup document fragment until env vars are fixed later
7 years ago
Jason Vanderhoof 7c8e365dff Conjur Lookup Plugin (#34280)
* Imported lookup plugin from Role

* Plugin cleanup, including:
* Use existing Python YAML parsing
* Remove environment variables as connection options
* Added initial debugging information

* Reworked the lookup plugin using the Python Request library.  As it's available through Ansible, it makes communication with Conjur much more straight forward.

* Removed un-used libraries

* Fixed linting issues

* Standardized output on `format` and insure it works for 2.6, 2.7, and 3.x.

* Use quote_plus from the six library for improved python 2/3 behavior.

* Refactored identity & configuration to prefer user's file. This also includes a refactor to remove an un-needed dictionary merge method.

* Removed `requests` in favor of `ansible.module_utils.urls`.

* Refactored netrc loading to warn if host is not present.

* Tests and a refactor to support easier testing.

* Added reference to website

* Fixed two linting errors

* Fixed an extra line found by linting

* Updated file write to use binary to insure config files are written correctly

* Resolved linting issues

* Refactored config & identity loading to take advantage of plugin options

* Cleanup a bunch of small items caught by linting

* Removed extra line caught by linting

* Swapped in pytest and added some tests with mocked network responses

* Pushing to see if this approach works better...

* Refactored be open_url mocking based on feedback

* Fixed a couple linting issues & refactored mocking into each method to attempt to resolve a failing test

* Use a generic MagicMock for python 2.6

* Fixes doc typo

require -> required

* Use `type: path` in identity_file and config_file

Also removes `expanduser` calls below (which will now be called automatically on
paths.)

* Defines maintainers for conjur_variable plugin

* BOTMETA.yml:
** defines $team_cyberark_conjur as maintainers of Conjur Variable plugin
** adds myself and @jvanderhoof to that team

* Adds URLs to relevant documentation for Conjur Variable lookup plugin

* Clarifies "the server," "the machine" -> "controlling host"

The machine identity used is that of the Ansible controlling host, not any
server being provisioned or instructed. This documentation change aims to make
that relationship clear.

* Adds response code to exception message on authentication failure

* Enhances exception messages to specify the controlling host

These error messages are less likely to confuse a user as to which machine is
associated with the files, identities, and configurations being described.

* Adds ANSIBLE_METADATA for Conjur variable lookup plugin
7 years ago
Brian Coca f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
7 years ago
3onyc 554496c404 [passwordstore] Use builtin _random_password function instead of pwgen (#25843)
* [password] _random_password -> random_password and moved to util/encrypt.py
* [passwordstore] Use built-in random_password instead of pwgen utility
* [passwordstore] Add integration tests
7 years ago
Dag Wieers 4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
8 years ago
Toshio Kuratomi 2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
8 years ago
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
8 years ago
Matt Clay 95789f3949 PEP 8 whitespace cleanup. (#20783)
* PEP 8 E271 whitespace cleanup.
* PEP 8 W293 whitespace cleanup.
* Fix whitespace issue from recent PR.
8 years ago
azenk f2458140f6 Lastpass lookup plugin (#16285)
This plugin can be used with the lpass cli interface for lastpass.
[lastpass-cli](https://github.com/lastpass/lastpass-cli)

Example:
Add a lookup to your playbooks/variables somewhere:
```
some_variable: "{{ lookup('lastpass','Some Lastpass entry name or ID', field='username') }}"
```
Usage:
* start a lpass session prior to using ansible
* run ansible
* logout when finished

```
lpass login user@domain.com
ansible-playbook foo.yml
lpass logout
```
8 years ago
Matt Clay 8c270ac75f Add empty-init code-smell script. (#18406)
Also removed boilerplate from otherwise empty __init__.py files
which should not contain any code (checked by empty-init script).
8 years ago
Toshio Kuratomi 5e9a8d9202 Fix password lookup py3 plus alikins unittest additions refactoring (#17626)
* Improve unit testing of 'password' lookup

The tests showed some UnicodeErrors for the
cases where the 'chars' param include unicode,
causing the 'getattr(string, c, c)' to fail.
So the candidate char generation code try/excepts
UnicodeErrors there now.

Some refactoring of the password.py module to make
it easier to test, and some new tests that cover more
of the password and salt generation.

* More refactoring and fixes.

* manual merge of text enc fixes from pr17475

* moving methods to module scope

* more refactoring

* A few more text encoding fixes/merges

* remove now unused code

* Add test cases and data for _gen_candidate_chars

* more test coverage for password lookup

* wip

* More text encoding fixes and test coverage

* cleanups

* reenable text_type assert

* Remove unneeded conditional in _random_password

* Add docstring for _gen_candidate_chars

* remove redundant to_text and list comphenesion

* Move set of 'chars' default in _random_password

on py2, C.DEFAULT_PASSWORD_CHARS is a regular str
type, so the assert here fails. Move setting the
default into the method and to_text(DEFAULT_PASSWORD_CHARS)
if it's needed.

* combine _random_password and _gen_password

* s/_create_password_file/_create_password_file_dir

* native strings for exception msgs

* move password to_text to _read_password_file

* move to_bytes(content) to _write_password_file

* add more test assertions about genned pw's

* Some cleanups to alikins and abadger's password lookup refactoring:

* Make DEFAULT_PASSWORD_CHARS into a text string in constants.py
  - Move this into the nonconfigurable section of constants.
* Make utils.encrypt.do_encrypt() return a text string because all the
  hashes in passlib should be returning ascii-only strings and they are
  text strings in python3.
* Make the split up of functions more sane:
   - Don't split such that conditionals have to occur in two separate functions.
   - Don't go overboard: Good to split file system manipulation from parsing
     but we don't need to do every file manipulation in a separate
     function.
  - Don't split so that creation of the password store happens in two
    parts.
  - Don't split in such a way that no decisions are made in run.
* Organize functions by when it gets called from run().
* Run all potential characters through the gen_candidate_chars function
  because it does both normalization and validation.
* docstrings for functions
* Change when we store salt slightly.  Store it whenever it was already
  present in the file as well as when encrypt is requested.  This will
  head of potential idempotence bugs where a user has two playbook tasks
  using the same password and in one they need it encrypted but in the
  other they need it plaintext.
* Reorganize tests to follow the order of the functions so it's easier
  to figure out if/where a function has been tested.
* Add tests for the functions that read and write the password file.
* Add tests of run() when the password has already been created.
* Test coverage currently at 100%
8 years ago
jctanner 4ba60d00c8 Refactor the parameter splitting in ini lookup to handle more path formats (#16477)
Refactor the parameter splitting in ini lookup to handle more path formats.

Fixes #16468
9 years ago
Toshio Kuratomi e2c49b4ef4 Fix problem with "=" in the initial file path. 9 years ago
James Cammarata 21fa385ce7 Reorganizing plugin unit tests and adding start of strategy tests (v2) 10 years ago