Commit Graph

130 Commits (24d70469ab6f32aa2041a9b2794b4cc5f3e814ef)

Author SHA1 Message Date
Matt Martz 66a83314b9
Modernize install (#76021)
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Davis <mrd@redhat.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
3 years ago
Matt Martz 724800cd3f
Python 3.8 Controller Minimum (#74013) 3 years ago
Sam Doran c0cb353ce1
Remove PyCrypto (#74699)
* Remove PyCrypto from setup.py and packaging script
* Remove mention of pycrpto from installation docs
* Remove PyCrypto from vault
* Remove pycryto constraint and unit test requirement
* Remove PyCrypto tests from unit tests
* Add docs and fix warning message
* Remove section about cryptography library in Ansible Vault docs
4 years ago
Matt Martz 949b2cc74c
Update CI URL in setup.py (#73750)
* Update CI URL in setup.py

* AZP
4 years ago
Sviatoslav Sydorenko 5b4f0b3370 Declare support for Python 3.9 in the metadata
Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Rick Elrod 6894ae7d1d
Rename to ansible-core (#72594)
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base

Test Plan:
- ci_complete
4 years ago
Matt Martz 959af7d90b
Don't do conflict check on sdist and egg_info (#71310)
* Don't do conflict check on sdist and egg_info. Fixes #71279

* Add changelog

* adjust changelog PR link

Co-authored-by: Rick Elrod <rick@elrod.me>

* add warning about bdist_wheel

Co-authored-by: Rick Elrod <rick@elrod.me>
4 years ago
Matt Martz 54b002e1ac
Guard against allowing ansible to ansible-base upgrades (#70529)
* Guard against allowing ansible to ansible-base upgrades

* newline

* use alias

* Add an explicit line detailing this is a 1 time thing

* period

* Read __version__ and __author__ rather than import, update working, and add ability to skip conflict checks

* Remove commented code

* Re introduce removed changes from rebase

* Just use open

* Nuke unused import
4 years ago
Sviatoslav Sydorenko 918388b85f
Fix building Ansible dist w/ setuptools>=48,<49.1 (#70525)
* Fix building Ansible dist w/ setuptools>=48,<49.1

This change addresses the deprecation of the use of stdlib
`distutils`. It's a short-term hotfix for the problem and we'll
need to consider dropping the use of `distutils` from our `setup.py`.

Refs:
* https://github.com/ansible/ansible/issues/70456
* https://github.com/pypa/setuptools/issues/2230
* https://github.com/pypa/setuptools/commit/bd110264

Co-Authored-By: Jason R. Coombs <jaraco@jaraco.com>

* Add a change note for PR #70525

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
4 years ago
Matt Clay 234994fc07 Fix boilerplate in setup.py and lib/ansible/ dir. 5 years ago
Toshio Kuratomi 2268a37d4d Rename package to ansible-base 5 years ago
Matt Davis ed9de94ad9
remove azure extras and extras_require support (#67822)
* remove azure extras and extras_require support

* Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now.
* Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history.

* Mark azure-requirements as orhpaned.

This keeps the docs around so that existing links from old test runs remain valid.

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Sviatoslav Sydorenko ee290bf5b1 Declare support for Python 3.8 @ Trove classifiers 5 years ago
Toshio Kuratomi 5f227fe260 Install ansible-test (#60718)
* Install ansible-test

Modify the install script to install ansible-test and its supporting
code.  Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437

Also:
* No longer use package_data.  Everything in the package dirs is going
  to be installed.  Anything that shouldn't be installed needs to be
  moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
  instead of same directory

* Add ansible_test files to package-data sanity test

* MANIFEST.in cleanups

* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
  use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
  directories

* Change package-data test to be more complete

* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
  everything in the install is in the sdist in addition to comparing that
  everything in the repo that we want is in the install

* Leave out test artifacts

Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs

Remove test/utils files from the sdist as these are only needed for our CI

cleanup of docs in MANIFEST.in; getting rid of build files.

* Add the ability to output sdist and snapshot to specific directory

* Add a warning about modifying the heuristic to setup.py

* Address generated files

* Use make snapshot instead of sdist to generate changelog and man pages
  and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity

* Add cli web docs to make clean

This was causing problems when attempting to test that the sdist didn't
have extra files

* Fix bug constructing python names from __pycache__ names

* Create a clean repo to work from

* Exclude test/legacy and be more explicit on extensions

* Exclude the legacy directory from sdist
5 years ago
Matt Martz 95f4282c42
Verify package data in setup.py installs all files (#59537)
* Add sanity test to ensure all non-py files are installed

* Fix mode and regex

* Fix role skel inventory package_data

* Add docs

* Update package_data for inventory files

* Address pylint concerns

* Another tweak to package_data

* Address review feedback

* Change index to 1

* add to ansible-only.txt
5 years ago
Matt Martz 65bedaf074
Ensure package_data matches collections_galaxy_meta.yml. Fixes #59451 (#59452) 5 years ago
John R. Dennis 0a5a74f192 Install hidden galaxy data files (e.g. .travis.yml) (#56005)
The package_data globs in setup.py were missing the necessary glob
to install:

galaxy/data/default/.travis.yml

A Python glob pattern will not by default match a hidden file
(i.e. a file basename beginning with a dot). If you want a glob
to pick up a hidden file in a directory you must explicitly specify
a glob pattern with "/.*".

Closes: #1777
Signed-off-by: John Dennis <jdennis@redhat.com>
6 years ago
Toshio Kuratomi 5ba7063f4f
Build manpages as part of sdist (#53728)
* Build fixups

This is in pursuit of making snapshots easier.

* Allow overriding the python command used for generating man pages
* Build the changelog prior to creating the sdist
* Add the uninstalled ansible library to PYTHONPATH for changelog
  generation
* Warn that python setup.py sdist may be incomplete; use make sdist
  or make snapshot instead.
* Implement a snapshot make command

* Fix environ variable test to use a string

Co-Authored-By: abadger <a.badger@gmail.com>
6 years ago
Jordan Borean e972287c35 win_exec: refactor PS exec runner (#45334)
* win_exec: refactor PS exec runner

* more changes for PSCore compatibility

* made some changes based on the recent review

* split up module exec scripts for smaller payload

* removed C# module support to focus on just error msg improvement

* cleaned up c# test classifier code
6 years ago
Toshio Kuratomi ccf41bb2fe
Update the minimum python versions to install (#46230)
* Update the minimum python versions to install

This will prevent modern pip from installing ansible-2.7.x in
python-2.6.  Users will get the Ansible-2.6.x maintenance releases
instead.
6 years ago
mrmagooey 991f61c138 Improve symlink install error message in setup.py
PR #45132

Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
6 years ago
Alan Rominger 34b36d5178 add missing entry to package_data (#44585) 6 years ago
Sviatoslav Sydorenko fc25d280be
Add CoC and mailing lists links to PYPI
PR #42101
7 years ago
Sviatoslav Sydorenko 14cdf3b03c Suppress a UserWarning about unknown dist option 7 years ago
Sviatoslav Sydorenko 8912b43051 Refactor setup.py helper code 7 years ago
Sviatoslav Sydorenko 922e7039dc Shield setup module from accidental setup exec
This might happen when third-parties like analizers import this
module for testing purposes.
7 years ago
Sviatoslav Sydorenko 75092c9f29 Update python requirements in distribution meta
This has been discussed during #38413 review, but got missed.
7 years ago
Sviatoslav Sydorenko a173cf51e8 Add a list or extra URLs to distribution metadata
This should enable Warehouse PYPI implementation pypa/warehouse#3604
to pull data from GitHub and show it on the project page.
7 years ago
Toshio Kuratomi 988808c066 README.rst should always exist so no need to set a default value 7 years ago
Stephen Weber 7afa34ae8a Convert README from Markdown to ReStructured Text and use as longdesc (#22330)
* Convert README from Markdown to ReStructured Text and use as longdesc

Discussion in #13758 led to deciding to switch README to rst and having
setup.py consume it as the long_description.

* Fix long string in setup.py for pep8 compliance

* Open README.rst as read-only

* Update usages of root README.md to README.rst

Unsure about the file ./packaging/debian/docs as it (only) contains the
text README.md. I believe it's referencing ./packaging/debian/README.md
but maybe someone who knows debian packaging could review it?

* Pick up fixes that had been merged into README.md after the initial conversion to rst
7 years ago
Matt Martz bd43776acf
Add ansible-config and ansible-inventory to setup.py scripts (#37151) 7 years ago
Alan Rominger ef42eaeaa1 set the zip_safe flag to False (#32194)
The references to __file__ within code base make the zip_safe
method of installing not possible.
7 years ago
Toshio Kuratomi e132918b92 On Py3, nonexistent files raise OSError (#28899) 7 years ago
Brian Coca f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
7 years ago
Thomas Stringer e4cd899363 Add support for Azure 2.0.0 (#27920)
* Adapt azure_rm_resource_group to azure 2.0.0 + azure Cli support

* Fix exceptions in Azure ARM plugins

* update azure_rm_networkinterface documention to reflect required params

* change state param to not required for docs in azure_rm_subnet

* fix import to reflect azure==2.0.0 changes

* add aliases and fix docs for azure_rm_storageblob

* add resource_group_name alias to azure_rm_storageaccount_facts

* fix import bug due to change in azure==2.0.0

* fix args bug and enum modules issue

* update docs to reflect azure==2.0.0

* pin management clients to a specific api_version

* update docs to reflect the new azure-ansible-base python package

* add fallback for older api resource group listing

* rework azure dependencies installation

* refactor path joining to a cross-plat solution
7 years ago
Matt Davis dae8857d3d powershell setup fixes (#27516)
* fixes #27374
* recursively include top 2 levels of .psm1's under module_utils/powershell
* recursively include top 2 levels of .ps1's under modules/windows (for future restructuring)
7 years ago
Toshio Kuratomi eff757eeb8 Maintain symlinks in setup.py sdist, build, and install commands (#27149)
* Maintain symlinks in setup.py sdist, build, and install commands

Symlinks are meaningful for ansible modules.  They differentiate between
aliases and deprecated modules.  They're also useful for saving space
and where downstream patches should be applied to the bin scripts.

Fixes #27105

* Add a fallback for install and build to try to cache symlinks on their own

Needed when someone tries to invoke setup.py build or setup.py install
directly from the source checkout without an intermediate sdist.
7 years ago
Toshio Kuratomi e238ae999b Cyptography pr 20566 rebase (#25560)
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto

pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.

This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.

* Handle wrong password given for VaultAES format

* Do not display deprecation warning for cryptography on python-2.6

* Namespace all of the pycrypto imports and always import them

  Makes unittests better and the code less likely to get stupid mistakes
  (like using HMAC from cryptogrpahy when the one from pycrypto is needed)

* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko

* contrib/inventory/gce: Remove spurious require on pycrypto

(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)

* Add cryptography to ec2_win_password module requirements
  * Fix python3 bug which would pass text strings to a function which
    requires byte strings.

* Attempt to add pycrypto version to setup deps

* Change hacking README for dual pycrypto/cryptography

* update dependencies for various CI scripts

* additional CI dockerfile/script updates

* add paramiko to the windows and sanity requirement set

  This is needed because ansible lists it as a requirement. Previously
  the missing dep wasn't enforced, but cryptography imports pkg_resources
  so you can't ignore a requirement any more

* Add integration test cases for old vault and for wrong passwords

* helper script for manual testing of pycrypto/cryptography

* Skip the pycrypto tests so that users without it installed can still run the unittests

* Run unittests for vault with both cryptography and pycrypto backend
8 years ago
nemobis 5d28d76277 Set license GPLv3+ ("or any later version") in setup.py
Most file headers in the project contain the standard
> either version 3 of the License, or (at your option) any later version
https://github.com/ansible/ansible/search?utf8=%E2%9C%93&q=%22any+later+version%22

So this project appears to be GPLv3+, as stated in the classifiers function.
8 years ago
Brian Coca 74842adc07 1st part of ansible config, adds ansible-config to view/manage configs (#12797)
* Start of ansible config project

moved configuration definitions to external yaml file vs hardcoded
 * updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
 * prints green for default/unchanged and yellow for those that have been overriden
 * added list action to show all configurable settings and their associated ini and env var names
 * allows specifying config file to see what result would look like
 * TBD update, edit and view options

removed test for functions that have been removed

env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling

updated as per feedback

* pep8
8 years ago
Toshio Kuratomi 9bbd48ad5f Make setting of metadata for crypto backend settable at buildtime
Working on several separate ideas to make ansible work with both
cryptography or pycrypto.  This particular change will let a user
installing ansible change which crypto backend gets encoded in the
metadata (and hence, which backend is required by pkg_resource).

This works with both setup.py and pip::

  ANSIBLE_CRYPTO_BACKEND=cryptography pip install ansible
  ANSIBLE_CRYPTO_BACKEND=cryptography python2 ./setup.py

Note that the code to use cryptography is not yet merged so using
ANSIBLE_CRYPTO_BACKEND is not yet a good idea (as it can lead to not
installing pycrypto and hence the current vault code will fail).

Related to: #20566
8 years ago
Dag Wieers 4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
8 years ago
Adrian Likins 6f2cd64cff Include .git_keep files in role template skeleton (#24381)
setup.py will not install package_data this is just an empty
directory, even if the globs in package_data match it.

So the role skeleton that was being installed to galaxy/data/*/
was excluding the files/ and templates/ directories since they were
empty.

Since the skeleton dir doesnt include those dirs
'ansible-galaxy init' would not set them up.

So this adds a .* glob to those directories so the .git_keep
will be included as well, so that setup.py will create the otherwise
empty directoty. ansible-galaxy init already knows to ignore those
files, so no other changes are needed.

(Including the .git_keep files is a little odd, but the alternative
would be creating our own placeholder files. Since ansible-galaxy
already understands .git_keep files we just reuse that)

Fixes #23597
8 years ago
Toshio Kuratomi 57a7d7704f Make scripts into symlinks
pip can't handle zip files with symlinks.  Attempt to workaround that by
transforming the bin scripts into symlinks after the zip file has been
unarchived into the build tree.
8 years ago
Robin Schneider 3700bcb6dd Use HTTPS instead of legacy HTTP for ansible.com (#16870)
Mechanical edit done by this "one-liner":

```Shell
git ls-files -z "$(git rev-parse --show-toplevel)" | xargs --null -I '{}' find '{}' -type f -print0 | xargs --null sed --in-place --regexp-extended 's#http://(www\.|galaxy\.|)ansible\.com#https://\1ansible.com#g;'
```

Related to: https://github.com/ansible/ansible/issues/16869
8 years ago
Toshio Kuratomi d7b7cbac1a Move to using a requirements.txt to install the python packages. (#21430)
Move to using a requirements.txt to install the python packages.

This makes it easy to keep the documentation and actual package
dependencies in sync.

Fixes #18453
8 years ago
Matt Martz 87aa59af79 Legacy pep8 updates for setup.py and tests 8 years ago
Toshio Kuratomi d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
8 years ago
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
8 years ago
James Cammarata 4f55f032ea Updating setup.py to remove extras specific paths 8 years ago