PR #72591
This change:
* Adds an artifacts manager that abstracts away extracting the
metadata from artifacts, downloading and caching them in a
temporary location.
* Adds `resolvelib` to direct ansible-core dependencies[0].
* Implements a `resolvelib`-based dependency resolver for
`collection` subcommands that replaces the legacy
in-house code.
This is a dependency resolution library that pip 20.3+ uses
by default. It's now integrated for use for the collection
dependency resolution in ansible-galaxy CLI.
* Refactors of the `ansible-galaxy collection` CLI.
In particular, it:
- reimplements most of the `download`, `install`, `list` and
`verify` subcommands from scratch;
- reuses helper bits previously moved out into external modules;
- replaces the old in-house resolver with a more clear
implementation based on the resolvelib library[0][1][2].
* Adds a multi Galaxy API proxy layer that abstracts accessing the
version and dependencies via API or local artifacts manager.
* Makes `GalaxyAPI` instances sortable.
* Adds string representation methods to `GalaxyAPI`.
* Adds dev representation to `GalaxyAPI`.
* Removes unnecessary integration and unit tests.
* Aligns the tests with the new expectations.
* Adds more tests, integration ones in particular.
[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries
* remove dead code/config, update messages and porting guides
* Enable installing collections from git repositories
* Add tests for installing individual and multiple collections from git repositories
* Test to make sure recursive dependencies with different syntax are deduplicated
* Add documentation
* add a changelog
* Skip Python 2.6
* Only fail if no collections are located in a git repository
Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.
Add a note that the collection(s) in the repo must contain a galaxy.yml
* Add a warning about embedding credentials in SCM URLs
* Update with review suggestions
* suppress sanity compile failure for Python 2.6
* Unify ansible-galaxy install -r
* Minor nit fixes for docs
* Re-align warnings
* Fix up integration test
* Fix up test where no roles/collections were in file
* ansible-galaxy: support multiple servers on install
* Added docs for the server configuration file
* Fix up doc string for requirements file format
* Fix bugs after testing
* Fix kwarg doc and added version
* Fix typo and doc improvement
* Fix base64 encoding and allow --server to override list
* ansible-galaxy tidy up arg parse with better validation
* Add support back in for -v before sub aprser
* Added deprecation warning for manually parsed verbosity
* Generate galaxy.yml based on single source of truth
* Fix up tests and align file names
* Minor Makefile tweak
* Remove link in galaxy.yml file and make it a template file
* Moved collections docs to dev_guide
* change Makefile clean path
* Added readme to example meta file
* review fixes
* Use newer style for doc generation script
* Fix mistake in dev_guide index
* removed uneeded file, fixed links and added preview banner
* Moved banner for sanity test
* ansible-galaxy: add collection init sub command
* Fix changelog and other sanity issues
* Slim down skeleton structure, fix encoding issue on template
* Fix doc generation code to include sub commands
* Added build step
* Tidy up the build action
* Fixed up doc changes and slight testing tweaks
* Re-organise tests to use pytest
* Added publish step and fixed up issues after working with Galaxy
* Unit test improvments
* Fix unit test on 3.5
* Add remaining build tests
* Test fixes, make the integration tests clearer to debug on failures
* Removed unicode name tests until I've got further clarification
* Added publish unit tests
* Change expected length value
* Added collection install steps, tests forthcoming
* Added unit tests for collection install entrypoint
* Added some more tests for collection install
* follow proper encoding rules and added more tests
* Add remaining tests
* tidied up tests and code based on review
* exclude pre-release versions from galaxy API
* Start of migration to argparse
* various fixes and improvements
* Linting fixes
* Test fixes
* Fix vault_password_files
* Add PrependAction for argparse
* A bunch of additional tweak/fixes
* Fix ansible-config tests
* Fix man page generation
* linting fix
* More adhoc pattern fixes
* Add changelog fragment
* Add support for argcomplete
* Enable argcomplete global completion
* Rename PrependAction to PrependListAction to better describe what it does
* Add documentation for installing and configuring argcomplete
* Address rebase issues
* Fix display encoding for vault
* Fix line length
* Address rebase issues
* Handle rebase issues
* Use mutually exclusive group instead of handling manually
* Fix rebase issues
* Address rebase issue
* Update version added for argcomplete support
* -e must be given a value
* ci_complete
* Note: Python2 is not as intelligent at detecting false import loops as
Python3. context_objects.py cannot be added to cli/arguments because it
would set up an import loop between cli/__init__.py,
cli/arguments/context_objects.py, and context.py on Python2.
ci_complete
The goal of breaking apart the base_parser() function is to get rid of
a bunch of conditionals and parameters in the code and, instead, make
code look like simple composition.
When splitting, a choice had to be made as to whether this would operate
by side effect (modifying a passed in parser) or side effect-free
(returning a new parser everytime).
Making a version that's side-effect-free appears to be fighting with the
optparse API (it wants to work by creating a parser object, configuring
the object, and then parsing the arguments with it) so instead, make it
clear that our helper functions are modifying the passed in parser by
(1) not returning the parser and (2) changing the function names to be
more clear that it is operating by side-effect.
Also move all of the generic optparse code, along with the argument
context classes, into a new subdirectory.
* Once cli args are parsed, they're constant. So, save the parsed args
into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
* Got rid of the private self._options attribute
* Use context.CLIARGS in the individual callback plugins instead.
* Also output positional arguments in default and unixy plugins
* Code has been simplified since we're now dealing with a dict rather
than Optparse.Value
* Move ansible.compat.tests to test/units/compat/.
* Fix unit test references to ansible.compat.tests.
* Move builtins compat to separate file.
* Fix classification of test/units/compat/ dir.
* Fix CNOS unit test log usage.
* Use temp dir for Galaxy unit tests.
* Write to temp files in interfaces_file unit test.
* Fix log placement in netapp_e_ldap unit test.
* galaxy unit tests: add missing program name
* galaxy unit tests: remove incorrect switchs
- 'install' action doesn't support '--offline' switch
- 'remove' action doesn't support '--init-path' switch
finished normalizing of path handling
removed overloaded '-p' from init_paths option, it is for role_paths
removed expand_tilde and get_opt methods as both were redundant, adjusted rest of code
updated tests to match
* Start of ansible config project
moved configuration definitions to external yaml file vs hardcoded
* updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
* prints green for default/unchanged and yellow for those that have been overriden
* added list action to show all configurable settings and their associated ini and env var names
* allows specifying config file to see what result would look like
* TBD update, edit and view options
removed test for functions that have been removed
env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling
updated as per feedback
* pep8
* test/: PEP8 compliancy
- Make PEP8 compliant
* Python3 chokes on casting int to bytes (#24952)
But if we tell the formatter that the var is a number, it works
* Update module_utils.six to latest
We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility. Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
* Add role skeleton support
The default role skeleton used by ansible-galaxy is good, but it doesn't
allow organizations to customize it to suit their needs. This patch
addresses that by adding the option to point ansible-galaxy at a
role skeleton directory. The contents of this directory are then
copied (or rendered) into the output role directory. Appropriate
command line options and configuration entries are added to allow for
further customization from the role skeleton.
* Remove dependency on platforms list from test
Platforms are no longer provided to the template by galaxy init.
Removing the code in our test meta/main.yml template that relied on it.
* Fix whitespace issue
* Making tests more uniform
Removing unnecessary GalaxyCLI arguments/patching of the command line since parsing of the CLI args has been modified.
Run GalaxyCLI.parse() without saving the returned value to be uniform with the rest of the code.
Fix test_execute_remove to use the correct path
Use GalaxyCLI.run() instead of super(GalaxyCLI, gc).run() and GalaxyCLI.api = ansible.galaxy.api.GalaxyAPI(gc.galaxy).
* Refactor so one unit test checks one thing instead of multiple.
Improve readability by using a dict instead of lots of elifs
* Removing import used for debugging
* Fixing PEP 8 issues.
* Fix PEP 8 issues
Implement tag and skip_tag handling in the CLI() class. Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.
* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
default will change in 2.4 and go away in 2.5.
* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
Make some python3 fixes to make the unittests pass:
* galaxy imports
* dictionary iteration in role requirements
* swap_stdout helper for unittests
* Normalize to text string in a facts.py function
Adds a new test for GalaxyCLI. Tests cases when:
- no actions are provided
- invalid actions are provided
- each valid action is provided
When parser() runs successfully the method returns True, creates a
SortedOptParser instance, a Galaxy instance, and updates certain class data.
* Adding a test for GalaxyCLI and two methods on which it depends. Tests execute_info method.
* Adding a test for GalaxyCLI and two methods on which it depends. Tests execute_remove method.
* Update test_galaxy.py
* Revising test for GalaxyCLI execute_remove.
* Removing mocks
Matt Clay