18 Commits (54a795896ac6b234223650bc979fe65329b4405a)

Author	SHA1	Message	Date
Sloane Hertel	72ba2bdc82	ansible-galaxy - add a per-server validate_certs option (#75710) ... * Add a validate_certs option to galaxy server configuration * Add a unit test for cert validation * changelog	3 years ago
Sviatoslav Sydorenko	595413d113	Replace the inhouse collection dependency resolver with `resolvelib` ... PR #72591 This change: * Adds an artifacts manager that abstracts away extracting the metadata from artifacts, downloading and caching them in a temporary location. * Adds `resolvelib` to direct ansible-core dependencies[0]. * Implements a `resolvelib`-based dependency resolver for `collection` subcommands that replaces the legacy in-house code. This is a dependency resolution library that pip 20.3+ uses by default. It's now integrated for use for the collection dependency resolution in ansible-galaxy CLI. * Refactors of the `ansible-galaxy collection` CLI. In particular, it: - reimplements most of the `download`, `install`, `list` and `verify` subcommands from scratch; - reuses helper bits previously moved out into external modules; - replaces the old in-house resolver with a more clear implementation based on the resolvelib library[0][1][2]. * Adds a multi Galaxy API proxy layer that abstracts accessing the version and dependencies via API or local artifacts manager. * Makes `GalaxyAPI` instances sortable. * Adds string representation methods to `GalaxyAPI`. * Adds dev representation to `GalaxyAPI`. * Removes unnecessary integration and unit tests. * Aligns the tests with the new expectations. * Adds more tests, integration ones in particular. [0]: https://pypi.org/p/resolvelib [1]: https://github.com/sarugaku/resolvelib [2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing Co-Authored-By: Jordan Borean <jborean93@gmail.com> Co-Authored-By: Matt Clay <matt@mystile.com> Co-Authored-By: Sam Doran <sdoran@redhat.com> Co-Authored-By: Sloane Hertel <shertel@redhat.com> Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com> Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>	4 years ago
Jordan Borean	d30fc6c0b3	galaxy - preserve symlinks on build/install (#69959) ... * galaxy - preserve symlinks on build/install * Handle directory symlinks * py2 compat change * Updated changelog fragment	5 years ago
Sloane Hertel	e40889e711	Add support to install collections from git repositories (#69154) ... * Enable installing collections from git repositories * Add tests for installing individual and multiple collections from git repositories * Test to make sure recursive dependencies with different syntax are deduplicated * Add documentation * add a changelog * Skip Python 2.6 * Only fail if no collections are located in a git repository Add support for a 'type' key for collections in requirement.yml files. Update the changelog and document the supported keys and allowed values for the type. Add a note that the collection(s) in the repo must contain a galaxy.yml * Add a warning about embedding credentials in SCM URLs * Update with review suggestions * suppress sanity compile failure for Python 2.6	5 years ago
Jordan Borean	ecea15c508	Unify ansible-galaxy install -r (#67843) ... * Unify ansible-galaxy install -r * Minor nit fixes for docs * Re-align warnings * Fix up integration test * Fix up test where no roles/collections were in file	5 years ago
Sloane Hertel	343ffaa18b	better error for "ansible-galaxy collection verify" if there's no MANIFEST.json (#67498) ... * Add a better error for "ansible-galaxy verify" if the MANIFEST.json has been deleted from the installed collection or if the collection hasn't been installed via normal means * Fix unit tests for the remote collection If there's something wrong with the local collection's version it will fail before the remote collection is sought * Add a test for the new error msg * Prevent the duplicate warning Mock the new isfile call where needed * Update lib/ansible/galaxy/collection.py Co-Authored-By: Martin Krizek <martin.krizek@gmail.com> Co-authored-by: Martin Krizek <martin.krizek@gmail.com>	5 years ago
Jordan Borean	a20a527014	ansible-galaxy - Fix tar path traversal issue during install - CVE-2020-10691 (#68596)	5 years ago
Sloane Hertel	97f011cf94	Add verify subcommand to 'ansible-galaxy collection' (#65618) ... * [WIP] Add verify subcommand command to 'ansible-galaxy collection' * Fix pep8 and use consistent display order * WIP - docs * Remove some redundancy in verify display messages by using an error queue for each collection * Share common code and improve output format * clean up documentation * typo * Apply suggestions from code review Co-Authored-By: Sandra McCann <samccann@redhat.com> * Move ModifiedContent namedtuple to the global scope Add a public metadata property Rename function to _get_json_from_tar_file * Add some unit tests * fix using common functions after rebase * changelog * Improve efficiency finding specific installed collections Improve efficiency by only downloading the tar.gz from the galaxy server for comparison after checking that the collection has been installed Handle multiple collection paths Fix up tests * pep8 * reword that for accuracy * use more common code and verify collection name * Improve error message and add documentation * Update unit tests and add coverage for both local and remote collections that are unable to be located * Only validate collections using the format collection_namespace.collection_name Fix tests to reflect that Fix documentation * Fix a bug when a local collection does not contain a file * pep8 * Fix formatting Co-authored-by: Sandra McCann <samccann@redhat.com>	5 years ago
Sam Doran	9a51dff0b1	Rename private function (#66577) ... Make find_existing_collections() a public function since it is being used across mulitple files now to support listing and verifying collections	5 years ago
Jordan Borean	f8f7662850	Add the ability to ignore files and collection build (#64688)	5 years ago
Toshio Kuratomi	4cad7e479c	Galaxy publish fix (#63580) ... * Handle galaxy v2/v3 API diffs for artifact publish response For publishing a collection artifact (POST /v3/collections/artifacts/), the response format is different between v2 and v3. For v2 galaxy, the 'task' url returned is a full url with scheme: {"task": "https://galaxy-dev.ansible.com/api/v2/collection-imports/35573/"} For v3 galaxy, the task url is relative: {"task": "/api/automation-hub/v3/imports/collections/838d1308-a8f4-402c-95cb-7823f3806cd8/"} So check which API we are using and update the task url approriately. * Use full url for all wait_for_import messages Update unit tests to parameterize the expected responses and urls. * update explanatory comment * Rename n_url to full_url. * Fix issue with overwrite of the complete path * Fixes overwrite of the complete path in case there's extra path stored in self.api_sever * Normalizes the input to the wait_import_task function so it receives the same value on both v2 and v3 Builds on #63523 * Update unittests for new call signature * Add changelog for ansible-galaxy publish API fixes.	5 years ago
Jordan Borean	a7fd6e99d9	Refactor galaxy collection API for v3 support (#61510) ... * Refactor galaxy collection API for v3 support * Added unit tests for GalaxyAPI and starting to fix other failures * finalise tests * more unit test fixes	5 years ago
Adrian Likins	af01cb114c	Support galaxy v3/autohub API in ansible-galaxy (#60982) ... * Add galaxy collections API v3 support Issue: ansible/galaxy-dev#60 - Determine if server supports v3 Use 'available_versions' from `GET /api` to determine if 'v3' api is available on the server. - Support v3 pagination style ie, 'limit/offset style', with the paginated responses based on https://jsonapi.org/format/#fetching-pagination v2 galaxy uses pagination that is more or less 'django rest framework style' or 'page/page_size style', based on the default drf pagination described at https://www.django-rest-framework.org/api-guide/pagination/#pagenumberpagination - Support galaxy v3 style error response The error objects returned by the galaxy v3 api are based on the JSONAPI response/errors format (https://jsonapi.org/format/#errors). This handles that style response. At least for publish_collection for now. Needs extracting/generalizing. Handle HTTPError in CollectionRequirement.from_name() with _handle_http_error(). It will raise AnsibleError based on the json in an error response. - Update unit tests update test/unit/galaxy/test_collection* to paramaterize calls to test against mocked v2 and v3 servers apis. Update artifacts_versions_json() to tale an api version paramater. Add error_json() for generating v3/v3 style error responses. So now, the urls generated and the pagination schema of the response will use the v3 version if the passed in GalaxyAPI 'galaxy_api' instance has 'v3' in it's available_api_versions * Move checking of server avail versions to collections.py collections.py needs to know the server api versions supported before it makes collection related calls, so the 'lazy' server version check in api.GalaxyAPI is never called and isn't set, so 'v3' servers weren't found. Update unit tests to mock the return value of the request instead of GalaxyAPI itself.	5 years ago
Jordan Borean	e04b2a9697	ansible-galaxy - Add timeout and progress indicator for publish and install (#60660) ... * ansible-galaxy - Add timeout and progress indicator for publish * add progress indicator to install phase as well	5 years ago
Jordan Borean	e747487720	ansible-galaxy - define multiple galaxy instances in ansible.cfg (#60553) ... * ansible-galaxy: support multiple servers on install * Added docs for the server configuration file * Fix up doc string for requirements file format * Fix bugs after testing * Fix kwarg doc and added version * Fix typo and doc improvement * Fix base64 encoding and allow --server to override list	5 years ago
Jordan Borean	aa0de421d2	Ignore collection build release files in the root collection directory (#59121)	5 years ago
Jordan Borean	65049620ee	Generate galaxy.yml based on single source of truth (#59170) ... * Generate galaxy.yml based on single source of truth * Fix up tests and align file names * Minor Makefile tweak * Remove link in galaxy.yml file and make it a template file * Moved collections docs to dev_guide * change Makefile clean path * Added readme to example meta file * review fixes * Use newer style for doc generation script * Fix mistake in dev_guide index * removed uneeded file, fixed links and added preview banner * Moved banner for sanity test	5 years ago
Jordan Borean	b6791e6ae3	ansible-galaxy: add collection sub command (#57106) ... * ansible-galaxy: add collection init sub command * Fix changelog and other sanity issues * Slim down skeleton structure, fix encoding issue on template * Fix doc generation code to include sub commands * Added build step * Tidy up the build action * Fixed up doc changes and slight testing tweaks * Re-organise tests to use pytest * Added publish step and fixed up issues after working with Galaxy * Unit test improvments * Fix unit test on 3.5 * Add remaining build tests * Test fixes, make the integration tests clearer to debug on failures * Removed unicode name tests until I've got further clarification * Added publish unit tests * Change expected length value * Added collection install steps, tests forthcoming * Added unit tests for collection install entrypoint * Added some more tests for collection install * follow proper encoding rules and added more tests * Add remaining tests * tidied up tests and code based on review * exclude pre-release versions from galaxy API	5 years ago