Commit Graph

16 Commits (53847bd5291c6ddc8c67e8d63a52e351072d2c1e)

Author SHA1 Message Date
Adrian Likins 43cf40ffe0 [stable-2.9] Don't remove trailing / on galaxy api URLS (#63294)
This causes multiple unneeded http redirects.
Since each http redirect takes a few hundred
milliseconds, handling the redirects starts to
take up a significant amount of wall time.

Fixes: #63281
(cherry picked from commit d1c39b9068)

Co-authored-by: Adrian Likins <alikins@redhat.com>
5 years ago
Jordan Borean 44a5b20c67 ansible-galaxy - expand User-Agent string for Galaxy endpoints (#65578)
(cherry picked from commit eaba5572cd)
5 years ago
Jordan Borean 03b5a6f5d6 galaxy - print HTTP reason on unknown error (#65273) (#65439)
(cherry picked from commit 6586b7132c)
5 years ago
Jordan Borean 93d9a091a3 [stable-2.9] Fix up role version pagination for Galaxy install (#64373)
* Fix up role version pagination for Galaxy install

* Fix sanity issue
(cherry picked from commit 7acae62fa8)

Co-authored-by: Jordan Borean <jborean93@gmail.com>
5 years ago
Toshio Kuratomi db5382341d [stable-2.9] AH servers include automation-hub as part of the server configuration. (#63700)
* AH servers include automation-hub as part of the server configuration.

So we don't need to add it here.

Fixes #63699

* Update unittests for the fix to galaxy wait_import_task
(cherry picked from commit cc1ff57)

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
5 years ago
Toshio Kuratomi 741b6e65ef [stable-2.9] Galaxy publish fix (#63580)
* Handle galaxy v2/v3 API diffs for artifact publish response

For publishing a collection artifact
(POST /v3/collections/artifacts/), the response
format is different between v2 and v3.

For v2 galaxy, the 'task' url returned is
a full url with scheme:

        {"task": "https://galaxy-dev.ansible.com/api/v2/collection-imports/35573/"}

For v3 galaxy, the task url is relative:

        {"task": "/api/automation-hub/v3/imports/collections/838d1308-a8f4-402c-95cb-7823f3806cd8/"}

So check which API we are using and update the task url approriately.

* Use full url for all wait_for_import messages

Update unit tests to parameterize the expected
responses and urls.

* update explanatory comment

* Rename n_url to full_url.

* Fix issue with overwrite of the complete path

* Fixes overwrite of the complete path in case there's extra path stored
  in self.api_sever
* Normalizes the input to the wait_import_task function so it receives
  the same value on both v2 and v3

Builds on #63523

* Update unittests for new call signature

* Add changelog for ansible-galaxy publish API fixes.
(cherry picked from commit 4cad7e4)

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
5 years ago
Adrian Likins ae3a79fa58 backport of galaxy trailing api fix (63238) to stable-2.9 (#63293)
* Stop appending '/api' to galaxy server url (#63238)

* Stop appending '/api' to configured galaxy server url

Since not all galaxy REST api server URLs live
at '/api', stop always appending it to the
'url' value loaded from config.

* Add note about manually migrated galaxy configs and /api

* Add '/api/' to galaxy url and guessing if galaxy API

* Fix most unit tests (update to expect /api/)

* Fix test_initialise_unknown unit test

Since we retry now with an added /api/, mock it as well.

* Update fallback default avail_ver to new format

(cherry picked from commit bad72693e4)

* Add changelog fragment galaxy_api_config

* Fix galaxy url use everywhere when url is set in config. (#63286)

In addition to trying the configured url (for ex, a migrated
'https://galaxy.ansible.com/') there is an attempt at that
URL with '/api' postpended.

If the extended URL works, update GalaxyAPI.api_server to
use the extended URL. Previously it only used it for finding
the API root info ('available_versions', etc)

(cherry picked from commit ed203c5902)
5 years ago
Adrian Likins 6cdd8e5fc2 Add support for automation-hub authentication to ansible-galaxy (#63031)
Adds support for token authentication in Automation Hub. Fixes: ansible/galaxy-dev#96

(cherry picked from commit 239d639fee)
5 years ago
Jordan Borean a8023c52d9 Galaxy refactory 2.9 (#61581)
* Refactor galaxy collection API for v3 support (#61510)

* Refactor galaxy collection API for v3 support

* Added unit tests for GalaxyAPI and starting to fix other failures

* finalise tests

* more unit test fixes

(cherry picked from commit a7fd6e99d9)

* Added changelog fragment
5 years ago
Adrian Likins af01cb114c
Support galaxy v3/autohub API in ansible-galaxy (#60982)
* Add galaxy collections API v3 support

Issue: ansible/galaxy-dev#60

- Determine if server supports v3

Use 'available_versions' from `GET /api`
to determine if 'v3' api is available on
the server.

- Support v3 pagination style

ie, 'limit/offset style', with the paginated
responses based on https://jsonapi.org/format/#fetching-pagination

v2 galaxy uses pagination that is more or less
'django rest framework style' or 'page/page_size style',
based on the default drf pagination described
at https://www.django-rest-framework.org/api-guide/pagination/#pagenumberpagination

- Support galaxy v3 style error response

The error objects returned by the galaxy v3 api are based
on the JSONAPI response/errors format
(https://jsonapi.org/format/#errors).

This handles that style response. At least for publish_collection
for now. Needs extracting/generalizing.

Handle HTTPError in CollectionRequirement.from_name()
with _handle_http_error(). It will raise AnsibleError
based on the json in an error response.

- Update unit tests

update test/unit/galaxy/test_collection*
to paramaterize calls to test against
mocked v2 and v3 servers apis.

Update artifacts_versions_json() to tale an
api version paramater.

Add error_json() for generating v3/v3 style error
responses.

So now, the urls generated and the pagination schema
of the response will use the v3 version if
the passed in GalaxyAPI 'galaxy_api' instance
has 'v3' in it's available_api_versions

* Move checking of server avail versions to collections.py

collections.py needs to know the server api versions
supported before it makes collection related calls,
so the 'lazy' server version check in api.GalaxyAPI
is never called and isn't set, so 'v3' servers weren't
found.

Update unit tests to mock the return value of the
request instead of GalaxyAPI itself.
5 years ago
Jordan Borean e04b2a9697
ansible-galaxy - Add timeout and progress indicator for publish and install (#60660)
* ansible-galaxy - Add timeout and progress indicator for publish

* add progress indicator to install phase as well
5 years ago
Jordan Borean e747487720
ansible-galaxy - define multiple galaxy instances in ansible.cfg (#60553)
* ansible-galaxy: support multiple servers on install

* Added docs for the server configuration file

* Fix up doc string for requirements file format

* Fix bugs after testing

* Fix kwarg doc and added version

* Fix typo and doc improvement

* Fix base64 encoding and allow --server to override list
5 years ago
Jordan Borean ba7097cba3
Collection test (#59488)
* Add missing __init__.py for collection unit tests

* Added more tests for collection install and remove default path
5 years ago
Jordan Borean aa0de421d2
Ignore collection build release files in the root collection directory (#59121) 5 years ago
Jordan Borean 65049620ee
Generate galaxy.yml based on single source of truth (#59170)
* Generate galaxy.yml based on single source of truth

* Fix up tests and align file names

* Minor Makefile tweak

* Remove link in galaxy.yml file and make it a template file

* Moved collections docs to dev_guide

* change Makefile clean path

* Added readme to example meta file

* review fixes

* Use newer style for doc generation script

* Fix mistake in dev_guide index

* removed uneeded file, fixed links and added preview banner

* Moved banner for sanity test
5 years ago
Jordan Borean b6791e6ae3
ansible-galaxy: add collection sub command (#57106)
* ansible-galaxy: add collection init sub command

* Fix changelog and other sanity issues

* Slim down skeleton structure, fix encoding issue on template

* Fix doc generation code to include sub commands

* Added build step

* Tidy up the build action

* Fixed up doc changes and slight testing tweaks

* Re-organise tests to use pytest

* Added publish step and fixed up issues after working with Galaxy

* Unit test improvments

* Fix unit test on 3.5

* Add remaining build tests

* Test fixes, make the integration tests clearer to debug on failures

* Removed unicode name tests until I've got further clarification

* Added publish unit tests

* Change expected length value

* Added collection install steps, tests forthcoming

* Added unit tests for collection install entrypoint

* Added some more tests for collection install

* follow proper encoding rules and added more tests

* Add remaining tests

* tidied up tests and code based on review

* exclude pre-release versions from galaxy API
5 years ago